Статті в журналах: "Intrinsic dimensionality estimation"

1

LI, CHUN-GUANG, JUN GUO, and BO XIAO. "INTRINSIC DIMENSIONALITY ESTIMATION WITHIN NEIGHBORHOOD CONVEX HULL." International Journal of Pattern Recognition and Artificial Intelligence 23, no. 01 (February 2009): 31–44. http://dx.doi.org/10.1142/s0218001409007016.

Повний текст джерела

Анотація:

In this paper, a novel method to estimate the intrinsic dimensionality of high-dimensional data set is proposed. Based on neighborhood information, our method calculates the non-negative locally linear reconstruction coefficients from its neighbors for each data point, and the numbers of those dominant positive reconstruction coefficients are regarded as a faithful guide to the intrinsic dimensionality of data set. The proposed method requires no parametric assumption on data distribution and is easy to implement in the general framework of manifold learning. Experimental results on several synthesized data sets and real data sets have shown the benefits of the proposed method.

Стилі APA, Harvard, Vancouver, ISO та ін.

2

He, Jinrong, Lixin Ding, Lei Jiang, Zhaokui Li, and Qinghui Hu. "Intrinsic dimensionality estimation based on manifold assumption." Journal of Visual Communication and Image Representation 25, no. 5 (July 2014): 740–47. http://dx.doi.org/10.1016/j.jvcir.2014.01.006.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

3

Bruske, J., and G. Sommer. "Intrinsic dimensionality estimation with optimally topology preserving maps." IEEE Transactions on Pattern Analysis and Machine Intelligence 20, no. 5 (May 1998): 572–75. http://dx.doi.org/10.1109/34.682189.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

4

Cordes, Dietmar, and Rajesh R. Nandy. "Estimation of the intrinsic dimensionality of fMRI data." NeuroImage 29, no. 1 (January 2006): 145–54. http://dx.doi.org/10.1016/j.neuroimage.2005.07.054.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

5

Amsaleg, Laurent, Oussama Chelly, Teddy Furon, Stéphane Girard, Michael E. Houle, Ken-ichi Kawarabayashi, and Michael Nett. "Extreme-value-theoretic estimation of local intrinsic dimensionality." Data Mining and Knowledge Discovery 32, no. 6 (July 27, 2018): 1768–805. http://dx.doi.org/10.1007/s10618-018-0578-6.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

6

Altan, Ege, Sara A. Solla, Lee E. Miller, and Eric J. Perreault. "Estimating the dimensionality of the manifold underlying multi-electrode neural recordings." PLOS Computational Biology 17, no. 11 (November 29, 2021): e1008591. http://dx.doi.org/10.1371/journal.pcbi.1008591.

Повний текст джерела

Анотація:

It is generally accepted that the number of neurons in a given brain area far exceeds the number of neurons needed to carry any specific function controlled by that area. For example, motor areas of the human brain contain tens of millions of neurons that control the activation of tens or at most hundreds of muscles. This massive redundancy implies the covariation of many neurons, which constrains the population activity to a low-dimensional manifold within the space of all possible patterns of neural activity. To gain a conceptual understanding of the complexity of the neural activity within a manifold, it is useful to estimate its dimensionality, which quantifies the number of degrees of freedom required to describe the observed population activity without significant information loss. While there are many algorithms for dimensionality estimation, we do not know which are well suited for analyzing neural activity. The objective of this study was to evaluate the efficacy of several representative algorithms for estimating the dimensionality of linearly and nonlinearly embedded data. We generated synthetic neural recordings with known intrinsic dimensionality and used them to test the algorithms’ accuracy and robustness. We emulated some of the important challenges associated with experimental data by adding noise, altering the nature of the embedding of the low-dimensional manifold within the high-dimensional recordings, varying the dimensionality of the manifold, and limiting the amount of available data. We demonstrated that linear algorithms overestimate the dimensionality of nonlinear, noise-free data. In cases of high noise, most algorithms overestimated the dimensionality. We thus developed a denoising algorithm based on deep learning, the “Joint Autoencoder”, which significantly improved subsequent dimensionality estimation. Critically, we found that all algorithms failed when the intrinsic dimensionality was high (above 20) or when the amount of data used for estimation was low. Based on the challenges we observed, we formulated a pipeline for estimating the dimensionality of experimental neural data.

Стилі APA, Harvard, Vancouver, ISO та ін.

7

Heylen, Rob, and Paul Scheunders. "Hyperspectral Intrinsic Dimensionality Estimation With Nearest-Neighbor Distance Ratios." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 6, no. 2 (April 2013): 570–79. http://dx.doi.org/10.1109/jstars.2013.2256338.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

8

Wang, Xiaorong, and Aiqing Xu. "Intrinsic Dimensionality Estimation for Data Points in Local Region." Sankhya B 81, no. 1 (March 15, 2018): 123–32. http://dx.doi.org/10.1007/s13571-018-0156-3.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

9

Karbauskaitė, Rasa, and Gintautas Dzemyda. "Geodesic distances in the intrinsic dimensionality estimation using packing numbers." Nonlinear Analysis: Modelling and Control 19, no. 4 (December 10, 2014): 578–91. http://dx.doi.org/10.15388/na.2014.4.4.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

10

Benkő, Zsigmond, Marcell Stippinger, Roberta Rehus, Attila Bencze, Dániel Fabó, Boglárka Hajnal, Loránd G. Eröss, András Telcs, and Zoltán Somogyvári. "Manifold-adaptive dimension estimation revisited." PeerJ Computer Science 8 (January 6, 2022): e790. http://dx.doi.org/10.7717/peerj-cs.790.

Повний текст джерела

Анотація:

Data dimensionality informs us about data complexity and sets limit on the structure of successful signal processing pipelines. In this work we revisit and improve the manifold adaptive Farahmand-Szepesvári-Audibert (FSA) dimension estimator, making it one of the best nearest neighbor-based dimension estimators available. We compute the probability density function of local FSA estimates, if the local manifold density is uniform. Based on the probability density function, we propose to use the median of local estimates as a basic global measure of intrinsic dimensionality, and we demonstrate the advantages of this asymptotically unbiased estimator over the previously proposed statistics: the mode and the mean. Additionally, from the probability density function, we derive the maximum likelihood formula for global intrinsic dimensionality, if i.i.d. holds. We tackle edge and finite-sample effects with an exponential correction formula, calibrated on hypercube datasets. We compare the performance of the corrected median-FSA estimator with kNN estimators: maximum likelihood (Levina-Bickel), the 2NN and two implementations of DANCo (R and MATLAB). We show that corrected median-FSA estimator beats the maximum likelihood estimator and it is on equal footing with DANCo for standard synthetic benchmarks according to mean percentage error and error rate metrics. With the median-FSA algorithm, we reveal diverse changes in the neural dynamics while resting state and during epileptic seizures. We identify brain areas with lower-dimensional dynamics that are possible causal sources and candidates for being seizure onset zones.

Стилі APA, Harvard, Vancouver, ISO та ін.

11

Bac, Jonathan, Evgeny M. Mirkes, Alexander N. Gorban, Ivan Tyukin, and Andrei Zinovyev. "Scikit-Dimension: A Python Package for Intrinsic Dimension Estimation." Entropy 23, no. 10 (October 19, 2021): 1368. http://dx.doi.org/10.3390/e23101368.

Повний текст джерела

Анотація:

Dealing with uncertainty in applications of machine learning to real-life data critically depends on the knowledge of intrinsic dimensionality (ID). A number of methods have been suggested for the purpose of estimating ID, but no standard package to easily apply them one by one or all at once has been implemented in Python. This technical note introduces scikit-dimension, an open-source Python package for intrinsic dimension estimation. The scikit-dimension package provides a uniform implementation of most of the known ID estimators based on the scikit-learn application programming interface to evaluate the global and local intrinsic dimension, as well as generators of synthetic toy and benchmark datasets widespread in the literature. The package is developed with tools assessing the code quality, coverage, unit testing and continuous integration. We briefly describe the package and demonstrate its use in a large-scale (more than 500 datasets) benchmarking of methods for ID estimation for real-life and synthetic data.

Стилі APA, Harvard, Vancouver, ISO та ін.

12

Dengyao Mo and S. H. Huang. "Fractal-Based Intrinsic Dimension Estimation and Its Application in Dimensionality Reduction." IEEE Transactions on Knowledge and Data Engineering 24, no. 1 (January 2012): 59–71. http://dx.doi.org/10.1109/tkde.2010.225.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

13

Alizadeh Naeini, Amin, Saeid Homayouni, and Mohammad Saadatseresht. "Intrinsic Dimensionality Estimation in Hyperspectral Imagery Using Residual and Change-Point Analyses." IEEE Geoscience and Remote Sensing Letters 11, no. 11 (November 2014): 2005–9. http://dx.doi.org/10.1109/lgrs.2014.2317352.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

14

Yata, Kazuyoshi, and Makoto Aoshima. "Intrinsic Dimensionality Estimation of High-Dimension, Low Sample Size Data withD-Asymptotics." Communications in Statistics - Theory and Methods 39, no. 8-9 (April 21, 2010): 1511–21. http://dx.doi.org/10.1080/03610920903121999.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

15

Lei, Liang, TongQing Wang, Jun Peng, and Bo Yang. "Image Dimensionality Reduction Based on the Intrinsic Dimension and Parallel Genetic Algorithm." International Journal of Cognitive Informatics and Natural Intelligence 5, no. 2 (April 2011): 97–112. http://dx.doi.org/10.4018/jcini.2011040106.

Повний текст джерела

Анотація:

In the research of Web content-based image retrieval, how to reduce more of the image dimensions without losing the main features of the image is highlighted. Many features of dimensional reduction schemes are determined by the breaking of higher dimensional general covariance associated with the selection of a particular subset of coordinates. This paper starts with analysis of commonly used methods for the dimension reduction of Web images, followed by a new algorithm for nonlinear dimensionality reduction based on the HSV image features. The approach obtains intrinsic dimension estimation by similarity calculation of two images. Finally, some improvements were made on the Parallel Genetic Algorithm (APGA) by use of the image similarity function as the self-adaptive judgment function to improve the genetic operators, thus achieving a Web image dimensionality reduction and similarity retrieval. Experimental results illustrate the validity of the algorithm.

Стилі APA, Harvard, Vancouver, ISO та ін.

16

Campadelli, P., E. Casiraghi, C. Ceruti, and A. Rozza. "Intrinsic Dimension Estimation: Relevant Techniques and a Benchmark Framework." Mathematical Problems in Engineering 2015 (2015): 1–21. http://dx.doi.org/10.1155/2015/759567.

Повний текст джерела

Анотація:

When dealing with datasets comprising high-dimensional points, it is usually advantageous to discover some data structure. A fundamental information needed to this aim is the minimum number of parameters required to describe the data while minimizing the information loss. This number, usually called intrinsic dimension, can be interpreted as the dimension of the manifold from which the input data are supposed to be drawn. Due to its usefulness in many theoretical and practical problems, in the last decades the concept of intrinsic dimension has gained considerable attention in the scientific community, motivating the large number of intrinsic dimensionality estimators proposed in the literature. However, the problem is still open since most techniques cannot efficiently deal with datasets drawn from manifolds of high intrinsic dimension and nonlinearly embedded in higher dimensional spaces. This paper surveys some of the most interesting, widespread used, and advanced state-of-the-art methodologies. Unfortunately, since no benchmark database exists in this research field, an objective comparison among different techniques is not possible. Consequently, we suggest a benchmark framework and apply it to comparatively evaluate relevant state-of-the-art estimators.

Стилі APA, Harvard, Vancouver, ISO та ін.

17

Sang, Yu, Hong Wen Song, and Jun Zhao. "Intrinsic Dimensional Correlation Discretization for Mining Task." Applied Mechanics and Materials 404 (September 2013): 548–54. http://dx.doi.org/10.4028/www.scientific.net/amm.404.548.

Повний текст джерела

Анотація:

Discretization is a necessary pre-processing step of the mining task, and a way of performance improvement for many machine learning algorithms. Existing techniques mainly focus on 1-dimension discretization in lower dimensional data space. In this paper, we present an intrinsic dimensional correlation discretization technique in high-dimensional data space. The approach estimates the intrinsic dimensionality (ID) of the data by using maximum likelihood estimation (MLE). Further, we project data onto eigenspace of the estimated lower ID by using principle component analysis (PCA) that can discover the potential correlation structure in the multivariate data. Thus, all the dimensions of the data can be transformed into new independent eigenspace of the ID, and each dimension can be discretized separately in the eigenspace based on the promising Bayes discretization model by using outstanding MODL discretization method. We design a heuristic framework to find better discretization scheme. Our approach demonstrates that there is a significantly improvement on the mean learning accuracy of the classifiers than traditional discretization methods.

Стилі APA, Harvard, Vancouver, ISO та ін.

18

Sumet Mehta. "Generalized Multi-manifold Graph Ensemble Embedding for Multi-View Dimensionality Reduction." Lahore Garrison University Research Journal of Computer Science and Information Technology 4, no. 4 (December 28, 2020): 55–72. http://dx.doi.org/10.54692/lgurjcsit.2020.0404109.

Повний текст джерела

Анотація:

In this paper, we propose a new dimension reduction (DR) algorithm called ensemble graph-based locality preserving projections (EGLPP); to overcome the neighborhood size k sensitivity in locally preserving projections (LPP). EGLPP constructs a homogeneous ensemble of adjacency graphs by varying neighborhood size k and finally uses the integrated embedded graph to optimize the low-dimensional projections. Furthermore, to appropriately handle the intrinsic geometrical structure of the multi-view data and overcome the dimensionality curse, we propose a generalized multi-manifold graph ensemble embedding framework (MLGEE). MLGEE aims to utilize multi-manifold graphs for the adjacency estimation with automatically weight each manifold to derive the integrated heterogeneous graph. Experimental results on various computer vision databases verify the effectiveness of proposed EGLPP and MLGEE over existing comparative DR methods.

Стилі APA, Harvard, Vancouver, ISO та ін.

19

Alluri, Vinoo, and Petri Toiviainen. "Effect of Enculturation on the Semantic and Acoustic Correlates of Polyphonic Timbre." Music Perception 29, no. 3 (December 2011): 297–310. http://dx.doi.org/10.1525/mp.2012.29.3.297.

Повний текст джерела

Анотація:

polyphonic timbre perception was investigated in a cross-cultural context wherein Indian and Western nonmusicians rated short Indian and Western popular music excerpts (1.5 s, n = 200) on eight bipolar scales. Intrinsic dimensionality estimation revealed a higher number of perceptual dimensions in the timbre space for music from one's own culture. Factor analyses of Indian and Western participants' ratings resulted in highly similar factor solutions. The acoustic features that predicted the perceptual dimensions were similar across the two participant groups. Furthermore, both the perceptual dimensions and their acoustic correlates matched closely with the results of a previous study performed using Western musicians as participants. Regression analyses revealed relatively well performing models for the perceptual dimensions. The models displayed relatively high cross-validation performance. The findings suggest the presence of universal patterns in polyphonic timbre perception while demonstrating the increase of dimensionality of timbre space as a result of enculturation.

Стилі APA, Harvard, Vancouver, ISO та ін.

20

Hasanlou, Mahdi, and Farhad Samadzadegan. "Comparative Study of Intrinsic Dimensionality Estimation and Dimension Reduction Techniques on Hyperspectral Images Using K-NN Classifier." IEEE Geoscience and Remote Sensing Letters 9, no. 6 (November 2012): 1046–50. http://dx.doi.org/10.1109/lgrs.2012.2189547.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

21

Cui, Kai. "Semiparametric Gaussian Variance-Mean Mixtures for Heavy-Tailed and Skewed Data." ISRN Probability and Statistics 2012 (December 23, 2012): 1–18. http://dx.doi.org/10.5402/2012/345784.

Повний текст джерела

Анотація:

There is a need for new classes of flexible multivariate distributions that can capture heavy tails and skewness without being so flexible as to fully incur the curse of dimensionality intrinsic to nonparametric density estimation. We focus on the family of Gaussian variance-mean mixtures, which have received limited attention in multivariate settings beyond simple special cases. By using a Bayesian semiparametric approach, we allow the data to infer about the unknown mixing distribution. Properties are considered and an approach to posterior computation is developed relying on Markov chain Monte Carlo. The methods are evaluated through simulation studies and applied to a variety of applications, illustrating their flexible performance in characterizing heavy tails, tail dependence, and skewness.

Стилі APA, Harvard, Vancouver, ISO та ін.

22

Ding, Yiyang, Anyong Qin, Zhaowei Shang, and Jiye Qian. "Spatial distribution preserving-based sparse subspace clustering for hyperspectral image." International Journal of Wavelets, Multiresolution and Information Processing 17, no. 02 (March 2019): 1940010. http://dx.doi.org/10.1142/s0219691319400101.

Повний текст джерела

Анотація:

The high dimensionality and heterogeneity of the hyperspectral image (HSI) make a challenge to the application of machine learning methods, such as sparse subspace clustering (SSC). SSC is designed to represent data as an union of affine subspaces, while it cannot capture the latent structure of the given data. In Mosers theory, the distribution can represent the intrinsic structure efficiently. Hence, we propose a novel approach called spatial distribution preserving-based sparse subspace clustering (SSC-SDP) in this paper for HSI data, which can help sparse representation preserve the underlying manifold structure. Specifically, the density constraint is added by minimizing the inconsistency of the densities estimated in the HSI data and the corresponding sparse coefficient matrix. In addition, we incorporate spatial information into the density estimation of the original data, and the optimization solution based on alternating direction method of multipliers (ADMM) is devised. Three HSI data sets are conducted to evaluate the performance of our SSC-SDP compared with other state-of-art algorithms.

Стилі APA, Harvard, Vancouver, ISO та ін.

23

Deleforge, Antoine, Florence Forbes, and Radu Horaud. "Acoustic Space Learning for Sound-Source Separation and Localization on Binaural Manifolds." International Journal of Neural Systems 25, no. 01 (January 6, 2015): 1440003. http://dx.doi.org/10.1142/s0129065714400036.

Повний текст джерела

Анотація:

In this paper, we address the problems of modeling the acoustic space generated by a full-spectrum sound source and using the learned model for the localization and separation of multiple sources that simultaneously emit sparse-spectrum sounds. We lay theoretical and methodological grounds in order to introduce the binaural manifold paradigm. We perform an in-depth study of the latent low-dimensional structure of the high-dimensional interaural spectral data, based on a corpus recorded with a human-like audiomotor robot head. A nonlinear dimensionality reduction technique is used to show that these data lie on a two-dimensional (2D) smooth manifold parameterized by the motor states of the listener, or equivalently, the sound-source directions. We propose a probabilistic piecewise affine mapping model (PPAM) specifically designed to deal with high-dimensional data exhibiting an intrinsic piecewise linear structure. We derive a closed-form expectation-maximization (EM) procedure for estimating the model parameters, followed by Bayes inversion for obtaining the full posterior density function of a sound-source direction. We extend this solution to deal with missing data and redundancy in real-world spectrograms, and hence for 2D localization of natural sound sources such as speech. We further generalize the model to the challenging case of multiple sound sources and we propose a variational EM framework. The associated algorithm, referred to as variational EM for source separation and localization (VESSL) yields a Bayesian estimation of the 2D locations and time-frequency masks of all the sources. Comparisons of the proposed approach with several existing methods reveal that the combination of acoustic-space learning with Bayesian inference enables our method to outperform state-of-the-art methods.

Стилі APA, Harvard, Vancouver, ISO та ін.

24

Baydal, E., G. Andreu, and E. Vidal. "Estimating the intrinsic dimensionality of discrete utterances." IEEE Transactions on Acoustics, Speech, and Signal Processing 37, no. 5 (May 1989): 755–57. http://dx.doi.org/10.1109/29.17567.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

25

Karbauskaitė, Rasa, Gintautas Dzemyda, and Edmundas Mazėtis. "Geodesic distances in the maximum likelihood estimator of intrinsic dimensionality." Nonlinear Analysis: Modelling and Control 16, no. 4 (December 7, 2011): 387–402. http://dx.doi.org/10.15388/na.16.4.14084.

Повний текст джерела

Анотація:

While analyzing multidimensional data, we often have to reduce their dimensionality so that to preserve as much information on the analyzed data set as possible. To this end, it is reasonable to find out the intrinsic dimensionality of the data. In this paper, two techniques for the intrinsic dimensionality are analyzed and compared, i.e., the maximum likelihood estimator (MLE) and ISOMAP method. We also propose the way how to get good estimates of the intrinsic dimensionality by the MLE method.

Стилі APA, Harvard, Vancouver, ISO та ін.

26

Karbauskaitė, Rasa, and Gintautas Dzemyda. "Optimization of the Maximum Likelihood Estimator for Determining the Intrinsic Dimensionality of High–Dimensional Data." International Journal of Applied Mathematics and Computer Science 25, no. 4 (December 1, 2015): 895–913. http://dx.doi.org/10.1515/amcs-2015-0064.

Повний текст джерела

Анотація:

AbstractOne of the problems in the analysis of the set of images of a moving object is to evaluate the degree of freedom of motion and the angle of rotation. Here the intrinsic dimensionality of multidimensional data, characterizing the set of images, can be used. Usually, the image may be represented by a high-dimensional point whose dimensionality depends on the number of pixels in the image. The knowledge of the intrinsic dimensionality of a data set is very useful information in exploratory data analysis, because it is possible to reduce the dimensionality of the data without losing much information. In this paper, the maximum likelihood estimator (MLE) of the intrinsic dimensionality is explored experimentally. In contrast to the previous works, the radius of a hypersphere, which covers neighbours of the analysed points, is fixed instead of the number of the nearest neighbours in the MLE. A way of choosing the radius in this method is proposed. We explore which metric—Euclidean or geodesic—must be evaluated in the MLE algorithm in order to get the true estimate of the intrinsic dimensionality. The MLE method is examined using a number of artificial and real (images) data sets.

Стилі APA, Harvard, Vancouver, ISO та ін.

27

Ahram, Tareq Z., Pamela McCauley-Bush, and Waldemar Karwowski. "Estimating intrinsic dimensionality using the multi-criteria decision weighted model and the average standard estimator." Information Sciences 180, no. 15 (August 2010): 2845–55. http://dx.doi.org/10.1016/j.ins.2010.04.006.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

28

Xie, Xiaoping, Zhitong Cao, Xuchu Weng, and Dan Jin. "Estimating intrinsic dimensionality of fMRI dataset incorporating an AR(1) noise model with cubic spline interpolation." Neurocomputing 72, no. 4-6 (January 2009): 1042–55. http://dx.doi.org/10.1016/j.neucom.2008.04.003.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

29

Karbauskaitė, Rasa, and Gintautas Dzemyda. "Fractal-Based Methods as a Technique for Estimating the Intrinsic Dimensionality of High-Dimensional Data: A Survey." Informatica 27, no. 2 (January 1, 2016): 257–81. http://dx.doi.org/10.15388/informatica.2016.84.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

30

Sidorov, Sergey, and Nikolai Zolotykh. "Linear and Fisher Separability of Random Points in the d-Dimensional Spherical Layer and Inside the d-Dimensional Cube." Entropy 22, no. 11 (November 12, 2020): 1281. http://dx.doi.org/10.3390/e22111281.

Повний текст джерела

Анотація:

Stochastic separation theorems play important roles in high-dimensional data analysis and machine learning. It turns out that in high dimensional space, any point of a random set of points can be separated from other points by a hyperplane with high probability, even if the number of points is exponential in terms of dimensions. This and similar facts can be used for constructing correctors for artificial intelligent systems, for determining the intrinsic dimensionality of data and for explaining various natural intelligence phenomena. In this paper, we refine the estimations for the number of points and for the probability in stochastic separation theorems, thereby strengthening some results obtained earlier. We propose the boundaries for linear and Fisher separability, when the points are drawn randomly, independently and uniformly from a d-dimensional spherical layer and from the cube. These results allow us to better outline the applicability limits of the stochastic separation theorems in applications.

Стилі APA, Harvard, Vancouver, ISO та ін.

31

Erba, Vittorio, Marco Gherardi, and Pietro Rotondo. "Intrinsic dimension estimation for locally undersampled data." Scientific Reports 9, no. 1 (November 20, 2019). http://dx.doi.org/10.1038/s41598-019-53549-9.

Повний текст джерела

Анотація:

AbstractIdentifying the minimal number of parameters needed to describe a dataset is a challenging problem known in the literature as intrinsic dimension estimation. All the existing intrinsic dimension estimators are not reliable whenever the dataset is locally undersampled, and this is at the core of the so called curse of dimensionality. Here we introduce a new intrinsic dimension estimator that leverages on simple properties of the tangent space of a manifold and extends the usual correlation integral estimator to alleviate the extreme undersampling problem. Based on this insight, we explore a multiscale generalization of the algorithm that is capable of (i) identifying multiple dimensionalities in a dataset, and (ii) providing accurate estimates of the intrinsic dimension of extremely curved manifolds. We test the method on manifolds generated from global transformations of high-contrast images, relevant for invariant object recognition and considered a challenge for state-of-the-art intrinsic dimension estimators.

Стилі APA, Harvard, Vancouver, ISO та ін.

32

Denti, Francesco, Diego Doimo, Alessandro Laio, and Antonietta Mira. "The generalized ratios intrinsic dimension estimator." Scientific Reports 12, no. 1 (November 21, 2022). http://dx.doi.org/10.1038/s41598-022-20991-1.

Повний текст джерела

Анотація:

AbstractModern datasets are characterized by numerous features related by complex dependency structures. To deal with these data, dimensionality reduction techniques are essential. Many of these techniques rely on the concept of intrinsic dimension (), a measure of the complexity of the dataset. However, the estimation of this quantity is not trivial: often, the depends rather dramatically on the scale of the distances among data points. At short distances, the can be grossly overestimated due to the presence of noise, becoming smaller and approximately scale-independent only at large distances. An immediate approach to examining the scale dependence consists in decimating the dataset, which unavoidably induces non-negligible statistical errors at large scale. This article introduces a novel statistical method, , that allows estimating the as an explicit function of the scale without performing any decimation. Our approach is based on rigorous distributional results that enable the quantification of uncertainty of the estimates. Moreover, our method is simple and computationally efficient since it relies only on the distances among data points. Through simulation studies, we show that is asymptotically unbiased, provides comparable estimates to other state-of-the-art methods, and is more robust to short-scale noise than other likelihood-based approaches.

Стилі APA, Harvard, Vancouver, ISO та ін.

33

Einbeck, Jochen, and Zakiah Kalantana. "Intrinsic Dimensionality Estimation for High-dimensional Data Sets: New Approaches for the Computation of Correlation Dimension." Journal of Emerging Technologies in Web Intelligence 5, no. 2 (May 1, 2013). http://dx.doi.org/10.4304/jetwi.5.2.91-97.

Повний текст джерела

Стилі APA, Harvard, Vancouver, ISO та ін.

34

Vinga, Susana. "Structured sparsity regularization for analyzing high-dimensional omics data." Briefings in Bioinformatics, June 29, 2020. http://dx.doi.org/10.1093/bib/bbaa122.

Повний текст джерела

Анотація:

Abstract The development of new molecular and cell technologies is having a significant impact on the quantity of data generated nowadays. The growth of omics databases is creating a considerable potential for knowledge discovery and, concomitantly, is bringing new challenges to statistical learning and computational biology for health applications. Indeed, the high dimensionality of these data may hamper the use of traditional regression methods and parameter estimation algorithms due to the intrinsic non-identifiability of the inherent optimization problem. Regularized optimization has been rising as a promising and useful strategy to solve these ill-posed problems by imposing additional constraints in the solution parameter space. In particular, the field of statistical learning with sparsity has been significantly contributing to building accurate models that also bring interpretability to biological observations and phenomena. Beyond the now-classic elastic net, one of the best-known methods that combine lasso with ridge penalizations, we briefly overview recent literature on structured regularizers and penalty functions that have been applied in biomedical data to build parsimonious models in a variety of underlying contexts, from survival to generalized linear models. These methods include functions of $\ell _k$-norms and network-based penalties that take into account the inherent relationships between the features. The successful application to omics data illustrates the potential of sparse structured regularization for identifying disease’s molecular signatures and for creating high-performance clinical decision support systems towards more personalized healthcare. Supplementary information: Supplementary data are available at Briefings in Bioinformatics online.

Стилі APA, Harvard, Vancouver, ISO та ін.

35

Masum, Shakil A., Zhihong Zhang, Gailei Tian, and Mimnun Sultana. "Three-dimensional fully coupled hydro-mechanical-chemical model for solute transport under mechanical and osmotic loading conditions." Environmental Science and Pollution Research, August 20, 2022. http://dx.doi.org/10.1007/s11356-022-22600-0.

Повний текст джерела

Анотація:

Abstract Mechanical deformation and chemico-osmotic consolidation of clay liners can change its intrinsic transport properties in all direction and can alter fluid and solute transport processes in the entire model domain. These phenomena are described inadequately by lower-dimensional models. Based on the Biot’s consolidation theory, fluid and solute mass conservation equations, a three-dimensional (3D) fully-coupled hydro-mechanical-chemical (HMC) model has been proposed in this study. The impacts of mechanical consolidation and chemico-osmotic consolidation on permeability, hydrodynamic dispersion, solute sorption, membrane efficiency, and chemical osmosis are considered in the model. The model is applied to evaluate performances of a single compacted clay liner (CCL) and a damaged geomembrane-compacted clay composite liner (GMB/CCL) to contain a generic landfill contaminant. Effect of model dimensionality on solute spread for CCL is found to be marginal, but for GMB/CCL the effect is significantly large. After 50-year simulation period, solute concentration at the half-length of the GMB/CCL liner is predicted to be 40% of the source concentration during 1D simulation, which is only 6% during the 3D simulation. The results revealed approximately 74% over-estimation of liner settlement in 1D simulation than that of the 3D for GMB/CL system. Solute spread accelerates (over-estimates) vertically than horizontally since overburden load and consequent mechanical loading-induced solute convection occurs in the same direction. However, in homogeneous and isotropic soils, horizontal spread retards the overall migration of contaminants, and it highlights the importance of 3D models to study solute transports under mechanical and chemico-osmotic loading conditions in semi-permeable clays, especially, for damaged geomembrane-clay liners. The results show the utility of geomembranes to reduce soil settlement, undulation, and restriction of solute migration. Furthermore, application of geomembrane can inhibit development of elevated negative excess pore water pressure at deeper portion of a clay liner.

Стилі APA, Harvard, Vancouver, ISO та ін.

Статті в журналах з теми "Intrinsic dimensionality estimation"

Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями