Academic literature on the topic 'Variable sparsity kernel learning'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Variable sparsity kernel learning.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Variable sparsity kernel learning"
1
Chen, Jingxiang, Chong Zhang, Michael R. Kosorok, and Yufeng Liu. "Double sparsity kernel learning with automatic variable selection and data extraction." Statistics and Its Interface 11, no. 3 (2018): 401–20. http://dx.doi.org/10.4310/sii.2018.v11.n3.a1.
Full text
2
Huang, Yuan, and Shuangge Ma. "Discussion on “Double sparsity kernel learning with automatic variable selection and data extraction”." Statistics and Its Interface 11, no. 3 (2018): 421–22. http://dx.doi.org/10.4310/sii.2018.v11.n3.a2.
Full text
3
Liu, Meimei, and Guang Cheng. "Discussion on “Double sparsity kernel learning with automatic variable selection and data extraction”." Statistics and Its Interface 11, no. 3 (2018): 423–24. http://dx.doi.org/10.4310/sii.2018.v11.n3.a3.
Full text
4
Zhang, Hao Helen. "Discussion on “Doubly sparsity kernel learning with automatic variable selection and data extraction”." Statistics and Its Interface 11, no. 3 (2018): 425–28. http://dx.doi.org/10.4310/sii.2018.v11.n3.a4.
Full text
5
Chen, Jingxiang, Chong Zhang, Michael R. Kosorok, and Yufeng Liu. "Rejoinder of “Double sparsity kernel learning with automatic variable selection and data extraction”." Statistics and Its Interface 11, no. 3 (2018): 429–31. http://dx.doi.org/10.4310/sii.2018.v11.n3.a5.
Full text
6
Wang, Shuangyue, and Ziyan Luo. "Sparse Support Tensor Machine with Scaled Kernel Functions." Mathematics 11, no. 13 (June 24, 2023): 2829. http://dx.doi.org/10.3390/math11132829.
Full textAbstract:
As one of the supervised tensor learning methods, the support tensor machine (STM) for tensorial data classification is receiving increasing attention in machine learning and related applications, including remote sensing imaging, video processing, fault diagnosis, etc. Existing STM approaches lack consideration for support tensors in terms of data reduction. To address this deficiency, we built a novel sparse STM model to control the number of support tensors in the binary classification of tensorial data. The sparsity is imposed on the dual variables in the context of the feature space, which facilitates the nonlinear classification with kernel tricks, such as the widely used Gaussian RBF kernel. To alleviate the local risk associated with the constant width in the tensor Gaussian RBF kernel, we propose a two-stage classification approach; in the second stage, we advocate for a scaling strategy on the kernel function in a data-dependent way, using the information of the support tensors obtained from the first stage. The essential optimization models in both stages share the same type, which is non-convex and discontinuous, due to the sparsity constraint. To resolve the computational challenge, a subspace Newton method is tailored for the sparsity-constrained optimization for effective computation with local convergence. Numerical experiments were conducted on real datasets, and the numerical results demonstrate the effectiveness of our proposed two-stage sparse STM approach in terms of classification accuracy, compared with the state-of-the-art binary classification approaches.
7
Pan, Chao, Cheng Shi, Honglang Mu, Jie Li, and Xinbo Gao. "EEG-Based Emotion Recognition Using Logistic Regression with Gaussian Kernel and Laplacian Prior and Investigation of Critical Frequency Bands." Applied Sciences 10, no. 5 (February 29, 2020): 1619. http://dx.doi.org/10.3390/app10051619.
Full textAbstract:
Emotion plays a nuclear part in human attention, decision-making, and communication. Electroencephalogram (EEG)-based emotion recognition has developed a lot due to the application of Brain-Computer Interface (BCI) and its effectiveness compared to body expressions and other physiological signals. Despite significant progress in affective computing, emotion recognition is still an unexplored problem. This paper introduced Logistic Regression (LR) with Gaussian kernel and Laplacian prior for EEG-based emotion recognition. The Gaussian kernel enhances the EEG data separability in the transformed space. The Laplacian prior promotes the sparsity of learned LR regressors to avoid over-specification. The LR regressors are optimized using the logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm. For simplicity, the introduced method is noted as LORSAL. Experiments were conducted on the dataset for emotion analysis using EEG, physiological and video signals (DEAP). Various spectral features and features by combining electrodes (power spectral density (PSD), differential entropy (DE), differential asymmetry (DASM), rational asymmetry (RASM), and differential caudality (DCAU)) were extracted from different frequency bands (Delta, Theta, Alpha, Beta, Gamma, and Total) with EEG signals. The Naive Bayes (NB), support vector machine (SVM), linear LR with L1-regularization (LR_L1), linear LR with L2-regularization (LR_L2) were used for comparison in the binary emotion classification for valence and arousal. LORSAL obtained the best classification accuracies (77.17% and 77.03% for valence and arousal, respectively) on the DE features extracted from total frequency bands. This paper also investigates the critical frequency bands in emotion recognition. The experimental results showed the superiority of Gamma and Beta bands in classifying emotions. It was presented that DE was the most informative and DASM and DCAU had lower computational complexity with relatively ideal accuracies. An analysis of LORSAL and the recently deep learning (DL) methods is included in the discussion. Conclusions and future work are presented in the final section.
8
Koltchinskii, Vladimir, and Ming Yuan. "Sparsity in multiple kernel learning." Annals of Statistics 38, no. 6 (December 2010): 3660–95. http://dx.doi.org/10.1214/10-aos825.
Full text
9
Jiang, Zhengxiong, Yingsong Li, Xinqi Huang, and Zhan Jin. "A Sparsity-Aware Variable Kernel Width Proportionate Affine Projection Algorithm for Identifying Sparse Systems." Symmetry 11, no. 10 (October 1, 2019): 1218. http://dx.doi.org/10.3390/sym11101218.
Full text
10
Yuan, Ying, Weiming Lu, Fei Wu, and Yueting Zhuang. "Multiple kernel learning with NOn-conVex group spArsity." Journal of Visual Communication and Image Representation 25, no. 7 (October 2014): 1616–24. http://dx.doi.org/10.1016/j.jvcir.2014.08.001.
Full textDissertations / Theses on the topic "Variable sparsity kernel learning"
1
Kolar, Mladen. "Uncovering Structure in High-Dimensions: Networks and Multi-task Learning Problems." Research Showcase @ CMU, 2013. http://repository.cmu.edu/dissertations/229.
Full textAbstract:
Extracting knowledge and providing insights into complex mechanisms underlying noisy high-dimensional data sets is of utmost importance in many scientific domains. Statistical modeling has become ubiquitous in the analysis of high dimensional functional data in search of better understanding of cognition mechanisms, in the exploration of large-scale gene regulatory networks in hope of developing drugs for lethal diseases, and in prediction of volatility in stock market in hope of beating the market. Statistical analysis in these high-dimensional data sets is possible only if an estimation procedure exploits hidden structures underlying data.
This thesis develops flexible estimation procedures with provable theoretical guarantees for uncovering unknown hidden structures underlying data generating process. Of particular interest are procedures that can be used on high dimensional data sets where the number of samples n is much smaller than the ambient dimension p. Learning in high-dimensions is difficult due to the curse of dimensionality, however, the special problem structure makes inference possible. Due to its importance for scientific discovery, we put emphasis on consistent structure recovery throughout the thesis. Particular focus is given to two important problems, semi-parametric estimation of networks and feature selection in multi-task learning.
2
Le, Van Luong. "Identification de systèmes dynamiques hybrides : géométrie, parcimonie et non-linéarités." Phd thesis, Université de Lorraine, 2013. http://tel.archives-ouvertes.fr/tel-00874283.
Full textAbstract:
En automatique, l'obtention d'un modèle du système est la pierre angulaire des procédures comme la synthèse d'une commande, la détection des défaillances, la prédiction... Cette thèse traite de l'identification d'une classe de systèmes complexes, les systèmes dynamiques hybrides. Ces systèmes impliquent l'interaction de comportements continus et discrets. Le but est de construire un modèle à partir de mesures expérimentales d'entrée et de sortie. Une nouvelle approche pour l'identification de systèmes hybrides linéaires basée sur les propriétés géométriques des systèmes hybrides dans l'espace des paramètres est proposée. Un nouvel algorithme est ensuite proposé pour le calcul de la solution la plus parcimonieuse (ou creuse) de systèmes d'équations linéaires sous-déterminés. Celui-ci permet d'améliorer une approche d'identification basée sur l'optimisation de la parcimonie du vecteur d'erreur. De plus, de nouvelles approches, basées sur des modèles à noyaux, sont proposées pour l'identification de systèmes hybrides non linéaires et de systèmes lisses par morceaux.
3
Hakala, Tim. "Settling-Time Improvements in Positioning Machines Subject to Nonlinear Friction Using Adaptive Impulse Control." BYU ScholarsArchive, 2006. https://scholarsarchive.byu.edu/etd/1061.
Full textAbstract:
A new method of adaptive impulse control is developed to precisely and quickly control the position of machine components subject to friction. Friction dominates the forces affecting fine positioning dynamics. Friction can depend on payload, velocity, step size, path, initial position, temperature, and other variables. Control problems such as steady-state error and limit cycles often arise when applying conventional control techniques to the position control problem. Studies in the last few decades have shown that impulsive control can produce repeatable displacements as small as ten nanometers without limit cycles or steady-state error in machines subject to dry sliding friction. These displacements are achieved through the application of short duration, high intensity pulses. The relationship between pulse duration and displacement is seldom a simple function. The most dependable practical methods for control are self-tuning; they learn from online experience by adapting an internal control parameter until precise position control is achieved. To date, the best known adaptive pulse control methods adapt a single control parameter. While effective, the single parameter methods suffer from sub-optimal settling times and poor parameter convergence. To improve performance while maintaining the capacity for ultimate precision, a new control method referred to as Adaptive Impulse Control (AIC) has been developed. To better fit the nonlinear relationship between pulses and displacements, AIC adaptively tunes a set of parameters. Each parameter affects a different range of displacements. Online updates depend on the residual control error following each pulse, an estimate of pulse sensitivity, and a learning gain. After an update is calculated, it is distributed among the parameters that were used to calculate the most recent pulse. As the stored relationship converges to the actual relationship of the machine, pulses become more accurate and fewer pulses are needed to reach each desired destination. When fewer pulses are needed, settling time improves and efficiency increases. AIC is experimentally compared to conventional PID control and other adaptive pulse control methods on a rotary system with a position measurement resolution of 16000 encoder counts per revolution of the load wheel. The friction in the test system is nonlinear and irregular with a position dependent break-away torque that varies by a factor of more than 1.8 to 1. AIC is shown to improve settling times by as much as a factor of two when compared to other adaptive pulse control methods while maintaining precise control tolerances.
4
Sankaran, Raman. "Structured Regularization Through Convex Relaxations Of Discrete Penalties." Thesis, 2018. https://etd.iisc.ac.in/handle/2005/5456.
Full textAbstract:
Motivation. Empirical risk minimization(ERM) is a popular framework for learning predictive
models from data, which has been used in various domains such as computer vision, text
processing, bioinformatics, neuro-biology, temporal point processes, to name a few. We consider
the cases where one has apriori information regarding the model structure, the simplest
one being the sparsity of the model. The desired sparsity structure can be imputed into ERM
problems using a regularizer, which is typically a norm on the model vector. Popular choices of
the regularizers include the `1 norm (LASSO) which encourages sparse solutions, block-`1 which
encourages block level sparsity, among many others which induce more complicated structures.
To impute the structural prior, recently, many studies have considered combinatorial functions
on the model's support to be the regularizer. But this leads to an NP-hard problem, which
thus motivates us to consider convex relaxations of the corresponding combinatorial functions,
which are tractable.
Existing work and research gaps. The convex relaxations of combinatorial functions have
been studied recently in the context of structured sparsity, but they still lead to inefficient
computational procedures in general cases: e.g., even when the combinatorial function is submodular,
whose convex relaxations are well studied and easier to work with than the general
ones, the resulting problem is computationally expensive (the proximal operator takes O(d6)
time for d variables). Hence, the associated high expenses have limited the research interest
towards these these regularizers, despite the submodular functions which generate them are
expressive enough to encourage many of the structures such as those mentioned before. Hence
it remains open to design efficient optimization procedures to work with the submodular penalties,
and with combinatorial penalties in general. It is also desirable that the optimization
algorithms designed to be applicable across the possible choices of the loss functions arising
from various applications. We identify four such problems from these existing research gaps,
and address them through the contributions which are listed below. We provide the list of
publications related to this thesis following this abstract Contributions. First, we propose a novel kernel learning technique termed as Variable
sparsity kernel learning (VSKL) for support vector machines (SVM), which are applicable
when there is an apriori information regarding the grouping among the kernels. In such cases, we
propose a novel mixed-norm regularizer, which encourages sparse selection of the kernels within
a group while selecting all groups. This regularizer can also be viewed as the convex relaxation
of a specifi c discrete penalty on the model's support. The resulting non-smooth optimization
problem is difficult, where o -the-shelf techniques are not applicable. We propose a mirror
descent based optimization algorithm to solve this problem, which has a guaranteed convergence
rate of O(1=
p
T) over T iterations. We demonstrate the efficiency of the proposed algorithm in
various scaling experiments, and applicability of the regularizer in an object recognition task.
Second, we introduce a family of regularizers termed as Smoothed Ordered Weighted
L1 (SOWL) norms, which are derived as the convex relaxation of non-decreasing cardinalitybased
submodular penalties, which form an important special class of the general discrete
penalties. Considering linear regression, where the presence of correlated predictors cause
the traditional regularizers such as Lasso to fail recovering the true support, SOWL has the
property of selecting the correlated predictors as groups. While Ordered Weighted `1 (OWL)
norms address similar use cases, they are biased to promote undesirable piece-wise constant
solutions, which SOWL does not have. SOWL is also shown equivalent to group-lasso, but
without specifying the groups explicitly. We develop efficient proximal operators for SOWL,
and illustrate its computational and theoretical benefi ts through various simulations.
Third, we discuss Hawkes-OWL, an application of OWL regularizers for the setting of
multidimensional Hawkes processes. Hawkes process is a multi-dimensional point process (collection
of multiple event streams) with self and mutual in
fluences between the event streams.
While the popular `1 regularizer fails to recover the true models in the presence of strongly
correlated event streams, OWL regularization address this issue and groups the correlated predictors.
This is the first instance in the literature, where OWL norms, which predominantly
have been studied with respect to simple loss functions such as the squared loss, are extended
to the Hawkes process with similar theoretical and computational guarantees.
In the fourth part, we discuss generic first-order algorithms for learning with Subquadraic
norms, a special sub-family of convex relaxations of submodular penalties. We consider subquadratic
norms regularization in a very general setting, covering all loss functions, and propose
different reformulations of the original problem. The reformulations enable us to propose two
different primal-dual algorithms, CP- and ADMM- , both of which having a guaranteed convergence
rate of O(1=T ). This study thus provides the rst ever algorithms with e cient
convergence rates for learning with subquadratic norms.
5
Naudé, Johannes Jochemus. "Aircraft recognition using generalised variable-kernel similarity metric learning." Thesis, 2014. http://hdl.handle.net/10210/13113.
Full textAbstract:
M.Ing.Nearest neighbour classifiers are well suited for use in practical pattern recognition applications for a number of reasons, including ease of implementation, rapid training, justifiable decisions and low computational load. However their generalisation performance is perceived to be inferior to that of more complex methods such as neural networks or support vector machines. Closer inspection shows however that the generalisation performance actually varies widely depending on the dataset used. On certain problems they outperform all other known classifiers while on others they fail dismally. In this thesis we allege that their sensitivity to the metric used is the reason for their mercurial performance. We also discuss some of the remedies for this problem that have been suggested in the past, most notably the variable-kernel similarity metric learning technique, and introduce our own extension to this technique. Finally these metric learning techniques are evaluated on an aircraft recognition task and critically compared.
6
Hwang, Sung Ju. "Discriminative object categorization with external semantic knowledge." 2013. http://hdl.handle.net/2152/21320.
Full textAbstract:
Visual object category recognition is one of the most challenging problems in computer vision. Even assuming that we can obtain a near-perfect instance level representation with the advances in visual input devices and low-level vision techniques, object categorization still remains as a difficult problem because it requires drawing boundaries between instances in a continuous world, where the boundaries are solely defined by human conceptualization. Object categorization is essentially a perceptual process that takes place in a human-defined semantic space. In this semantic space, the categories reside not in isolation, but in relation to others. Some categories are similar, grouped, or co-occur, and some are not. However, despite this semantic nature of object categorization, most of the today's automatic visual category recognition systems rely only on the category labels for training discriminative recognition with statistical machine learning techniques. In many cases, this could result in the recognition model being misled into learning incorrect associations between visual features and the semantic labels, from essentially overfitting to training set biases. This limits the model's prediction power when new test instances are given. Using semantic knowledge has great potential to benefit object category recognition. First, semantic knowledge could guide the training model to learn a correct association between visual features and the categories. Second, semantics provide much richer information beyond the membership information given by the labels, in the form of inter-category and category-attribute distances, relations, and structures. Finally, the semantic knowledge scales well as the relations between categories become larger with an increasing number of categories. My goal in this thesis is to learn discriminative models for categorization that leverage semantic knowledge for object recognition, with a special focus on the semantic relationships among different categories and concepts. To this end, I explore three semantic sources, namely attributes, taxonomies, and analogies, and I show how to incorporate them into the original discriminative model as a form of structural regularization. In particular, for each form of semantic knowledge I present a feature learning approach that defines a semantic embedding to support the object categorization task. The regularization penalizes the models that deviate from the known structures according to the semantic knowledge provided. The first semantic source I explore is attributes, which are human-describable semantic characteristics of an instance. While the existing work treated them as mid-level features which did not introduce new information, I focus on their potential as a means to better guide the learning of object categories, by enforcing the object category classifiers to share features with attribute classifiers, in a multitask feature learning framework. This approach essentially discovers the common low-dimensional features that support predictions in both semantic spaces. Then, I move on to the semantic taxonomy, which is another valuable source of semantic knowledge. The merging and splitting criteria for the categories on a taxonomy are human-defined, and I aim to exploit this implicit semantic knowledge. Specifically, I propose a tree of metrics (ToM) that learns metrics that capture granularity-specific similarities at different nodes of a given semantic taxonomy, and uses a regularizer to isolate granularity-specific disjoint features. This approach captures the intuition that the features used for the discrimination of the parent class should be different from the features used for the children classes. Such learned metrics can be used for hierarchical classification. The use of a single taxonomy can be limited in that its structure is not optimal for hierarchical classification, and there may exist no single optimal semantic taxonomy that perfectly aligns with visual distributions. Thus, I next propose a way to overcome this limitation by leveraging multiple taxonomies as semantic sources to exploit, and combine the acquired complementary information across multiple semantic views and granularities. This allows us, for example, to synthesize semantics from both 'Biological', and 'Appearance'-based taxonomies when learning the visual features. Finally, as a further exploration of more complex semantic relations different from the previous two pairwise similarity-based models, I exploit analogies, which encode the relational similarities between two related pairs of categories. Specifically, I use analogies to regularize a discriminatively learned semantic embedding space for categorization, such that the displacements between the two category embeddings in both category pairs of the analogy are enforced to be the same. Such a constraint allows for a more confusing pair of categories to benefit from a clear separation in the matched pair of categories that share the same relation. All of these methods are evaluated on challenging public datasets, and are shown to effectively improve the recognition accuracy over purely discriminative models, while also guiding the recognition to be more semantic to human perception. Further, the applications of the proposed methods are not limited to visual object categorization in computer vision, but they can be applied to any classification problems where there exists some domain knowledge about the relationships or structures between the classes. Possible applications of my methods outside the visual recognition domain include document classification in natural language processing, and gene-based animal or protein classification in computational biology.text
Book chapters on the topic "Variable sparsity kernel learning"
1
Koltchinskii, Vladimir, Dmitry Panchenko, and Savina Andonova. "Generalization Bounds for Voting Classifiers Based on Sparsity and Clustering." In Learning Theory and Kernel Machines, 492–505. Berlin, Heidelberg: Springer Berlin Heidelberg, 2003. http://dx.doi.org/10.1007/978-3-540-45167-9_36.
Full text
2
Naudé, Johannes J., Michaël A. van Wyk, and Barend J. van Wyk. "Generalized Variable-Kernel Similarity Metric Learning." In Lecture Notes in Computer Science, 788–96. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004. http://dx.doi.org/10.1007/978-3-540-27868-9_86.
Full text
3
U. Torun, Mustafa, Onur Yilmaz, and Ali N. Akansu. "Explicit Kernel and Sparsity of Eigen Subspace for the AR(1) Process." In Financial Signal Processing and Machine Learning, 67–99. Chichester, UK: John Wiley & Sons, Ltd, 2016. http://dx.doi.org/10.1002/9781118745540.ch5.
Full text
4
Gregorová, Magda, Jason Ramapuram, Alexandros Kalousis, and Stéphane Marchand-Maillet. "Large-Scale Nonlinear Variable Selection via Kernel Random Features." In Machine Learning and Knowledge Discovery in Databases, 177–92. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-10928-8_11.
Full text
5
Connolly, Andrew J., Jacob T. VanderPlas, Alexander Gray, Andrew J. Connolly, Jacob T. VanderPlas, and Alexander Gray. "Regression and Model Fitting." In Statistics, Data Mining, and Machine Learning in Astronomy. Princeton University Press, 2014. http://dx.doi.org/10.23943/princeton/9780691151687.003.0008.
Full textAbstract:
Regression is a special case of the general model fitting and selection procedures discussed in chapters 4 and 5. It can be defined as the relation between a dependent variable, y, and a set of independent variables, x, that describes the expectation value of y given x: E [y¦x]. The purpose of obtaining a “best-fit” model ranges from scientific interest in the values of model parameters (e.g., the properties of dark energy, or of a newly discovered planet) to the predictive power of the resulting model (e.g., predicting solar activity). This chapter starts with a general formulation for regression, list various simplified cases, and then discusses methods that can be used to address them, such as regression for linear models, kernel regression, robust regression and nonlinear regression.
6
T., Subbulakshmi. "Combating Cyber Security Breaches in Digital World Using Misuse Detection Methods." In Advances in Digital Crime, Forensics, and Cyber Terrorism, 85–92. IGI Global, 2016. http://dx.doi.org/10.4018/978-1-5225-0193-0.ch006.
Full textAbstract:
Intrusion Detection Systems (IDS) play a major role in the area of combating security breaches for information security. Current IDS are developed with Machine learning techniques like Artificial Neural Networks, C 4.5, KNN, Naïve Bayes classifiers, Genetic algorithms Fuzzy logic and SVMs. The objective of this paper is to apply Artificial Neural Networks and Support Vector Machines for intrusion detection. Artificial Neural Networks are applied along with faster training methods like variable learning rate and scaled conjugate gradient. Support Vector Machines use various kernel functions to improve the performance. From the kddcup'99 dataset 45,657 instances are taken and used in our experiment. The speed is compared for various training functions. The performance of various kernel functions is assessed. The detection rate of Support Vector Machines is found to be greater than Artificial Neural Networks with less number of false positives and with less time of detection.
7
Wong, Andrew K. C., Yang Wang, and Gary C. L. Li. "Pattern Discovery as Event Association." In Machine Learning, 1924–32. IGI Global, 2012. http://dx.doi.org/10.4018/978-1-60960-818-7.ch804.
Full textAbstract:
A basic task of machine learning and data mining is to automatically uncover <b>patterns</b> that reflect regularities in a data set. When dealing with a large database, especially when domain knowledge is not available or very weak, this can be a challenging task. The purpose of <b>pattern discovery</b> is to find non-random relations among events from data sets. For example, the “exclusive OR” (XOR) problem concerns 3 binary variables, A, B and C=A<img src="http://resources.igi-global.com/Marketing/Preface_Figures/x_symbol.png">B, i.e. C is true when either A or B, but not both, is true. Suppose not knowing that it is the XOR problem, we would like to check whether or not the occurrence of the compound event [A=T, B=T, C=F] is just a random happening. If we could estimate its frequency of occurrences under the random assumption, then we know that it is not random if the observed frequency deviates significantly from that assumption. We refer to such a compound event as an event association pattern, or simply a <b>pattern</b>, if its frequency of occurrences significantly deviates from the default random assumption in the statistical sense. For instance, suppose that an XOR database contains 1000 samples and each primary event (e.g. [A=T]) occurs 500 times. The expected frequency of occurrences of the compound event [A=T, B=T, C=F] under the independence assumption is 0.5×0.5×0.5×1000 = 125. Suppose that its observed frequency is 250, we would like to see whether or not the difference between the observed and expected frequencies (i.e. 250 – 125) is significant enough to indicate that the compound event is not a random happening.<div><br></div><div>In statistics, to test the correlation between random variables, <b>contingency table</b> with chi-squared statistic (Mills, 1955) is widely used. Instead of investigating variable correlations, pattern discovery shifts the traditional correlation analysis in statistics at the variable level to association analysis at the event level, offering an effective method to detect statistical association among events.</div><div><br></div><div>In the early 90’s, this approach was established for second order event associations (Chan & Wong, 1990). A higher order <b>pattern discovery</b> algorithm was devised in the mid 90’s for discrete-valued data sets (Wong & Yang, 1997). In our methods, patterns inherent in data are defined as statistically significant associations of two or more primary events of different attributes if they pass a statistical test for deviation significance based on <b>residual analysis</b>. The discovered high order patterns can then be used for classification (Wang & Wong, 2003). With continuous data, events are defined as Borel sets and the pattern discovery process is formulated as an optimization problem which recursively partitions the sample space for the best set of significant events (patterns) in the form of high dimension intervals from which probability density can be estimated by Gaussian kernel fit (Chau & Wong, 1999). Classification can then be achieved using Bayesian classifiers. For data with a mixture of discrete and continuous data (Wong & Yang, 2003), the latter is categorized based on a global optimization discretization algorithm (Liu, Wong & Yang, 2004). As demonstrated in numerous real-world and commercial applications (Yang, 2002), pattern discovery is an ideal tool to uncover subtle and useful patterns in a database. </div><div><br></div><div>In pattern discovery, three open problems are addressed. The first concerns learning where noise and uncertainty are present. In our method, noise is taken as inconsistent samples against statistically significant patterns. Missing attribute values are also considered as noise. Using a standard statistical <b>hypothesis testing</b> to confirm statistical patterns from the candidates, this method is a less ad hoc approach to discover patterns than most of its contemporaries. The second problem concerns the detection of polythetic patterns without relying on exhaustive search. Efficient systems for detecting monothetic patterns between two attributes exist (e.g. Chan & Wong, 1990). However, for detecting polythetic patterns, an exhaustive search is required (Han, 2001). In many problem domains, polythetic assessments of feature combinations (or higher order relationship detection) are imperative for robust learning. Our method resolves this problem by directly constructing polythetic concepts while screening out non-informative pattern candidates, using statisticsbased heuristics in the discovery process. The third problem concerns the representation of the detected patterns. Traditionally, if-then rules and graphs, including networks and trees, are the most popular ones. However, they have shortcomings when dealing with multilevel and multiple order patterns due to the non-exhaustive and unpredictable hierarchical nature of the inherent patterns. We adopt <b>attributed hypergraph</b> (AHG) (Wang & Wong, 1996) as the representation of the detected patterns. It is a data structure general enough to encode information at many levels of abstraction, yet simple enough to quantify the information content of its organized structure. It is able to encode both the qualitative and the quantitative characteristics and relations inherent in the data set.<br></div>
Conference papers on the topic "Variable sparsity kernel learning"
1
Dellacasagrande, Matteo, Davide Lengani, Pietro Paliotta, Daniele Petronio, Daniele Simoni, and Francesco Bertini. "Evaluation of Different Regression Models Tuned With Experimental Turbine Cascade Data." In ASME Turbo Expo 2022: Turbomachinery Technical Conference and Exposition. American Society of Mechanical Engineers, 2022. http://dx.doi.org/10.1115/gt2022-81357.
Full textAbstract:
Abstract In the present work linear and non-linear regression functions have been tuned with an extensive database describing the unsteady aerodynamic efficiency of low-pressure-turbine cascades. The learning strategy has been first defined using a dataset published in a previous work concerning the loss coefficient measured in a large-scale cascade for a large variation of the Reynolds number, the reduced frequency, and the flow coefficient. Linear models have been educated accounting for the Occam’s razor parsimony criterion, condensing the effects due to the parameter variation in few predictors. Then, these predictors have been used to generate an extended polynomial-base function, and an ℓ1-norm constrain has been included into the optimization process to promote sparsity. Different non-linear models have been also evaluated, introducing the formulation of Gaussian Processes for regression for different kernel functions. The capabilities of the models here tuned are compared by means of cross-validation global and local criteria. Cross-validated error and leverage distribution have been analyzed. The proper compromise between model accuracy and generalizability is identified as a Pareto front in the space of the cross-validation indicators. In addition, a new variance-based indicator for the identification of the best model among the candidate ones has been introduced and its capability in complementing the cross-validation analysis is here discussed. The learning strategy has been finally replicated adopting an extremely large database, experimentally acquired in the framework of the extensive collaboration between the University of Genova and AvioAero. For confidentiality, results concerning this database are not shown in terms of response surface, but model performances are discussed in order to strengthen the strategy here adopted, that could be useful also to other research groups adopting their own data.
2
Yokoi, Sho, Daichi Mochihashi, Ryo Takahashi, Naoaki Okazaki, and Kentaro Inui. "Learning Co-Substructures by Kernel Dependence Maximization." In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/465.
Full textAbstract:
Modeling associations between items in a dataset is a problem that is frequently encountered in data and knowledge mining research. Most previous studies have simply applied a predefined fixed pattern for extracting the substructure of each item pair and then analyzed the associations between these substructures. Using such fixed patterns may not, however, capture the significant association. We, therefore, propose the novel machine learning task of extracting a strongly associated substructure pair (co-substructure) from each input item pair. We call this task dependent co-substructure extraction (DCSE), and formalize it as a dependence maximization problem. Then, we discuss critical issues with this task: the data sparsity problem and a huge search space. To address the data sparsity problem, we adopt the Hilbert--Schmidt independence criterion as an objective function. To improve search efficiency, we adopt the Metropolis--Hastings algorithm. We report the results of empirical evaluations, in which the proposed method is applied for acquiring and predicting narrative event pairs, an active task in the field of natural language processing.
3
Vahdat, Arash, Kevin Cannons, Greg Mori, Sangmin Oh, and Ilseo Kim. "Compositional Models for Video Event Detection: A Multiple Kernel Learning Latent Variable Approach." In 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, 2013. http://dx.doi.org/10.1109/iccv.2013.463.
Full text
4
He, Jia, Changying Du, Changde Du, Fuzhen Zhuang, Qing He, and Guoping Long. "Nonlinear Maximum Margin Multi-View Learning with Adaptive Kernel." In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/254.
Full textAbstract:
Existing multi-view learning methods based on kernel function either require the user to select and tune a single predefined kernel or have to compute and store many Gram matrices to perform multiple kernel learning. Apart from the huge consumption of manpower, computation and memory resources, most of these models seek point estimation of their parameters, and are prone to overfitting to small training data. This paper presents an adaptive kernel nonlinear max-margin multi-view learning model under the Bayesian framework. Specifically, we regularize the posterior of an efficient multi-view latent variable model by explicitly mapping the latent representations extracted from multiple data views to a random Fourier feature space where max-margin classification constraints are imposed. Assuming these random features are drawn from Dirichlet process Gaussian mixtures, we can adaptively learn shift-invariant kernels from data according to Bochners theorem. For inference, we employ the data augmentation idea for hinge loss, and design an efficient gradient-based MCMC sampler in the augmented space. Having no need to compute the Gram matrix, our algorithm scales linearly with the size of training set. Extensive experiments on real-world datasets demonstrate that our method has superior performance.
5
Garcia-Vega, S., E. A. Leon-Gomez, and G. Castellanos-Dominguez. "Time Series Prediction for Kernel-based Adaptive Filters Using Variable Bandwidth, Adaptive Learning-rate, and Dimensionality Reduction." In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. http://dx.doi.org/10.1109/icassp.2019.8683117.
Full text
6
Sclavounos, Paul D., and Yu Ma. "Artificial Intelligence Machine Learning in Marine Hydrodynamics." In ASME 2018 37th International Conference on Ocean, Offshore and Arctic Engineering. American Society of Mechanical Engineers, 2018. http://dx.doi.org/10.1115/omae2018-77599.
Full textAbstract:
Artificial Intelligence (AI) Support Vector Machine (SVM) learning algorithms have enjoyed rapid growth in recent years with applications in a wide range of disciplines often with impressive results. The present paper introduces this machine learning technology to the field of marine hydrodynamics for the study of complex potential and viscous flow problems. Examples considered include the forecasting of the seastate elevations and vessel responses using their past time records as “explanatory variables” or “features” and the development of a nonlinear model for the roll restoring, added moment of inertia and viscous damping using the vessel response kinematics from free decay tests as “features”. A key innovation of AI-SVM kernel algorithms is that the nonlinear dependence of the dependent variable on the “features” is embedded into the SVM kernel and its selection plays a key role in the performance of the algorithms. The kernel selection is discussed and its relation to the physics of the marine hydrodynamic flows considered in the present paper is addressed.
7
Hu, Chao, Gaurav Jain, Craig Schmidt, Carrie Strief, and Melani Sullivan. "Online Estimation of Lithium-Ion Battery Capacity Using Sparse Bayesian Learning." In ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2015. http://dx.doi.org/10.1115/detc2015-46964.
Full textAbstract:
Lithium-ion (Li-ion) rechargeable batteries are used as one of the major energy storage components for implantable medical devices. Reliability of Li-ion batteries used in these devices has been recognized as of high importance from a broad range of stakeholders, including medical device manufacturers, regulatory agencies, patients and physicians. To ensure a Li-ion battery operates reliably, it is important to develop health monitoring techniques that accurately estimate the capacity of the battery throughout its life-time. This paper presents a sparse Bayesian learning method that utilizes the charge voltage and current measurements to estimate the capacity of a Li-ion battery used in an implantable medical device. Relevance Vector Machine (RVM) is employed as a probabilistic kernel regression method to learn the complex dependency of the battery capacity on the characteristic features that are extracted from the charge voltage and current measurements. Owing to the sparsity property of RVM, the proposed method generates a reduced-scale regression model that consumes only a small fraction of the CPU time required by a full-scale model, which makes online capacity estimation computationally efficient. 10 years’ continuous cycling data and post-explant cycling data obtained from Li-ion prismatic cells are used to verify the performance of the proposed method.
8
Liu, Yanchi, Tan Yan, and Haifeng Chen. "Exploiting Graph Regularized Multi-dimensional Hawkes Processes for Modeling Events with Spatio-temporal Characteristics." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/343.
Full textAbstract:
Multi-dimensional Hawkes processes (MHP) has been widely used for modeling temporal events. However, when MHP was used for modeling events with spatio-temporal characteristics, the spatial information was often ignored despite its importance. In this paper, we introduce a framework to exploit MHP for modeling spatio-temporal events by considering both temporal and spatial information. Specifically, we design a graph regularization method to effectively integrate the prior spatial structure into MHP for learning influence matrix between different locations. Indeed, the prior spatial structure can be first represented as a connection graph. Then, a multi-view method is utilized for the alignment of the prior connection graph and influence matrix while preserving the sparsity and low-rank properties of the kernel matrix. Moreover, we develop an optimization scheme using an alternating direction method of multipliers to solve the resulting optimization problem. Finally, the experimental results show that we are able to learn the interaction patterns between different geographical areas more effectively with prior connection graph introduced for regularization.
9
Cheng, Hongliang, Weilin Yi, and Luchen Ji. "Multi-Point Optimization Design of High Pressure-Ratio Centrifugal Impeller Based on Machine Learning." In ASME Turbo Expo 2020: Turbomachinery Technical Conference and Exposition. American Society of Mechanical Engineers, 2020. http://dx.doi.org/10.1115/gt2020-14576.
Full textAbstract:
Abstract The traditional optimization method on turbomachinery has the problem as time-consuming and difficult to solve well multi-parameter optimization in a short time. So an accurate surrogate model that is used to estimate the functional relationship between the independent variable and objective value is important to accelerate computationally expensive CFD-based optimization. Many of them had been developed and proven their reliability, such as the Kriging model, Back Propagation Neural Network (BPNN), Artificial Neural Network (ANN) and Support Vector Regression (SVR), etc. Because SVR is based on the principle of structural risk minimization, it has advantages in dealing with a small database, high dimensional and non-linear problems. The reliability of SVR is depended on its kernel parameter and penalty factor, and it can be improved by getting optimal parameters with some optimization algorithms. In this paper, a machine learning model based on SVR combined with a multi-point genetic algorithm (MPGA) is applied to the optimization of a centrifugal impeller with 41 parameters. The optimization objectives are to maximize efficiency at design point and pressure-ratio at near stall point and to minimize the variation of choked mass flow. Results show that (1) time costs reduced significantly. (2) The maximum efficiency increases by 1.24%. (3) It is verified that the reliability of the SVR-MPGA model for multi parameters optimization by comparing to the results of the traditional optimization method — Design 3D. The efforts of this study expand the application of machine learning and provide an idea for multi-point optimization of turbomachinery as well.
10
Reda Ali, Ahmed, Makky Sandra Jaya, and Ernest A. Jones. "Machine Learning Strategies for Accurate Log Prediction in Reservoir Characterization: Self-Calibrating Versus Domain-Knowledge." In SPE/IATMI Asia Pacific Oil & Gas Conference and Exhibition. SPE, 2021. http://dx.doi.org/10.2118/205602-ms.
Full textAbstract:
Abstract Petrophysical evaluation is a crucial task for reservoir characterization but it is often complicated, time-consuming and associated with uncertainties. Moreover, this job is subjective and ambiguous depending on the petrophysicist's experience. Utilizing the flourishing Artificial Intelligence (AI)/Machine Learning (ML) is a way to build an automating process with minimal human intervention, improving consistency and efficiency of well log prediction and interpretation. Nowadays, the argument is whether AI-ML should base on a statistically self-calibrating or knowledge-based prediction framework! In this study, we develop a petrophysically knowledge-based AI-ML workflow that upscale sparsely-sampled core porosity and permeability into continuous curves along the entire well interval. AI-ML focuses on making predictions from analyzing data by learning and identifying patterns. The accuracy of the self-calibrating statistical models is heavily dependent on the volume of training data. The proposed AI-ML workflow uses raw well logs (gamma-ray, neutron and density) to predict porosity and permeability over the well interval using sparsely core data. The challenge in building the AI-ML model is the number of data points used for training showed an imbalance in the relative sampling of plugs, i.e. the number of core data (used as target variable) is less than 10%. Ensemble learning and stacking ML approaches are used to obtain maximum predictive performance of self-calibrating learning strategy. Alternatively, a new petrophysical workflow is established to debrief the domain experience in the feature selection that is used as an important weight in the regression problem. This helps ML model to learn more accurately by discovering hidden relationships between independent and target variables. This workflow is the inference engine of the AI-ML model to extract relevant domain-knowledge within the system that leads to more accurate predictions. The proposed knowledge-driven ML strategy achieved a prediction accuracy of R2 score = 87% (Correlation Coefficient (CC) of 96%). This is a significant improvement by R2 = 57% (CC = 62%) compared to the best performing self-calibrating ML models. The predicted properties are upscaled automatically to predict uncored intervals, improving data coverage and property population in reservoir models leading to the improvement of the model robustness. The high prediction accuracy demonstrates the potential of knowledge-driven AI-ML strategy in predicting rock properties under data sparsity and limitations and saving significant cost and time. This paper describes an AI-ML workflow that predicts high-resolution continuous porosity and permeability logs from imbalanced and sparse core plug data. The method successfully incorporates new type petrophysical facies weight as a feature augmentation engine for ML domain-knowledge framework. The workflow consisted of petrophysical treatment of raw data includes log quality control, preconditioning, processing, features augmentation and labelling, followed by feature selection to impersonate domain experience.