Academic literature on the topic 'Variable sparsity kernel learning'

Regression is a special case of the general model fitting and selection procedures discussed in chapters 4 and 5. It can be defined as the relation between a dependent variable, y, and a set of independent variables, x, that describes the expectation value of y given x: E [y¦x]. The purpose of obtaining a “best-fit” model ranges from scientific interest in the values of model parameters (e.g., the properties of dark energy, or of a newly discovered planet) to the predictive power of the resulting model (e.g., predicting solar activity). This chapter starts with a general formulation for regression, list various simplified cases, and then discusses methods that can be used to address them, such as regression for linear models, kernel regression, robust regression and nonlinear regression.

Intrusion Detection Systems (IDS) play a major role in the area of combating security breaches for information security. Current IDS are developed with Machine learning techniques like Artificial Neural Networks, C 4.5, KNN, Naïve Bayes classifiers, Genetic algorithms Fuzzy logic and SVMs. The objective of this paper is to apply Artificial Neural Networks and Support Vector Machines for intrusion detection. Artificial Neural Networks are applied along with faster training methods like variable learning rate and scaled conjugate gradient. Support Vector Machines use various kernel functions to improve the performance. From the kddcup'99 dataset 45,657 instances are taken and used in our experiment. The speed is compared for various training functions. The performance of various kernel functions is assessed. The detection rate of Support Vector Machines is found to be greater than Artificial Neural Networks with less number of false positives and with less time of detection.

A basic task of machine learning and data mining is to automatically uncover <b>patterns</b> that reflect regularities in a data set. When dealing with a large database, especially when domain knowledge is not available or very weak, this can be a challenging task. The purpose of <b>pattern discovery</b> is to find non-random relations among events from data sets. For example, the “exclusive OR” (XOR) problem concerns 3 binary variables, A, B and C=A<img src="http://resources.igi-global.com/Marketing/Preface_Figures/x_symbol.png">B, i.e. C is true when either A or B, but not both, is true. Suppose not knowing that it is the XOR problem, we would like to check whether or not the occurrence of the compound event [A=T, B=T, C=F] is just a random happening. If we could estimate its frequency of occurrences under the random assumption, then we know that it is not random if the observed frequency deviates significantly from that assumption. We refer to such a compound event as an event association pattern, or simply a <b>pattern</b>, if its frequency of occurrences significantly deviates from the default random assumption in the statistical sense. For instance, suppose that an XOR database contains 1000 samples and each primary event (e.g. [A=T]) occurs 500 times. The expected frequency of occurrences of the compound event [A=T, B=T, C=F] under the independence assumption is 0.5×0.5×0.5×1000 = 125. Suppose that its observed frequency is 250, we would like to see whether or not the difference between the observed and expected frequencies (i.e. 250 – 125) is significant enough to indicate that the compound event is not a random happening.<div><br></div><div>In statistics, to test the correlation between random variables, <b>contingency table</b> with chi-squared statistic (Mills, 1955) is widely used. Instead of investigating variable correlations, pattern discovery shifts the traditional correlation analysis in statistics at the variable level to association analysis at the event level, offering an effective method to detect statistical association among events.</div><div><br></div><div>In the early 90’s, this approach was established for second order event associations (Chan & Wong, 1990). A higher order <b>pattern discovery</b> algorithm was devised in the mid 90’s for discrete-valued data sets (Wong & Yang, 1997). In our methods, patterns inherent in data are defined as statistically significant associations of two or more primary events of different attributes if they pass a statistical test for deviation significance based on <b>residual analysis</b>. The discovered high order patterns can then be used for classification (Wang & Wong, 2003). With continuous data, events are defined as Borel sets and the pattern discovery process is formulated as an optimization problem which recursively partitions the sample space for the best set of significant events (patterns) in the form of high dimension intervals from which probability density can be estimated by Gaussian kernel fit (Chau & Wong, 1999). Classification can then be achieved using Bayesian classifiers. For data with a mixture of discrete and continuous data (Wong & Yang, 2003), the latter is categorized based on a global optimization discretization algorithm (Liu, Wong & Yang, 2004). As demonstrated in numerous real-world and commercial applications (Yang, 2002), pattern discovery is an ideal tool to uncover subtle and useful patterns in a database. </div><div><br></div><div>In pattern discovery, three open problems are addressed. The first concerns learning where noise and uncertainty are present. In our method, noise is taken as inconsistent samples against statistically significant patterns. Missing attribute values are also considered as noise. Using a standard statistical <b>hypothesis testing</b> to confirm statistical patterns from the candidates, this method is a less ad hoc approach to discover patterns than most of its contemporaries. The second problem concerns the detection of polythetic patterns without relying on exhaustive search. Efficient systems for detecting monothetic patterns between two attributes exist (e.g. Chan & Wong, 1990). However, for detecting polythetic patterns, an exhaustive search is required (Han, 2001). In many problem domains, polythetic assessments of feature combinations (or higher order relationship detection) are imperative for robust learning. Our method resolves this problem by directly constructing polythetic concepts while screening out non-informative pattern candidates, using statisticsbased heuristics in the discovery process. The third problem concerns the representation of the detected patterns. Traditionally, if-then rules and graphs, including networks and trees, are the most popular ones. However, they have shortcomings when dealing with multilevel and multiple order patterns due to the non-exhaustive and unpredictable hierarchical nature of the inherent patterns. We adopt <b>attributed hypergraph</b> (AHG) (Wang & Wong, 1996) as the representation of the detected patterns. It is a data structure general enough to encode information at many levels of abstraction, yet simple enough to quantify the information content of its organized structure. It is able to encode both the qualitative and the quantitative characteristics and relations inherent in the data set.<br></div>

Abstract In the present work linear and non-linear regression functions have been tuned with an extensive database describing the unsteady aerodynamic efficiency of low-pressure-turbine cascades. The learning strategy has been first defined using a dataset published in a previous work concerning the loss coefficient measured in a large-scale cascade for a large variation of the Reynolds number, the reduced frequency, and the flow coefficient. Linear models have been educated accounting for the Occam’s razor parsimony criterion, condensing the effects due to the parameter variation in few predi

Modeling associations between items in a dataset is a problem that is frequently encountered in data and knowledge mining research. Most previous studies have simply applied a predefined fixed pattern for extracting the substructure of each item pair and then analyzed the associations between these substructures. Using such fixed patterns may not, however, capture the significant association. We, therefore, propose the novel machine learning task of extracting a strongly associated substructure pair (co-substructure) from each input item pair. We call this task dependent co-substructure extrac

Existing multi-view learning methods based on kernel function either require the user to select and tune a single predefined kernel or have to compute and store many Gram matrices to perform multiple kernel learning. Apart from the huge consumption of manpower, computation and memory resources, most of these models seek point estimation of their parameters, and are prone to overfitting to small training data. This paper presents an adaptive kernel nonlinear max-margin multi-view learning model under the Bayesian framework. Specifically, we regularize the posterior of an efficient multi-view la

Artificial Intelligence (AI) Support Vector Machine (SVM) learning algorithms have enjoyed rapid growth in recent years with applications in a wide range of disciplines often with impressive results. The present paper introduces this machine learning technology to the field of marine hydrodynamics for the study of complex potential and viscous flow problems. Examples considered include the forecasting of the seastate elevations and vessel responses using their past time records as “explanatory variables” or “features” and the development of a nonlinear model for the roll restoring, added momen

Lithium-ion (Li-ion) rechargeable batteries are used as one of the major energy storage components for implantable medical devices. Reliability of Li-ion batteries used in these devices has been recognized as of high importance from a broad range of stakeholders, including medical device manufacturers, regulatory agencies, patients and physicians. To ensure a Li-ion battery operates reliably, it is important to develop health monitoring techniques that accurately estimate the capacity of the battery throughout its life-time. This paper presents a sparse Bayesian learning method that utilizes t

Multi-dimensional Hawkes processes (MHP) has been widely used for modeling temporal events. However, when MHP was used for modeling events with spatio-temporal characteristics, the spatial information was often ignored despite its importance. In this paper, we introduce a framework to exploit MHP for modeling spatio-temporal events by considering both temporal and spatial information. Specifically, we design a graph regularization method to effectively integrate the prior spatial structure into MHP for learning influence matrix between different locations. Indeed, the prior spatial structure c

Abstract The traditional optimization method on turbomachinery has the problem as time-consuming and difficult to solve well multi-parameter optimization in a short time. So an accurate surrogate model that is used to estimate the functional relationship between the independent variable and objective value is important to accelerate computationally expensive CFD-based optimization. Many of them had been developed and proven their reliability, such as the Kriging model, Back Propagation Neural Network (BPNN), Artificial Neural Network (ANN) and Support Vector Regression (SVR), etc. Because SVR

Abstract Petrophysical evaluation is a crucial task for reservoir characterization but it is often complicated, time-consuming and associated with uncertainties. Moreover, this job is subjective and ambiguous depending on the petrophysicist's experience. Utilizing the flourishing Artificial Intelligence (AI)/Machine Learning (ML) is a way to build an automating process with minimal human intervention, improving consistency and efficiency of well log prediction and interpretation. Nowadays, the argument is whether AI-ML should base on a statistically self-calibrating or knowledge-based predicti