Academic literature on the topic 'Selection of hyperparameters'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Selection of hyperparameters.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Selection of hyperparameters"
1
Sun, Yunlei, Huiquan Gong, Yucong Li, and Dalin Zhang. "Hyperparameter Importance Analysis based on N-RReliefF Algorithm." International Journal of Computers Communications & Control 14, no. 4 (August 5, 2019): 557–73. http://dx.doi.org/10.15837/ijccc.2019.4.3593.
Full textAbstract:
Hyperparameter selection has always been the key to machine learning. The Bayesian optimization algorithm has recently achieved great success, but it has certain constraints and limitations in selecting hyperparameters. In response to these constraints and limitations, this paper proposed the N-RReliefF algorithm, which can evaluate the importance of hyperparameters and the importance weights between hyperparameters. The N-RReliefF algorithm estimates the contribution of a single hyperparameter to the performance according to the influence degree of each hyperparameter on the performance and calculates the weight of importance between the hyperparameters according to the improved normalization formula. The N-RReliefF algorithm analyses the hyperparameter configuration and performance set generated by Bayesian optimization, and obtains the important hyperparameters in random forest algorithm and SVM algorithm. The experimental results verify the effectiveness of the N-RReliefF algorithm.
2
Bengio, Yoshua. "Gradient-Based Optimization of Hyperparameters." Neural Computation 12, no. 8 (August 1, 2000): 1889–900. http://dx.doi.org/10.1162/089976600300015187.
Full textAbstract:
Many machine learning algorithms can be formulated as the minimization of a training criterion that involves a hyperparameter. This hyperparameter is usually chosen by trial and error with a model selection criterion. In this article we present a methodology to optimize several hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to the hyperparameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyperparameters is efficiently computed by backpropagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient involving second derivatives of the training criterion.
3
Lohvithee, Manasavee, Wenjuan Sun, Stephane Chretien, and Manuchehr Soleimani. "Ant Colony-Based Hyperparameter Optimisation in Total Variation Reconstruction in X-ray Computed Tomography." Sensors 21, no. 2 (January 15, 2021): 591. http://dx.doi.org/10.3390/s21020591.
Full textAbstract:
In this paper, a computer-aided training method for hyperparameter selection of limited data X-ray computed tomography (XCT) reconstruction was proposed. The proposed method employed the ant colony optimisation (ACO) approach to assist in hyperparameter selection for the adaptive-weighted projection-controlled steepest descent (AwPCSD) algorithm, which is a total-variation (TV) based regularisation algorithm. During the implementation, there was a colony of artificial ants that swarm through the AwPCSD algorithm. Each ant chose a set of hyperparameters required for its iterative CT reconstruction and the correlation coefficient (CC) score was given for reconstructed images compared to the reference image. A colony of ants in one generation left a pheromone through its chosen path representing a choice of hyperparameters. Higher score means stronger pheromones/probabilities to attract more ants in the next generations. At the end of the implementation, the hyperparameter configuration with the highest score was chosen as an optimal set of hyperparameters. In the experimental results section, the reconstruction using hyperparameters from the proposed method was compared with results from three other cases: the conjugate gradient least square (CGLS), the AwPCSD algorithm using the set of arbitrary hyperparameters and the cross-validation method.The experiments showed that the results from the proposed method were superior to those of the CGLS algorithm and the AwPCSD algorithm using the set of arbitrary hyperparameters. Although the results of the ACO algorithm were slightly inferior to those of the cross-validation method as measured by the quantitative metrics, the ACO algorithm was over 10 times faster than cross—Validation. The optimal set of hyperparameters from the proposed method was also robust against an increase of noise in the data and can be applicable to different imaging samples with similar context. The ACO approach in the proposed method was able to identify optimal values of hyperparameters for a dataset and, as a result, produced a good quality reconstructed image from limited number of projection data. The proposed method in this work successfully solves a problem of hyperparameters selection, which is a major challenge in an implementation of TV based reconstruction algorithms.
4
Adewole, Ayoade I., and Olusoga A. Fasoranbaku. "Determination of Quantile Range of Optimal Hyperparameters Using Bayesian Estimation." Tanzania Journal of Science 47, no. 3 (August 13, 2021): 988–98. http://dx.doi.org/10.4314/tjs.v47i3.10.
Full textAbstract:
Bayesian estimations have the advantages of taking into account the uncertainty of all parameter estimates which allows virtually the use of vague priors. This study focused on determining the quantile range at which optimal hyperparameter of normally distributed data with vague information could be obtained in Bayesian estimation of linear regression models. A Monte Carlo simulation approach was used to generate a sample size of 200 data-set. Observation precisions and posterior precisions were estimated from the regression output to determine the posterior means estimate for each model to derive the new dependent variables. The variances were divided into 10 equal parts to obtain the hyperparameters of the prior distribution. Average absolute deviation for model selection was used to validate the adequacy of each model. The study revealed the optimal hyperparameters located at 5th and 7th deciles. The research simplified the process of selecting the hyperparameters of prior distribution from the data with vague information in empirical Bayesian inferences.
Keywords: Optimal Hyperparameters; Quantile Ranges; Bayesian Estimation; Vague prior
5
Johnson, Kara Layne, and Nicole Bohme Carnegie . "Calibration of an Adaptive Genetic Algorithm for Modeling Opinion Diffusion." Algorithms 15, no. 2 (January 28, 2022): 45. http://dx.doi.org/10.3390/a15020045.
Full textAbstract:
Genetic algorithms mimic the process of natural selection in order to solve optimization problems with minimal assumptions and perform well when the objective function has local optima on the search space. These algorithms treat potential solutions to the optimization problem as chromosomes, consisting of genes which undergo biologically-inspired operators to identify a better solution. Hyperparameters or control parameters determine the way these operators are implemented. We created a genetic algorithm in order to fit a DeGroot opinion diffusion model using limited data, making use of selection, blending, crossover, mutation, and survival operators. We adapted the algorithm from a genetic algorithm for design of mixture experiments, but the new algorithm required substantial changes due to model assumptions and the large parameter space relative to the design space. In addition to introducing new hyperparameters, these changes mean the hyperparameter values suggested for the original algorithm cannot be expected to result in optimal performance. To make the algorithm for modeling opinion diffusion more accessible to researchers, we conduct a simulation study investigating hyperparameter values. We find the algorithm is robust to the values selected for most hyperparameters and provide suggestions for initial, if not default, values and recommendations for adjustments based on algorithm output.
6
Raji, Ismail Damilola, Habeeb Bello-Salau, Ime Jarlath Umoh, Adeiza James Onumanyi, Mutiu Adesina Adegboye, and Ahmed Tijani Salawudeen. "Simple Deterministic Selection-Based Genetic Algorithm for Hyperparameter Tuning of Machine Learning Models." Applied Sciences 12, no. 3 (January 24, 2022): 1186. http://dx.doi.org/10.3390/app12031186.
Full textAbstract:
Hyperparameter tuning is a critical function necessary for the effective deployment of most machine learning (ML) algorithms. It is used to find the optimal hyperparameter settings of an ML algorithm in order to improve its overall output performance. To this effect, several optimization strategies have been studied for fine-tuning the hyperparameters of many ML algorithms, especially in the absence of model-specific information. However, because most ML training procedures need a significant amount of computational time and memory, it is frequently necessary to build an optimization technique that converges within a small number of fitness evaluations. As a result, a simple deterministic selection genetic algorithm (SDSGA) is proposed in this article. The SDSGA was realized by ensuring that both chromosomes and their accompanying fitness values in the original genetic algorithm are selected in an elitist-like way. We assessed the SDSGA over a variety of mathematical test functions. It was then used to optimize the hyperparameters of two well-known machine learning models, namely, the convolutional neural network (CNN) and the random forest (RF) algorithm, with application on the MNIST and UCI classification datasets. The SDSGA’s efficiency was compared to that of the Bayesian Optimization (BO) and three other popular metaheuristic optimization algorithms (MOAs), namely, the genetic algorithm (GA), particle swarm optimization (PSO) and biogeography-based optimization (BBO) algorithms. The results obtained reveal that the SDSGA performed better than the other MOAs in solving 11 of the 17 known benchmark functions considered in our study. While optimizing the hyperparameters of the two ML models, it performed marginally better in terms of accuracy than the other methods while taking less time to compute.
7
Lu, Wanjie, Hongpeng Mao, Fanhao Lin, Zilin Chen, Hua Fu, and Yaosong Xu. "Recognition of rolling bearing running state based on genetic algorithm and convolutional neural network." Advances in Mechanical Engineering 14, no. 4 (April 2022): 168781322210956. http://dx.doi.org/10.1177/16878132221095635.
Full textAbstract:
In this study, the GA-CNN model is proposed to realize the automatic recognition of rolling bearing running state. Firstly, to avoid the over-fitting and gradient dispersion in the training process of the CNN model, the BN layer and Dropout technology are introduced into the LeNet-5 model. Secondly, to obtain the automatic selection of hyperparameters in CNN model, a method of hyperparameter selection combined with genetic algorithm (GA) is proposed. In the proposed method, each hyperparameter is encoded as a chromosome, and each hyperparameter has a mapping relationship with the corresponding gene position on the chromosome. After the process of chromosome selection, crossover and variation, the fitness value is calculated to present the superiority of the current chromosome. The chromosomes with high fitness values are more likely to be selected in the next genetic iteration, that is, the optimal hyperparameters of the CNN model are obtained. Then, vibration signals from CWRU are used for the time-frequency analysis, and the obtained time-frequency image set is used to train and test the proposed GA-CNN model, and the accuracy of the proposed model can reach 99.85% on average, and the training speed is four times faster than the model LeNet-5. Finally, the result of the experiment on the laboratory test platform The experimental results confirm the superiority of the method and the transplantability of the optimization model.
8
Han, Junjie, Cedric Gondro, and Juan Steibel. "98 Using differential evolution to improve predictive accuracy of deep learning models applied to pig production data." Journal of Animal Science 98, Supplement_3 (November 2, 2020): 27. http://dx.doi.org/10.1093/jas/skaa054.048.
Full textAbstract:
Abstract Deep learning (DL) is being used for prediction in precision livestock farming and in genomic prediction. However, optimizing hyperparameters in DL models is critical for their predictive performance. Grid search is the traditional approach to select hyperparameters in DL, but it requires exhaustive search over the parameter space. We propose hyperparameter selection using differential evolution (DE), which is a heuristic algorithm that does not require exhaustive search. The goal of this study was to design and apply DE to optimize hyperparameters of DL models for genomic prediction and image analysis in pig production systems. One dataset consisted of 910 pigs genotyped with 28,916 SNP markers to predict their post-mortem meat pH. Another dataset consisted of 1,334 images of pigs eating inside a single-spaced feeder classified as: “single pig” or “multiple pigs.” The accuracy of genomic prediction was defined as the correlation between the predicted pH and the observed pH. The image classification prediction accuracy was the proportion of correctly classified images. For genomic prediction, a multilayer perceptron (MLP) was optimized. For image classification, MLP and convolutional neural networks (CNN) were optimized. For genomic prediction, the initial hyperparameter set resulted in an accuracy of 0.032 and for image classification, the initial accuracy was between 0.72 and 0.76. After optimization using DE, the genomic prediction accuracy was 0.3688 compared to 0.334 using GBLUP. The top selected models included one layer, 60 neurons, sigmoid activation and L2 penalty = 0.3. The accuracy of image classification after optimization was between 0.89 and 0.92. Selected models included three layers, adamax optimizer and relu or elu activation for the MLP, and one layer, 64 filters and 5×5 filter size for the CNN. DE can adapt the hyperparameter selection to each problem, dataset and model, and it significantly increased prediction accuracy with minimal user input.
9
Wang, Chung-Ying, Chien-Yao Huang, and Yen-Han Chiang. "Solutions of Feature and Hyperparameter Model Selection in the Intelligent Manufacturing." Processes 10, no. 5 (April 27, 2022): 862. http://dx.doi.org/10.3390/pr10050862.
Full textAbstract:
In the era of Industry 4.0, numerous AI technologies have been widely applied. However, implementation of the AI technology requires observation, analysis, and pre-processing of the obtained data, which takes up 60–90% of total time after data collection. Next, sensors and features are selected. Finally, the AI algorithms are used for clustering or classification. Despite the completion of data pre-processing, the subsequent feature selection and hyperparameter tuning in the AI model affect the sensitivity, accuracy, and robustness of the system. In this study, two novel approaches of sensor and feature selecting system, and hyperparameter tuning mechanisms are proposed. In the sensor and feature selecting system, the Shapley Additive ExPlanations model is used to calculate the contribution of individual features or sensors and to make the black-box AI model transparent, whereas, in the hyperparameter tuning mechanism, Hyperopt is used for tuning to improve model performance. Implementation of these two new systems is expected to reduce the problems in the processes of selection of the most sensitive features in the pre-processing stage, and tuning of hyperparameters, which are the most frequently occurring problems. Meanwhile, these methods are also applicable to the field of tool wear monitoring systems in intelligent manufacturing.
10
Hendriks, Jacob, and Patrick Dumond. "Exploring the Relationship between Preprocessing and Hyperparameter Tuning for Vibration-Based Machine Fault Diagnosis Using CNNs." Vibration 4, no. 2 (April 3, 2021): 284–309. http://dx.doi.org/10.3390/vibration4020019.
Full textAbstract:
This paper demonstrates the differences between popular transformation-based input representations for vibration-based machine fault diagnosis. This paper highlights the dependency of different input representations on hyperparameter selection with the results of training different configurations of classical convolutional neural networks (CNNs) with three common benchmarking datasets. Raw temporal measurement, Fourier spectrum, envelope spectrum, and spectrogram input types are individually used to train CNNs. Many configurations of CNNs are trained, with variable input sizes, convolutional kernel sizes and stride. The results show that each input type favors different combinations of hyperparameters, and that each of the datasets studied yield different performance characteristics. The input sizes are found to be the most significant determiner of whether overfitting will occur. It is demonstrated that CNNs trained with spectrograms are less dependent on hyperparameter optimization over all three datasets. This paper demonstrates the wide range of performance achieved by CNNs when preprocessing method and hyperparameters are varied as well as their complex interaction, providing researchers with useful background information and a starting place for further optimization.
Dissertations / Theses on the topic "Selection of hyperparameters"
1
Ndiaye, Eugene. "Safe optimization algorithms for variable selection and hyperparameter tuning." Thesis, Université Paris-Saclay (ComUE), 2018. http://www.theses.fr/2018SACLT004/document.
Full textAbstract:
Le traitement massif et automatique des données requiert le développement de techniques de filtration des informations les plus importantes. Parmi ces méthodes, celles présentant des structures parcimonieuses se sont révélées idoines pour améliorer l’efficacité statistique et computationnelle des estimateurs, dans un contexte de grandes dimensions. Elles s’expriment souvent comme solution de la minimisation du risque empirique régularisé s’écrivant comme une somme d’un terme lisse qui mesure la qualité de l’ajustement aux données, et d’un terme non lisse qui pénalise les solutions complexes. Cependant, une telle manière d’inclure des informations a priori, introduit de nombreuses difficultés numériques pour résoudre le problème d’optimisation sous-jacent et pour calibrer le niveau de régularisation. Ces problématiques ont été au coeur des questions que nous avons abordées dans cette thèse.Une technique récente, appelée «Screening Rules», propose d’ignorer certaines variables pendant le processus d’optimisation en tirant bénéfice de la parcimonie attendue des solutions. Ces règles d’élimination sont dites sûres lorsqu’elles garantissent de ne pas rejeter les variables à tort. Nous proposons un cadre unifié pour identifier les structures importantes dans ces problèmes d’optimisation convexes et nous introduisons les règles «Gap Safe Screening Rules». Elles permettent d’obtenir des gains considérables en temps de calcul grâce à la réduction de la dimension induite par cette méthode. De plus, elles s’incorporent facilement aux algorithmes itératifs et s’appliquent à un plus grand nombre de problèmes que les méthodes précédentes.Pour trouver un bon compromis entre minimisation du risque et introduction d’un biais d’apprentissage, les algorithmes d’homotopie offrent la possibilité de tracer la courbe des solutions en fonction du paramètre de régularisation. Toutefois, ils présentent des instabilités numériques dues à plusieurs inversions de matrice, et sont souvent coûteux en grande dimension. Aussi, ils ont des complexités exponentielles en la dimension du modèle dans des cas défavorables. En autorisant des solutions approchées, une approximation de la courbe des solutions permet de contourner les inconvénients susmentionnés. Nous revisitons les techniques d’approximation des chemins de régularisation pour une tolérance prédéfinie, et nous analysons leur complexité en fonction de la régularité des fonctions de perte en jeu. Il s’ensuit une proposition d’algorithmes optimaux ainsi que diverses stratégies d’exploration de l’espace des paramètres. Ceci permet de proposer une méthode de calibration de la régularisation avec une garantie de convergence globale pour la minimisation du risque empirique sur les données de validation.Le Lasso, un des estimateurs parcimonieux les plus célèbres et les plus étudiés, repose sur une théorie statistique qui suggère de choisir la régularisation en fonction de la variance des observations. Ceci est difficilement utilisable en pratique car, la variance du modèle est une quantité souvent inconnue. Dans de tels cas, il est possible d’optimiser conjointement les coefficients de régression et le niveau de bruit. Ces estimations concomitantes, apparues dans la littérature sous les noms de Scaled Lasso, Square-Root Lasso, fournissent des résultats théoriques aussi satisfaisants que celui du Lasso tout en étant indépendant de la variance réelle. Bien que présentant des avancées théoriques et pratiques importantes, ces méthodes sont aussi numériquement instables et les algorithmes actuellement disponibles sont coûteux en temps de calcul. Nous illustrons ces difficultés et nous proposons à la fois des modifications basées sur des techniques de lissage pour accroitre la stabilité numérique de ces estimateurs, ainsi qu’un algorithme plus efficace pour les obtenirMassive and automatic data processing requires the development of techniques able to filter the most important information. Among these methods, those with sparse structures have been shown to improve the statistical and computational efficiency of estimators in a context of large dimension. They can often be expressed as a solution of regularized empirical risk minimization and generally lead to non differentiable optimization problems in the form of a sum of a smooth term, measuring the quality of the fit, and a non-smooth term, penalizing complex solutions. Although it has considerable advantages, such a way of including prior information, unfortunately introduces many numerical difficulties both for solving the underlying optimization problem and to calibrate the level of regularization. Solving these issues has been at the heart of this thesis. A recently introduced technique, called "Screening Rules", proposes to ignore some variables during the optimization process by benefiting from the expected sparsity of the solutions. These elimination rules are said to be safe when the procedure guarantees to not reject any variable wrongly. In this work, we propose a unified framework for identifying important structures in these convex optimization problems and we introduce the "Gap Safe Screening Rules". They allows to obtain significant gains in computational time thanks to the dimensionality reduction induced by this method. In addition, they can be easily inserted into iterative algorithms and apply to a large number of problems.To find a good compromise between minimizing risk and introducing a learning bias, (exact) homotopy continuation algorithms offer the possibility of tracking the curve of the solutions as a function of the regularization parameters. However, they exhibit numerical instabilities due to several matrix inversions and are often expensive in large dimension. Another weakness is that a worst-case analysis shows that they have exact complexities that are exponential in the dimension of the model parameter. Allowing approximated solutions makes possible to circumvent the aforementioned drawbacks by approximating the curve of the solutions. In this thesis, we revisit the approximation techniques of the regularization paths given a predefined tolerance and we propose an in-depth analysis of their complexity w.r.t. the regularity of the loss functions involved. Hence, we propose optimal algorithms as well as various strategies for exploring the parameters space. We also provide calibration method (for the regularization parameter) that enjoys globalconvergence guarantees for the minimization of the empirical risk on the validation data.Among sparse regularization methods, the Lasso is one of the most celebrated and studied. Its statistical theory suggests choosing the level of regularization according to the amount of variance in the observations, which is difficult to use in practice because the variance of the model is oftenan unknown quantity. In such case, it is possible to jointly optimize the regression parameter as well as the level of noise. These concomitant estimates, appeared in the literature under the names of Scaled Lasso or Square-Root Lasso, and provide theoretical results as sharp as that of theLasso while being independent of the actual noise level of the observations. Although presenting important advances, these methods are numerically unstable and the currently available algorithms are expensive in computation time. We illustrate these difficulties and we propose modifications based on smoothing techniques to increase stability of these estimators as well as to introduce a faster algorithm
2
Thornton, Chris. "Auto-WEKA : combined selection and hyperparameter optimization of supervised machine learning algorithms." Thesis, University of British Columbia, 2014. http://hdl.handle.net/2429/46177.
Full textAbstract:
Many different machine learning algorithms exist; taking into account each algorithm's set of hyperparameters, there is a staggeringly large number of possible choices. This project considers the problem of simultaneously selecting a learning algorithm and setting its hyperparameters. Previous works attack these issues separately, but this problem can be addressed by a fully automated approach, in particular by leveraging recent innovations in Bayesian optimization. The WEKA software package provides an implementation for a number of feature selection and supervised machine learning algorithms, which we use inside our automated tool, Auto-WEKA. Specifically, we examined the 3 search and 8 evaluator methods for feature selection, as well as all of the classification and regression methods, spanning 2 ensemble methods, 10 meta-methods, 27 base algorithms, and their associated hyperparameters. On 34 popular datasets from the UCI repository, the Delve repository, the KDD Cup 09, variants of the MNIST dataset and CIFAR-10, our method produces classification and regression performance often much better than obtained using state-of-the-art algorithm selection and hyperparameter optimization methods from the literature. Using this integrated approach, users can more effectively identify not only the best machine learning algorithm, but also the corresponding hyperparameter settings and feature selection methods appropriate for that algorithm, and hence achieve improved performance for their specific classification or regression task.
3
Thomas, Janek [Verfasser], and Bernd [Akademischer Betreuer] Bischl. "Gradient boosting in automatic machine learning: feature selection and hyperparameter optimization / Janek Thomas ; Betreuer: Bernd Bischl." München : Universitätsbibliothek der Ludwig-Maximilians-Universität, 2019. http://d-nb.info/1189584808/34.
Full text
4
Радюк, Павло Михайлович, and Pavlo Radiuk. "Інформаційна технологія раннього діагностування пневмонії за індивідуальним підбором параметрів моделі класифікації медичних зображень легень." Дисертація, Хмельницький національний університет, 2021. http://elar.khnu.km.ua/jspui/handle/123456789/11937.
Full textAbstract:
Дисертаційна робота присвячена розв’язанню актуальної науково-прикладної задачі автоматизації процесу діагностування вірусного пневмонічного запалення за медичними зображеннями легень через розроблення інформаційної технології раннього діагностування пневмонії за індивідуальним підбором параметрів моделі класифікації медичних зображень легень. Застосування розробленої інформаційної технології раннього діагностування пневмонії в клінічній практиці дає змогу підвищити точність та надійність ідентифікації пневмонії на ранніх стадіях за медичними зображеннями грудної клітини людини.
Об’єктом дослідження є процес діагностування пневмонії за медичними зображеннями грудної клітини людини.
Предметом дослідження є моделі, методи та засоби інформаційної технології для раннього діагностування пневмонії за медичними зображеннями грудної клітини людини.
У дисертаційній роботі визначено актуальність застосування інформаційних технологій у галузі цифрового діагностування захворювань легень за медичними зображеннями грудної клітини. На основі проведено аналізу методів та підходів до виявлення пневмонії встановлено, що нейромережеві моделі є найкращим рішенням для розроблення інформаційної технології раннього діагностування. Досліджено методи для налаштування нейромережевої моделі та підходи до пояснення та інтерпретування результатів ідентифікації захворювання легень. За аналізом сучасних підходів, методів та інформаційних технологій для діагностування захворювання легень на ранніх стадіях за медичними зображеннями грудної клітини обґрунтовано потребу в створенні інформаційної технології раннього діагностування пневмонії.The present thesis is devoted to solving the topical scientific and applied problem of automating the process of diagnosing viral pneumonia by medical images of the lungs through the development of information technology for early diagnosis of pneumonia by the individual selection of parameters of the classification model by medical images of the lungs. Applying the developed information technology for the early diagnosis of pneumonia in clinical practice by medical images of the human chest increases the accuracy and reliability of pneumonia identification in the early stages
5
Maillard, Guillaume. "Hold-out and Aggregated hold-out Aggregated Hold-Out Aggregated hold-out for sparse linear regression with a robust loss function." Thesis, université Paris-Saclay, 2020. http://www.theses.fr/2020UPASM005.
Full textAbstract:
En statistiques, il est fréquent d'avoir à choisir entre plusieurs estimateurs (sélection d'estimateurs) ou à les combiner. Cela permet notamment d'adapter la complexité d'un modèle statistique en fonction des données (compromis biais-variance). Pour les problèmes de minimisation de risque, une méthode simple et générale, la validation ou hold-out, consiste à consacrer une partie de l'échantillon à l'estimation du risque des estimateurs, dans le but de choisir celui de risque minimal. Cette procédure nécessite de choisir arbitrairement un sous-échantillon "de validation". Afin de réduire l'influence de ce choix, il est possible d'agréger plusieurs estimateurs hold-out en les moyennant (Agrégation d'hold-out). Dans cette thèse, le hold-out et l'agrégation d'hold-out sont étudiés dans différents cadres. Dans un premier temps, les garanties théoriques sur le hold-out sont étendues à des cas où le risque n'est pas borné: les méthodes à noyaux et la régression linéaire parcimonieuse. Dans un deuxième temps, une étude précise du risque de ces méthodes est menée dans un cadre particulier: l'estimation de densité L² par des séries de Fourier. Il est démontré que l'agrégation de hold-out peut faire mieux que le meilleur des estimateurs qu'elle agrège, ce qui est impossible pour une méthode qui, comme le hold-out ou la validation croisée, sélectionne un seul estimateurIn statistics, it is often necessary to choose between different estimators (estimator selection) or to combine them (agregation). For risk-minimization problems, a simple method, called hold-out or validation, is to leave out some of the data, using it to estimate the risk of the estimators, in order to select the estimator with minimal risk. This method requires the statistician to arbitrarily select a subset of the data to form the "validation sample". The influence of this choice can be reduced by averaging several hold-out estimators (Aggregated hold-out, Agghoo). In this thesis, the hold-out and Agghoo are studied in various settings. First, theoretical guarantees for the hold-out (and Agghoo) are extended to two settings where the risk is unbounded: kernel methods and sparse linear regression. Secondly, a comprehensive analysis of the risk of both methods is carried out in a particular case: least-squares density estimation using Fourier series. It is proved that aggregated hold-out can perform better than the best estimator in the given collection, something that is clearly impossible for a procedure, such as hold-out or cross-validation, which selects only one estimator
6
Rusch, Thomas, Patrick Mair, and Kurt Hornik. "The STOPS framework for structure-based hyperparameter selection in multidimensional scaling." 2018. http://epub.wu.ac.at/6399/4/stops%2Ddssv18.pdf.
Full text
7
Chen, Jing-Wun, and 陳靖玟. "Exploring Effects of Optimizer Selection and Their Hyperparameter Tuning on Performance of Deep Neural Networks for Image Recognition." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/fpx347.
Full textAbstract:
碩士國立中央大學
數學系
107
In recent years, deep learning has flourished and people have begun to use deep learning to solve problems. Deep neural networks can be used for speech recognition, image recognition, object detection, face recognition, or driverless. The most basic neural network is the Multilayer Perceptron (MLP), which consists of multiple node layers, each layer is fully connected to each other, and one of the drawbacks of MLP is that it ignores the shape of the data which is important for image data. Compare to traditional neural networks, the convolutional neural network (CNN) has additional convolution and pooling layers which are used for preserving and capturing image features. The accuracy rate for prediction using neural network depends on many factors, such as the architecture of neural networks, the cost functions, and the selection of an optimizer. The goal of this work is to investigate the effects of optimizer selection and their hyperparameter tuning on the performance of deep neural networks for image recognition problems. We use three data sets including MNIST, CIFAR-10 and train route scenarios as test problems and test six optimizers (Gradient descent, Momentum, Adaptive gradient algorithm, Adadelta, Root Mean Square Propagation, and Adam). Our numerical results show that Adam is a good choice because of its efficiency and robustness.
8
Rawat, Waseem. "Optimization of convolutional neural networks for image classification using genetic algorithms and bayesian optimization." Diss., 2018. http://hdl.handle.net/10500/24977.
Full textAbstract:
Notwithstanding the recent successes of deep convolutional neural networks for classification tasks, they are sensitive to the selection of their hyperparameters, which impose an exponentially large search space on modern convolutional models. Traditional hyperparameter selection methods include manual, grid, or random search, but these require expert knowledge or are computationally burdensome. Divergently, Bayesian optimization and evolutionary inspired techniques have surfaced as viable alternatives to the hyperparameter problem. Thus, an alternative hybrid approach that combines the advantages of these techniques is proposed. Specifically, the search space is partitioned into discrete-architectural, and continuous and categorical hyperparameter subspaces, which are respectively traversed by a stochastic genetic search, followed by a genetic-Bayesian search. Simulations on a prominent image classification task reveal that the proposed method results in an overall classification accuracy improvement of 0.87% over unoptimized baselines, and a greater than 97% reduction in computational costs compared to a commonly employed brute force approach.Electrical and Mining Engineering
M. Tech. (Electrical Engineering)
Book chapters on the topic "Selection of hyperparameters"
1
Brazdil, Pavel, Jan N. van Rijn, Carlos Soares, and Joaquin Vanschoren. "Metalearning Approaches for Algorithm Selection I (Exploiting Rankings)." In Metalearning, 19–37. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-67024-5_2.
Full textAbstract:
SummaryThis chapter discusses an approach to the problem of algorithm selection, which exploits the performance metadata of algorithms (workflows) on prior tasks to generate recommendations for a given target dataset. The recommendations are in the form of rankings of candidate algorithms. The methodology involves two phases. In the first one, rankings of algorithms/workflows are elaborated on the basis of historical performance data on different datasets. These are subsequently aggregated into a single ranking (e.g. average ranking). In the second phase, the average ranking is used to schedule tests on the target dataset with the objective of identifying the best performing algorithm. This approach requires that an appropriate evaluation measure, such as accuracy, is set beforehand. In this chapter we also describe a method that builds this ranking based on a combination of accuracy and runtime, yielding good anytime performance. While this approach is rather simple, it can still provide good recommendations to the user. Although the examples in this chapter are from the classification domain, this approach can be applied to other tasks besides algorithm selection, namely hyperparameter optimization (HPO), as well as the combined algorithm selection and hyperparameter optimization (CASH) problem. As this approach works with discrete data, continuous hyperparameters need to be discretized first.
2
Efimova, Valeria, Andrey Filchenkov, and Anatoly Shalyto. "Reinforcement-Based Simultaneous Algorithm and Its Hyperparameters Selection." In Communications in Computer and Information Science, 15–27. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-35400-8_2.
Full text
3
Cheng, Jian, Jiansheng Qian, and Yi-nan Guo. "Adaptive Chaotic Cultural Algorithm for Hyperparameters Selection of Support Vector Regression." In Emerging Intelligent Computing Technology and Applications. With Aspects of Artificial Intelligence, 286–93. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009. http://dx.doi.org/10.1007/978-3-642-04020-7_31.
Full text
4
Dernoncourt, Franck, Shamim Nemati, Elias Baedorf Kassis, and Mohammad Mahdi Ghassemi. "Hyperparameter Selection." In Secondary Analysis of Electronic Health Records, 419–27. Cham: Springer International Publishing, 2016. http://dx.doi.org/10.1007/978-3-319-43742-2_29.
Full text
5
Lehrer, Steven F., Tian Xie, and Guanxi Yi. "Do the Hype of the Benefits from Using New Data Science Tools Extend to Forecasting Extremely Volatile Assets?" In Data Science for Economics and Finance, 287–330. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-66891-4_13.
Full textAbstract:
AbstractThis chapter first provides an illustration of the benefits of using machine learning for forecasting relative to traditional econometric strategies. We consider the short-term volatility of the Bitcoin market by realized volatility observations. Our analysis highlights the importance of accounting for nonlinearities to explain the gains of machine learning algorithms and examines the robustness of our findings to the selection of hyperparameters. This provides an illustration of how different machine learning estimators improve the development of forecast models by relaxing the functional form assumptions that are made explicit when writing up an econometric model. Our second contribution is to illustrate how deep learning can be used to measure market-level sentiment from a 10% random sample of Twitter users. This sentiment variable significantly improves forecast accuracy for every econometric estimator and machine algorithm considered in our forecasting application. This provides an illustration of the benefits of new tools from the natural language processing literature at creating variables that can improve the accuracy of forecasting models.
6
Montesinos López, Osval Antonio, Abelardo Montesinos López, and Jose Crossa. "Random Forest for Genomic Prediction." In Multivariate Statistical Machine Learning Methods for Genomic Prediction, 633–81. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-89010-0_15.
Full textAbstract:
AbstractWe give a detailed description of random forest and exemplify its use with data from plant breeding and genomic selection. The motivations for using random forest in genomic-enabled prediction are explained. Then we describe the process of building decision trees, which are a key component for building random forest models. We give (1) the random forest algorithm, (2) the main hyperparameters that need to be tuned, and (3) different splitting rules that are key for implementing random forest models for continuous, binary, categorical, and count response variables. In addition, many examples are provided for training random forest models with different types of response variables with plant breeding data. The random forest algorithm for multivariate outcomes is provided and its most popular splitting rules are also explained. In this case, some examples are provided for illustrating its implementation even with mixed outcomes (continuous, binary, and categorical). Final comments about the pros and cons of random forest are provided.
7
Ting, Michael. "Hyperparameter Selection Using the SURE Criterion." In Molecular Imaging in Nano MRI, 43–52. Hoboken, USA: John Wiley & Sons, Inc., 2014. http://dx.doi.org/10.1002/9781118760949.ch4.
Full text
8
Brazdil, Pavel, Jan N. van Rijn, Carlos Soares, and Joaquin Vanschoren. "Metalearning for Hyperparameter Optimization." In Metalearning, 103–22. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-67024-5_6.
Full textAbstract:
SummaryThis chapter describes various approaches for the hyperparameter optimization (HPO) and combined algorithm selection and hyperparameter optimization problems (CASH). It starts by presenting some basic hyperparameter optimization methods, including grid search, random search, racing strategies, successive halving and hyperband. Next, it discusses Bayesian optimization, a technique that learns from the observed performance of previously tried hyperparameter settings on the current task. This knowledge is used to build a meta-model (surrogate model) that can be used to predict which unseen configurations may work better on that task. This part includes the description sequential model-based optimization (SMBO). This chapter also covers metalearning techniques that extend the previously discussed optimization techniques with the ability to transfer knowledge across tasks. This includes techniques such as warm-starting the search, or transferring previously learned meta-models that were trained on prior (similar) tasks. A key question here is how to establish how similar prior tasks are to the new task. This can be done on the basis of past experiments, but can also exploit the information gained from recent experiments on the target task. This chapter presents an overview of some recent methods proposed in this area.
9
Pacula, Maciej, Jason Ansel, Saman Amarasinghe, and Una-May O’Reilly. "Hyperparameter Tuning in Bandit-Based Adaptive Operator Selection." In Applications of Evolutionary Computation, 73–82. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. http://dx.doi.org/10.1007/978-3-642-29178-4_8.
Full text
10
Lendasse, Amaury, Yongnan Ji, Nima Reyhani, and Michel Verleysen. "LS-SVM Hyperparameter Selection with a Nonparametric Noise Estimator." In Lecture Notes in Computer Science, 625–30. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. http://dx.doi.org/10.1007/11550907_99.
Full textConference papers on the topic "Selection of hyperparameters"
1
Owoyele, Opeoluwa, Pinaki Pal, and Alvaro Vidal Torreira. "An Automated Machine Learning-Genetic Algorithm (AutoML-GA) Framework With Active Learning for Design Optimization." In ASME 2020 Internal Combustion Engine Division Fall Technical Conference. American Society of Mechanical Engineers, 2020. http://dx.doi.org/10.1115/icef2020-3000.
Full textAbstract:
Abstract The use of machine learning (ML) based surrogate models is a promising technique to significantly accelerate simulation-based design optimization of IC engines, due to the high computational cost of running computational fluid dynamics (CFD) simulations. However, surrogate-based optimization for IC engine applications suffers from two main issues. First, training ML models requires hyperparameter selection, often involving trial-and-error combined with domain expertise. The second issue is that the data required to train these models is often unknown a priori. In this work, we present an automated hyperparameter selection technique coupled with an active learning approach to address these challenges. The technique presented in this study involves the use of a Bayesian approach to optimize the hyperparameters of the base learners that make up a Super Learner model to obtain better test performance. In addition to performing hyperparameter optimization (HPO), an active learning approach is employed, where the process of data generation using simulations, ML training, and surrogate optimization, is performed repeatedly to refine the solution in the vicinity of the predicted optimum. The proposed approach is applied to the optimization of a compression ignition engine with control parameters relating to fuel injection, in-cylinder flow, and thermodynamic conditions. It is demonstrated that by automatically selecting the best values of the hyperparameters, a 1.6% improvement in merit value is obtained, compared to an improvement of 1.0% with default hyperparameters. Overall, the framework introduced in this study reduces the need for technical expertise in training ML models for optimization, while also reducing the number of simulations needed for performing surrogate-based design optimization.
2
Ashrafi, Parivash, Yi Sun, Neil Davey, Rod Adams, Marc B. Brown, Maria Prapopoulou, and Gary Moss. "The importance of hyperparameters selection within small datasets." In 2015 International Joint Conference on Neural Networks (IJCNN). IEEE, 2015. http://dx.doi.org/10.1109/ijcnn.2015.7280645.
Full text
3
Egorov, Aleksej D., and Maksim S. Reznik. "Selection of Hyperparameters and Data Augmentation Method for Diverse Backbone Models Mask R-CNN." In 2021 IV International Conference on Control in Technical Systems (CTS). IEEE, 2021. http://dx.doi.org/10.1109/cts53513.2021.9562845.
Full text
4
Salvador, Rodolfo C., Elmer P. Dadios, Irister M. Javel, and Antipas T. Teologo. "PULSE: A Pulsar Searching Model with Genetic Algorithm Implementation for Best Pipeline Selection and Hyperparameters Optimization." In 2019 IEEE 11th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management ( HNICEM ). IEEE, 2019. http://dx.doi.org/10.1109/hnicem48295.2019.9072764.
Full text
5
Alghamdi, Muataz. "AI Driven Approach to Predict Sonic Response Utilizing Typical Formation Evaluation Logs." In International Petroleum Technology Conference. IPTC, 2022. http://dx.doi.org/10.2523/iptc-22132-ea.
Full textAbstract:
Abstract Artificial intelligence is used to capitalize on commonly run well logs to build and train an artificial neural network model to predict the sonic log curve. The dataset used to develop the model is made up of a total of 50 well logs for an oilfield that is located in the eastern part of US. It includes actual recorded sonic logs, which allows a comparison between model driven results and actual sonic curves to evaluate accuracy. Commonly run logs are used to train prediction models. Log selection targets meaningful petrophysical correlations to sonic properties. Well logs were preprocessed for data cleanup and scaling. After that, the data was broken down to geologically continuous chunks, and splitted to training, validation, and testing subsets. Artificial neuronal network models that use different combinations of neural network layers, hyperparameters, optimizers, and architectures to train the model are investigated to decide on the best approach to construct the predictive model. Automated hyperparameters tuning is employed to further enhance the model accuracy. After evaluation, a total of four models are selected to train. The resulted sonic logs produced by the models are assessed for accuracy, and are visually plotted and compared to actual logs. The models are ranked based on their accuracy and ability to detect geological features. The models show variations in prediction capabilities. They show that models that are built based on recurrent neural network performed the best. In addition, introducing convolutional neural layers to the models further enhanced accuracy. It is observed that the models are doing a better job of predicting when there is a clear change in values, and are less effective when actual values rate of change is low. Overall, the model that is built employing Gated Recurrent Unit combined with convolutional layers performed the best. In comparison, models that are purely built on dense neural network structure didn't yield good results. Also, it is proven that automated hyperparameter functions are effective in improving models’ accuracy by tuning of parameters through several iterations. The paper exhibits the effectiveness of utilizing evolving AI techniques to capitalize on expanding existing formation evaluation methods to extract more data from commonly run well logs. In addition, developing a model to predict sonic response negates the need to run actual sonic logs, which saves cost and operational time. Also, AI derived data from previously acquired logs in the wells of interest can help in calibration of models in applications such as geomechanics and wellbore stability studies. This approach can be potentially used for real-time predications without running sonic tools in wells.
6
Rakhshani, Hojjat, Lhassane Idoumghar, Julien Lepagnot, Mathieu Brevilliers, and Edward Keedwell. "Automatic hyperparameter selection in Autodock." In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2018. http://dx.doi.org/10.1109/bibm.2018.8621172.
Full text
7
Moore, Gregory M., Charles Bergeron, and Kristin P. Bennett. "Nonsmooth Bilevel Programming for Hyperparameter Selection." In 2009 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 2009. http://dx.doi.org/10.1109/icdmw.2009.74.
Full text
8
Zhang, Hongbao, Baoping Lu, Lulu Liao, Hongzhi Bao, Zhifa Wang, Xutian Hou, Amol Mulunjkar, and Xin Jin. "Combining Machine Learning and Classic Drilling Theories to Improve Rate of Penetration Prediction." In SPE/IADC Middle East Drilling Technology Conference and Exhibition. SPE, 2021. http://dx.doi.org/10.2118/202202-ms.
Full textAbstract:
Abstract Theoretically, rate of penetration (ROP) model is the basic to drilling parameters design, ROP improvement tools selection and drill time & cost estimation. Currently, ROP modelling is mainly conducted by two approaches: equation-based approach and machine learning approach, and machine learning performs better because of the capacity in high-dimensional and non-linear process modelling. However, in deep or deviated wells, the ROP prediction accuracy of machine learning is always unsatisfied mainly because the energy loss along the wellbore and drill string is non-negligible and it's difficult to consider the effect of wellbore geometry in machine learning models by pure data-driven methods. Therefore, it's necessary to develop robust ROP modelling method for different scenarios. In the paper, the performance of several equation-based methods and machine learning methods are evaluated by data from 82 wells, the technical features and applicable scopes of different methods are analysed. A new machine learning based ROP modelling method suitable for different well path types was proposed. Integrated data processing pipeline was designed to dealing with data noises, data missing, and discrete variables. ROP effecting factors were analysed, including mechanical parameters, hydraulic parameters, bit characteristics, rock properties, wellbore geometry, etc. Several new features were created by classic drilling theories, such as downhole weight on bit (DWOB), hydraulic impact force, formation heterogeneity index, etc. to improve the efficiency of learning from data. A random forest model was trained by cross validation and hyperparameters optimization methods. Field test results shows that the model could predict the ROP in different hole sections (vertical, deviated and horizontal) and different drilling modes (sliding and rotating drilling) and the average accuracy meets the requirement of well planning. A novel data processing and feature engineering workflow was designed according the characteristics of ROP modelling in different well path types. An integrated data-driven ROP modelling and optimization software was developed, including functions of mechanical specific energy analysis, bit wear analysis and predict, 2D & 3D ROP sensitivity analysis, offset wells benchmark, ROP prediction, drilling parameters constraints analysis, cost per meter prediction, etc. and providing quantitative evidences for drilling parameters optimization, drilling tools selection and well time estimation.
9
Smets, Koen, Brigitte Verdonk, and Elsa M. Jordaan. "Evaluation of Performance Measures for SVR Hyperparameter Selection." In 2007 International Joint Conference on Neural Networks. IEEE, 2007. http://dx.doi.org/10.1109/ijcnn.2007.4371031.
Full text
10
Kronvall, Ted, and Andreas Jakobsson. "Hyperparameter-selection for sparse regression: A probablistic approach." In 2017 51st Asilomar Conference on Signals, Systems, and Computers. IEEE, 2017. http://dx.doi.org/10.1109/acssc.2017.8335469.
Full text