Journal articles on the topic 'ENSEMBLE LEARNING MODELS'
To see the other types of publications on this topic, follow the link: ENSEMBLE LEARNING MODELS.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'ENSEMBLE LEARNING MODELS.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
GURBYCH, A. "METHOD SUPER LEARNING FOR DETERMINATION OF MOLECULAR RELATIONSHIP." Herald of Khmelnytskyi National University. Technical sciences 307, no. 2 (May 2, 2022): 14–24. http://dx.doi.org/10.31891/2307-5732-2022-307-2-14-24.
Full textAbstract:
This paper uses the Super Learning principle to predict the molecular affinity between the receptor (large biomolecule) and ligands (small organic molecules). Meta-models study the optimal combination of individual basic models in two consecutive ensembles – classification and regression. Each costume contains six models of machine learning, which are combined by stacking. Base models include the reference vector method, random forest, gradient boosting, neural graph networks, direct propagation, and transformers. The first ensemble predicts binding probability and classifies all candidate molecules to the selected receptor into active and inactive. Ligands recognized as involved by the first ensemble are fed to the second ensemble, which assumes the degree of their affinity for the receptor in the form of an inhibition factor (Ki). A feature of the method is the rejection of the use of atomic coordinates of individual molecules and their complexes – thus eliminating experimental errors in sample preparation and measurement of nuclear coordinates and the method to determine the affinity of biomolecules with unknown spatial configurations. It is shown that meta-learning increases the response (Recall) of the classification ensemble by 34.9% and the coefficient of determination (R2) of the regression ensemble by 21% compared to the average values. This paper shows that an ensemble with meta-stacking is an asymptotically optimal system for learning. The feature of Super Learning is to use k-fold cross-validation to form first-level predictions that teach second-level models — or meta-models — that combine first-level models optimally. The ability to predict the molecular affinity of six machine learning models is studied, and the efficiency improvement is due to the combination of models in the ensemble by the stacking method. Models that are combined into two consecutive ensembles are shown.
2
ACOSTA-MENDOZA, NIUSVEL, ALICIA MORALES-REYES, HUGO JAIR ESCALANTE, and ANDRÉS GAGO-ALONSO. "LEARNING TO ASSEMBLE CLASSIFIERS VIA GENETIC PROGRAMMING." International Journal of Pattern Recognition and Artificial Intelligence 28, no. 07 (October 14, 2014): 1460005. http://dx.doi.org/10.1142/s0218001414600052.
Full textAbstract:
This paper introduces a novel approach for building heterogeneous ensembles based on genetic programming (GP). Ensemble learning is a paradigm that aims at combining individual classifier's outputs to improve their performance. Commonly, classifiers outputs are combined by a weighted sum or a voting strategy. However, linear fusion functions may not effectively exploit individual models' redundancy and diversity. In this research, a GP-based approach to learn fusion functions that combine classifiers outputs is proposed. Heterogeneous ensembles are aimed in this study, these models use individual classifiers which are based on different principles (e.g. decision trees and similarity-based techniques). A detailed empirical assessment is carried out to validate the effectiveness of the proposed approach. Results show that the proposed method is successful at building very effective classification models, outperforming alternative ensemble methodologies. The proposed ensemble technique is also applied to fuse homogeneous models' outputs with results also showing its effectiveness. Therefore, an in-depth analysis from different perspectives of the proposed strategy to build ensembles is presented with a strong experimental support.
3
Siswoyo, Bambang, Zuraida Abal Abas, Ahmad Naim Che Pee, Rita Komalasari, and Nano Suryana. "Ensemble machine learning algorithm optimization of bankruptcy prediction of bank." IAES International Journal of Artificial Intelligence (IJ-AI) 11, no. 2 (June 1, 2022): 679. http://dx.doi.org/10.11591/ijai.v11.i2.pp679-686.
Full textAbstract:
The ensemble consists of a single set of individually trained models, the predictions of which are combined when classifying new cases, in building a good classification model requires the diversity of a single model. The algorithm, logistic regression, support vector machine, random forest, and neural network are single models as alternative sources of diversity information. Previous research has shown that ensembles are more accurate than single models. Single model and modified ensemble bagging model are some of the techniques we will study in this paper. We experimented with the banking industry’s financial ratios. The results of his observations are: First, an ensemble is always more accurate than a single model. Second, we observe that modified ensemble bagging models show improved classification model performance on balanced datasets, as they can adjust behavior and make them more suitable for relatively small datasets. The accuracy rate is 97% in the bagging ensemble learning model, an increase in the accuracy level of up to 16% compared to other models that use unbalanced datasets.
4
Huang, Haifeng, Lei Huang, Rongjia Song, Feng Jiao, and Tao Ai. "Bus Single-Trip Time Prediction Based on Ensemble Learning." Computational Intelligence and Neuroscience 2022 (August 11, 2022): 1–24. http://dx.doi.org/10.1155/2022/6831167.
Full textAbstract:
The prediction of bus single-trip time is essential for passenger travel decision-making and bus scheduling. Since many factors could influence bus operations, the accurate prediction of the bus single-trip time faces a great challenge. Moreover, bus single-trip time has obvious nonlinear and seasonal characteristics. Hence, in order to improve the accuracy of bus single-trip time prediction, five prediction algorithms including LSTM (Long Short-term Memory), LR (Linear Regression), KNN (K-Nearest Neighbor), XGBoost (Extreme Gradient Boosting), and GRU (Gate Recurrent Unit) are used and examined as the base models, and three ensemble models are further constructed by using various ensemble methods including Random Forest (bagging), AdaBoost (boosting), and Linear Regression (stacking). A data-driven bus single-trip time prediction framework is then proposed, which consists of three phases including traffic data analysis, feature extraction, and ensemble model prediction. Finally, the data features and the proposed ensembled models are analyzed using real-world datasets that are collected from the Beijing Transportation Operations Coordination Center (TOCC). Through comparing the predicting results, the following conclusions are drawn: (1) the accuracy of predicting by using the three ensemble models constructed is better than the corresponding prediction results by using the five sub-models; (2) the Random Forest ensemble model constructed based on the bagging method has the best prediction accuracy among the three ensemble models; and (3) in terms of the five sub-models, the prediction accuracy of LR is better than that of the other four models.
5
Ruaud, Albane, Niklas Pfister, Ruth E. Ley, and Nicholas D. Youngblut. "Interpreting tree ensemble machine learning models with endoR." PLOS Computational Biology 18, no. 12 (December 14, 2022): e1010714. http://dx.doi.org/10.1371/journal.pcbi.1010714.
Full textAbstract:
Tree ensemble machine learning models are increasingly used in microbiome science as they are compatible with the compositional, high-dimensional, and sparse structure of sequence-based microbiome data. While such models are often good at predicting phenotypes based on microbiome data, they only yield limited insights into how microbial taxa may be associated. We developed endoR, a method to interpret tree ensemble models. First, endoR simplifies the fitted model into a decision ensemble. Then, it extracts information on the importance of individual features and their pairwise interactions, displaying them as an interpretable network. Both the endoR network and importance scores provide insights into how features, and interactions between them, contribute to the predictive performance of the fitted model. Adjustable regularization and bootstrapping help reduce the complexity and ensure that only essential parts of the model are retained. We assessed endoR on both simulated and real metagenomic data. We found endoR to have comparable accuracy to other common approaches while easing and enhancing model interpretation. Using endoR, we also confirmed published results on gut microbiome differences between cirrhotic and healthy individuals. Finally, we utilized endoR to explore associations between human gut methanogens and microbiome components. Indeed, these hydrogen consumers are expected to interact with fermenting bacteria in a complex syntrophic network. Specifically, we analyzed a global metagenome dataset of 2203 individuals and confirmed the previously reported association between Methanobacteriaceae and Christensenellales. Additionally, we observed that Methanobacteriaceae are associated with a network of hydrogen-producing bacteria. Our method accurately captures how tree ensembles use features and interactions between them to predict a response. As demonstrated by our applications, the resultant visualizations and summary outputs facilitate model interpretation and enable the generation of novel hypotheses about complex systems.
6
Khanna, Samarth, and Kabir Nagpal. "Sign Language Interpretation using Ensembled Deep Learning Models." ITM Web of Conferences 53 (2023): 01003. http://dx.doi.org/10.1051/itmconf/20235301003.
Full textAbstract:
Communication is an integral part of our day-to-day lives. People experiencing difficulty in speaking or hearing often feel neglected in our society. While Automatic Speech Recognition Systems have now progressed to the purpose of being commercially viable, Signed Language Recognition Systems are still in the early stages. Currently, all such interpretations are administered by humans. Here, we present an approach using ensembled architecture for the classification of Sign Language characters. The novel ensemble of InceptionV3 and ResNet101 achieved an accuracy of 97.24% on the ASL dataset.
7
Alazba, Amal, and Hamoud Aljamaan. "Software Defect Prediction Using Stacking Generalization of Optimized Tree-Based Ensembles." Applied Sciences 12, no. 9 (April 30, 2022): 4577. http://dx.doi.org/10.3390/app12094577.
Full textAbstract:
Software defect prediction refers to the automatic identification of defective parts of software through machine learning techniques. Ensemble learning has exhibited excellent prediction outcomes in comparison with individual classifiers. However, most of the previous work utilized ensemble models in the context of software defect prediction with the default hyperparameter values, which are considered suboptimal. In this paper, we investigate the applicability of a stacking ensemble built with fine-tuned tree-based ensembles for defect prediction. We used grid search to optimize the hyperparameters of seven tree-based ensembles: random forest, extra trees, AdaBoost, gradient boosting, histogram-based gradient boosting, XGBoost and CatBoost. Then, a stacking ensemble was built utilizing the fine-tuned tree-based ensembles. The ensembles were evaluated using 21 publicly available defect datasets. Empirical results showed large impacts of hyperparameter optimization on extra trees and random forest ensembles. Moreover, our results demonstrated the superiority of the stacking ensemble over all fine-tuned tree-based ensembles.
8
Sonawane, Deepkanchan Nanasaheb. "Ensemble Learning For Increasing Accuracy Data Models." IOSR Journal of Computer Engineering 9, no. 1 (2013): 35–37. http://dx.doi.org/10.9790/0661-0913537.
Full text
9
Li, Ziyue, Kan Ren, Yifan Yang, Xinyang Jiang, Yuqing Yang, and Dongsheng Li. "Towards Inference Efficient Deep Ensemble Learning." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 7 (June 26, 2023): 8711–19. http://dx.doi.org/10.1609/aaai.v37i7.26048.
Full textAbstract:
Ensemble methods can deliver surprising performance gains but also bring significantly higher computational costs, e.g., can be up to 2048X in large-scale ensemble tasks. However, we found that the majority of computations in ensemble methods are redundant. For instance, over 77% of samples in CIFAR-100 dataset can be correctly classified with only a single ResNet-18 model, which indicates that only around 23% of the samples need an ensemble of extra models. To this end, we propose an inference efficient ensemble learning method, to simultaneously optimize for effectiveness and efficiency in ensemble learning. More specifically, we regard ensemble of models as a sequential inference process and learn the optimal halting event for inference on a specific sample. At each timestep of the inference process, a common selector judges if the current ensemble has reached ensemble effectiveness and halt further inference, otherwise filters this challenging sample for the subsequent models to conduct more powerful ensemble. Both the base models and common selector are jointly optimized to dynamically adjust ensemble inference for different samples with various hardness, through the novel optimization goals including sequential ensemble boosting and computation saving. The experiments with different backbones on real-world datasets illustrate our method can bring up to 56% inference cost reduction while maintaining comparable performance to full ensemble, achieving significantly better ensemble utility than other baselines. Code and supplemental materials are available at https://seqml.github.io/irene.
10
Abdillah, Abid Famasya, Cornelius Bagus Purnama Putra, Apriantoni Apriantoni, Safitri Juanita, and Diana Purwitasari. "Ensemble-based Methods for Multi-label Classification on Biomedical Question-Answer Data." Journal of Information Systems Engineering and Business Intelligence 8, no. 1 (April 26, 2022): 42–50. http://dx.doi.org/10.20473/jisebi.8.1.42-50.
Full textAbstract:
Background: Question-answer (QA) is a popular method to seek health-related information and biomedical data. Such questions can refer to more than one medical entity (multi-label) so determining the correct tags is not easy. The question classification (QC) mechanism in a QA system can narrow down the answers we are seeking. Objective: This study develops a multi-label classification using the heterogeneous ensembles method to improve accuracy in biomedical data with long text dimensions. Methods: We used the ensemble method with heterogeneous deep learning and machine learning for multi-label extended text classification. There are 15 various single models consisting of three deep learning (CNN, LSTM, and BERT) and four machine learning algorithms (SVM, kNN, Decision Tree, and Naïve Bayes) with various text representations (TF-IDF, Word2Vec, and FastText). We used the bagging approach with a hard voting mechanism for the decision-making. Results: The result shows that deep learning is more powerful than machine learning as a single multi-label biomedical data classification method. Moreover, we found that top-three was the best number of base learners by combining the ensembles method. Heterogeneous-based ensembles with three learners resulted in an F1-score of 82.3%, which is better than the best single model by CNN with an F1-score of 80%. Conclusion: A multi-label classification of biomedical QA using ensemble models is better than single models. The result shows that heterogeneous ensembles are more potent than homogeneous ensembles on biomedical QA data with long text dimensions. Keywords: Biomedical Question Classification, Ensemble Method, Heterogeneous Ensembles, Multi-Label Classification, Question Answering
11
Saphal, Rohan, Balaraman Ravindran, Dheevatsa Mudigere, Sasikanth Avancha, and Bharat Kaul. "ERLP: Ensembles of Reinforcement Learning Policies (Student Abstract)." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 10 (April 3, 2020): 13905–6. http://dx.doi.org/10.1609/aaai.v34i10.7225.
Full textAbstract:
Reinforcement learning algorithms are sensitive to hyper-parameters and require tuning and tweaking for specific environments for improving performance. Ensembles of reinforcement learning models on the other hand are known to be much more robust and stable. However, training multiple models independently on an environment suffers from high sample complexity. We present here a methodology to create multiple models from a single training instance that can be used in an ensemble through directed perturbation of the model parameters at regular intervals. This allows training a single model that converges to several local minima during the optimization process as a result of the perturbation. By saving the model parameters at each such instance, we obtain multiple policies during training that are ensembled during evaluation. We evaluate our approach on challenging discrete and continuous control tasks and also discuss various ensembling strategies. Our framework is substantially sample efficient, computationally inexpensive and is seen to outperform state of the art (SOTA) approaches
12
Qutub, Aseel, Asmaa Al-Mehmadi, Munirah Al-Hssan, Ruyan Aljohani, and Hanan S. Alghamdi. "Prediction of Employee Attrition Using Machine Learning and Ensemble Methods." International Journal of Machine Learning and Computing 11, no. 2 (March 2021): 110–14. http://dx.doi.org/10.18178/ijmlc.2021.11.2.1022.
Full textAbstract:
Employees are the most valuable resources for any organization. The cost associated with professional training, the developed loyalty over the years and the sensitivity of some organizational positions, all make it very essential to identify who might leave the organization. Many reasons can lead to employee attrition. In this paper, several machine learning models are developed to automatically and accurately predict employee attrition. IBM attrition dataset is used in this work to train and evaluate machine learning models; namely Decision Tree, Random Forest Regressor, Logistic Regressor, Adaboost Model, and Gradient Boosting Classifier models. The ultimate goal is to accurately detect attrition to help any company to improve different retention strategies on crucial employees and boost those employee satisfactions.
13
Quartulli, Marco, Amaia Gil, Ane Miren Florez-Tapia, Pablo Cereijo, Elixabete Ayerbe, and Igor G. Olaizola. "Ensemble Surrogate Models for Fast LIB Performance Predictions." Energies 14, no. 14 (July 8, 2021): 4115. http://dx.doi.org/10.3390/en14144115.
Full textAbstract:
Battery Cell design and control have been widely explored through modeling and simulation. On the one hand, Doyle’s pseudo-two-dimensional (P2D) model and Single Particle Models are among the most popular electrochemical models capable of predicting battery performance and therefore guiding cell characterization. On the other hand, empirical models obtained, for example, by Machine Learning (ML) methods represent a simpler and computationally more efficient complement to electrochemical models and have been widely used for Battery Management System (BMS) control purposes. This article proposes ML-based ensemble models to be used for the estimation of the performance of an LIB cell across a wide range of input material characteristics and parameters and evaluates 1. Deep Learning ensembles for simulation convergence classification and 2. structured regressors for battery energy and power predictions. The results represent an improvement on state-of-the-art LIB surrogate models and indicate that deep ensembles represent a promising direction for battery modeling and design.
14
Shen, Zhiqiang, Zhankui He, and Xiangyang Xue. "MEAL: Multi-Model Ensemble via Adversarial Learning." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4886–93. http://dx.doi.org/10.1609/aaai.v33i01.33014886.
Full textAbstract:
Often the best performing deep neural models are ensembles of multiple base-level networks. Unfortunately, the space required to store these many networks, and the time required to execute them at test-time, prohibits their use in applications where test sets are large (e.g., ImageNet). In this paper, we present a method for compressing large, complex trained ensembles into a single network, where knowledge from a variety of trained deep neural networks (DNNs) is distilled and transferred to a single DNN. In order to distill diverse knowledge from different trained (teacher) models, we propose to use adversarial-based learning strategy where we define a block-wise training loss to guide and optimize the predefined student network to recover the knowledge in teacher models, and to promote the discriminator network to distinguish teacher vs. student features simultaneously. The proposed ensemble method (MEAL) of transferring distilled knowledge with adversarial learning exhibits three important advantages: (1) the student network that learns the distilled knowledge with discriminators is optimized better than the original model; (2) fast inference is realized by a single forward pass, while the performance is even better than traditional ensembles from multi-original models; (3) the student network can learn the distilled knowledge from a teacher model that has arbitrary structures. Extensive experiments on CIFAR-10/100, SVHN and ImageNet datasets demonstrate the effectiveness of our MEAL method. On ImageNet, our ResNet-50 based MEAL achieves top-1/5 21.79%/5.99% val error, which outperforms the original model by 2.06%/1.14%.
15
Stevens, Christophe AT, Alexander RM Lyons, Kanika I. Dharmayat, Alireza Mahani, Kausik K. Ray, Antonio J. Vallejo-Vaz, and Mansour TA Sharabiani. "Ensemble machine learning methods in screening electronic health records: A scoping review." DIGITAL HEALTH 9 (January 2023): 205520762311732. http://dx.doi.org/10.1177/20552076231173225.
Full textAbstract:
Background Electronic health records provide the opportunity to identify undiagnosed individuals likely to have a given disease using machine learning techniques, and who could then benefit from more medical screening and case finding, reducing the number needed to screen with convenience and healthcare cost savings. Ensemble machine learning models combining multiple prediction estimates into one are often said to provide better predictive performances than non-ensemble models. Yet, to our knowledge, no literature review summarises the use and performances of different types of ensemble machine learning models in the context of medical pre-screening. Method We aimed to conduct a scoping review of the literature reporting the derivation of ensemble machine learning models for screening of electronic health records. We searched EMBASE and MEDLINE databases across all years applying a formal search strategy using terms related to medical screening, electronic health records and machine learning. Data were collected, analysed, and reported in accordance with the PRISMA scoping review guideline. Results A total of 3355 articles were retrieved, of which 145 articles met our inclusion criteria and were included in this study. Ensemble machine learning models were increasingly employed across several medical specialties and often outperformed non-ensemble approaches. Ensemble machine learning models with complex combination strategies and heterogeneous classifiers often outperformed other types of ensemble machine learning models but were also less used. Ensemble machine learning models methodologies, processing steps and data sources were often not clearly described. Conclusions Our work highlights the importance of deriving and comparing the performances of different types of ensemble machine learning models when screening electronic health records and underscores the need for more comprehensive reporting of machine learning methodologies employed in clinical research.
16
Chang-You Zhang, Chang-You Zhang, Jing-Jing Wang Chang-You Zhang, Li-Xia Wan Jing-Jing Wang, and Ruo-Xue Yu Li-Xia Wan. "An Emotional Analysis Method Based on Multi Model Ensemble Learning." 電腦學刊 34, no. 1 (February 2023): 001–11. http://dx.doi.org/10.53106/199115992023023401001.
Full textAbstract:
<p>Traditional machine learning models generally use weak supervision model, which is difficult to adapt to the scene of multi classification for emotional text. Therefore, a multi model ensemble learning algorithm for emotional text classification is proposed. The algorithm takes the labeled emotional text data as the training sample, uses the improved TF-IDF algorithm to train the word vector space model, selects three weakly supervised machine learning algorithms, linear SVC, xgboost and logistic regression, to construct the base classifier, and uses the random forest algorithm to construct the meta classifier. It realizes the function of dividing emotional text into three categories: positive, neutral and negative. From the simulation and test results, the AUC values of the multi model ensemble learning algorithm model for each category are 0.93, 0.94 and 1.00, and the AP values are 0.87, 0.86 and 1.00, and the indicators of accuracy and recall are better than the single machine learning model, which realizes the high performance and high accuracy for emotional text classification.</p> <p> </p>
17
Deore, Bhushan, Aditya Kyatham, and Shubham Narkhede. "A novel approach to ensemble MLP and random forest for network security." ITM Web of Conferences 32 (2020): 03003. http://dx.doi.org/10.1051/itmconf/20203203003.
Full textAbstract:
The following paper provides a novel approach for Network Intrusion Detection System using Machine Learning and Deep Learning. This approach uses two MLP (Multi-Layer Perceptron) models one having 3 layers and other having 6 layers. Random Forest is also used for classification. These models are ensembled in such a way that the final accuracy is boosted and also the testing time is reduced. Researchers have implemented various ways for the ensemble of multiple models but we are using contradiction management concept to ensemble machine learning models. Contradiction Management concept means if two machine learning models are contradicting in their decisions (in our case 3-layer MLP and Random Forest), then the third model’s (6-layer MLP) decision is considered whose accuracy is higher than the previous models. The third model is only used for testing when the previous two models contradict in their decision because the testing time of third model is higher than the two previous models as the third model has complex architecture. This approach increased the final accuracy as ensemble of multiple models is done and also testing time has reduced. The novelty of this paper is the choice and the combination of the models for the purpose of Network security.
18
Kapil, Divya. "Enhancing MNIST Digit Recognition with Ensemble Learning Techniques." Mathematical Statistician and Engineering Applications 70, no. 2 (February 26, 2021): 1362–71. http://dx.doi.org/10.17762/msea.v70i2.2328.
Full textAbstract:
Abstract
The classification task known as MNIST digit recognition involves identifying handwritten numbers into their corresponding values. Although there are numerous approaches proposed for this type of task, they typically face issues in achieving high accuracy. One method that can improve single models' performance is through ensemble learning. The goal of this study is to explore the use of various learning techniques, such as boosting and bagging, in combination with random forest models and decision trees, to improve the performance of MNIST digit recognition with regard to accuracy. We then perform evaluations on these methods using various metrics, such as recall, precision, accuracy, and F1. The findings of this study provide valuable insight into the various advantages of ensemble methods for the MNIST digit recognition task. It also highlights the need to explore these techniques in the context of machine learning. The objective of this study is to investigate the use of ensembles in improving the accuracy of MNIST digit recognition. We performed evaluations on two popular methods, namely boosting and bagging, with random forest and decision tree models. The evaluation parameters included F1 score, recall, accuracy, and precision. The results of the evaluations revealed that both boosting and bagging methods performed well in terms of their evaluation metrics. In most cases, the decision tree performed better than the random forest. However, the random forest method was able to achieve the highest accuracy, which is 99 percent. The findings of the evaluation revealed that ensembles can help improve single models' accuracy in MNIST digit recognition. On the other hand, the random forest method is a promising option for this task. The exact results of the evaluations will vary depending on the evaluation and implementation metrics. More research is needed to confirm their generalizability. The study emphasizes the value of exploring ensembles in machine learning systems, as well as the potential advantages of performing MNIST digit recognition using them.
19
Wu, Li-Ya, and Sung-Shun Weng. "Ensemble Learning Models for Food Safety Risk Prediction." Sustainability 13, no. 21 (November 7, 2021): 12291. http://dx.doi.org/10.3390/su132112291.
Full textAbstract:
Ensemble learning was adopted to design risk prediction models with the aim of improving border inspection methods for food imported into Taiwan. Specifically, we constructed a set of prediction models to enhance the hit rate of non-conforming products, thus strengthening the border control of food products to safeguard public health. Using five algorithms, we developed models to provide recommendations for the risk assessment of each imported food batch. The models were evaluated by constructing a confusion matrix to calculate predictive performance indicators, including the positive prediction value (PPV), recall, harmonic mean of PPV and recall (F1 score), and area under the curve. Our results showed that ensemble learning achieved better and more stable prediction results than any single algorithm. When the results of comparable data periods were examined, the non-conformity hit rate was found to increase significantly after online implementation of the ensemble learning models, indicating that ensemble learning was effective at risk prediction. In addition to enhancing the inspection hit rate of non-conforming food, the results of this study can serve as a reference for the improvement of existing random inspection methods, thus strengthening capabilities in food risk management.
20
Campos, David, Miao Zhang, Bin Yang, Tung Kieu, Chenjuan Guo, and Christian S. Jensen. "LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation." Proceedings of the ACM on Management of Data 1, no. 2 (June 13, 2023): 1–27. http://dx.doi.org/10.1145/3589316.
Full textAbstract:
Due to the sweeping digitalization of processes, increasingly vast amounts of time series data are being produced. Accurate classification of such time series facilitates decision making in multiple domains. State-of-the-art classification accuracy is often achieved by ensemble learning where results are synthesized from multiple base models. This characteristic implies that ensemble learning needs substantial computing resources, preventing their use in resource-limited environments, such as in edge devices. To extend the applicability of ensemble learning, we propose the LightTS framework that compresses large ensembles into lightweight models while ensuring competitive accuracy. First, we propose adaptive ensemble distillation that assigns adaptive weights to different base models such that their varying classification capabilities contribute purposefully to the training of the lightweight model. Second, we propose means of identifying Pareto optimal settings w.r.t. model accuracy and model size, thus enabling users with a space budget to select the most accurate lightweight model. We report on experiments using 128 real-world time series sets and different types of base models that justify key decisions in the design of LightTS and provide evidence that LightTS is able to outperform competitors.
21
Chopra, Anjali, and Priyanka Bhilare. "Application of Ensemble Models in Credit Scoring Models." Business Perspectives and Research 6, no. 2 (April 17, 2018): 129–41. http://dx.doi.org/10.1177/2278533718765531.
Full textAbstract:
Loan default is a serious problem in banking industries. Banking systems have strong processes in place for identification of customers with poor credit risk scores; however, most of the credit scoring models need to be constantly updated with newer variables and statistical techniques for improved accuracy. While totally eliminating default is almost impossible, loan risk teams, however, minimize the rate of default, thereby protecting banks from the adverse effects of loan default. Credit scoring models have used logistic regression and linear discriminant analysis for identification of potential defaulters. Newer and contemporary machine learning techniques have the ability to outperform classic old age techniques. This article aims to conduct empirical analysis on publically available bank loan dataset to study banking loan default using decision tree as the base learner and comparing it with ensemble tree learning techniques such as bagging, boosting, and random forests. The results of the empirical analysis suggest that the gradient boosting model outperforms the base decision tree learner, indicating that ensemble model works better than individual models. The study recommends that the risk team should adopt newer contemporary techniques to achieve better accuracy resulting in effective loan recovery strategies.
22
Kim, Yong-Woon, Yung-Cheol Byun, and Addapalli V. N. Krishna. "Portrait Segmentation Using Ensemble of Heterogeneous Deep-Learning Models." Entropy 23, no. 2 (February 5, 2021): 197. http://dx.doi.org/10.3390/e23020197.
Full textAbstract:
Image segmentation plays a central role in a broad range of applications, such as medical image analysis, autonomous vehicles, video surveillance and augmented reality. Portrait segmentation, which is a subset of semantic image segmentation, is widely used as a preprocessing step in multiple applications such as security systems, entertainment applications, video conferences, etc. A substantial amount of deep learning-based portrait segmentation approaches have been developed, since the performance and accuracy of semantic image segmentation have improved significantly due to the recent introduction of deep learning technology. However, these approaches are limited to a single portrait segmentation model. In this paper, we propose a novel approach using an ensemble method by combining multiple heterogeneous deep-learning based portrait segmentation models to improve the segmentation performance. The Two-Models ensemble and Three-Models ensemble, using a simple soft voting method and weighted soft voting method, were experimented. Intersection over Union (IoU) metric, IoU standard deviation and false prediction rate were used to evaluate the performance. Cost efficiency was calculated to analyze the efficiency of segmentation. The experiment results show that the proposed ensemble approach can perform with higher accuracy and lower errors than single deep-learning-based portrait segmentation models. The results also show that the ensemble of deep-learning models typically increases the use of memory and computing power, although it also shows that the ensemble of deep-learning models can perform more efficiently than a single model with higher accuracy using less memory and less computing power.
23
Kuo, Ming-Tse, Benny Wei-Yun Hsu, Yi Sheng Lin, Po-Chiung Fang, Hun-Ju Yu, Yu-Ting Hsiao, and Vincent S. Tseng. "Deep Learning Approach in Image Diagnosis of Pseudomonas Keratitis." Diagnostics 12, no. 12 (November 25, 2022): 2948. http://dx.doi.org/10.3390/diagnostics12122948.
Full textAbstract:
This investigation aimed to explore deep learning (DL) models’ potential for diagnosing Pseudomonas keratitis using external eye images. In the retrospective research, the images of bacterial keratitis (BK, n = 929), classified as Pseudomonas (n = 618) and non-Pseudomonas (n = 311) keratitis, were collected. Eight DL algorithms, including ResNet50, DenseNet121, ResNeXt50, SE-ResNet50, and EfficientNets B0 to B3, were adopted as backbone models to train and obtain the best ensemble 2-, 3-, 4-, and 5-DL models. Five-fold cross-validation was used to determine the ability of single and ensemble models to diagnose Pseudomonas keratitis. The EfficientNet B2 model had the highest accuracy (71.2%) of the eight single-DL models, while the best ensemble 4-DL model showed the highest accuracy (72.1%) among the ensemble models. However, no statistical difference was shown in the area under the receiver operating characteristic curve and diagnostic accuracy among these single-DL models and among the four best ensemble models. As a proof of concept, the DL approach, via external eye photos, could assist in identifying Pseudomonas keratitis from BK patients. All the best ensemble models can enhance the performance of constituent DL models in diagnosing Pseudomonas keratitis, but the enhancement effect appears to be limited.
24
Oner, Mahir, and Alp Ustundag. "Combining predictive base models using deep ensemble learning." Journal of Intelligent & Fuzzy Systems 39, no. 5 (November 19, 2020): 6657–68. http://dx.doi.org/10.3233/jifs-189126.
Full textAbstract:
Since information science and communication technologies had improved significantly, data volumes had expanded. As a result of that situation, advanced pre-processing and analysis of collected data became a crucial topic for extracting meaningful patterns hidden in the data. Therefore, traditional machine learning algorithms generally fail to gather satisfactory results when analyzing complex data. The main reason of this situation is the difficulty of capturing multiple characteristics of the high dimensional data. Within this scope, ensemble learning enables the integration of diversified single models to produce weak predictive results. The final combination is generally achieved by various voting schemes. On the other hand, if a large amount of single models are utilized, voting mechanism cannot be able to combine these results. At this point, Deep Learning (DL) provides the combination of the ensemble results in a considerable time. Apart from previous studies, we determine various predictive models in order to forecast the outcome of two different case studies. Consequently, data cleaning and feature selection are conducted in advance and three predictive models are defined to be combined. DL based integration is applied substituted for voting mechanism. The weak predictive results are fused based on Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM) using different parameters and datasets and best predictors are extracted. After that, different experimental combinations are evaluated for gathering better prediction results. For comparison, grouped individual results (clusters) with proper parameters are compared with DL based ensemble results.
25
Nandhini, A. Sunitha, J. Balakrishna, R. Bala Manikandan, and S. Bharath Kumar. "Advanced flood severity detection using ensemble learning models." Journal of Physics: Conference Series 1916, no. 1 (May 1, 2021): 012048. http://dx.doi.org/10.1088/1742-6596/1916/1/012048.
Full text
26
Hu, Pingfan, Zeren Jiao, Zhuoran Zhang, and Qingsheng Wang. "Development of Solubility Prediction Models with Ensemble Learning." Industrial & Engineering Chemistry Research 60, no. 30 (July 21, 2021): 11627–35. http://dx.doi.org/10.1021/acs.iecr.1c02142.
Full text
27
Lee, Junho, Wu Wang, Fouzi Harrou, and Ying Sun. "Wind Power Prediction Using Ensemble Learning-Based Models." IEEE Access 8 (2020): 61517–27. http://dx.doi.org/10.1109/access.2020.2983234.
Full text
28
Asadi, Nazanin, Abdolreza Mirzaei, and Ehsan Haghshenas. "Multiple Observations HMM Learning by Aggregating Ensemble Models." IEEE Transactions on Signal Processing 61, no. 22 (November 2013): 5767–76. http://dx.doi.org/10.1109/tsp.2013.2280179.
Full text
29
Livieris, Ioannis E., Emmanuel Pintelas, Stavros Stavroyiannis, and Panagiotis Pintelas. "Ensemble Deep Learning Models for Forecasting Cryptocurrency Time-Series." Algorithms 13, no. 5 (May 10, 2020): 121. http://dx.doi.org/10.3390/a13050121.
Full textAbstract:
Nowadays, cryptocurrency has infiltrated almost all financial transactions; thus, it is generally recognized as an alternative method for paying and exchanging currency. Cryptocurrency trade constitutes a constantly increasing financial market and a promising type of profitable investment; however, it is characterized by high volatility and strong fluctuations of prices over time. Therefore, the development of an intelligent forecasting model is considered essential for portfolio optimization and decision making. The main contribution of this research is the combination of three of the most widely employed ensemble learning strategies: ensemble-averaging, bagging and stacking with advanced deep learning models for forecasting major cryptocurrency hourly prices. The proposed ensemble models were evaluated utilizing state-of-the-art deep learning models as component learners, which were comprised by combinations of long short-term memory (LSTM), Bi-directional LSTM and convolutional layers. The ensemble models were evaluated on prediction of the cryptocurrency price on the following hour (regression) and also on the prediction if the price on the following hour will increase or decrease with respect to the current price (classification). Additionally, the reliability of each forecasting model and the efficiency of its predictions is evaluated by examining for autocorrelation of the errors. Our detailed experimental analysis indicates that ensemble learning and deep learning can be efficiently beneficial to each other, for developing strong, stable, and reliable forecasting models.
30
Karim, Zainoolabadien, and Terence L. van Zyl. "Deep/Transfer Learning with Feature Space Ensemble Networks (FeatSpaceEnsNets) and Average Ensemble Networks (AvgEnsNets) for Change Detection Using DInSAR Sentinel-1 and Optical Sentinel-2 Satellite Data Fusion." Remote Sensing 13, no. 21 (October 31, 2021): 4394. http://dx.doi.org/10.3390/rs13214394.
Full textAbstract:
Differential interferometric synthetic aperture radar (DInSAR), coherence, phase, and displacement are derived from processing SAR images to monitor geological phenomena and urban change. Previously, Sentinel-1 SAR data combined with Sentinel-2 optical imagery has improved classification accuracy in various domains. However, the fusing of Sentinel-1 DInSAR processed imagery with Sentinel-2 optical imagery has not been thoroughly investigated. Thus, we explored this fusion in urban change detection by creating a verified balanced binary classification dataset comprising 1440 blobs. Machine learning models using feature descriptors and non-deep learning classifiers, including a two-layer convolutional neural network (ConvNet2), were used as baselines. Transfer learning by feature extraction (TLFE) using various pre-trained models, deep learning from random initialization, and transfer learning by fine-tuning (TLFT) were all evaluated. We introduce a feature space ensemble family (FeatSpaceEnsNet), an average ensemble family (AvgEnsNet), and a hybrid ensemble family (HybridEnsNet) of TLFE neural networks. The FeatSpaceEnsNets combine TLFE features directly in the feature space using logistic regression. AvgEnsNets combine TLFEs at the decision level by aggregation. HybridEnsNets are a combination of FeatSpaceEnsNets and AvgEnsNets. Several FeatSpaceEnsNets, AvgEnsNets, and HybridEnsNets, comprising a heterogeneous mixture of different depth and architecture models, are defined and evaluated. We show that, in general, TLFE outperforms both TLFT and classic deep learning for the small dataset used and that larger ensembles of TLFE models do not always improve accuracy. The best performing ensemble is an AvgEnsNet (84.862%) comprised of a ResNet50, ResNeXt50, and EfficientNet B4. This was matched by a similarly composed FeatSpaceEnsNet with an F1 score of 0.001 and variance of 0.266 less. The best performing HybridEnsNet had an accuracy of 84.775%. All of the ensembles evaluated outperform the best performing single model, ResNet50 with TLFE (83.751%), except for AvgEnsNet 3, AvgEnsNet 6, and FeatSpaceEnsNet 5. Five of the seven similarly composed FeatSpaceEnsNets outperform the corresponding AvgEnsNet.
31
Wang, Yiren, Lijun Wu, Yingce Xia, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. "Transductive Ensemble Learning for Neural Machine Translation." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 6291–98. http://dx.doi.org/10.1609/aaai.v34i04.6097.
Full textAbstract:
Ensemble learning, which aggregates multiple diverse models for inference, is a common practice to improve the accuracy of machine learning tasks. However, it has been observed that the conventional ensemble methods only bring marginal improvement for neural machine translation (NMT) when individual models are strong or there are a large number of individual models. In this paper, we study how to effectively aggregate multiple NMT models under the transductive setting where the source sentences of the test set are known. We propose a simple yet effective approach named transductive ensemble learning (TEL), in which we use all individual models to translate the source test set into the target language space and then finetune a strong model on the translated synthetic corpus. We conduct extensive experiments on different settings (with/without monolingual data) and different language pairs (English↔{German, Finnish}). The results show that our approach boosts strong individual models with significant improvement and benefits a lot from more individual models. Specifically, we achieve the state-of-the-art performances on the WMT2016-2018 English↔German translations.
32
Gagne, David John, Amy McGovern, and Ming Xue. "Machine Learning Enhancement of Storm-Scale Ensemble Probabilistic Quantitative Precipitation Forecasts." Weather and Forecasting 29, no. 4 (July 22, 2014): 1024–43. http://dx.doi.org/10.1175/waf-d-13-00108.1.
Full textAbstract:
Abstract Probabilistic quantitative precipitation forecasts challenge meteorologists due to the wide variability of precipitation amounts over small areas and their dependence on conditions at multiple spatial and temporal scales. Ensembles of convection-allowing numerical weather prediction models offer a way to produce improved precipitation forecasts and estimates of the forecast uncertainty. These models allow for the prediction of individual convective storms on the model grid, but they often displace the storms in space, time, and intensity, which results in added uncertainty. Machine learning methods can produce calibrated probabilistic forecasts from the raw ensemble data that correct for systemic biases in the ensemble precipitation forecast and incorporate additional uncertainty information from aggregations of the ensemble members and additional model variables. This study utilizes the 2010 Center for Analysis and Prediction of Storms Storm-Scale Ensemble Forecast system and the National Severe Storms Laboratory National Mosaic & Multi-Sensor Quantitative Precipitation Estimate as input data for training logistic regressions and random forests to produce a calibrated probabilistic quantitative precipitation forecast. The reliability and discrimination of the forecasts are compared through verification statistics and a case study.
33
Joshi, Gaurav. "Implementation of Isotension Ensemble in Deep Learning." Turkish Journal of Computer and Mathematics Education (TURCOMAT) 9, no. 2 (December 30, 2018): 576–86. http://dx.doi.org/10.17762/turcomat.v9i2.13861.
Full textAbstract:
The implementation of the isotension ensemble in deep learning is a novel approach that aims to enhance the performance and robustness of deep learning models. This abstract provides a detailed overview of the implementation and its key components, highlighting its significance and potential impact on the field of deep learning. Deep learning has achieved remarkable success in various domains, including computer vision, natural language processing, and pattern recognition. However, deep neural networks are known to suffer from overfitting and lack of generalization when trained on limited datasets or when faced with complex and diverse data distributions. These limitations hinder their performance and reliability in real-world applications.
The isotension ensemble approach addresses these challenges by integrating the concept of isotension into the training process of deep learning models. Isotension refers to a state in which the tensions between different parts of a model are balanced, promoting overall stability and robustness. By incorporating isotension, the ensemble aims to improve generalization capabilities, reduce overfitting, and enhance the model's ability to handle diverse data distributions. The implementation of the isotension ensemble involves several key components. The ensemble is constructed by training multiple deep neural networks with different initializations or hyperparameter configurations. Each network is designed to capture different aspects of the data and learn diverse representations. Sean isotension constraint is introduced during the training process to balance the tensions between the networks, ensuring that they collectively converge to a stable and robust solution. This constraint can be achieved through various techniques such as isotonic regression or loss function regularization.
The implementation of the isotension ensemble in deep learning has shown promising results in various applications. Experimental evaluations demonstrate improved generalization capabilities, enhanced model performance, and increased robustness compared to conventional deep learning approaches. The isotension ensemble has been successfully applied in tasks such as image classification, object detection, and natural language processing, achieving state-of-the-art results and demonstrating its potential impact in real-world scenarios.
The significance of the isotension ensemble lies in its ability to address the limitations of deep learning models, providing a framework for enhanced performance and reliability. By integrating the concept of isotension into the training process, the ensemble promotes stability, robustness, and improved generalization capabilities. This approach opens up new possibilities for tackling complex and diverse datasets, advancing the field of deep learning, and enabling the deployment of more reliable and efficient models in practical applications.
The implementation of the isotension ensemble in deep learning offers a promising approach to overcome the limitations of conventional deep learning models. By leveraging the concept of isotension, the ensemble enhances generalization capabilities, reduces overfitting, and improves model performance and robustness. The successful application of the isotension ensemble in various tasks demonstrates its potential impact and paves the way for future research and development in the field of deep learning.
34
Strobach, Ehud, and Golan Bel. "Decadal Climate Predictions Using Sequential Learning Algorithms." Journal of Climate 29, no. 10 (May 6, 2016): 3787–809. http://dx.doi.org/10.1175/jcli-d-15-0648.1.
Full textAbstract:
Abstract Ensembles of climate models are commonly used to improve decadal climate predictions and assess the uncertainties associated with them. Weighting the models according to their performances holds the promise of further improving their predictions. Here, an ensemble of decadal climate predictions is used to demonstrate the ability of sequential learning algorithms (SLAs) to reduce the forecast errors and reduce the uncertainties. Three different SLAs are considered, and their performances are compared with those of an equally weighted ensemble, a linear regression, and the climatology. Predictions of four different variables—the surface temperature, the zonal and meridional wind, and pressure—are considered. The spatial distributions of the performances are presented, and the statistical significance of the improvements achieved by the SLAs is tested. The reliability of the SLAs is also tested, and the advantages and limitations of the different measures of the performance are discussed. It was found that the best performances of the SLAs are achieved when the learning period is comparable to the prediction period. The spatial distribution of the SLAs performance showed that they are skillful and better than the other forecasting methods over large continuous regions. This finding suggests that, despite the fact that each of the ensemble models is not skillful, they were able to capture some physical processes that resulted in deviations from the climatology and that the SLAs enabled the extraction of this additional information.
35
Nai-Arun, Nongyao, and Punnee Sittidech. "Ensemble Learning Model for Diabetes Classification." Advanced Materials Research 931-932 (May 2014): 1427–31. http://dx.doi.org/10.4028/www.scientific.net/amr.931-932.1427.
Full textAbstract:
This paper proposed data mining techniques to improve efficiency and reliability in diabetes classification. The real data set collected from Sawanpracharak Regional Hospital, Thailand, was fist analyzed by using gain-ratio feature selection techniques. Three well known algorithms; naïve bayes, k-nearest neighbors and decision tree, were used to construct classification models on the selected features. Then, the popular ensemble learning; bagging and boosting were applied using the three base classifiers. The results revealed that the best model with the highest accuracy was bagging with base classifier decision tree algorithm (95.312%). The experiments also showed that ensemble classifier models performed better than the base classifiers alone.
36
Nakata, Norio, and Tsuyoshi Siina. "Ensemble Learning of Multiple Models Using Deep Learning for Multiclass Classification of Ultrasound Images of Hepatic Masses." Bioengineering 10, no. 1 (January 5, 2023): 69. http://dx.doi.org/10.3390/bioengineering10010069.
Full textAbstract:
Ultrasound (US) is often used to diagnose liver masses. Ensemble learning has recently been commonly used for image classification, but its detailed methods are not fully optimized. The purpose of this study is to investigate the usefulness and comparison of some ensemble learning and ensemble pruning techniques using multiple convolutional neural network (CNN) trained models for image classification of liver masses in US images. Dataset of the US images were classified into four categories: benign liver tumor (BLT) 6320 images, liver cyst (LCY) 2320 images, metastatic liver cancer (MLC) 9720 images, primary liver cancer (PLC) 7840 images. In this study, 250 test images were randomly selected for each class, for a total of 1000 images, and the remaining images were used as the training. 16 different CNNs were used for training and testing ultrasound images. The ensemble learning used soft voting (SV), weighted average voting (WAV), weighted hard voting (WHV) and stacking (ST). All four types of ensemble learning (SV, ST, WAV, and WHV) showed higher values of accuracy than the single CNN. All four types also showed significantly higher deep learning (DL) performance than ResNeXt101 alone. For image classification of liver masses using US images, ensemble learning improved the performance of DL over a single CNN.
37
A, Prof Ajil, Tanvi Jain, T. M. Namratha, Vismaya S, and Thummaluru Ganga Lakshmi. "Detection of PCOS using Ensemble Models." International Journal for Research in Applied Science and Engineering Technology 11, no. 5 (May 31, 2023): 420–25. http://dx.doi.org/10.22214/ijraset.2023.51426.
Full textAbstract:
Abstract: Polycystic ovary syndrome (PCOS) is a complex endocrine disorder that affects women of childbearing age. It is characterized by a range of symptoms, including irregular monthly cycles, hirsutism, and childlessness. Early diagnosis and detection of PCOS is vital for successful management of this condition. In the last few years, machine learning algorithms have shown great results in the diagnosis of various medical conditions. The proposed model is an ensemble model consisting of XG Boost and Random Forest to detect PCOS in women by analysing a dataset of 541 women, including 177 patients with PCOS. We analyse a dataset of clinical and demographic variables from women with and without PCOS and use various machine learning algorithms such as Ada boost, Random forest ,XG boost, Decision tree and a hybrid model to predict the presence of the condition. We evaluate the accuracy of our models by comparing the performance of the above listed algorithms. The proposed method consists of a hybrid model which is a combination of two algorithms that is Random forest and XGBoost and is yielding one of the highest accuracies of 97.2%.This early detection could potentially improve the level of care for women with this condition.
38
Kurilová, Veronika, Szabolcs Rajcsányi, Zuzana Rábeková, Jarmila Pavlovičová, Miloš Oravec, and Nora Majtánová. "Detecting glaucoma from fundus images using ensemble learning." Journal of Electrical Engineering 74, no. 4 (August 1, 2023): 328–35. http://dx.doi.org/10.2478/jee-2023-0040.
Full textAbstract:
Abstract Glaucomatous changes of the optic nerve head could be detected from fundus images. Focusing on optic nerve head appearance, and its difference from healthy images, altogether with the availability of plenty of such images in public fundus image databases, these images are ideal sources for artificial intelligence methods applications. In this work, we used ensemble learning methods and compared them with various single CNN models (VGG-16, ResNet-50, and MobileNet). The models were trained on images from REFUGE public dataset. The average voting ensemble method outperformed all mentioned models with 0.98 accuracy. In the AUC metric, the average voting ensemble method outperformed VGG-16 and MobileNet models, which had significantly weaker performance when used alone. The best results were observed using the ResNet-50 model. These results confirmed the significant potential of ensemble learning in enhancing the overall predictive performance in glaucomatous changes detection, but the overall performance could be negatively affected when models with weaker prediction performance are included.
39
Jaruskova, K., and S. Vallecorsa. "Ensemble Models for Calorimeter Simulations." Journal of Physics: Conference Series 2438, no. 1 (February 1, 2023): 012080. http://dx.doi.org/10.1088/1742-6596/2438/1/012080.
Full textAbstract:
Abstract Foreseen increasing demand for simulations of particle transport through detectors in High Energy Physics motivated the search for faster alternatives to Monte Carlo-based simulations. Deep learning approaches provide promising results in terms of speed up and accuracy, among which generative adversarial networks (GANs) appear to be particularly successful in reproducing realistic detector data. However, the GANs tend to suffer from different issues such as not reproducing the full variability of the training data, missing modes problem, and unstable convergence. Various ensemble techniques applied to image generation proved that these issues can be moderated either by deploying multiple generators or multiple discriminators. This work follows a development of a GAN with two-dimensional convolutions that reproduces 3D images of an electromagnetic calorimeter. We build on top of this model and construct an ensemble of generators. With each new generator, the ensemble shows better agreement with the Monte Carlo images in terms of shower shapes and the sampling fraction.
40
Hozhyi, O. P., O. O. Zhebko, I. O. Kalinina, and T. A. Hannichenko. "Іntelligent classification system based on ensemble methods." System technologies 3, no. 146 (May 11, 2023): 61–75. http://dx.doi.org/10.34185/1562-9945-3-146-2023-07.
Full textAbstract:
In the paper, based on machine learning methods, the solution of the classification task was investigated using a two-level structure of ensembles of models. To improve forecasting results, an ensemble approach was used: several basic models were trained to solve the same problem, with subsequent aggregation and improvement of the ob-tained results. The problem of classification was studied. The architecture of the intelli-gent classification system is proposed. The system consists of the following components: a subsystem of preprocessing and data analysis, a subsystem of data distribution, a subsystem of building basic models, a subsystem of building and evaluating ensembles of models. A two-level ensemble structure was used to find a compromise between bias and variance inherent in machine learning models. At the first level, an ensemble based on stacking is implemented using a logistic regression model as a metamodel. The pre-dictions that are generated by the underlying models are used as input for training in the first layer. The following basic models of the first layer were chosen: decision trees (DecisionTree), naive Bayesian classifier (NB), quadratic discriminant analysis (QDA), logistic regression (LR), support vector method (SVM), random forest model (RF). The bagging method based on the Bagged CART algorithm was used in the second layer. The algorithm creates N regression trees using M initial training sets and averages the re-sulting predictions. As the basic models of the second layer, the following were chosen: the first-level model (Stacking LR), the model of artificial neural networks (ANN); the linear discriminant analysis (LDA) model and the nearest neighbor (KNN) model. A study of basic classification models and ensemble models based on stacking and bag-ging, as well as metrics for evaluating the effectiveness of the use of basic classifiers and models of the first and second level, was conducted. The following parameters were de-termined for all the methods in the work: prediction accuracy and error rate, Kappa statistic, sensitivity and specificity, accuracy and completeness, F-measure and area under the ROC curve. The advantages and effectiveness of the ensemble of models in comparison with each basic model are determined.
41
Farias, G., E. Fabregas, I. Martínez, J. Vega, S. Dormido-Canto, and H. Vargas. "Nuclear Fusion Pattern Recognition by Ensemble Learning." Complexity 2021 (June 29, 2021): 1–9. http://dx.doi.org/10.1155/2021/1207167.
Full textAbstract:
Nuclear fusion is the process by which two or more atomic nuclei join together to form a single heavier nucleus. This is usually accompanied by the release of large quantities of energy. This energy could be cheaper, cleaner, and safer than other technology currently in use. Experiments in nuclear fusion generate a large number of signals that are stored in huge databases. It is impossible to do a complete analysis of this data manually, and it is essential to automate this process. That is why machine learning models have been used to this end in previous years. In the literature, several popular algorithms can be found to carry out the automatic classification of signals. Among these, ensemble methods provide a good balance between success rate and internal information about models. Specifically, AdaBoost algorithm will allow obtaining an explicit set of rules that explains the class for each input data, adding interpretability to the models. In this paper, an innovative approach to perform an online classification, that is, to identify the discharge before it actually ends, using interpretable models is presented. In order to evaluate and reveal the benefits of rule-based models, an illustrative example has been implemented to perform an online classification of five different signals of the TJ-II stellarator fusion device located in Madrid, Spain.
42
Whitaker, Tim, and Darrell Whitley. "Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (June 28, 2022): 8638–46. http://dx.doi.org/10.1609/aaai.v36i8.20842.
Full textAbstract:
Ensemble Learning is an effective method for improving generalization in machine learning. However, as state-of-the-art neural networks grow larger, the computational cost associated with training several independent networks becomes expensive. We introduce a fast, low-cost method for creating diverse ensembles of neural networks without needing to train multiple models from scratch. We do this by first training a single parent network. We then create child networks by cloning the parent and dramatically pruning the parameters of each child to create an ensemble of members with unique and diverse topologies. We then briefly train each child network for a small number of epochs, which now converge significantly faster when compared to training from scratch. We explore various ways to maximize diversity in the child networks, including the use of anti-random pruning and one-cycle tuning. This diversity enables "Prune and Tune" ensembles to achieve results that are competitive with traditional ensembles at a fraction of the training cost. We benchmark our approach against state of the art low-cost ensemble methods and display marked improvement in both accuracy and uncertainty estimation on CIFAR-10 and CIFAR-100.
43
Saleh, Hager, Sherif Mostafa, Lubna Abdelkareim Gabralla, Ahmad O. Aseeri, and Shaker El-Sappagh. "Enhanced Arabic Sentiment Analysis Using a Novel Stacking Ensemble of Hybrid and Deep Learning Models." Applied Sciences 12, no. 18 (September 7, 2022): 8967. http://dx.doi.org/10.3390/app12188967.
Full textAbstract:
Sentiment analysis (SA) is a machine learning application that drives people’s opinions from text using natural language processing (NLP) techniques. Implementing Arabic SA is challenging for many reasons, including equivocation, numerous dialects, lack of resources, morphological diversity, lack of contextual information, and hiding of sentiment terms in the implicit text. Deep learning models such as convolutional neural networks (CNN) and long short-term memory (LSTM) have significantly improved in the Arabic SA domain. Hybrid models based on CNN combined with long short-term memory (LSTM) or gated recurrent unit (GRU) have further improved the performance of single DL models. In addition, the ensemble of deep learning models, especially stacking ensembles, is expected to increase the robustness and accuracy of the previous DL models. In this paper, we proposed a stacking ensemble model that combined the prediction power of CNN and hybrid deep learning models to predict Arabic sentiment accurately. The stacking ensemble algorithm has two main phases. Three DL models were optimized in the first phase, including deep CNN, hybrid CNN-LSTM, and hybrid CNN-GRU. In the second phase, these three separate pre-trained models’ outputs were integrated with a support vector machine (SVM) meta-learner. To extract features for DL models, the continuous bag of words (CBOW) and the skip-gram models with 300 dimensions of the word embedding were used. Arabic health services datasets (Main-AHS and Sub-AHS) and the Arabic sentiment tweets dataset were used to train and test the models (ASTD). A number of well-known deep learning models, including DeepCNN, hybrid CNN-LSTM, hybrid CNN-GRU, and conventional ML algorithms, have been used to compare the performance of the proposed ensemble model. We discovered that the proposed deep stacking model achieved the best performance compared to the previous models. Based on the CBOW word embedding, the proposed model achieved the highest accuracy of 92.12%, 95.81%, and 81.4% for Main-AHS, Sub-AHS, and ASTD datasets, respectively.
44
Bilotserkovskyy, V. V., S. G. Udovenko, and L. E. Chala. "Method of neural network recognition of falsified images." Bionics of Intelligence 2, no. 95 (December 2, 2020): 32–42. http://dx.doi.org/10.30837/bi.2020.2(95).05.
Full textAbstract:
Methods for generating images falsified using Deepfake technologies and methods for detecting them are considered. A method for detecting falsified images is proposed, based on the joint use of an ensemble of convolutional neural models, the Attention mechanism and a Siamese network learning strategy. The ensembles of models were formed in different ways (using two, three or more components). The result was calculated as the average value of the AUC and LogLoss indices from all the models included in the ensemble. This approach improves the accuracy of convolutional neural network classifiers for detecting static and dynamic images created using Deepfake technologies.
45
Thapa, Niraj, Zhipeng Liu, Addison Shaver, Albert Esterline, Balakrishna Gokaraju, and Kaushik Roy. "Secure Cyber Defense: An Analysis of Network Intrusion-Based Dataset CCD-IDSv1 with Machine Learning and Deep Learning Models." Electronics 10, no. 15 (July 21, 2021): 1747. http://dx.doi.org/10.3390/electronics10151747.
Full textAbstract:
Anomaly detection and multi-attack classification are major concerns for cyber defense. Several publicly available datasets have been used extensively for the evaluation of Intrusion Detection Systems (IDSs). However, most of the publicly available datasets may not contain attack scenarios based on evolving threats. The development of a robust network intrusion dataset is vital for network threat analysis and mitigation. Proactive IDSs are required to tackle ever-growing threats in cyberspace. Machine learning (ML) and deep learning (DL) models have been deployed recently to detect the various types of cyber-attacks. However, current IDSs struggle to attain both a high detection rate and a low false alarm rate. To address these issues, we first develop a Center for Cyber Defense (CCD)-IDSv1 labeled flow-based dataset in an OpenStack environment. Five different attacks with normal usage imitating real-life usage are implemented. The number of network features is increased to overcome the shortcomings of the previous network flow-based datasets such as CIDDS and CIC-IDS2017. Secondly, this paper presents a comparative analysis on the effectiveness of different ML and DL models on our CCD-IDSv1 dataset. In this study, we consider both cyber anomaly detection and multi-attack classification. To improve the performance, we developed two DL-based ensemble models: Ensemble-CNN-10 and Ensemble-CNN-LSTM. Ensemble-CNN-10 combines 10 CNN models developed from 10-fold cross-validation, whereas Ensemble-CNN-LSTM combines base CNN and LSTM models. This paper also presents feature importance for both anomaly detection and multi-attack classification. Overall, the proposed ensemble models performed well in both the 10-fold cross-validation and independent testing on our dataset. Together, these results suggest the robustness and effectiveness of the proposed IDSs based on ML and DL models on the CCD-IDSv1 intrusion detection dataset.
46
Nithin, V. Joe, and Prof S. Pallam Setty. "Prediction of Diabetes Using Ensemble Learning." International Journal for Research in Applied Science and Engineering Technology 10, no. 10 (October 31, 2022): 932–35. http://dx.doi.org/10.22214/ijraset.2022.47114.
Full textAbstract:
Abstract: Diabetes mellitus is a chronic condition that influences everyday life of the individual having this disease. Diabetes can only be treated to maintain controlled blood glucose levels than to achieve a permanent cure to lead a normal life. As the proverb goes, “prevention is better than cure”, this model aims at “predicting the probability”, of getting this condition, which help early prognosis enough to either avoid it or delay it. Ensemble method is used for prediction of probability of getting diabetes. Classification models in machine learning are used for decision making and enlisted in sequence of accuracy. Hyperparameters are tuned for top five accurate models. Comparison of different classifiers are carried out and then subjected to voting to choose the best possible method of prediction. Voting is carried out in hard voting and soft voting procedures. The results obtained are better compared to general classifiers individually
47
Iskanderani, Ahmed I., Ibrahim M. Mehedi, Abdulah Jeza Aljohani, Mohammad Shorfuzzaman, Farzana Akther, Thangam Palaniswamy, Shaikh Abdul Latif, Abdul Latif, and Aftab Alam. "Artificial Intelligence and Medical Internet of Things Framework for Diagnosis of Coronavirus Suspected Cases." Journal of Healthcare Engineering 2021 (May 28, 2021): 1–7. http://dx.doi.org/10.1155/2021/3277988.
Full textAbstract:
The world has been facing the COVID-19 pandemic since December 2019. Timely and efficient diagnosis of COVID-19 suspected patients plays a significant role in medical treatment. The deep transfer learning-based automated COVID-19 diagnosis on chest X-ray is required to counter the COVID-19 outbreak. This work proposes a real-time Internet of Things (IoT) framework for early diagnosis of suspected COVID-19 patients by using ensemble deep transfer learning. The proposed framework offers real-time communication and diagnosis of COVID-19 suspected cases. The proposed IoT framework ensembles four deep learning models such as InceptionResNetV2, ResNet152V2, VGG16, and DenseNet201. The medical sensors are utilized to obtain the chest X-ray modalities and diagnose the infection by using the deep ensemble model stored on the cloud server. The proposed deep ensemble model is compared with six well-known transfer learning models over the chest X-ray dataset. Comparative analysis revealed that the proposed model can help radiologists to efficiently and timely diagnose the COVID-19 suspected patients.
48
Rajaraman, Sivaramakrishnan, Feng Yang, Ghada Zamzmi, Zhiyun Xue, and Sameer K. Antani. "A Systematic Evaluation of Ensemble Learning Methods for Fine-Grained Semantic Segmentation of Tuberculosis-Consistent Lesions in Chest Radiographs." Bioengineering 9, no. 9 (August 24, 2022): 413. http://dx.doi.org/10.3390/bioengineering9090413.
Full textAbstract:
Automated segmentation of tuberculosis (TB)-consistent lesions in chest X-rays (CXRs) using deep learning (DL) methods can help reduce radiologist effort, supplement clinical decision-making, and potentially result in improved patient treatment. The majority of works in the literature discuss training automatic segmentation models using coarse bounding box annotations. However, the granularity of the bounding box annotation could result in the inclusion of a considerable fraction of false positives and negatives at the pixel level that may adversely impact overall semantic segmentation performance. This study evaluates the benefits of using fine-grained annotations of TB-consistent lesions toward training the variants of U-Net models and constructing their ensembles for semantically segmenting TB-consistent lesions in both original and bone-suppressed frontal CXRs. The segmentation performance is evaluated using several ensemble methods such as bitwise- AND, bitwise-OR, bitwise-MAX, and stacking. Extensive empirical evaluations showcased that the stacking ensemble demonstrated superior segmentation performance (Dice score: 0.5743, 95% confidence interval: (0.4055, 0.7431)) compared to the individual constituent models and other ensemble methods. To the best of our knowledge, this is the first study to apply ensemble learning to improve fine-grained TB-consistent lesion segmentation performance.
49
Ko, Hyungjin, Jaewook Lee, Junyoung Byun, Bumho Son, and Saerom Park. "Loss-Driven Adversarial Ensemble Deep Learning for On-Line Time Series Analysis." Sustainability 11, no. 12 (June 25, 2019): 3489. http://dx.doi.org/10.3390/su11123489.
Full textAbstract:
Developing a robust and sustainable system is an important problem in which deep learning models are used in real-world applications. Ensemble methods combine diverse models to improve performance and achieve robustness. The analysis of time series data requires dealing with continuously incoming instances; however, most ensemble models suffer when adapting to a change in data distribution. Therefore, we propose an on-line ensemble deep learning algorithm that aggregates deep learning models and adjusts the ensemble weight based on loss value in this study. We theoretically demonstrate that the ensemble weight converges to the limiting distribution, and, thus, minimizes the average total loss from a new regret measure based on adversarial assumption. We also present an overall framework that can be applied to analyze time series. In the experiments, we focused on the on-line phase, in which the ensemble models predict the binary class for the simulated data and the financial and non-financial real data. The proposed method outperformed other ensemble approaches. Moreover, our method was not only robust to the intentional attacks but also sustainable in data distribution changes. In the future, our algorithm can be extended to regression and multiclass classification problems.
50
Klaar, Anne Carolina Rodrigues, Stefano Frizzo Stefenon, Laio Oriel Seman, Viviana Cocco Mariani, and Leandro dos Santos Coelho. "Structure Optimization of Ensemble Learning Methods and Seasonal Decomposition Approaches to Energy Price Forecasting in Latin America: A Case Study about Mexico." Energies 16, no. 7 (March 31, 2023): 3184. http://dx.doi.org/10.3390/en16073184.
Full textAbstract:
The energy price influences the interest in investment, which leads to economic development. An estimate of the future energy price can support the planning of industrial expansions and provide information to avoid times of recession. This paper evaluates adaptive boosting (AdaBoost), bootstrap aggregation (bagging), gradient boosting, histogram-based gradient boosting, and random forest ensemble learning models for forecasting energy prices in Latin America, especially in a case study about Mexico. Seasonal decomposition of the time series is used to reduce unrepresentative variations. The Optuna using tree-structured Parzen estimator, optimizes the structure of the ensembles through a voter by combining several ensemble frameworks; thus an optimized hybrid ensemble learning method is proposed. The results show that the proposed method has a higher performance than the state-of-the-art ensemble learning methods, with a mean squared error of 3.37 × 10−9 in the testing phase.