To see the other types of publications on this topic, follow the link: AdaDepth.
Journal articles on the topic 'AdaDepth'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'AdaDepth.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Gulland, A. "Ameyo Adadevoh." BMJ 349, dec16 14 (December 16, 2014): g7558. http://dx.doi.org/10.1136/bmj.g7558.
Full text
2
Mandasari, Sartika, Desi Irfan, Wanayumini Wanayumini, and Rika Rosnelly. "COMPARISON OF SGD, ADADELTA, ADAM OPTIMIZATION IN GENDER CLASSIFICATION USING CNN." JURTEKSI (Jurnal Teknologi dan Sistem Informasi) 9, no. 3 (June 7, 2023): 345–54. http://dx.doi.org/10.33330/jurteksi.v9i3.2067.
Full textAbstract:
Abstract: Gender classification is one of the most important tasks of video analysis. A machine learning-based approach was presented to identify male and female facial images with a data set of 2000 images taken from kaggles. This method plays a role in finding the weight value that gives the best output value. This study uses the most appropriate learning rate of each optimization method as a criterion for stopping training. The results showed that the Artificial Neural Network with Adam optimization produced the highest accuracy, which was 91.5% compared to the SGD and ADADELTA optimization methods. Deep Learning techniques that are applied extensively to image recognition used utilize Adam's optimizer method. Keywords: artificial neural networks; adadelta; adam; gender; sgm; Abstrak: Klasifikasi gender adalah salah satu tugas analisis video yang paling penting. Pendekatan berbasis machine learning disajikan untuk mengidentifikasi gambar wajah Pria dan Wanita dengan kumpulan data sebanyak 2000 gambar yang diambil dari kaggle. Metode ini berperan dalam menemukan nilai bobot yang memberikan nilai keluaran terbaik. Penelitian ini menggunakan learning rate yang paling sesuai dari masing-masing metode optimasi sebagai kriteria pemberhentian pelatihan. Hasil penelitian menunjukkan Jaringan Saraf Tiruan dengan optimasi Adam menghasilkan akurasi tertinggi yaitu 91,5 % dibandingkan dengan dengan metode optimasi SGD dan ADADELTA. Teknik Deep Learning yang diterapkan secara ekstensif pada pengenalan gambar yang digunakan memanfaatkan metode optimizer Adam. Kata kunci: Adadelta; Adam; Jaringan Syaraf; Gender; Tiruan; SGM;
3
Shete, Saurabh, and Avinash Marbhal. "Experiments on Gaussian Dropout and Adadelta for Hepatocellular Carcinoma." International Journal for Research in Applied Science and Engineering Technology, no. 12 (December 31, 2022): 396–403. http://dx.doi.org/10.22214/ijraset.2022.47883.
Full textAbstract:
Abstract: Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer. Hepatocellular carcinoma occurs most often in people with chronic liver diseases, such as cirrhosis caused by hepatitis B or hepatitis C infection. Early detection and accurate predictive analysis play a pivotal role in the totality of the human population and are of extreme importance for enhanced life expectancy. With the advent of computation, there are well-defined publicly available datasets that can be leveraged for an accurate and temporarily efficient understanding of HCC. There exists preliminary work on these data samples that leverage classical machine learning algorithms, however, the state of the art is heavily skewed towards the deep neural networks. To improve the existing approaches, this paper seeks to leverage Gaussian Dropout, a variant of the standard dropout, for its remedial action on overfitting and related qualities. The pipeline is also tested and experimented with Adadelta, to obtain the applicability of these additions to a standard feed-forward network. These experiments and the methodologies considered for appendage to the baseline network are thoroughly assessed and validated by using the accepted metrics on an iteratively imputed dataset on multiple train-test data distributions.
4
Сіряк, Р. В., І. С. Скарга-Бандурова, and T. O. Білобородова. "Towards an empirical hyperparameters optimization in CNN." ВІСНИК СХІДНОУКРАЇНСЬКОГО НАЦІОНАЛЬНОГО УНІВЕРСИТЕТУ імені Володимира Даля, no. 5(253) (September 5, 2019): 87–91. http://dx.doi.org/10.33216/1998-7927-2019-253-5-87-91.
Full textAbstract:
The necessity of creating a model of recognition of gestures based on convolutional neural network that effective not only in pattern recognition, but also in terms of learning speed and resource intensity, is substantiated. In this regard, the work solved the problem of optimization of hyperparameters and the selection of the best optimizer backpropagation errors. To implement the tasks, a model was created that can recognize hand gestures, both from a single image and from streaming video.When choosing an optimizer, two adaptive methods were tested - Adadelta and Adam. The experiments confirmed the high efficiency of Adadelta, however, when compared with Adam, it showed more than twice as long network training.
5
Irfan, Desi, Rika Rosnelly, Masri Wahyuni, Jaka Tirta Samudra, and Aditia Rangga. "PERBANDINGAN OPTIMASI SGD, ADADELTA, DAN ADAM DALAM KLASIFIKASI HYDRANGEA MENGGUNAKAN CNN." JOURNAL OF SCIENCE AND SOCIAL RESEARCH 5, no. 2 (June 27, 2022): 244. http://dx.doi.org/10.54314/jssr.v5i2.789.
Full textAbstract:
Abstract - invasive species are threatening indigenous species habitat in many countries around the world. Nowadays, the monitoring method relies on scientists. Scientists are skilled to see the determined areas and record the living species. Applying high skill labors requires high cost, inefficient time and limited scope as the large area cannot be reached by the man. In this research, engine based learning approach was presented to identify the image of invasive hydrangea (indigenous species from Asia) with data collection around 800 images taken form the Brazil national forest and Hydrangea appears in some images. Gradient Descent optimization method is frequently used for artificial neural network. This method roles to discover standard grade for the best output. The Gradient Descent method role play is minimizing the cost function grade by changing the parameter grade step by step. Three optimization methods have been implemented namely Stochastic Gradient Descent (SGD), ADADELTA, and Adam in the artificial neural network (Ann) for classifying aritmia data [32]. This research used the most suitable error grade limitation from each optimization method as the indicators at the end of the training. The result of this research showed that artificial nerve tissue using Adam optimization gets the highest accuration compared with SDG and ADADELTA optimization methods. Deep Learning Technique applied extensively in image introduction is Adam optimization. The training model has reached accuration to 83, 5 % and showed properness of approach conducted. Keyword: SGD, Adadelta, Adam, Optimizer FunctionAbstrak— Spesies invasif mengancam habitat spesies asli di banyak negara di dunia. Saat ini dalam metode pemantauan mereka tergantung pada pengetahuan ahli. Ilmuwan terlatih mengunjungi area yang ditentukan dan mencatat spesies yang menghuninya. Menggunakan tenaga kerja berkualifikasi tinggi seperti itu membutuhkan biaya yang mahal, tidak efisien waktu dan jangkauan yang terbatas karena manusia tidak dapat mencakup area yang luas. Dalam makalah ini, pendekatan berbasis pembelajaran mesin disajikan untuk mengidentifikasi gambar hydrangea invasif (spesies invasif asli Asia) dengan kumpulan data yang berisi sekitar 800 gambar yang diambil di hutan nasional Brasil dan di beberapa gambar terdapat Hydrangea.  Metode optimasi Gradient Descent sering digunakan untuk pelatihan Jaringan Syaraf Tiruan (JST). Metode ini berperan dalam menemukan nilai bobot yang memberikan nilai keluaran terbaik. Prinsip kerja metode Gradient Descent adalah memperkecil nilai fungsi biaya dengan mengubah nilai parameter selangkah demi selangkah. Telah diimplementasikan tiga buah metode optimasi yaitu Stochastic Gradient Descent (SGD), ADADELTA, dan Adam pada sistem Jaringan Saraf Tiruan untuk klasifikasi data aritmia [32]. Penelitian ini menggunakan batas nilai kesalahan yang paling sesuai dari masing-masing metode optimasi  sebagai kriteria pemberhentian pelatihan. Hasil penelitian menunjukkan Jaringan Saraf Tiruan dengan optimasi Adam menghasilkan akurasi tertinggi dibandingkan dengan dengan metode optimasi SGD dan ADADELTA.Teknik Deep Learning  yang diterapkan secara ekstensif pada pengenalan gambar yang digunakan memanfaatkan metode optimizer Adam  . Model yang kita latih menggunakan fungsi optimisasi Adam mencapai akurasi 83,5% pada tes yang lakukan, menunjukkan kelayakan pada  pendekatan yang dilakukan .Kata Kunci— SGD, Adadelta, Adam, Fungsi Optimasi
6
Assodiky, Hilmy, Iwan Syarif, and Tessy Badriyah. "Arrhythmia Classification Using Long Short-Term Memory with Adaptive Learning Rate." EMITTER International Journal of Engineering Technology 6, no. 1 (July 10, 2018): 75–91. http://dx.doi.org/10.24003/emitter.v6i1.265.
Full textAbstract:
Arrhythmia is a heartbeat abnormality that can be harmless or harmful. It depends on what kind of arrhythmia that the patient suffers. People with arrhythmia usually feel the same physical symptoms but every arrhythmia requires different treatments. For arrhythmia detection, the cardiologist uses electrocardiogram that represents the cardiac electrical activity. And it is a kind of sequential data with high complexity. So the high performance classification method to help the arrhythmia detection is needed. In this paper, Long Short-Term Memory (LSTM) method was used to classify the arrhythmia. The performance was boosted by using AdaDelta as the adaptive learning rate method. As a comparison, it was compared to LSTM without adaptive learning rate. And the best result that showed high accuracy was obtained by using LSTM with AdaDelta. The correct classification rate was 98% for train data and 97% for test data.
7
Shedriko, Shedriko, and Muhammad Firdaus. "Perbandingan Optimizer Adagrad, Adadelta dan Adam dalam Klasifikasi Gambar Menggunakan Deep Learning." STRING (Satuan Tulisan Riset dan Inovasi Teknologi) 8, no. 1 (August 5, 2023): 103. http://dx.doi.org/10.30998/string.v8i1.16564.
Full text
8
Azar, Ahmad Taher, Mohamed Tounsi, Suliman Mohamed Fati, Yasir Javed, Syed Umar Amin, Zafar Iqbal Khan, Shrooq Alsenan, and Jothi Ganesan. "Automated System for Colon Cancer Detection and Segmentation Based on Deep Learning Techniques." International Journal of Sociotechnology and Knowledge Development 15, no. 1 (July 24, 2023): 1–28. http://dx.doi.org/10.4018/ijskd.326629.
Full textAbstract:
Colon cancer is one of the world's three most deadly and severe cancers. As with any cancer, the key priority is early detection. Deep learning (DL) applications have recently gained popularity in medical image analysis due to the success they have achieved in the early detection and screening of cancerous tissues or organs. This paper aims to explore the potential of deep learning techniques for colon cancer classification. This research will aid in the early prediction of colon cancer in order to provide effective treatment in the most timely manner. In this exploratory study, many deep learning optimizers were investigated, including stochastic gradient descent (SGD), Adamax, AdaDelta, root mean square prop (RMSprop), adaptive moment estimation (Adam), and the Nesterov and Adam optimizer (Nadam). According to the empirical results, the CNN-Adam technique produced the highest accuracy with an average score of 82% when compared to other models for four colon cancer datasets. Similarly, Dataset_1 produced better results, with CNN-Adam, CNN-RMSprop, and CNN-Adadelta achieving accuracy scores of 0.95, 0.76, and 0.96, respectively.
9
Chen, Jiayu, Jinsheng Liu, Long Han, Mingru Ci, Dongbo Che, Lihong Guo, and Hongjun Yu. "Theory of AdaDelSPGD Algorithm in Fiber Laser-Phased Array Multiplex Communication Systems." Applied Sciences 12, no. 6 (March 16, 2022): 3009. http://dx.doi.org/10.3390/app12063009.
Full textAbstract:
Stochastic parallel gradient descent (SPGD) algorithm is one of the most promising methods for effective coherent beam combination. However, the algorithm also has some disadvantages, such as slow convergence speed and local extremum. This paper proposes an AdaDelSPGD algorithm, which combines an AdaDelta algorithm with a SPGD algorithm, and improves the traditional AdaDelta algorithm with adaptive gain coefficient. It is worth noting that the adaptive gain coefficient can be adjusted in real time to improve the convergence rate. The effectiveness of the proposed algorithm is verified by relevant simulation experiments, and the results show that the proposed algorithm can significantly improve the convergence speed. Following the experiments with the fiber laser-phased array multiplex communication system, we can draw the conclusion that the addition of communication modulation reduces the beam quality, and the higher the modulation frequency, the worse the beam quality. However, adding the SPGD algorithm can improve the beam quality. The AdaDelSPGD algorithm proposed in this paper can further improve the beam quality, and the bit error rate of communication is also decreased after testing. This provides a foundation for further research on the fiber laser-phased array multiplex communication system.
10
Dogo, Eustace M., Oluwatobi J. Afolabi, and Bhekisipho Twala. "On the Relative Impact of Optimizers on Convolutional Neural Networks with Varying Depth and Width for Image Classification." Applied Sciences 12, no. 23 (November 23, 2022): 11976. http://dx.doi.org/10.3390/app122311976.
Full textAbstract:
The continued increase in computing resources is one key factor that is allowing deep learning researchers to scale, design and train new and complex convolutional neural network (CNN) architectures in terms of varying width, depth, or both width and depth to improve performance for a variety of problems. The contributions of this study include an uncovering of how different optimization algorithms impact CNN architectural setups with variations in width, depth, and both width/depth. Specifically in this study, three different CNN architectural setups in combination with nine different optimization algorithms—namely SGD vanilla, with momentum, and with Nesterov momentum, RMSProp, ADAM, ADAGrad, ADADelta, ADAMax, and NADAM—are trained and evaluated using three publicly available benchmark image classification datasets. Through extensive experimentation, we analyze the output predictions of the different optimizers with the CNN architectures using accuracy, convergence speed, and loss function as performance metrics. Findings based on the overall results obtained across the three image classification datasets show that ADAM and NADAM achieved superior performances with wider and deeper/wider setups, respectively, while ADADelta was the worst performer, especially with the deeper CNN architectural setup.
11
AL Smadi, Ahmad, Atif Mehmood, Ahed Abugabah, Eiad Almekhlafi, and Ahmad Mohammad Al-smadi. "Deep convolutional neural network-based system for fish classification." International Journal of Electrical and Computer Engineering (IJECE) 12, no. 2 (April 1, 2022): 2026. http://dx.doi.org/10.11591/ijece.v12i2.pp2026-2039.
Full textAbstract:
<p>In computer vision, image classification is one of the potential image processing tasks. Nowadays, fish classification is a wide considered issue within the areas of machine learning and image segmentation. Moreover, it has been extended to a variety of domains, such as marketing strategies. This paper presents an effective fish classification method based on convolutional neural networks (CNNs). The experiments were conducted on the new dataset of Bangladesh’s indigenous fish species with three kinds of splitting: 80-20%, 75-25%, and 70-30%. We provide a comprehensive comparison of several popular optimizers of CNN. In total, we perform a comparative analysis of 5 different state-of-the-art gradient descent-based optimizers, namely adaptive delta (AdaDelta), stochastic gradient descent (SGD), adaptive momentum (Adam), adaptive max pooling (Adamax), Root mean square propagation (Rmsprop), for CNN. Overall, the obtained experimental results show that Rmsprop, Adam, Adamax performed well compared to the other optimization techniques used, while AdaDelta and SGD performed the worst. Furthermore, the experimental results demonstrated that Adam optimizer attained the best results in performance measures for 70-30% and 80-20% splitting experiments, while the Rmsprop optimizer attained the best results in terms of performance measures of 70-25% splitting experiments. Finally, the proposed model is then compared with state-of-the-art deep CNNs models. Therefore, the proposed model attained the best accuracy of 98.46% in enhancing the CNN ability in classification, among others.</p>
12
Fatima, Noor. "Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms." ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal 9, no. 2 (June 20, 2020): 79–90. http://dx.doi.org/10.14201/adcaij2020927990.
Full textAbstract:
Adopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimization algorithms namely: sgd, rmsprop, adagrad, adadelta, adam, adamax and nadam on four unrelated datasets discretely, to conclude which one dispenses the best accuracy, efficiency and performance to our deep neural network. This work will provide insightful analysis to a data scientist in choosing the best optimizer while modelling their deep neural network.
13
Njima, Wafa, Rafik Zayani, Iness Ahriz, Michel Terre, and Ridha Bouallegue. "Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization." Applied Sciences 9, no. 12 (June 13, 2019): 2414. http://dx.doi.org/10.3390/app9122414.
Full textAbstract:
In this paper, we propose a high accuracy fingerprint-based localization scheme for the Internet of Things (IoT). The proposed scheme employs mathematical concepts based on sparse representation and matrix completion theories. Specifically, the proposed indoor localization scheme is formulated as a simple optimization problem which enables efficient and reliable algorithm implementations. Many approaches, like Nesterov accelerated gradient (Nesterov), Adaptative Moment Estimation (Adam), Adadelta, Root Mean Square Propagation (RMSProp) and Adaptative gradient (Adagrad), have been implemented and compared in terms of localization accuracy and complexity. Simulation results demonstrate that Adam outperforms all other algorithms in terms of localization accuracy and computational complexity.
14
Qu, Zhijian, Shengao Yuan, Rui Chi, Liuchen Chang, and Liang Zhao. "Genetic Optimization Method of Pantograph and Catenary Comprehensive Monitor Status Prediction Model Based on Adadelta Deep Neural Network." IEEE Access 7 (2019): 23210–21. http://dx.doi.org/10.1109/access.2019.2899074.
Full text
15
Wadas, Maciej, and Jakub Smołka. "Performance analysis of the TensorFlow library with different optimisation algorithms." Journal of Computer Sciences Institute 21 (December 30, 2021): 330–35. http://dx.doi.org/10.35784/jcsi.2738.
Full textAbstract:
This paper presents the results of performance analysis of the Tensorflow library used in machine learning and deep neural networks. The analysis focuses on comparing the parameters obtained when training the neural network model for optimization algorithms: Adam, Nadam, AdaMax, AdaDelta, AdaGrad. Special attention has been paid to the differences between the training efficiency on tasks using microprocessor and graphics card. For the study, neural network models were created in order to recognise Polish handwritten characters. The results obtained showed that the most efficient algorithm is AdaMax, while the computer component used during the research only affects the training time of the neural network model used.
16
USMAN, KOREDIANTO, NOR KUMALASARI CAECAR PRATIWI, NUR IBRAHIM, HERI SYAHRIAN, and VITRIA PUSPITASARI RAHADI. "Evaluasi Optimizer pada Residual Network untuk Klasifikasi Klon Teh Seri GMB Berbasis Citra Daun." ELKOMIKA: Jurnal Teknik Energi Elektrik, Teknik Telekomunikasi, & Teknik Elektronika 9, no. 4 (October 10, 2021): 841. http://dx.doi.org/10.26760/elkomika.v9i4.841.
Full textAbstract:
ABSTRAKKomoditas teh berperan strategis terhadap pertumbuhan perekonomian Indonesia, salah satunya dari teh klon Gambung (GMB). Klon GMB memiliki beberapa karakter khas, dengan tingkat kemiripan morfologi yang sangat tinggi. Hal ini berdampak pada proses pengenalan klon GMB dilakukan melalui pengamatan visual oleh tenaga ahli sangat rentan terhadap kesalahan identifikasi. Sehingga, dalam penelitian ini dirancang suatu sistem identifikasi terhadap 11 klon teh seri GMB (GMB-1 hingga GMB-11) dengan menggunakan arsitektur ResNet101. Evaluasi sistem akan dilakukan dengan membandingkan tujuh algoritma optimizer yang berbeda, yaitu; Adam, SGD, RMSProp, AdaGrad, AdaMax, AdaDelta dan Nadam. Hasil pengujian menunjukkan bahwa Adam dan SGD memberikan nilai rata-rata presisi, recall dan F1-score terbaik. Selain itu, Adam memberikan nilai akurasi yang cenderung stabil sejak iterasi pertama. Metode yang diusulkan memberikan tingkat presisi, recall, F1-score sebesar 96% dan akurasi terbaik sebesar 97%.Kata kunci: klasifikasi daun teh, seri Gambung (GMB), CNN, ResNet101 ABSTRACTGambung (GMB) tea is one of the tea commodities that plays a key role in Indonesia's economic development. GMB clones have a number of distinguishing characteristics, including a high degree of morphological similarities. This has an impact on the process of identifying GMB clones through visual observation by experts who are subject to mistakes. In this study, ResNet101 architecture was used to create an identification scheme for 11 tea clones from the GMB series (GMB-1 to GMB-11). System evaluation will be carried out by comparing seven different optimizer; Adam, SGD, RMSProp, AdaGrad, AdaMax, AdaDelta, and Nadam. The test results indicate that Adam and SGD have the highest average accuracy, recall, and f1-score values. Adam also has an accuracy value that has remained consistent since the first iteration. The proposed method provides highest precision, recall, F1-score of 96% and accuracy of 97%.Keywords: tea leaves classification, GMB series, CNN, ResNet101
17
F. Alkhalid, Farah. "The effect of optimizers in fingerprint classification model utilizing deep learning." Indonesian Journal of Electrical Engineering and Computer Science 20, no. 2 (November 1, 2020): 1098. http://dx.doi.org/10.11591/ijeecs.v20.i2.pp1098-1102.
Full textAbstract:
<p>Fingerprint is the most popular way to identify persons, it is assumed a unique identity, which enable us to return the record of specific person through his fingerprint, and could be useful in many applications; such as military applications, social applications, criminal applications… etc. In this paper, the study of a new model based deep learning is suggested. The focus is directed on how to enhance the training model with the increase of the testing accuracy by applying four scenarios and comparing among them. The effects of two dedicated optimizers are shown and their contrast enhancement is tested. The results prove that the testing accuracy is 85.61% for “Adadelta” optimizer, whereas for “Adam” optimizer, it is 91.73%.</p>
18
Tinaliah, Tinaliah. "Penerapan Convolutional Neural Network Untuk Klasifikasi Citra Ekspresi Wajah Manusia Pada MMA Facial Expression Dataset." JATISI (Jurnal Teknik Informatika dan Sistem Informasi) 8, no. 4 (December 14, 2021): 2051–59. http://dx.doi.org/10.35957/jatisi.v8i4.1437.
Full textAbstract:
Ekspresi wajah manusia secara umum mewakili emosi atau perasaan yang sedang dirasakannya saat itu. Klasifikasi citra ekspresi wajah dapat membantu untuk mengetahui apakah emosi yang sedang dirasakan seseorang. CNN adalah jenis neural network yang digunakan untuk mengekstrak fitur – fitur dari sebuah citra dan sangat unggul apabila diterapkan pada data citra. Pada penelitian ini akan dilakukan klasifikasi citra ekspresi wajah dengan menerapkan Convolution Neural Network pada dataset MMA Facial Expression. Dimana data akan dibagi menjadi 2 kelas, yaitu happy dan sad. Pengujian dilakukan menggunakan data testing untuk masing – masing kelas dari model CNN yang telah dibuat menggunakan optimizer yang telah ditentukan, yaitu : Adadelta, Adagrad, Adam, Adamax, Nadam, Rmsprop, dan SGD. Berdasarkan hasil pengujian dapat disimpulkan bahwa CNN dapat melakukan klasifikasi citra ekspresi wajah manusia dengan baik menggunakan optimizer SGD dengan nilai akurasi, yaitu 63%.
19
Septiadi, Jaka, Budi Warsito, and Adi Wibowo. "Human Activity Prediction using Long Short Term Memory." E3S Web of Conferences 202 (2020): 15008. http://dx.doi.org/10.1051/e3sconf/202020215008.
Full textAbstract:
Early symptoms of dementia is one of the causes decrease in quality of life. Human activity recognition (HAR) system is proposed to recognize the daily routines which has an important role in detecting early symptoms of dementia. Long Short Term Memory (LSTM) is very useful for sequence analysis that can find the pattern of activities that carried out in daily routines. However, the LSTM model is slow to achieving convergence and take a long time during training. In this paper, we investigated the sequence of actions recorded in smart home sensors data using LSTM model, then the model will be optimized using several optimization methods. The optimization methods were Stochastic Gradient Descent (SGD), Adagrad, Adadelta, RMSProp, and Adam. The results showed that using Adam to optimized LSTM is better than other optimization methods.
20
Tian, Yingjie, Yuqi Zhang, and Haibin Zhang. "Recent Advances in Stochastic Gradient Descent in Deep Learning." Mathematics 11, no. 3 (January 29, 2023): 682. http://dx.doi.org/10.3390/math11030682.
Full textAbstract:
In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective. This study provides a detailed analysis of contemporary state-of-the-art deep learning applications, such as natural language processing (NLP), visual data processing, and voice and audio processing. Following that, this study introduces several versions of SGD and its variant, which are already in the PyTorch optimizer, including SGD, Adagrad, adadelta, RMSprop, Adam, AdamW, and so on. Finally, we propose theoretical conditions under which these methods are applicable and discover that there is still a gap between theoretical conditions under which the algorithms converge and practical applications, and how to bridge this gap is a question for the future.
21
Sana, Danish, Ul Rahman Jamshaid, and Haider Gulfam. "Performance analysis of convolutional neural networks for image classification with appropriate optimizers." i-manager’s Journal on Mathematics 12, no. 1 (2023): 1. http://dx.doi.org/10.26634/jmat.12.1.19398.
Full textAbstract:
Optimizers in Convolutional Neural Networks play an important role in many advanced deep learning models. Studies on advanced optimizers and modifications of existing optimizers continue to hold significant importance in the study of machine tools and algorithms. There are a number of studies to defend and the selection of these optimizers illustrate some of the challenges on the effectiveness of these optimizers. Comprehensive analysis on the optimizers and alteration with famous activation function Rectified Linear Unit (ReLU) offered to protect effectiveness. Significance is determined based on the adjustment with the original Softmax and ReLU. Experiments were performed with Adam, Root Mean Squared Propagation (RMSprop), Adaptive Learning Rate Method (Adadelta), Adaptive Gradient Algorithm (Adagrad) and Stochastic Gradient Descent (SGD) to examine the performance of Convolutional Neural Networks for image classification using the Canadian Institute for Advanced Research dataset (CIFAR-10).
22
Hung, Maosheng, and Meijin Hsiao. "Application of Adaptive Neural Network Algorithm Model in English Text Analysis." Computational Intelligence and Neuroscience 2022 (May 26, 2022): 1–12. http://dx.doi.org/10.1155/2022/4866531.
Full textAbstract:
Based on the existing optimization neural network algorithm, this paper introduces a simple and computationally efficient adaptive mechanism (adaptive exponential decay rate). By applying the adaptive mechanism to the Adadelta algorithm, it can be seen that AEDR-Adadelta acquires the learning rate dynamically and adaptively. At the same time, by proposing an adaptive exponential decay rate, the number and method of configuring hyperparameters can be reduced, and different learning rates can be effectively obtained for different parameters. The model is based on the encoder-decoder structure and adopts a dual-encoder structure. The transformer encoder is used to extract the context information of the sentence; the Bi-GRU encoder is used to extract the information of the source sentence; and the gated structure is used at the decoder side. The input information is integrated, and each part is matched with different attention mechanisms, which improves the model’s ability to extract and analyze relevant features in sentences. In order to accurately capture the coherence features in English texts, an improved subgraph matching algorithm is used to mine frequently occurring subgraph patterns in sentence semantic graphs, which are used to simulate the unique coherence patterns in English texts, and then analyze the overall coherence of English texts. According to the frequency of occurrence of different subgraph patterns in the sentence semantic graph, the subgraphs are filtered to generate frequent subgraph sets, and the subgraph frequency of each frequent subgraph is calculated separately. The overall coherence quality of English text is quantitatively analyzed by extracting the distribution characteristics of frequent subgraphs and the semantic values of subgraphs in the sentence semantic graph. According to the experimental results, the algorithm using the adaptive mechanism can reduce the error of the training set and the test set, improve the classification accuracy to a certain extent, and has a faster convergence speed and better text generalization ability. The semantic coherence diagnosis model of English text in this paper performs well in various tasks and has a good effect on improving the automatic correction system of English composition and providing reference for English teachers’ composition correction.
23
Misra, Ananya, Emmanuel Okewu, Sanjay Misra, and Luis Fernández-Sanz. "Deep Neural Network Model for Evaluating and Achieving the Sustainable Development Goal 16." Applied Sciences 12, no. 18 (September 15, 2022): 9256. http://dx.doi.org/10.3390/app12189256.
Full textAbstract:
The decision-making process for attaining Sustainable Development Goals (SDGs) can be enhanced through the use of predictive modelling. The application of predictive tools like deep neural networks (DNN) empowers stakeholders with quality information and promotes open data policy for curbing corruption. The anti-corruption drive is a cardinal component of SDG 16 which is aimed at strengthening state institutions and promoting social justice for the attainment of all 17 SDGs. This study examined the implementation of the SDGs in Nigeria and modelled the 2017 national corruption survey data using a DNN. We experimentally tested the efficacy of DNN optimizers using a standard image dataset from the Modified National Institute of Standards and Technology (MNIST). The outcomes validated our claims that predictive analytics could enhance decision-making through high-level accuracies as posted by the optimizers: Adam 98.2%; Adadelta 98.4%; SGD 94.9%; RMSProp 98.1%; Adagrad 98.1%.
24
Elameer, Amer S., Mustafa Musa Jaber, and Sura Khalil Abd. "Radiography image analysis using cat swarm optimized deep belief networks." Journal of Intelligent Systems 31, no. 1 (November 30, 2021): 40–54. http://dx.doi.org/10.1515/jisys-2021-0172.
Full textAbstract:
Abstract Radiography images are widely utilized in the health sector to recognize the patient health condition. The noise and irrelevant region information minimize the entire disease detection accuracy and computation complexity. Therefore, in this study, statistical Kolmogorov–Smirnov test has been integrated with wavelet transform to overcome the de-noising issues. Then the cat swarm-optimized deep belief network is applied to extract the features from the affected region. The optimized deep learning model reduces the feature training cost and time and improves the overall disease detection accuracy. The network learning process is enhanced according to the AdaDelta learning process, which replaces the learning parameter with a delta value. This process minimizes the error rate while recognizing the disease. The efficiency of the system evaluated using image retrieval in medical application dataset. This process helps to determine the various diseases such as breast, lung, and pediatric studies.
25
Hakim, Arif Rahman, Dewi Marini Umi Atmaja, Tugiman Tugiman, and Amat Basri. "Android-Based Herpes Disease Detection Application using Image Processing." Sinkron 8, no. 1 (January 2, 2023): 305–13. http://dx.doi.org/10.33395/sinkron.v8i1.11913.
Full textAbstract:
Herpes is a viral infection that causes a skin disease that is widespread throughout the world. Herpes virus is a DNA virus transmitted via infected skin, saliva, and other body fluids. Herpes is characterized by chickenpox-like nodules in one area of the skin, swollen tissue surrounding the nodule, and blister formation on the nodule. Digital image processing that can detect herpes disease is anticipated to reduce physical contact between physicians and patients during skin disease diagnosis. This study's methodology includes collecting data on herpes disease, developing machine-learning models using the CNN algorithm, and deploying the model as an Android application. This study makes use of actual data collected via smartphones, Pocket Cameras, and internet-sourced photographs. The data include 12,645 images of skin affected by herpes and normal skin. Using 100 epochs and the Adadelta optimizer, the accuracy of this study is 85 percent.
26
Hamidi, Faiçal, Severus Constantin Olteanu, Dumitru Popescu, Houssem Jerbi, Ingrid Dincă, Sondess Ben Aoun, and Rabeh Abbassi. "Model Based Optimisation Algorithm for Maximum Power Point Tracking in Photovoltaic Panels." Energies 13, no. 18 (September 14, 2020): 4798. http://dx.doi.org/10.3390/en13184798.
Full textAbstract:
Extracting maximum energy from photovoltaic (PV) systems at varying conditions is crucial. It represents a problem that is being addressed by researchers who are using several techniques to obtain optimal outcomes in real-life scenarios. Among the many techniques, Maximum Power Point Tracking (MPPT) is one category that is not extensively researched upon. MPPT uses mathematical models to achieve gradient optimisation in the context of PV panels. This study proposes an enhanced maximisation problem based on gradient optimisation techniques to achieve better performance. In the context of MPPT in photovoltaic panels, an equality restriction applies, which is solved by employing the Dual Lagrangian expression. Considering this dual problem and its mathematical form, the Nesterov Accelerated Gradient (NAG) framework is used. Additionally, since it is challenging to ascertain the step size, its approximate value is taken using the Adadelta approach. A basic MPPT framework, along with a DC-to-DC convertor, was simulated to validate the results.
27
Qiu, Ningjia, Zhuorui Shen, Xiaojuan Hu, and Peng Wang. "A novel sentiment classification model based on online learning." Journal of Algorithms & Computational Technology 13 (January 2019): 174830261984576. http://dx.doi.org/10.1177/1748302619845764.
Full textAbstract:
Memory limitation and slow training speed are two important problems in sentiment analysis. In this paper, we propose a sentiment classification model based on online learning to improve the training speed of the sentiment classification. First, combining the adaptive adjustment of learning rate of the Adadelta algorithm and the characteristics of avoid frequent jitter of Adam algorithm in the later stage of training, we present a novel Adamdelta algorithm. It solves the problem that learning rate of traditional follow the regularized leader (FTRL)-Proximal online learning algorithm will disappear with the increase of training times. Moreover, we gain an optimized logistic regression (LR) model and use it to the sentiment classification of online learning. Finally, we compare the proposed algorithm with five similar models with the experimental data of the IMDb movie review dataset. Experimental results show that the improved algorithm has better classification effect and can effectively improve the precision and recall of the classifier.
28
Ezgi, Ahmet, and Aytuğ Onan. "Automatic Knee Osteoarthritis Severity Grading using Deep Neural Networks: Comparative Analysis of Network Architectures and Optimization Functions." International Conference on Applied Engineering and Natural Sciences 1, no. 1 (July 20, 2023): 197–203. http://dx.doi.org/10.59287/icaens.992.
Full textAbstract:
Knee osteoarthritis (OA) is a prevalent degenerative joint disease that requires accurate assessment of its severity for effective treatment planning. In this study, we propose an automatic knee OA severity-grading system based on deep neural networks. Specifically, we explore various network architectures, including VGG-16, VGG-19, ResNet-101, EfficientNet-B7, and EfficientNet-B6, along with different optimization functions such as SGD, ADAM, Nadam, AdamW, and AdaDelta. Furthermore, we investigate two loss functions, namely, the novel ordinal loss and the cross-entropy loss. The proposed system is evaluated on a carefully curated dataset, and comprehensive experimental settings are employed to ensure reliable results. Our findings indicate that the combination of the EfficientNet-B7 network with the Nadam optimizer yields the best performance, achieving an accuracy of 70.1% in knee OA severity grading. These results demonstrate the potential of deep neural networks in automating the grading process, offering a valuable tool for clinicians and researchers in the field of knee osteoarthritis management.
29
Alves Oliveira, Raquel, José Marcato Junior, Celso Soares Costa, Roope Näsi, Niko Koivumäki, Oiva Niemeläinen, Jere Kaivosoja, Laura Nyholm, Hemerson Pistori, and Eija Honkavaara. "Silage Grass Sward Nitrogen Concentration and Dry Matter Yield Estimation Using Deep Regression and RGB Images Captured by UAV." Agronomy 12, no. 6 (June 1, 2022): 1352. http://dx.doi.org/10.3390/agronomy12061352.
Full textAbstract:
Agricultural grasslands are globally important for food production, biodiversity, and greenhouse gas mitigation. Effective strategies to monitor grass sward properties, such as dry matter yield (DMY) and nitrogen concentration, are crucial when aiming to improve the sustainable use of grasslands in the context of food production. UAV-borne spectral imaging and traditional machine learning methods have already shown the potential to estimate DMY and nitrogen concentration for the grass swards. In this study, convolutional neural networks (CNN) were trained using low-cost RGB images, captured from a UAV, and agricultural reference measurements collected in an experimental grass field in Finland. Four different deep regression network architectures and three different optimizers were assessed. The best average results of the cross-validation were achieved by the VGG16 architecture with optimizer Adadelta: r2 of 0.79 for DMY and r2 of 0.73 for nitrogen concentration. The results demonstrate that this is a promising and effective tool for practical applications since the sensor is low-cost and the computational processing is not time-consuming in comparison to more complex sensors.
30
Dalli, Anouar. "Impact of Hyperparameters on Deep Learning Model for Customer Churn Prediction in Telecommunication Sector." Mathematical Problems in Engineering 2022 (February 9, 2022): 1–11. http://dx.doi.org/10.1155/2022/4720539.
Full textAbstract:
In this paper, in order to predict a customer churn in the telecommunication sector, we have analysed several published articles that had used machine learning (ML) techniques. Significant predictive performance had been seen by utilising deep learning techniques. However, we have seen a tremendous lack of empirically derived heuristic information where we had to influence the hyperparameters consequently. Here, we had demonstrated three experimental findings, where a Relu activation function was embedded and utilised successfully in the hidden layers of the deep network. We can also see that the output layer had the service ability of a sigmoid function, in which we had seen a significant performance of the neural network model and obviously it was improved. Furthermore, we had also seen that the model's performance was noticed to be even better, but it was only considered better though when the batch size in the model was taken less than the test dataset’s size, respectively. In terms of accuracy, the RemsProp optimizer beat out the other algorithms such as stochastic gradient descent (SGD). RemsProp was seen even better from the Adadelta algorithm, the Adam algorithm, the AdaGrad algorithm, and AdaMax algorithm as well.
31
P, Shanmugavadivu, Mary Shanthi Rani M, Chitra P, Lakshmanan S, Nagaraja P, and Vignesh U. "Bio-Optimization of Deep Learning Network Architectures." Security and Communication Networks 2022 (September 20, 2022): 1–11. http://dx.doi.org/10.1155/2022/3718340.
Full textAbstract:
Deep learning is reaching new heights as a result of its cutting-edge performance in a variety of fields, including computer vision, natural language processing, time series analysis, and healthcare. Deep learning is implemented using batch and stochastic gradient descent methods, as well as a few optimizers; however, this led to subpar model performance. However, there is now a lot of effort being done to improve deep learning’s performance using gradient optimization methods. The suggested work analyses convolutional neural networks (CNN) and deep neural networks (DNN) using several cutting-edge optimizers to enhance the performance of architectures. This work uses specific optimizers (SGD, RMSprop, Adam, Adadelta, etc.) to enhance the performance of designs using different types of datasets for result matching. A thorough report on the optimizers’ performance across a variety of architectures and datasets finishes the study effort. This research will be helpful to researchers in developing their framework and appropriate architecture optimizers. The proposed work involves eight new optimizers using four CNN and DNN architectures. The experimental results exploit breakthrough results for improving the efficiency of CNN and DNN architectures using various datasets.
32
Parlys, Albert, Ajub Ajulian Zahra, and Achmad Hidayatno. "PENGGOLONGAN LAGU BERDASARKAN SPEKTOGRAM DENGAN CONVOLUTION NEURAL NETWORK." TRANSIENT 7, no. 1 (March 12, 2018): 28. http://dx.doi.org/10.14710/transient.7.1.28-33.
Full textAbstract:
Saat ini terdapat banyak lagu yang sudah diproduksi di dunia. Lagu-lagu tersebut digolongkan ke dalam genre berbeda. Ada berbagai macam genre mulai dari pop, rock, classic, reggae, dubstep, dan lain-lain. Perbedaan genre ini disebabkan adanya ketidaksamaan melodi, ketukan, intonasi, dan ekspresi pada masing-masing genre. Saat ini terdapat banyak metode yang digunakan untuk mengenali sebuah lagu, di antaranya audioprint, penggolongan genre, pengenalan ketukan lagu, pengenalan lirik lagu, dan lain-lain. Metode yang dipakai selama ini menggunakan database dengan ciri dari jutaan lagu. Salah satu metode lain adalah dengan mengembangkan sistem identifikasi lagu dengan suatu jaringan saraf terlatih. Penelitian ini akan membahas perancangan sebuah sistem untuk menggolongan lagu berdasarkan spektogram. Masukan sistem berupa lagu dengan format audio MP3 yang diubah ke dalam bentuk spektogram kemudian dilatih menggunakan Convolutional Neural Network. Ciri lagu akan diperoleh kemudian diklasifikan ke dalam lima genre berbeda yaitu pop, rock, classic, dubstep, dan reggae. Berdasarkan hasil pelatihan dan pengujian dengan filter 3x3 didapat nilai akurasi penggolongan lagu sebesar 100% pada 750 data latih dan 98% pada 50 lagu data uji. Algoritme pembelajaran terbaik pada pelatihan dengan filter yang sama adalah algoritme Adam yang lebih cepat dibandingkan dengan Adadelta, Adagrad, dan SGD.
33
Irfan, Desi, Teddy Surya Gunawan, and Wanayumini Wanayumini. "Comparison Of SGD, Rmsprop, and Adam Optimation In Animal Classification Using CNNs." International Conference on Information Science and Technology Innovation (ICoSTEC) 2, no. 1 (March 5, 2023): 45–51. http://dx.doi.org/10.35842/icostec.v2i1.35.
Full textAbstract:
Many measures have been taken to protect endangered species by using "camera trap" technology which is widespread in the field of technology-based nature protection field research. In this study, a machine learning-based approach is presented to identify endangered wildlife images with a data set containing 5000 images taken from Kaggle and some other sources. The Gradient Descent optimization method is often used for Artificial Neural Network (ANN) training. This method plays a role in finding the weight values that give the best output value. Three optimization methods have been implemented, namely Stochastic Gradient Descent (SGD), ADADELTA, and Adam on the Artificial Neural Network system for animal data classification. In some of the studies reviewed there are differences in the results of SGD and ADAM, which on the one hand SGD is superior, and on the one hand ADAM is superior with the appropriate learning rate. The results of this study show that the CNN method with the Adam optimization function produces the highest accuracy compared to the SGD and RMSprop optimization methods. The model trained using Adam's optimization function achieved an accuracy of 89.81% on the test, showing the feasibility of the approach.
34
Yang, Szu-Yueh, Hsin-Che Jan, Chun-Yu Chen, and Ming-Shyan Wang. "CNN-Based QR Code Reading of Package for Unmanned Aerial Vehicle." Sensors 23, no. 10 (May 12, 2023): 4707. http://dx.doi.org/10.3390/s23104707.
Full textAbstract:
This paper plans to establish a warehouse management system based on an unmanned aerial vehicle (UAV) to scan the QR codes printed on packages. This UAV consists of a positive cross quadcopter drone and a variety of sensors and components, such as flight controllers, single-board computers, optical flow sensors, ultrasonic sensors and cameras, etc. The UAV stabilizes itself by proportional-integral-derivative (PID) control and takes pictures of the package as it reaches ahead of the shelf. Through convolutional neural networks (CNNs), the placement angle of the package can be accurately identified. Some optimization functions are applied to compare system performance. When the angle is 90°, that is, the package is placed normally and correctly, the QR code will be read directly. Otherwise, image processing techniques that include Sobel edge computing, minimum circumscribed rectangle, perspective transformation, and image enhancement is required to assist in reading the QR code. The experimental results showed that the proposed algorithm provided good performance of a recognition rate of 94% for the stochastic gradient descent (SGD) and 95% for Adadelta optimization functions. After that, successful QR code reading was presented.
35
Jasim, Sarah S., Alia K. Abdul Hassan, and Scott Turner. "Driver Drowsiness Detection Using Gray Wolf Optimizer Based on Face and Eye Tracking." ARO-THE SCIENTIFIC JOURNAL OF KOYA UNIVERSITY 10, no. 1 (May 5, 2022): 49–56. http://dx.doi.org/10.14500/aro.10928.
Full textAbstract:
It is critical today to provide safe and collision-free transport. As a result, identifying the driver’s drowsiness before their capacity to drive is jeopardized. An automated hybrid drowsiness classification method that incorporates the artificial neural network (ANN) and the gray wolf optimizer (GWO) is presented to discriminate human drowsiness and fatigue for this aim. The proposed method is evaluated in alert and sleep-deprived settings on the driver drowsiness detection of video dataset from the National Tsing Hua University Computer Vision Lab. The video was subjected to various video and image processing techniques to detect the drivers’ eye condition. Four features of the eye were extracted to determine the condition of drowsiness, the percentage of eyelid closure (PERCLOS), blink frequency, maximum closure duration of the eyes, and eye aspect ratio (ARE). These parameters were then integrated into an ANN and combined with the proposed method (gray wolf optimizer with ANN [GWOANN]) for drowsiness classification. The accuracy of these models was calculated, and the results demonstrate that the proposed method is the best. An Adadelta optimizer with 3 and 4 hidden layer networks of (13, 9, 7, and 5) and (200, 150, 100, 50, and 25) neurons was utilized. The GWOANN technique had 91.18% and 97.06% accuracy, whereas the ANN model had 82.35% and 86.76%.
36
Zahara, Soffa, Sugianto, and M. Bahril Ilmiddafiq. "Prediksi Indeks Harga Konsumen Menggunakan Metode Long Short Term Memory (LSTM) Berbasis Cloud Computing." Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) 3, no. 3 (December 2, 2019): 357–63. http://dx.doi.org/10.29207/resti.v3i3.1086.
Full textAbstract:
Long Short Term Memory (LSTM) is known as optimized Recurrent Neural Network (RNN) architectures that overcome RNN’s lact about maintaining long period of memories. As part of machine learning networks, LSTM also notable as the right choice for time-series prediction. Currently, machine learning is a burning issue in economic world, abundant studies such predicting macroeconomic and microeconomics indicators are emerge. Inflation rate has been used for decision making for central banks also private sector. In Indonesia, CPI (Consumer Price Index) is one of best practice inflation indicators besides Wholesale Price Index and The Gross Domestic Product (GDP). Since CPI data could be used as a direction for next inflation move, we conducted CPI prediction model using LSTM method. The network model input consists of 28 variables of staple price in Surabaya and the output is CPI value, also the entire development of prediction model are done in Amazon Web Service (AWS) Cloud. In the interest of accuracy improvement, we used several optimization algorithm i.e. Stochastic Gradient Descent (sgd), Root Mean Square Propagation (RMSProp), Adaptive Gradient(AdaGrad), Adaptive moment (Adam), Adadelta, Nesterov Adam (Nadam) and Adamax. The results indicate that Nadam has 4,008 RMSE’s value, less than other algorithm which indicate the most accurate optimization algorithm to predict CPI value.
37
Lin, Cuifang. "Application of Traditional Cultural Symbols in Art Design under the Background of Artificial Intelligence." Mathematical Problems in Engineering 2021 (October 26, 2021): 1–11. http://dx.doi.org/10.1155/2021/1258080.
Full textAbstract:
In order to solve the declining influence of traditional cultural symbols, the research on traditional cultural symbols has become more meaningful. This article aims to study the application of traditional cultural symbols in art design under the background of artificial intelligence. In this paper, a fractal model with self-combined nonlinear function changes is constructed. By combining nonlinear transformations and multiparameter adjustments, various types of fractal models can be automatically rendered. The convolutional neural network algorithm is used to extract the characteristics of the style picture, and it is compared with the trained picture many times to avoid the problem of excessive tendency of the image with improper weight. The improved L-BFGS algorithm is also used to optimize the loss of the traditional L-BFGS, which improves the quality of the generated pictures and reduces the noise of the chessboard. The experimental results in this paper show that the improved L-BFGS algorithm has the least loss and the shortest time in the time used for more than 500 s. Compared with the traditional AdaGrad method, its loss is reduced by about 62%; compared with the traditional AdaDelta method, its loss is reduced by 46%. Its loss is reduced by about 8% compared with the newly optimized Adam method, which is a great improvement.
38
Li, Yanan, Xuebin Ren, Fangyuan Zhao, and Shusen Yang. "A Zeroth-Order Adaptive Learning Rate Method to Reduce Cost of Hyperparameter Tuning for Deep Learning." Applied Sciences 11, no. 21 (October 30, 2021): 10184. http://dx.doi.org/10.3390/app112110184.
Full textAbstract:
Due to powerful data representation ability, deep learning has dramatically improved the state-of-the-art in many practical applications. However, the utility highly depends on fine-tuning of hyper-parameters, including learning rate, batch size, and network initialization. Although many first-order adaptive methods (e.g., Adam, Adagrad) have been proposed to adjust learning rate based on gradients, they are susceptible to the initial learning rate and network architecture. Therefore, the main challenge of using deep learning in practice is how to reduce the cost of tuning hyper-parameters. To address this, we propose a heuristic zeroth-order learning rate method, Adacomp, which adaptively adjusts the learning rate based only on values of the loss function. The main idea is that Adacomp penalizes large learning rates to ensure the convergence and compensates small learning rates to accelerate the training process. Therefore, Adacomp is robust to the initial learning rate. Extensive experiments, including comparison to six typically adaptive methods (Momentum, Adagrad, RMSprop, Adadelta, Adam, and Adamax) on several benchmark datasets for image classification tasks (MNIST, KMNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100), were conducted. Experimental results show that Adacomp is not only robust to the initial learning rate but also to the network architecture, network initialization, and batch size.
39
Kumar Singh, Koushlendra, Suraj Kumar, Marios Antonakakis, Konstantina Moirogiorgou, Anirudh Deep, Kanchan Lata Kashyap, Manish Kumar Bajpai, and Michalis Zervakis. "Deep Learning Capabilities for the Categorization of Microcalcification." International Journal of Environmental Research and Public Health 19, no. 4 (February 14, 2022): 2159. http://dx.doi.org/10.3390/ijerph19042159.
Full textAbstract:
Breast cancer is the most common cancer in women worldwide. It is the most frequently diagnosed cancer among women in 140 countries out of 184 reporting countries. Lesions of breast cancer are abnormal areas in the breast tissues. Various types of breast cancer lesions include (1) microcalcifications, (2) masses, (3) architectural distortion, and (4) bilateral asymmetry. Microcalcification can be classified as benign, malignant, and benign without a callback. In the present manuscript, we propose an automatic pipeline for the detection of various categories of microcalcification. We performed deep learning using convolution neural networks (CNNs) for the automatic detection and classification of all three categories of microcalcification. CNN was applied using four different optimizers (ADAM, ADAGrad, ADADelta, and RMSProp). The input images of a size of 299 × 299 × 3, with fully connected RELU and SoftMax output activation functions, were utilized in this study. The feature map was obtained using the pretrained InceptionResNetV2 model. The performance evaluation of our classification scheme was tested on a curated breast imaging subset of the DDSM mammogram dataset (CBIS–DDSM), and the results were expressed in terms of sensitivity, specificity, accuracy, and area under the curve (AUC). Our proposed classification scheme outperforms the ability of previously used deep learning approaches and classical machine learning schemes.
40
Martinez, Fernando, Holman Montiel, and Fredy Martinez. "Comparative study of optimization algorithms on convolutional network for autonomous driving." International Journal of Electrical and Computer Engineering (IJECE) 12, no. 6 (December 1, 2022): 6363. http://dx.doi.org/10.11591/ijece.v12i6.pp6363-6372.
Full textAbstract:
<p>he last 10 years have been the decade of autonomous vehicles. Advances in intelligent sensors and control schemes have shown the possibility of real applications. Deep learning, and in particular convolutional networks have become a fundamental tool in the solution of problems related to environment identification, path planning, vehicle behavior, and motion control. In this paper, we perform a comparative study of the most used optimization strategies on the convolutional architecture residual neural network (ResNet) for an autonomous driving problem as a previous step to the development of an intelligent sensor. This sensor, part of our research in reactive systems for autonomous vehicles, aims to become a system for direct mapping of sensory information to control actions from real-time images of the environment. The optimization techniques analyzed include stochastic gradient descent (SGD), adaptive gradient (Adagrad), adaptive learning rate (Adadelta), root mean square propagation (RMSProp), Adamax, adaptive moment estimation (Adam), nesterov-accelerated adaptive moment estimation (Nadam), and follow the regularized leader (Ftrl). The training of the deep model is evaluated in terms of convergence, accuracy, recall, and F1-score metrics. Preliminary results show a better performance of the deep network when using the SGD function as an optimizer, while the Ftrl function presents the poorest performances.</p>
41
Tang, Mingxing, Zhen Huang, Yuan Yuan, Changjian Wang, and Yuxing Peng. "A Bounded Scheduling Method for Adaptive Gradient Methods." Applied Sciences 9, no. 17 (September 1, 2019): 3569. http://dx.doi.org/10.3390/app9173569.
Full textAbstract:
Many adaptive gradient methods have been successfully applied to train deep neural networks, such as Adagrad, Adadelta, RMSprop and Adam. These methods perform local optimization with an element-wise scaling learning rate based on past gradients. Although these methods can achieve an advantageous training loss, some researchers have pointed out that their generalization capability tends to be poor as compared to stochastic gradient descent (SGD) in many applications. These methods obtain a rapid initial training process but fail to converge to an optimal solution due to the unstable and extreme learning rates. In this paper, we investigate the adaptive gradient methods and get the insights on various factors that may lead to poor performance of Adam. To overcome that, we propose a bounded scheduling algorithm for Adam, which can not only improve the generalization capability but also ensure the convergence. To validate our claims, we carry out a series of experiments on the image classification and the language modeling tasks on several standard benchmarks such as ResNet, DenseNet, SENet and LSTM on typical data sets such as CIFAR-10, CIFAR-100 and Penn Treebank. Experimental results show that our method can eliminate the generalization gap between Adam and SGD, meanwhile maintaining a relative high convergence rate during training.
42
Jembre, Yalew Zelalem, Yuniarto Wimbo Nugroho, Muhammad Toaha Raza Khan, Muhammad Attique, Rajib Paul, Syed Hassan Ahmed Shah, and Beomjoon Kim. "Evaluation of Reinforcement and Deep Learning Algorithms in Controlling Unmanned Aerial Vehicles." Applied Sciences 11, no. 16 (August 6, 2021): 7240. http://dx.doi.org/10.3390/app11167240.
Full textAbstract:
Unmanned Aerial Vehicles (UAVs) are abundantly becoming a part of society, which is a trend that is expected to grow even further. The quadrotor is one of the drone technologies that is applicable in many sectors and in both military and civilian activities, with some applications requiring autonomous flight. However, stability, path planning, and control remain significant challenges in autonomous quadrotor flights. Traditional control algorithms, such as proportional-integral-derivative (PID), have deficiencies, especially in tuning. Recently, machine learning has received great attention in flying UAVs to desired positions autonomously. In this work, we configure the quadrotor to fly autonomously by using agents (the machine learning schemes being used to fly the quadrotor autonomously) to learn about the virtual physical environment. The quadrotor will fly from an initial to a desired position. When the agent brings the quadrotor closer to the desired position, it is rewarded; otherwise, it is punished. Two reinforcement learning models, Q-learning and SARSA, and a deep learning deep Q-network network are used as agents. The simulation is conducted by integrating the robot operating system (ROS) and Gazebo, which allowed for the implementation of the learning algorithms and the physical environment, respectively. The result has shown that the Deep Q-network network with Adadelta optimizer is the best setting to fly the quadrotor from the initial to desired position.
43
Li, Ning, and Md Jais Bin Ismail. "Application of Artificial Intelligence Technology in the Teaching of Complex Situations of Folk Music under the Vision of New Media Art." Wireless Communications and Mobile Computing 2022 (April 14, 2022): 1–10. http://dx.doi.org/10.1155/2022/5816067.
Full textAbstract:
Enhancement in information technology has made the online teaching-learning process easier. However, this process is still a challenging task for teaching courses of theoretical type, music, dance, and arts. For the classes of these types, the traditional system of teaching and learning is well suited, but, in particularly complex situations, accomplishing the task is highly difficult. Hence, new media art technology is introduced to overcome the difficulties. In this research, Chinese folk music is taught online with the aid of new media art with the support of artificial intelligence. The analysis of the proposed work is carried out on the folk music dataset, which considers folk music of ethnic minority groups. A novel Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) art algorithm is implemented to perform the analysis. The performance is compared with the existing gradient descent, Adam, and AdaDelta algorithms. L-BFGS algorithm is essentially a particular recipe for designing and possibly executing an artwork, including algorithms, functions, facial expression, and other input that ultimately decides the structure the folk music and media art would then take. This contribution could be numerical, information processing, or formative. From the obtained results, it can be shown that the proposed system has provided 97% and 98% of accuracy on training and testing data, which is higher when compared to the existing algorithms.
44
Yaqub, Muhammad, Jinchao Feng, M. Sultan Zia, Kaleem Arshid, Kebin Jia, Zaka Ur Rehman, and Atif Mehmood. "State-of-the-Art CNN Optimizer for Brain Tumor Segmentation in Magnetic Resonance Images." Brain Sciences 10, no. 7 (July 3, 2020): 427. http://dx.doi.org/10.3390/brainsci10070427.
Full textAbstract:
Brain tumors have become a leading cause of death around the globe. The main reason for this epidemic is the difficulty conducting a timely diagnosis of the tumor. Fortunately, magnetic resonance images (MRI) are utilized to diagnose tumors in most cases. The performance of a Convolutional Neural Network (CNN) depends on many factors (i.e., weight initialization, optimization, batches and epochs, learning rate, activation function, loss function, and network topology), data quality, and specific combinations of these model attributes. When we deal with a segmentation or classification problem, utilizing a single optimizer is considered weak testing or validity unless the decision of the selection of an optimizer is backed up by a strong argument. Therefore, optimizer selection processes are considered important to validate the usage of a single optimizer in order to attain these decision problems. In this paper, we provides a comprehensive comparative analysis of popular optimizers of CNN to benchmark the segmentation for improvement. In detail, we perform a comparative analysis of 10 different state-of-the-art gradient descent-based optimizers, namely Adaptive Gradient (Adagrad), Adaptive Delta (AdaDelta), Stochastic Gradient Descent (SGD), Adaptive Momentum (Adam), Cyclic Learning Rate (CLR), Adaptive Max Pooling (Adamax), Root Mean Square Propagation (RMS Prop), Nesterov Adaptive Momentum (Nadam), and Nesterov accelerated gradient (NAG) for CNN. The experiments were performed on the BraTS2015 data set. The Adam optimizer had the best accuracy of 99.2% in enhancing the CNN ability in classification and segmentation.
45
Nurcahyati, Afivah Dwi, Ronny Makhfuddin Akbar, and Soffa Zahara. "Klasifikasi Citra Penyakit pada Daun Jagung Menggunakan Deep Learning dengan Metode Convolution Neural Network (CNN)." SUBMIT: Jurnal Ilmiah Teknologi Infomasi dan Sains 2, no. 2 (June 29, 2022): 43–51. http://dx.doi.org/10.36815/submit.v2i2.1877.
Full textAbstract:
Di kecamatan Gedeg, kabupaten Mojokerto mayoritas masyarakat memiliki mata pencaharian sebagai seorang petani tanaman Jagung, namun banyak kendala yang telah dihadapi oleh semua petani yakni gagal panen dikarenakan jenis penyakit yang tidak diketahui jenisnya yang berakibat gagal panen. Permasalahan tersebut dapat diatasi dengan Deep Learning yang menggunakan metode klasifikasi algoritma Convolutional Neural Network (CNN). Menggunakan citra fisik pada daun tanaman jagung, metode CNN dapat membuat klasifikasi melalui model yang dibuat. Peneliti membuat sebuah model untuk dilakukan klasifikasi dengan bagian terdiri dari 4 convolution layer, 2 pooling layer dengan ukuran 2×2, 3 dropout layer, 2 dense layer serta 1 flatten layer. Untuk melakukan aktivasi menggunakan ReLu, beserta 32 dan 64 filter menggunakan 4 macam ukuran kernel yakni 3x2, 3x3, 3x4, 4x4. Dan dilakukan pengujian dengan 900 data gambar yang di mana 720 digunakan sebagai data train dan 180 sebagai data Test. Dengan learning rate sebesar 0.004, 100 epoch serta 6 algoritma performansi sebagai perbandingan yakni algoritma Root Mean Square Propagation (RMSProp), Adaptive Gradient (AdaGrad), Stochastic Gradient descent (SGD), Adaptive Moment (Adam), Adamax, Adadelta. Dan dihasilkan tingkat akurasi tertinggi yang dihasilkan oleh ukuran kernel 3x3 dengan algoritma optimasi Adaptive Moment (Adam) dengan hasil tingkat akurasinya sebesar 84% untuk data test dan 89% untuk data train, pada pengujian Testing dilakukan dengan jumlah 180 data yang didapatkan hasil tertinggi dengan model ukuran kernel 3x3 dengan jumlah true 175 dan jumlah false 5 didapatkan nilai presisi yang dihasilkan sebesar 94%, berdasarkan dengan komposisi warna pada citra.
46
Mambang and Finki Dona Marleny. "Image Prediction of Exact Science and Social Science Learning Content with Convolutional Neural Network." JOIV : International Journal on Informatics Visualization 6, no. 4 (December 31, 2022): 749. http://dx.doi.org/10.30630/joiv.6.4.923.
Full textAbstract:
Learning content can be identified through text, images, and videos. This study aims to predict the learning content contained on YouTube. The images used are images contained in the learning content of the exact sciences, such as mathematics, and social science fields, such as culture. Prediction of images on learning content is done by creating a model on CNN. The collection of datasets carried out on learning content is found on YouTube. The first assessment was performed with an RMSProp optimizer with a learning rate of 0.001, which is used for all optimizers. Several other optimizers were used in this experiment, such as Adam, Nadam, SGD, Adamax, Adadelta, Adagrad, and Ftrl. The CNN model used in the dataset training process tested the image with multiple optimizers and obtained high accuracy results on RMSprop, Adam, and Adamax. There are still many shortcomings in the experiments we conducted in this study, such as not using the momentum component. The momentum component is carried out to improve the speed and quality of neural networks. We can develop a CNN model using the momentum component to obtain good training results and accuracy in later studies. All optimizers contained in Keras and TensorFlow can be used as a comparison. This study concluded that images of learning content on YouTube could be modeled and classified. Further research can add image variables and a momentum component in the testing of CNN models.
47
Mehmood, Faisal, Shabir Ahmad, and Taeg Keun Whangbo. "An Efficient Optimization Technique for Training Deep Neural Networks." Mathematics 11, no. 6 (March 10, 2023): 1360. http://dx.doi.org/10.3390/math11061360.
Full textAbstract:
Deep learning is a sub-branch of artificial intelligence that acquires knowledge by training a neural network. It has many applications in the field of banking, automobile industry, agriculture, and healthcare industry. Deep learning has played a significant role in solving complex tasks related to computer vision, such as image classification, natural language processing, and object detection. On the other hand, optimizers also play an intrinsic role in training the deep learning model. Recent studies have proposed many deep learning models, such as VGG, ResNet, DenseNet, and ImageNet. In addition, there are many optimizers such as stochastic gradient descent (SGD), Adam, AdaDelta, Adabelief, and AdaMax. In this study, we have selected those models that require lower hardware requirements and shorter training times, which facilitates the overall training process. We have modified the Adam based optimizers and minimized the cyclic path. We have removed an additional hyper-parameter from RMSProp and observed that the optimizer works with various models. The learning rate is set to minimum and constant. The initial weights are updated after each epoch, which helps to improve the accuracy of the model. We also changed the position of the epsilon in the default Adam optimizer. By changing the position of the epsilon, it accumulates the updating process. We used various models with SGD, Adam, RMSProp, and the proposed optimization technique. The results indicate that the proposed method is effective in achieving the accuracy and works well with the state-of-the-art architectures.
48
Putro, Nugroho Adi, Rendi Septian, Widiastuti Widiastuti, Mawadatul Maulidah, and Hilman Ferdinandus Pardede. "PREDICTION OF HOTEL BOOKING CANCELLATION USING DEEP NEURAL NETWORK AND LOGISTIC REGRESSION ALGORITHM." Jurnal Techno Nusa Mandiri 18, no. 1 (March 15, 2021): 1–8. http://dx.doi.org/10.33480/techno.v18i1.2056.
Full textAbstract:
Booking cancellation is a key aspect of hotel revenue management as it affects the room reservation system. Booking cancellation has a significant effect on revenue which has a significant impact on demand management decisions in the hotel industry. In order to reduce the cancellation effect, the hotel applies the cancellation model as the key to addressing this problem with the machine learning-based system developed. In this study, using a data collection from the Kaggle website with the name hotel-booking-demand dataset. The research objective was to see the performance of the deep neural network method which has two classification classes, namely cancel and not. Then optimized with optimizers and learning rate. And to see which attribute has the most role in determining the level of accuracy using the Logistic Regression algorithm. The results obtained are the Encoder-Decoder Layer by adamax optimizer which is higher than that of the Decoder-Encoder by adadelta optimizer. After adding the learning rate, the adamax accuracy for the encoders and encoders decreased for a learning rate of 0.001. The results of the top three ranks of each neural network after adding the learning rate show that the smaller the learning rate, the higher the accuracy, but we don't know what the optimal value for the learning rate is. By using the Logistic Regression algorithm by eliminating several attributes, the most influential level of accuracy is the state attribute and total_of_special_requests, where accuracy increases when the state attribute is removed because there are 177 variations in these attributes
49
Tahir, Sheikh Badar ud din, Ahmad Jalal, and Kibum Kim. "Wearable Inertial Sensors for Daily Activity Analysis Based on Adam Optimization and the Maximum Entropy Markov Model." Entropy 22, no. 5 (May 20, 2020): 579. http://dx.doi.org/10.3390/e22050579.
Full textAbstract:
Advancements in wearable sensors technologies provide prominent effects in the daily life activities of humans. These wearable sensors are gaining more awareness in healthcare for the elderly to ensure their independent living and to improve their comfort. In this paper, we present a human activity recognition model that acquires signal data from motion node sensors including inertial sensors, i.e., gyroscopes and accelerometers. First, the inertial data is processed via multiple filters such as Savitzky–Golay, median and hampel filters to examine lower/upper cutoff frequency behaviors. Second, it extracts a multifused model for statistical, wavelet and binary features to maximize the occurrence of optimal feature values. Then, adaptive moment estimation (Adam) and AdaDelta are introduced in a feature optimization phase to adopt learning rate patterns. These optimized patterns are further processed by the maximum entropy Markov model (MEMM) for empirical expectation and highest entropy, which measure signal variances for outperformed accuracy results. Our model was experimentally evaluated on University of Southern California Human Activity Dataset (USC-HAD) as a benchmark dataset and on an Intelligent Mediasporting behavior (IMSB), which is a new self-annotated sports dataset. For evaluation, we used the “leave-one-out” cross validation scheme and the results outperformed existing well-known statistical state-of-the-art methods by achieving an improved recognition accuracy of 91.25%, 93.66% and 90.91% when compared with USC-HAD, IMSB, and Mhealth datasets, respectively. The proposed system should be applicable to man–machine interface domains, such as health exercises, robot learning, interactive games and pattern-based surveillance.
50
Hung, Truong Viet, Vu Quang Viet, and Dinh Van Thuat. "A deep learning-based procedure for estimation of ultimate load carrying of steel trusses using advanced analysis." Journal of Science and Technology in Civil Engineering (STCE) - NUCE 13, no. 3 (August 31, 2019): 113–23. http://dx.doi.org/10.31814/stce.nuce2019-13(3)-11.
Full textAbstract:
In the present study, Deep Learning (DL) algorithm or Deep Neural Networks (DNN), one of the most powerful techniques in Machine Learning (ML), is employed for estimation of ultimate load factor of nonlinear inelastic steel truss. Datasets consisting of training and test data are created based on advanced analysis. In datasets, input data are the member cross-sections of the truss members and output data is the ultimate load factor of the whole structure. An example of a planar 39-bar steel truss is studied to demonstrate the efficiency and accuracy of the DL method. Five optimizers such as Adadelta, Adam, Nadam, RMSprop and SGD and five activation functions such as ELU, LeakyReLU, Sigmoid, Softplus, and Tanh are considered. Based on analysis results, it is proven that DL algorithm shows very high accuracy in the regression of the ultimate load factor of the planar 39-bar nonlinear inelastic steel truss. The number of layers can be selected with a small value such as 1, 2 or 3 layers and the number of neurons in each layer can be chosen in the range [Ni, 3Ni] with Ni is the number of input variables of the model. The activation functions ELU and LeakyReLU have better convergence speed of the training process compared to Sigmoid, Softplus and Tanh. The optimizer Adam works well with all activation functions considered and produces better MSE values regarding both training and test data.
Keywords:
deep learning; artificial neural networks; nonlinear inelastic analysis; steel truss; machine learning.