To see the other types of publications on this topic, follow the link: Mini-Batch Optimization.
Journal articles on the topic 'Mini-Batch Optimization'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Mini-Batch Optimization.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Gultekin, San, Avishek Saha, Adwait Ratnaparkhi, and John Paisley. "MBA: Mini-Batch AUC Optimization." IEEE Transactions on Neural Networks and Learning Systems 31, no. 12 (December 2020): 5561–74. http://dx.doi.org/10.1109/tnnls.2020.2969527.
Full text
2
Feyzmahdavian, Hamid Reza, Arda Aytekin, and Mikael Johansson. "An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization." IEEE Transactions on Automatic Control 61, no. 12 (December 2016): 3740–54. http://dx.doi.org/10.1109/tac.2016.2525015.
Full text
3
Banerjee, Subhankar, and Shayok Chakraborty. "Deterministic Mini-batch Sequencing for Training Deep Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 8 (May 18, 2021): 6723–31. http://dx.doi.org/10.1609/aaai.v35i8.16831.
Full textAbstract:
Recent advancements in the field of deep learning have dramatically improved the performance of machine learning models in a variety of applications, including computer vision, text mining, speech processing and fraud detection among others. Mini-batch gradient descent is the standard algorithm to train deep models, where mini-batches of a fixed size are sampled randomly from the training data and passed through the network sequentially. In this paper, we present a novel algorithm to generate a deterministic sequence of mini-batches to train a deep neural network (rather than a random sequence). Our rationale is to select a mini-batch by minimizing the Maximum Mean Discrepancy (MMD) between the already selected mini-batches and the unselected training samples. We pose the mini-batch selection as a constrained optimization problem and derive a linear programming relaxation to determine the sequence of mini-batches. To the best of our knowledge, this is the first research effort that uses the MMD criterion to determine a sequence of mini-batches to train a deep neural network. The proposed mini-batch sequencing strategy is deterministic and independent of the underlying network architecture and prediction task. Our extensive empirical analyses on three challenging datasets corroborate the merit of our framework over competing baselines. We further study the performance of our framework on two other applications besides classification (regression and semantic segmentation) to validate its generalizability.
4
Simanungkalit, F. R. J., H. Hanifah, G. Ardaneswari, N. Hariadi, and B. D. Handari. "Prediction of students’ academic performance using ANN with mini-batch gradient descent and Levenberg-Marquardt optimization algorithms." Journal of Physics: Conference Series 2106, no. 1 (November 1, 2021): 012018. http://dx.doi.org/10.1088/1742-6596/2106/1/012018.
Full textAbstract:
Abstract Online learning indirectly increases stress, thereby reducing social interaction among students and leading to physical and mental fatigue, which in turn reduced students’ academic performance. Therefore, the prediction of academic performance is required sooner to identify at-risk students with declining performance. In this paper, we use artificial neural networks (ANN) to predict this performance. ANNs with two optimization algorithms, mini-batch gradient descent and Levenberg-Marquardt, are implemented on students’ learning activity data in course X, which is recorded on LMS UI. Data contains 232 students and consists of two periods: the first month and second month of study. Before ANNs are implemented, both normalization and usage of ADASYN are conducted. The results of ANN implementation using two optimization algorithms within 10 trials each are compared based on the average accuracy, sensitivity, and specificity values. We then determine the best period to predict unsuccessful students correctly. The results show that both algorithms give better predictions over two months instead of one. ANN with mini-batch gradient descent has an average sensitivity of 78%; the corresponding values for ANN with Levenberg-Marquardt are 75%. Therefore, ANN with mini-batch gradient descent as its optimization algorithm is more suitable for predicting students that have potential to fail.
5
van Herwaarden, Dirk Philip, Christian Boehm, Michael Afanasiev, Solvi Thrastarson, Lion Krischer, Jeannot Trampert, and Andreas Fichtner. "Accelerated full-waveform inversion using dynamic mini-batches." Geophysical Journal International 221, no. 2 (February 21, 2020): 1427–38. http://dx.doi.org/10.1093/gji/ggaa079.
Full textAbstract:
SUMMARY We present an accelerated full-waveform inversion based on dynamic mini-batch optimization, which naturally exploits redundancies in observed data from different sources. The method rests on the selection of quasi-random subsets (mini-batches) of sources, used to approximate the misfit and the gradient of the complete data set. The size of the mini-batch is dynamically controlled by the desired quality of the gradient approximation. Within each mini-batch, redundancy is minimized by selecting sources with the largest angular differences between their respective gradients, and spatial coverage is maximized by selecting candidate events with Mitchell’s best-candidate algorithm. Information from sources not included in a specific mini-batch is incorporated into each gradient calculation through a quasi-Newton approximation of the Hessian, and a consistent misfit measure is achieved through the inclusion of a control group of sources. By design, the dynamic mini-batch approach has several main advantages: (1) The use of mini-batches with adaptive size ensures that an optimally small number of sources is used in each iteration, thus potentially leading to significant computational savings; (2) curvature information is accumulated and exploited during the inversion, using a randomized quasi-Newton method; (3) new data can be incorporated without the need to re-invert the complete data set, thereby enabling an evolutionary mode of full-waveform inversion. We illustrate our method using synthetic and real-data inversions for upper-mantle structure beneath the African Plate. In these specific examples, the dynamic mini-batch approach requires around 20 per cent of the computational resources in order to achieve data and model misfits that are comparable to those achieved by a standard full-waveform inversion where all sources are used in each iteration.
6
Ghadimi, Saeed, Guanghui Lan, and Hongchao Zhang. "Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization." Mathematical Programming 155, no. 1-2 (December 11, 2014): 267–305. http://dx.doi.org/10.1007/s10107-014-0846-1.
Full text
7
Kervazo, C., T. Liaudat, and J. Bobin. "Faster and better sparse blind source separation through mini-batch optimization." Digital Signal Processing 106 (November 2020): 102827. http://dx.doi.org/10.1016/j.dsp.2020.102827.
Full text
8
Dimitriou, Neofytos, and Ognjen Arandjelović. "Sequential Normalization: Embracing Smaller Sample Sizes for Normalization." Information 13, no. 7 (July 12, 2022): 337. http://dx.doi.org/10.3390/info13070337.
Full textAbstract:
Normalization as a layer within neural networks has over the years demonstrated its effectiveness in neural network optimization across a wide range of different tasks, with one of the most successful approaches being that of batch normalization. The consensus is that better estimates of the BatchNorm normalization statistics (μ and σ2) in each mini-batch result in better optimization. In this work, we challenge this belief and experiment with a new variant of BatchNorm known as GhostNorm that, despite independently normalizing batches within the mini-batches, i.e., μ and σ2 are independently computed and applied to groups of samples in each mini-batch, outperforms BatchNorm consistently. Next, we introduce sequential normalization (SeqNorm), the sequential application of the above type of normalization across two dimensions of the input, and find that models trained with SeqNorm consistently outperform models trained with BatchNorm or GhostNorm on multiple image classification data sets. Our contributions are as follows: (i) we uncover a source of regularization that is unique to GhostNorm, and not simply an extension from BatchNorm, and illustrate its effects on the loss landscape, (ii) we introduce sequential normalization (SeqNorm) a new normalization layer that improves the regularization effects of GhostNorm, (iii) we compare both GhostNorm and SeqNorm against BatchNorm alone as well as with other regularization techniques, (iv) for both GhostNorm and SeqNorm models, we train models whose performance is consistently better than our baselines, including ones with BatchNorm, on the standard image classification data sets of CIFAR–10, CIFAR-100, and ImageNet ((+0.2%, +0.7%, +0.4%), and (+0.3%, +1.7%, +1.1%) for GhostNorm and SeqNorm, respectively).
9
Bakurov, Illya, Marco Buzzelli, Mauro Castelli, Leonardo Vanneschi, and Raimondo Schettini. "General Purpose Optimization Library (GPOL): A Flexible and Efficient Multi-Purpose Optimization Library in Python." Applied Sciences 11, no. 11 (May 23, 2021): 4774. http://dx.doi.org/10.3390/app11114774.
Full textAbstract:
Several interesting libraries for optimization have been proposed. Some focus on individual optimization algorithms, or limited sets of them, and others focus on limited sets of problems. Frequently, the implementation of one of them does not precisely follow the formal definition, and they are difficult to personalize and compare. This makes it difficult to perform comparative studies and propose novel approaches. In this paper, we propose to solve these issues with the General Purpose Optimization Library (GPOL): a flexible and efficient multipurpose optimization library that covers a wide range of stochastic iterative search algorithms, through which flexible and modular implementation can allow for solving many different problem types from the fields of continuous and combinatorial optimization and supervised machine learning problem solving. Moreover, the library supports full-batch and mini-batch learning and allows carrying out computations on a CPU or GPU. The package is distributed under an MIT license. Source code, installation instructions, demos and tutorials are publicly available in our code hosting platform (the reference is provided in the Introduction).
10
Li, Zhiyuan, Xun Jian, Yue Wang, Yingxia Shao, and Lei Chen. "DAHA: Accelerating GNN Training with Data and Hardware Aware Execution Planning." Proceedings of the VLDB Endowment 17, no. 6 (February 2024): 1364–76. http://dx.doi.org/10.14778/3648160.3648176.
Full textAbstract:
Graph neural networks (GNNs) have been gaining a reputation for effective modeling of graph data. Yet, it is challenging to train GNNs efficiently. Many frameworks have been proposed but most of them suffer from high batch preparation cost and data transfer cost for mini-batch training. In addition, existing works have limitations on the device utilization pattern, which results in fewer opportunities for pipeline parallelism. In this paper, we present DAHA, a GNN training framework with data and hardware aware execution planning to accelerate end-to-end GNN training. We first propose a data and hardware aware cost model that is lightweight and gives accurate estimates on per-operation time cost for arbitrary input and hardware settings. Based on the cost model, we further explore the optimal execution plan for the data and hardware with three optimization strategies with pipeline parallelism: (1) group-based in-turn pipelining of batch preparation neural training to explore more optimization opportunities and prevent batch preparation bottlenecks; (2) data and hardware aware rewriting for intra-batch execution planning to improve computation efficiency and create more opportunities for pipeline parallelism; and (3) inter-batch scheduling to further boost the training efficiency. Extensive experiments demonstrate that DAHA can consistently and significantly accelerate end-to-end GNN training and generalize to different message-passing GNN models.
11
Panteleev, Andrei V., and Aleksandr V. Lobanov. "Application of Mini-Batch Metaheuristic Algorithms in Problems of Optimization of Deterministic Systems with Incomplete Information about the State Vector." Algorithms 14, no. 11 (November 14, 2021): 332. http://dx.doi.org/10.3390/a14110332.
Full textAbstract:
In this paper, we consider the application of the zero-order mini-batch optimization method in the problem of finding optimal control of a pencil of trajectories of nonlinear deterministic systems in the case of incomplete information about the state vector. The pencil of trajectories originates from a given set of initial states. To solve the problem, the structure of a feedback system is proposed, which contains models of the plant, measuring system, nonlinear state observer and control law of the fixed structure with unknown coefficients. The objective function proposed considers the quality of pencil of trajectories control, which is estimated by the average value of the Bolz functional over the given set of initial states. Unknown control laws of a plant and an observer are found in the form of expansions in terms of orthonormal systems of basis functions, which are specified on the set of possible states of a dynamical system. The original pencil of trajectories control problem is reduced to a global optimization problem, which is solved using the well-proven zero-order method, which uses a modified mini-batch approach in a random search procedure with adaptation. An algorithm for solving the problem is proposed. The satellite stabilization problem with incomplete information is solved.
12
Fan, Shengping, Jun Li, Linyong Li, and Zhigang Chu. "Noise Annoyance Prediction of Urban Substation Based on Transfer Learning and Convolutional Neural Network." Energies 15, no. 3 (January 20, 2022): 749. http://dx.doi.org/10.3390/en15030749.
Full textAbstract:
The noise pollution caused by urban substations is an increasingly serious problem, as is the issue of local residents being disturbed by substation noise. To accurately assess the degree of noise annoyance caused by substations to surrounding residents, we established a noise annoyance prediction model based on transfer learning and a convolution neural network. Using the model, we took the noise spectrum as the input, the subjective evaluation result as the target output, and the AlexNet network model with a modified output layer and corresponding parameters as the pre-training model. In a fixed learning rate and epoch setting, the influence of different mini-batch size values on the prediction accuracy of the model was compared and analyzed. The results showed that when the mini-batch size was set to 4, 8, 16, and 32, all the data sets had convergence after 90 iterations. The root mean square error (RMSE) of all validation sets was lower than 0.355, and the loss of all validation sets was lower than 0.067. As the mini-batch size increased, the RMSE, loss, and mean absolute error (MAE) of the verification set gradually increased, while the number of iterations and the training duration decreased gradually. In this test, a mini-batch size value of four was appropriate. The resultant convolutional neural network model showed high accuracy and robustness, and the error between the prediction result and the subjective evaluation result was between 2% and 7%. The model comprehensively reflects the objective metrics affecting subjective perception, and accurately describes the subjective perception of urban substation noise on human ears.
13
Messaoud, Seifeddine, Abbas Bradai, and Emmanuel Moulay. "Online GMM Clustering and Mini-Batch Gradient Descent Based Optimization for Industrial IoT 4.0." IEEE Transactions on Industrial Informatics 16, no. 2 (February 2020): 1427–35. http://dx.doi.org/10.1109/tii.2019.2945012.
Full text
14
Lin, Zhenwei, Jingfan Xia, Qi Deng, and Luo Luo. "Decentralized Gradient-Free Methods for Stochastic Non-smooth Non-convex Optimization." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 16 (March 24, 2024): 17477–86. http://dx.doi.org/10.1609/aaai.v38i16.29697.
Full textAbstract:
We consider decentralized gradient-free optimization of minimizing Lipschitz continuous functions that satisfy neither smoothness nor convexity assumption. We propose two novel gradient-free algorithms, the Decentralized Gradient-Free Method (DGFM) and its variant, the Decentralized Gradient-Free Method+ (DGFM+). Based on the techniques of randomized smoothing and gradient tracking, DGFM requires the computation of the zeroth-order oracle of a single sample in each iteration, making it less demanding in terms of computational resources for individual computing nodes. Theoretically, DGFM achieves a complexity of O(d^(3/2)δ^(-1)ε^(-4)) for obtaining a (δ,ε)-Goldstein stationary point. DGFM+, an advanced version of DGFM, incorporates variance reduction to further improve the convergence behavior. It samples a mini-batch at each iteration and periodically draws a larger batch of data, which improves the complexity to O(d^(3/2)δ^(-1)ε^(-3)). Moreover, experimental results underscore the empirical advantages of our proposed algorithms when applied to real-world datasets.
15
Yang, Wei, Qiheng Yuan, Yongli Wang, Fei Zheng, Xin Shi, and Yi Li. "Carbon Emission Forecasting Study Based on Influence Factor Mining and Mini-Batch Stochastic Gradient Optimization." Energies 17, no. 1 (December 29, 2023): 188. http://dx.doi.org/10.3390/en17010188.
Full textAbstract:
With the increasing prominence of the global carbon emission problem, the accurate prediction of carbon emissions has become an increasingly urgent need. Existing carbon emission prediction methods have the problems of slow calculation speed, inaccurate prediction, and insufficient deep mining of influencing factors when dealing with large-scale data. In this study, a comprehensive carbon emission prediction method is proposed. Firstly, multiple influencing factors including economic factors and demographic factors are considered, and a pathway analysis method is introduced to mine the long-term relationship between these factors and carbon emissions. Then, indirect influence terms are added to the multiple regression equation, and the variable is used to represent the indirect influence relationship. Finally, this study proposes the PCA-PA-MBGD method, which applies the results of principal component analysis to the pathway analysis. By reducing the data dimensions and extracting the main influencing factors, and optimizing the carbon emission prediction model by using a mini-batch stochastic gradient descent algorithm, the results show that this method can process a large amount of data quickly and efficiently, and realize an accurate prediction of carbon emissions. This provides strong support for solving the carbon emission problem and offers new ideas and methods for future related research.
16
Huang, Wendi. "Implementation of Parallel Optimization Algorithms for NLP: Mini-batch SGD, SGD with Momentum, AdaGrad Adam." Applied and Computational Engineering 81, no. 1 (November 8, 2024): 226–33. http://dx.doi.org/10.54254/2755-2721/81/20241146.
Full textAbstract:
Abstract. With the rapid development of machine learning technology, optimization algorithms and optimizers have become key to the development of related technologies contemporarily. Models need the help of optimizers to meet other performance indicators while saving computing resources. This research focuses on comparisons between optimizers, in the context of text sentiment classification tasks. The optimizers mainly compared in this article are mini batch SGD, momentum SGD, Adagrad and Adam. Through comparative experiments, it was found that SGD and its variants have a high dependence on the initial learning rate setting, while the performance of Adagrad and Adam is relatively balanced. Although the training time of Adagrad is shorter than that of Adam, its principal formula has flaws, which are not reflected in this task. The conclusions drawn in this article through comparison can point out the advantages and disadvantages of each optimizer, and can help realize better optimizers in subsequent research.
17
Li, Jing, Xiaorun Li, and Liaoying Zhao. "Unmixing of large-scale hyperspectral data based on projected mini-batch gradient descent." International Journal of Wavelets, Multiresolution and Information Processing 15, no. 06 (November 2017): 1750059. http://dx.doi.org/10.1142/s021969131750059x.
Full textAbstract:
The minimization problem of reconstruction error over large hyperspectral image data is one of the most important problems in unsupervised hyperspectral unmixing. A variety of algorithms based on nonnegative matrix factorization (NMF) have been proposed in the literature to solve this minimization problem. One popular optimization method for NMF is the projected gradient descent (PGD). However, as the algorithm must compute the full gradient on the entire dataset at every iteration, the PGD suffers from high computational cost in the large-scale real hyperspectral image. In this paper, we try to alleviate this problem by introducing a mini-batch gradient descent-based algorithm, which has been widely used in large-scale machine learning. In our method, the endmember can be updated pixel set by pixel set while abundance can be updated band set by band set. Thus, the computational cost is lowered to a certain extent. The performance of the proposed algorithm is quantified in the experiment on synthetic and real data.
18
Song, Hwa Jeon, Ho Young Jung, and Jeon Gue Park. "Implementation of CNN in the view of mini-batch DNN training for efficient second order optimization." Journal of the Korean society of speech sciences 8, no. 2 (June 30, 2016): 23–30. http://dx.doi.org/10.13064/ksss.2016.8.2.023.
Full text
19
Batista-Silva, João, Diana Gomes, Jorge Barroca-Ferreira, Eugénia Gallardo, Ângela Sousa, and Luís A. Passarinha. "Specific Six-Transmembrane Epithelial Antigen of the Prostate 1 Capture with Gellan Gum Microspheres: Design, Optimization and Integration." International Journal of Molecular Sciences 24, no. 3 (January 18, 2023): 1949. http://dx.doi.org/10.3390/ijms24031949.
Full textAbstract:
This work demonstrates the potential of calcium- and nickel-crosslinked Gellan Gum (GG) microspheres to capture the Six-Transmembrane Epithelial Antigen of the Prostate 1 (STEAP1) directly from complex Komagataella pastoris mini-bioreactor lysates in a batch method. Calcium-crosslinked microspheres were applied in an ionic exchange strategy, by manipulation of pH and ionic strength, whereas nickel-crosslinked microspheres were applied in an affinity strategy, mirroring a standard immobilized metal affinity chromatography. Both formulations presented small diameters, with appreciable crosslinker content, but calcium-crosslinked microspheres were far smoother. The most promising results were obtained for the ionic strategy, wherein calcium-crosslinked GG microspheres were able to completely bind 0.1% (v/v) DM solubilized STEAP1 in lysate samples (~7 mg/mL). The target protein was eluted in a complexed state at pH 11 with 500 mM NaCl in 10 mM Tris buffer, in a single step with minimal losses. Coupling the batch clarified sample with a co-immunoprecipitation polishing step yields a sample of monomeric STEAP1 with a high degree of purity. For the first time, we demonstrate the potential of a gellan batch method to function as a clarification and primary capture method towards STEAP1, a membrane protein, simplifying and reducing the costs of standard purification workflows.
20
Kim, Yonghoon, and and Mokdong Chung. "An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning." Electronics 8, no. 11 (November 1, 2019): 1267. http://dx.doi.org/10.3390/electronics8111267.
Full textAbstract:
In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.
21
Kazei, Vladimir, Hong Liang, and Ali AlDawood. "Acquisition and near-surface impacts on VSP mini-batch FWI and RTM imaging in desert environment." Leading Edge 42, no. 3 (March 2023): 165–72. http://dx.doi.org/10.1190/tle42030165.1.
Full textAbstract:
The SEG Advanced Modeling (SEAM) Arid benchmark model was designed to simulate an extremely heterogeneous low-velocity near surface (NS), which is typical of desert environments and typically not well characterized or imaged. Imaging of land seismic data is highly sensitive to errors in the NS velocity model. Vertical seismic profiling (VSP) partly alleviates the impact of the NS as the receivers are located at depth in the borehole. Deep learning (DL) offers a flexible optimization framework for full-waveform inversion (FWI), often outperforming typically used optimization methods. We investigate the quality of images that can be obtained from SEAM Arid VSP data by acoustic mini-batch reverse time migration (RTM) and full-waveform imaging. First, we focus on the effects of seismic vibrator and receiver array positioning and imperfect knowledge of the NS model when inverting 2D acoustic data. FWI imaging expectedly and consistently outperforms RTM in our tests. We find that the acquisition density is critical for RTM imaging and less so for FWI, while NS model accuracy is critical for FWI and has less effect on RTM imaging. Distributed acoustic sensing along the full length of the well provides noticeable improvement over a limited aperture array of geophones in imaging deep targets in both RTM and FWI imaging scenarios. Finally, we compare DL-based FWI imaging with inverse scattering RTM using the upgoing wavefield from the original SEAM data. Use of significantly more realistic 3D elastic physics for the simulated data generation and simple 2D acoustic inversion engine makes our inverse problem more realistic. We observe that FWI imaging in this case produces an image with fewer artifacts.
22
Panteleev, A. V., and A. V. Lobanov. "The mini-batch adaptive method of random search (MAMRS) for parameters optimization in the tracking control problem." IOP Conference Series: Materials Science and Engineering 927 (September 26, 2020): 012025. http://dx.doi.org/10.1088/1757-899x/927/1/012025.
Full text
23
Chakraborty, Arya. "Perceptron Collaborative Filtering." International Journal for Research in Applied Science and Engineering Technology 11, no. 2 (February 28, 2023): 437–47. http://dx.doi.org/10.22214/ijraset.2023.49044.
Full textAbstract:
Abstract: While multivariate logistic regression classifiers are a great way of implementing collaborative filtering - a method of making automatic predictions about the interests of a user by collecting preferences or taste information from many other users, we can also achieve similar results using neural networks. A recommender system is a subclass of information filtering system that provide suggestions for items that are most pertinent to a particular user. A perceptron or a neural network is a machine learning model designed for fitting complex datasets using backpropagation and gradient descent. When coupled with advanced optimization techniques, the model may prove to be a great substitute for classical logistic classifiers. The optimizations include feature scaling, mean normalization, regularization, hyperparameter tuning and using stochastic/mini-batch gradient descent instead of regular gradient descent. In this use case, we will use the perceptron in the recommender system to fit the parameters i.e., the data from a multitude of users and use it to predict the preference/interest of a particular user
24
Ao, Wenqi, Wenbin Li, and Jianliang Qian. "A data and knowledge driven approach for SPECT using convolutional neural networks and iterative algorithms." Journal of Inverse and Ill-posed Problems 29, no. 4 (March 26, 2021): 543–55. http://dx.doi.org/10.1515/jiip-2020-0056.
Full textAbstract:
Abstract We propose a data and knowledge driven approach for SPECT by combining a classical iterative algorithm of SPECT with a convolutional neural network. The classical iterative algorithm, such as ART and ML-EM, is employed to provide the model knowledge of SPECT. A modified U-net is then connected to exploit further features of reconstructed images and data sinograms of SPECT. We provide mathematical formulations for the architecture of the proposed networks. The networks are trained by supervised learning using the technique of mini-batch optimization. We apply the trained networks to the problems of simulated lung perfusion imaging and simulated myocardial perfusion imaging, and numerical results demonstrate their effectiveness of reconstructing source images from noisy data measurements.
25
Kalaiselvi, K., and M. Kasthuri. "Tuning VGG19 hyperparameters for improved pneumonia classification." Scientific Temper 15, no. 02 (June 15, 2024): 2231–37. http://dx.doi.org/10.58414/scientifictemper.2024.15.2.36.
Full textAbstract:
This research focuses on the classification of chest X-ray (CXR) images using powerful VGG19 convolutional neural network (CNN)architecture. The classification task involves distinguishing between various chest conditions present in X-ray images, with the aim of assisting medical professionals in achieving accurate and efficient diagnoses. This research work explores the use of the VGG19 model for classifying CXR images using three optimization algorithms: Stochastic gradient descent with momentum (SGDM), root mean square propagation (RMSprop), and adaptive moment estimation (Adam). This study investigates the impact of various factors on hyperparameter adjustments, including a learning rate (LR), mini-batch size (MBS) and training epochs. Additionally, two dropout layers are introduced for weight decay with an L2 factor, and data augmentation techniques are applied with various activation functions. This study not only helps optimize for medical image analysis but also offers valuable insights into the comparative efficacy of popular optimization algorithms in deep learning (DL) applications
26
Mančev, Dejan, and Branimir Todorović. "A primal sub-gradient method for structured classification with the averaged sum loss." International Journal of Applied Mathematics and Computer Science 24, no. 4 (December 1, 2014): 917–30. http://dx.doi.org/10.2478/amcs-2014-0067.
Full textAbstract:
Abstract We present a primal sub-gradient method for structured SVM optimization defined with the averaged sum of hinge losses inside each example. Compared with the mini-batch version of the Pegasos algorithm for the structured case, which deals with a single structure from each of multiple examples, our algorithm considers multiple structures from a single example in one update. This approach should increase the amount of information learned from the example. We show that the proposed version with the averaged sum loss has at least the same guarantees in terms of the prediction loss as the stochastic version. Experiments are conducted on two sequence labeling problems, shallow parsing and part-of-speech tagging, and also include a comparison with other popular sequential structured learning algorithms.
27
Bilal, Muhammad Atif, Yongzhi Wang, Yanju Ji, Muhammad Pervez Akhter, and Hengxi Liu. "Earthquake Detection Using Stacked Normalized Recurrent Neural Network (SNRNN)." Applied Sciences 13, no. 14 (July 12, 2023): 8121. http://dx.doi.org/10.3390/app13148121.
Full textAbstract:
Earthquakes threaten people, homes, and infrastructure. Earthquake detection is a complex task because it does not show any specific pattern, unlike object detection from images. Convolutional neural networks have been widely used for earthquake detection but have problems like vanishing gradients, exploding, and parameter optimization. The ensemble learning approach combines multiple models, each of which attempts to compensate for the shortcomings of the others to enhance performance. This article proposes an ensemble learning model based on a stacked normalized recurrent neural network (SNRNN) for earthquake detection. The proposed model uses three recurrent neural network models (RNN, GRU, and LSTM) with batch normalization and layer normalization. After preprocessing the waveform data, the RNN, GRU, and LSTM extract the feature map sequentially. Batch normalization and layer normalization methods take place in mini-batches and input layers for stable and faster training of the model and improving its performance. We trained and tested the proposed model on 6574 events from 2000 to 2018 (18 years) in Turkey, a highly targeted region. The SNRNN achieves RMSE values of 3.16 and 3.24 for magnitude and depth detection. The SNRNN model outperforms the three baseline models, as seen by their low RMSE values.
28
Ghosh, Bishwamittra, Dmitry Malioutov, and Kuldeep S. Meel. "Efficient Learning of Interpretable Classification Rules." Journal of Artificial Intelligence Research 74 (August 30, 2022): 1823–63. http://dx.doi.org/10.1613/jair.1.13482.
Full textAbstract:
Machine learning has become omnipresent with applications in various safety-critical domains such as medical, law, and transportation. In these domains, high-stake decisions provided by machine learning necessitate researchers to design interpretable models, where the prediction is understandable to a human. In interpretable machine learning, rule-based classifiers are particularly effective in representing the decision boundary through a set of rules comprising input features. Examples of such classifiers include decision trees, decision lists, and decision sets. The interpretability of rule-based classifiers is in general related to the size of the rules, where smaller rules are considered more interpretable. To learn such a classifier, the brute-force direct approach is to consider an optimization problem that tries to learn the smallest classification rule that has close to maximum accuracy. This optimization problem is computationally intractable due to its combinatorial nature and thus, the problem is not scalable in large datasets. To this end, in this paper we study the triangular relationship among the accuracy, interpretability, and scalability of learning rule-based classifiers. The contribution of this paper is an interpretable learning framework IMLI, that is based on maximum satisfiability (MaxSAT) for synthesizing classification rules expressible in proposition logic. IMLI considers a joint objective function to optimize the accuracy and the interpretability of classification rules and learns an optimal rule by solving an appropriately designed MaxSAT query. Despite the progress of MaxSAT solving in the last decade, the straightforward MaxSAT-based solution cannot scale to practical classification datasets containing thousands to millions of samples. Therefore, we incorporate an efficient incremental learning technique inside the MaxSAT formulation by integrating mini-batch learning and iterative rule-learning. The resulting framework learns a classifier by iteratively covering the training data, wherein in each iteration, it solves a sequence of smaller MaxSAT queries corresponding to each mini-batch. In our experiments, IMLI achieves the best balance among prediction accuracy, interpretability, and scalability. For instance, IMLI attains a competitive prediction accuracy and interpretability w.r.t. existing interpretable classifiers and demonstrates impressive scalability on large datasets where both interpretable and non-interpretable classifiers fail. As an application, we deploy IMLI in learning popular interpretable classifiers such as decision lists and decision sets. The source code is available at https://github.com/meelgroup/mlic.
29
Oelgemöller, Michael, Norbert Hoffmann, and Oksana Shvydkiv. "From 'Lab & Light on a Chip' to Parallel Microflow Photochemistry." Australian Journal of Chemistry 67, no. 3 (2014): 337. http://dx.doi.org/10.1071/ch13591.
Full textAbstract:
Continuous-flow microreactors offer major advantages for photochemical applications. This mini-review summarizes the technological development of microflow devices in the Applied and Green Photochemistry Group at James Cook University, and its associates, from fixed microchips for microscale synthesis to flexible multicapillary systems for parallel photochemistry. Whereas the enclosed microchip offered high space–time-yields, the open capillary-type reactor showed a greater potential for further modifications. Consequently, a 10-microcapillary reactor was constructed and used successfully for process optimization, reproducibility studies, scale-up, and library synthesis. To demonstrate the superiority of microflow photochemistry over conventional batch processes, the reactors were systematically evaluated using alcohol additions to furanones as model reactions. In all cases, the microreactor systems furnished faster conversions, improved product qualities, and higher yields. UVC-induced [2+2] cycloadditions of furanone with alkenes were exemplarily examined in a capillary reactor, thus proving the broad applicability of this reactor type.
30
Wei, Pengzhi, Yanqiu Li, Tie Li, Naiyuan Sheng, Enze Li, and Yiyu Sun. "Multi-Objective Defocus Robust Source and Mask Optimization Using Sensitive Penalty." Applied Sciences 9, no. 10 (May 27, 2019): 2151. http://dx.doi.org/10.3390/app9102151.
Full textAbstract:
The continuous decrease in the size of lithographic technology nodes has led to the development of source and mask optimization (SMO) and also to the control of defocus becoming stringent in the actual lithography process. Due to multi-factor impact, defocusing is always changeable and uncertain in the real exposure process. But conventional SMO assumes the lithography system is ideal, which only compensates the optical proximity effect (OPE) in the best focus plane. Therefore, to solve the inverse lithography problem with more uniformity of pattern in different defocus variations, we proposed a defocus robust SMO (DRSMO) approach that is driven by a defocus sensitivity penalty function for the first time. This multi-objective optimization samples a wide range of defocus disturbances and it can be proceeded by the mini-batch gradient descent (MBGD) algorithm effectively. The simulation results showed that a more robust defocus source and mask can be designed through DRSMO optimization. The defocus sensitivity factor sβ maximally decreased 63.5% compared to conventional SMO, and due to the low error sensitivity and the depth of defocus (DOF), the process window (PW) was further enlarged effectively. Compared to conventional SMO, the exposure latitude (EL) maximally increased from 4.5% to 10.5% and DOF maximally increased 54.5% (EL = 5%), which proved the validity of the DRSMO method in improving the focusing performance.
31
Liu, Pingping, Guixia Gou, Xue Shan, Dan Tao, and Qiuzhan Zhou. "Global Optimal Structured Embedding Learning for Remote Sensing Image Retrieval." Sensors 20, no. 1 (January 4, 2020): 291. http://dx.doi.org/10.3390/s20010291.
Full textAbstract:
A rich line of works focus on designing elegant loss functions under the deep metric learning (DML) paradigm to learn a discriminative embedding space for remote sensing image retrieval (RSIR). Essentially, such embedding space could efficiently distinguish deep feature descriptors. So far, most existing losses used in RSIR are based on triplets, which have disadvantages of local optimization, slow convergence and insufficient use of similarity structure in a mini-batch. In this paper, we present a novel DML method named as global optimal structured loss to deal with the limitation of triplet loss. To be specific, we use a softmax function rather than a hinge function in our novel loss to realize global optimization. In addition, we present a novel optimal structured loss, which globally learn an efficient deep embedding space with mined informative sample pairs to force the positive pairs within a limitation and push the negative ones far away from a given boundary. We have conducted extensive experiments on four public remote sensing datasets and the results show that the proposed global optimal structured loss with pairs mining scheme achieves the state-of-the-art performance compared with the baselines.
32
Liu, Fangyu, Rongtian Ye, Xun Wang, and Shuaipeng Li. "HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 07 (April 3, 2020): 11563–71. http://dx.doi.org/10.1609/aaai.v34i07.6823.
Full textAbstract:
The hubness problem widely exists in high-dimensional embedding space and is a fundamental source of error for cross-modal matching tasks. In this work, we study the emergence of hubs in Visual Semantic Embeddings (VSE) with application to text-image matching. We analyze the pros and cons of two widely adopted optimization objectives for training VSE and propose a novel hubness-aware loss function (Hal) that addresses previous methods' defects. Unlike (Faghri et al. 2018) which simply takes the hardest sample within a mini-batch, Hal takes all samples into account, using both local and global statistics to scale up the weights of “hubs”. We experiment our method with various configurations of model architectures and datasets. The method exhibits exceptionally good robustness and brings consistent improvement on the task of text-image matching across all settings. Specifically, under the same model architectures as (Faghri et al. 2018) and (Lee et al. 2018), by switching only the learning objective, we report a maximum R@1 improvement of 7.4% on MS-COCO and 8.3% on Flickr30k.1
33
Brouard, Céline, Antoine Bassé, Florence d’Alché-Buc, and Juho Rousu. "Improved Small Molecule Identification through Learning Combinations of Kernel Regression Models." Metabolites 9, no. 8 (August 1, 2019): 160. http://dx.doi.org/10.3390/metabo9080160.
Full textAbstract:
In small molecule identification from tandem mass (MS/MS) spectra, input–output kernel regression (IOKR) currently provides the state-of-the-art combination of fast training and prediction and high identification rates. The IOKR approach can be simply understood as predicting a fingerprint vector from the MS/MS spectrum of the unknown molecule, and solving a pre-image problem to find the molecule with the most similar fingerprint. In this paper, we bring forward the following improvements to the IOKR framework: firstly, we formulate the IOKRreverse model that can be understood as mapping molecular structures into the MS/MS feature space and solving a pre-image problem to find the molecule whose predicted spectrum is the closest to the input MS/MS spectrum. Secondly, we introduce an approach to combine several IOKR and IOKRreverse models computed from different input and output kernels, called IOKRfusion. The method is based on minimizing structured Hinge loss of the combined model using a mini-batch stochastic subgradient optimization. Our experiments show a consistent improvement of top-k accuracy both in positive and negative ionization mode data.
34
AL-Hawamleh, Ahmad M. "Advanced Spam Filtering in Electronic Mail Using Hybrid the Mini Batch K-Means Normalized Mutual Information Feature Elimination with Elephant Herding Optimization Technique." International Journal of Computing and Digital Systems 13, no. 1 (May 30, 2023): 1409–22. http://dx.doi.org/10.12785/ijcds/1301114.
Full text
35
Tsalera, Eleni, Andreas Papadakis, and Maria Samarakou. "Comparison of Pre-Trained CNNs for Audio Classification Using Transfer Learning." Journal of Sensor and Actuator Networks 10, no. 4 (December 10, 2021): 72. http://dx.doi.org/10.3390/jsan10040072.
Full textAbstract:
The paper investigates retraining options and the performance of pre-trained Convolutional Neural Networks (CNNs) for sound classification. CNNs were initially designed for image classification and recognition, and, at a second phase, they extended towards sound classification. Transfer learning is a promising paradigm, retraining already trained networks upon different datasets. We selected three ‘Image’- and two ‘Sound’-trained CNNs, namely, GoogLeNet, SqueezeNet, ShuffleNet, VGGish, and YAMNet, and applied transfer learning. We explored the influence of key retraining parameters, including the optimizer, the mini-batch size, the learning rate, and the number of epochs, on the classification accuracy and the processing time needed in terms of sound preprocessing for the preparation of the scalograms and spectrograms as well as CNN training. The UrbanSound8K, ESC-10, and Air Compressor open sound datasets were employed. Using a two-fold criterion based on classification accuracy and time needed, we selected the ‘champion’ transfer-learning parameter combinations, discussed the consistency of the classification results, and explored possible benefits from fusing the classification estimations. The Sound CNNs achieved better classification accuracy, reaching an average of 96.4% for UrbanSound8K, 91.25% for ESC-10, and 100% for the Air Compressor dataset.
36
Chen, Yung-Ting, Yao-Liang Chen, Yi-Yun Chen, Yu-Ting Huang, Ho-Fai Wong, Jiun-Lin Yan, and Jiun-Jie Wang. "Deep Learning–Based Brain Computed Tomography Image Classification with Hyperparameter Optimization through Transfer Learning for Stroke." Diagnostics 12, no. 4 (March 25, 2022): 807. http://dx.doi.org/10.3390/diagnostics12040807.
Full textAbstract:
Brain computed tomography (CT) is commonly used for evaluating the cerebral condition, but immediately and accurately interpreting emergent brain CT images is tedious, even for skilled neuroradiologists. Deep learning networks are commonly employed for medical image analysis because they enable efficient computer-aided diagnosis. This study proposed the use of convolutional neural network (CNN)-based deep learning models for efficient classification of strokes based on unenhanced brain CT image findings into normal, hemorrhage, infarction, and other categories. The included CNN models were CNN-2, VGG-16, and ResNet-50, all of which were pretrained through transfer learning with various data sizes, mini-batch sizes, and optimizers. Their performance in classifying unenhanced brain CT images was tested thereafter. This performance was then compared with the outcomes in other studies on deep learning–based hemorrhagic or ischemic stroke diagnoses. The results revealed that among our CNN-2, VGG-16, and ResNet-50 analyzed by considering several hyperparameters and environments, the CNN-2 and ResNet-50 outperformed the VGG-16, with an accuracy of 0.9872; however, ResNet-50 required a longer time to present the outcome than did the other networks. Moreover, our models performed much better than those reported previously. In conclusion, after appropriate hyperparameter optimization, our deep learning–based models can be applied to clinical scenarios where neurologist or radiologist may need to verify whether their patients have a hemorrhage stroke, an infarction, and any other symptom.
37
Vassileva, Maria, Eligio Malusà, Lidia Sas-Paszt, Pawel Trzcinski, Antonia Galvez, Elena Flor-Peregrin, Stefan Shilev, Loredana Canfora, Stefano Mocali, and Nikolay Vassilev. "Fermentation Strategies to Improve Soil Bio-Inoculant Production and Quality." Microorganisms 9, no. 6 (June 9, 2021): 1254. http://dx.doi.org/10.3390/microorganisms9061254.
Full textAbstract:
The application of plant beneficial microorganisms has been widely accepted as an efficient alternative to chemical fertilizers and pesticides. Isolation and selection of efficient microorganisms, their characterization and testing in soil-plant systems are well studied. However, the production stage and formulation of the final products are not in the focus of the research, which affects the achievement of stable and consistent results in the field. Recent analysis of the field of plant beneficial microorganisms suggests a more integrated view on soil inoculants with a special emphasis on the inoculant production process, including fermentation, formulation, processes, and additives. This mini-review describes the different groups of fermentation processes and their characteristics, bearing in mind different factors, both nutritional and operational, which affect the biomass/spores yield and microbial metabolite activity. The characteristics of the final products of fermentation process optimization strategies determine further steps of development of the microbial inoculants. Submerged liquid and solid-state fermentation processes, fed-batch operations, immobilized cell systems, and production of arbuscular mycorrhiza are presented and their advantages and disadvantages are discussed. Recommendations for further development of the fermentation strategies for biofertilizer production are also considered.
38
Wu, Ming-Yu, Yan Wu, Xin-Yi Yuan, Zhi-Hua Chen, Wei-Tao Wu, and Nadine Aubry. "Fast Prediction of Flow Field around Airfoils Based on Deep Convolutional Neural Network." Applied Sciences 12, no. 23 (November 25, 2022): 12075. http://dx.doi.org/10.3390/app122312075.
Full textAbstract:
We propose a steady-state aerodynamic data-driven method to predict the incompressible flow around airfoils of NACA (National Advisory Committee for Aeronautics) 0012-series. Using the Signed Distance Function (SDF) to parameterize the geometric and flow condition setups, the prediction core of the method is constructed essentially by a consecutive framework of a convolutional neural network (CNN) and a deconvolutional neural network (DCNN). Impact of training parameters on the behavior of the proposed CNN-DCNN model is studied, so that appropriate learning rate, mini-batch size, and random deactivation rate are specified. Tested by “unseen” airfoil geometries and far-field velocities, it is found that the prediction process is three orders of magnitudes faster than a corresponding Computational Fluid Dynamics (CFD) simulation, while relative errors are maintained lower than 1% on most of the sample points. The proposed model manages to capture the essential dynamics of the flow field, as its predictions correspond reasonably with the reconstructed field by proper orthogonal decomposition (POD). The performance and accuracy of the proposed model indicate that the deep learning-based approach has great potential as a robust predictive tool for aerodynamic design and optimization.
39
Khunratchasana, Kheamparit, and Tassanan Treenuntharath. "Thai digit handwriting image classification with convolution neuron networks." Indonesian Journal of Electrical Engineering and Computer Science 27, no. 1 (July 1, 2022): 110. http://dx.doi.org/10.11591/ijeecs.v27.i1.pp110-117.
Full textAbstract:
This paper aims to determine the efficiency in classifying and recognizing Thai digit handwritten using convolutional neural networks (CNN). We created a new dataset called the Thai digit dataset. The performance test was divided into two parts: the first part determines the exact number of epochs, and the second part examines the occurrence of overfits in the model with Keras library's EarlyStoping() function, processed through Cloud Computing with Google Colaboratory, and used a Python programming language. The main parameters for the model were a dropout of 0.75, mini-batch size of 128, the learning rate of 0.0001, and using an Adam optimizer. This study found the model's predictive accuracy was 96.88 and the loss was 0.1075. The results showed that using CNN in image classification and recognition. It has a high level of prediction efficiency. However, the parameters in the model must be adjusted accordingly.
40
Li, Jie, Boyu Zhao, Kai Wu, Zhicheng Dong, Xuerui Zhang, and Zhihao Zheng. "A Representation Generation Approach of Transmission Gear Based on Conditional Generative Adversarial Network." Actuators 10, no. 5 (April 23, 2021): 86. http://dx.doi.org/10.3390/act10050086.
Full textAbstract:
Gear reliability assessment of vehicle transmission has been a challenging issue of determining vehicle safety in the transmission industry due to a significant amount of classification errors with high-coupling gear parameters and insufficient high-density data. In terms of the preprocessing of gear reliability assessment, this paper presents a representation generation approach based on generative adversarial networks (GAN) to advance the performance of reliability evaluation as a classification problem. First, with no need for complex modeling and massive calculations, a conditional generative adversarial net (CGAN) based model is established to generate gear representations through discovering inherent mapping between features with gear parameters and gear reliability. Instead of producing intact samples like other GAN techniques, the CGAN based model is designed to learn features of gear data. In this model, to raise the diversity of produced features, a mini-batch strategy of randomly sampling from the combination of raw and generated representations is used in the discriminator, instead of using all of the data features. Second, in order to overcome the unlabeled ability of CGAN, a Wasserstein labeling (WL) scheme is proposed to tag the created representations from our model for classification. Lastly, original and produced representations are fused to train classifiers. Experiments on real-world gear data from the industry indicate that the proposed approach outperforms other techniques on operational metrics.
41
Walls, Laura Ellen, José L. Martinez, and Leonardo Rios-Solis. "Enhancing Saccharomyces cerevisiae Taxane Biosynthesis and Overcoming Nutritional Stress-Induced Pseudohyphal Growth." Microorganisms 10, no. 1 (January 13, 2022): 163. http://dx.doi.org/10.3390/microorganisms10010163.
Full textAbstract:
The recent technological advancements in synthetic biology have demonstrated the extensive potential socio-economic benefits at laboratory scale. However, translations of such technologies to industrial scale fermentations remains a major bottleneck. The existence and lack of understanding of the major discrepancies in cultivation conditions between scales often leads to the selection of suboptimal bioprocessing conditions, crippling industrial scale productivity. In this study, strategic design of experiments approaches were coupled with state-of-the-art bioreactor tools to characterize and overcome nutritional stress for the enhanced production of precursors to the blockbuster chemotherapy drug, Taxol, in S. cerevisiae cell factories. The batch-to-batch variation in yeast extract composition was found to trigger nutritional stress at a mini-bioreactor scale, resulting in profound changes in cellular morphology and the inhibition of taxane production. The cells shifted from the typical budding morphology into striking pseudohyphal cells. Doubling initial yeast extract and peptone concentrations (2×YP) delayed filamentous growth, and taxane accumulation improved to 108 mg/L. Through coupling a statistical definitive screening design approach with the state-of-the-art high-throughput micro-bioreactors, the total taxane titers were improved a further two-fold, compared to the 2×YP culture, to 229 mg/L. Filamentous growth was absent in nutrient-limited microscale cultures, underlining the complex and multifactorial nature of yeast stress responses. Validation of the optimal microscale conditions in 1L bioreactors successfully alleviated nutritional stress and improved the titers to 387 mg/L. Production of the key Taxol precursor, T5αAc, was improved two-fold to 22 mg/L compared to previous maxima. The present study highlights the importance of following an interdisciplinary approach combining synthetic biology and bioprocessing technologies for effective process optimization and scale-up.
42
K M, Prof Ramya, Pavan H, Darshan Gowda, Bhagavantray Hosamani, and Jagadeva A S. "MULTIMODAL BIOMETRIC IDENTIFICATION SYSTEM USING THE FUSION OF FINGERPRINT AND IRIS RECOGNITION WITH CNN APPROACH." International Journal of Engineering Applied Sciences and Technology 6, no. 8 (December 1, 2021): 213–20. http://dx.doi.org/10.33564/ijeast.2021.v06i08.036.
Full textAbstract:
Multimodal biometric systems are widely applied in many real-world applications because of its ability to accommodate variety of great limitations of unimodal biometric systems, including sensitivity to noise, population coverage, intra-class variability, nonuniversality, and vulnerability to spoofing. during this paper, an efficient and real-time multimodal biometric system is proposed supported building deep learning representations for images of both the correct and left irises of someone, and fusing the results obtained employing a ranking-level fusion method. The trained deep learning system proposed is named IrisConvNet whose architecture relies on a mix of Convolutional Neural Network (CNN) and Softmax classifier to extract discriminative features from the input image with none domain knowledge where the input image represents the localized iris region and so classify it into one amongst N classes. during this work, a discriminative CNN training scheme supported a mixture of back-propagation algorithm and mini-batch AdaGrad optimization method is proposed for weights updating and learning rate adaptation, respectively. additionally, other training strategies (e.g., dropout method, data augmentation) also are proposed so as to gauge different CNN architectures. The performance of the proposed system is tested on three public datasets collected under different conditions: SDUMLA-HMT, CASIA-IrisV3 Interval and IITD iris database
43
Ntakolia, Charis, and Dimitrios V. Lyridis. "Path Planning in the Case of Swarm Unmanned Surface Vehicles for Visiting Multiple Targets." Journal of Marine Science and Engineering 11, no. 4 (March 26, 2023): 719. http://dx.doi.org/10.3390/jmse11040719.
Full textAbstract:
In this study, we present a hybrid approach of Ant Colony Optimization algorithm (ACO) with fuzzy logic and clustering methods to solve multiobjective path planning problems in the case of swarm Unmanned Surface Vehicles (USVs). This study aims to further explore the performance of the ACO algorithm by integrating fuzzy logic in order to cope with the multiple contradicting objectives and generate quality solutions by in-parallel identifying the mission areas of each USV to reach the desired targets. The design of the operational areas for each USV in the swarm is performed by a comparative evaluation of three popular clustering algorithms: Mini Batch K-Means, Ward Clustering and Birch. Following the identification of the operational areas, the design of each USV path to perform the operation is performed based on the minimization of traveled distance and energy consumption, as well as the maximization of path smoothness. To solve this multiobjective path planning problem, a comparative evaluation is conducted among ACO and fuzzy inference systems, Mamdani (ACO-Mamdani) and Takagi–Sugeno–Kang (ACO-TSK). The results show that depending on the needs of the application, each methodology can contribute, respectively. ACO-Mamdani generates better paths, but ACO-TSK presents higher computation efficiency.
44
Ghimire, Nawaraj. "A Recognition System for Devanagari Handwritten Digits Using CNN." American Journal of Electrical and Computer Engineering 8, no. 2 (July 29, 2024): 21–30. http://dx.doi.org/10.11648/j.ajece.20240802.11.
Full textAbstract:
A Recognition System for Devanagari Handwritten Digits using CNN, a novel approach to recognizing transcribed digits in the Devanagari script using Convolutional Neural Networks (CNN). This framework represents a significant contribution to the field of pattern recognition and language processing objective of the research project is to perform a literature review, identify an algorithm for a digits recognition system implement the Devanagari digits recognition system for educational activities. In the first phase, a dataset of 150 transcribed digit images is curated, allocating 75% for training (113 images) and 25% for validation (37 images). A Convolutional Neural Network (CNN) is designed with five convolutional layers, each utilizing 3 × 3 filters with 16, 32, 64, 128, and 128 feature maps, respectively. The experiments conducted involve varying the number of epochs, with results captured at 5, 10, 20, and 100 epochs. This comprehensive evaluation aims to understand the model's convergence and performance over different training durations. The outcomes of this phase contribute to the fine-tuning and optimization of the model for subsequent phases. In the second phase, the dataset is expanded to 100*10 (1000) images, each resized to 28 × 28 pixels through cropping. The CNN architecture remains consistent, with the previously determined layer configuration. Similar experiments are conducted, assessing the model's performance over 5, 10, 20, and 100 epochs. This model with a data size of 1000 demonstrates superior accuracy (100% on mini-batches) compared to the 150 model, with consistently high validation accuracy, while both models exhibit decreasing trends in mini-batch and validation losses, favoring the larger dataset, and maintaining a constant learning rate at 0.0100, albeit with a slightly longer time elapsed for each epoch due to the increased data size. 98.37398 accuracy in the phase 2 experiment in 100 epochs. Similar research and contributions and Devanagari’s character and word recognition system.
45
Lu, Zhenglong, Jie Li, Xi Zhang, Kaiqiang Feng, Xiaokai Wei, Debiao Zhang, Jing Mi, and Yang Liu. "A New In-Flight Alignment Method with an Application to the Low-Cost SINS/GPS Integrated Navigation System." Sensors 20, no. 2 (January 16, 2020): 512. http://dx.doi.org/10.3390/s20020512.
Full textAbstract:
The optimization-based alignment (OBA) methods, which are implemented by the optimal attitude estimation using vector observations—also called double-vectors—have proven to be effective at solving the in-flight alignment (IFA) problem. However, the traditional OBA methods are not applicable for the low-cost strap-down inertial navigation system (SINS) since the error of double-vectors will be accumulated over time due to the substantial drift of micro-electronic- mechanical system (MEMS) gyroscope. Moreover, the existing optimal estimation method is subject to a large computation burden, which results in a low alignment speed. To address these issues, in this article we propose a new fast IFA method based on modified double-vectors construction and the gradient descent method. To be specific, the modified construction method is implemented by reducing the integration interval and identifying the gyroscope bias during the construction procedure, which improves the accuracy of double-vectors and IFA; the gradient descent scheme is adopted to estimate the optimal attitude of alignment without complex matrix operation, which results in the improvement of alignment speed. The effect of different sizes of mini-batch on the performance of the gradient descent method is also discussed. Extensive simulations and vehicle experiments demonstrate that the proposed method has better accuracy and faster alignment speed than the related traditional methods for the low-cost SINS/global positioning system (GPS) integrated navigation system
46
Wikamulia, Nathaniel, and Sani Muhamad Isa. "Predictive business intelligence dashboard for food and beverage business." Bulletin of Electrical Engineering and Informatics 12, no. 5 (October 1, 2023): 3016–26. http://dx.doi.org/10.11591/eei.v12i5.5162.
Full textAbstract:
This research was conducted to provide an example of predictive business intelligence (BI) dashboard implementation for the food and beverage business (businesses that sell fast-expired goods). This research was conducted using data from a bakery's transactional database. The data are used to perform demand forecasting using extreme gradient boosting (XGBoost), and recency, frequency, and monetary value (RFM) analysis using mini batch k-means (MBKM). The data are processed and displayed in a BI dashboard created using Microsoft Power BI. The XGBoost model created resulted in a root mean square error (RMSE) value of 0.188 and an R2 score of 0.931. The MBKM model created resulted in a Dunn index value of 0.4264, a silhouette score value of 0.4421, and a Davies-Bouldin index value of 0.8327. After the BI dashboard is evaluated by the end user using a questionnaire, the BI dashboard gets a final score of 4.77 out of 5. From the BI dashboard evaluation, it was concluded that the predictive BI dashboard succeeded in helping the analysis process in the bakery business by: accelerating the decision-making process, implementing a data-driven decision-making system, and helping businesses discover new insights.
47
Huang, Lingbo, Yushi Chen, Xin He, and Pedram Ghamisi. "Supervised Contrastive Learning-Based Classification for Hyperspectral Image." Remote Sensing 14, no. 21 (November 2, 2022): 5530. http://dx.doi.org/10.3390/rs14215530.
Full textAbstract:
Recently, deep learning methods, especially convolutional neural networks (CNNs), have achieved good performance for hyperspectral image (HSI) classification. However, due to limited training samples of HSIs and the high volume of trainable parameters in deep models, training deep CNN-based models is still a challenge. To address this issue, this study investigates contrastive learning (CL) as a pre-training strategy for HSI classification. Specifically, a supervised contrastive learning (SCL) framework, which pre-trains a feature encoder using an arbitrary number of positive and negative samples in a pair-wise optimization perspective, is proposed. Additionally, three techniques for better generalization in the case of limited training samples are explored in the proposed SCL framework. First, a spatial–spectral HSI data augmentation method, which is composed of multiscale and 3D random occlusion, is designed to generate diverse views for each HSI sample. Second, the features of the augmented views are stored in a queue during training, which enriches the positives and negatives in a mini-batch and thus leads to better convergence. Third, a multi-level similarity regularization method (MSR) combined with SCL (SCL–MSR) is proposed to regularize the similarities of the data pairs. After pre-training, a fully connected layer is combined with the pre-trained encoder to form a new network, which is then fine-tuned for final classification. The proposed methods (SCL and SCL–MSR) are evaluated on four widely used hyperspectral datasets: Indian Pines, Pavia University, Houston, and Chikusei. The experiment results show that the proposed SCL-based methods provide competitive classification accuracy compared to the state-of-the-art methods.
48
Shaaf, Zakarya Farea, Muhammad Mahadi Abdul Jamil, Radzi Ambar, Ahmed Abdu Alattab, Anwar Ali Yahya, and Yousef Asiri. "Automatic Left Ventricle Segmentation from Short-Axis Cardiac MRI Images Based on Fully Convolutional Neural Network." Diagnostics 12, no. 2 (February 5, 2022): 414. http://dx.doi.org/10.3390/diagnostics12020414.
Full textAbstract:
Background: Left ventricle (LV) segmentation using a cardiac magnetic resonance imaging (MRI) dataset is critical for evaluating global and regional cardiac functions and diagnosing cardiovascular diseases. LV clinical metrics such as LV volume, LV mass and ejection fraction (EF) are frequently extracted based on the LV segmentation from short-axis MRI images. Manual segmentation to assess such functions is tedious and time-consuming for medical experts to diagnose cardiac pathologies. Therefore, a fully automated LV segmentation technique is required to assist medical experts in working more efficiently. Method: This paper proposes a fully convolutional network (FCN) architecture for automatic LV segmentation from short-axis MRI images. Several experiments were conducted in the training phase to compare the performance of the network and the U-Net model with various hyper-parameters, including optimization algorithms, epochs, learning rate, and mini-batch size. In addition, a class weighting method was introduced to avoid having a high imbalance of pixels in the classes of image’s labels since the number of background pixels was significantly higher than the number of LV and myocardium pixels. Furthermore, effective image conversion with pixel normalization was applied to obtain exact features representing target organs (LV and myocardium). The segmentation models were trained and tested on a public dataset, namely the evaluation of myocardial infarction from the delayed-enhancement cardiac MRI (EMIDEC) dataset. Results: The dice metric, Jaccard index, sensitivity, and specificity were used to evaluate the network’s performance, with values of 0.93, 0.87, 0.98, and 0.94, respectively. Based on the experimental results, the proposed network outperforms the standard U-Net model and is an advanced fully automated method in terms of segmentation performance. Conclusion: This proposed method is applicable in clinical practice for doctors to diagnose cardiac diseases from short-axis MRI images.
49
Prozur, Vitalii. "Architecture and Formal-mathematical Justification of Generative Adversarial Networks." Vìsnik Nacìonalʹnogo unìversitetu "Lʹvìvsʹka polìtehnìka". Serìâ Ìnformacìjnì sistemi ta merežì 15 (July 15, 2024): 15–22. http://dx.doi.org/10.23939/sisn2024.15.015.
Full textAbstract:
The purpose of the work is to analyze the features of generative adversarial networks. The object of research is the process of machine learning algorithmization. The subject of the research is mathematical methods used in the generation of semantically related text. This article explores the architecture and mathematical justification of such a type of generative models as generative adversarial networks. Generative adversarial networks are a powerful tool in the field of artificial intelligence, capable of generating realistic data, including photos, videos, sounds, etc. The architecture of generative competition defines its structure, the interaction of components and a general description of the learning process. Mathematical justification, in turn, includes a theoretical analysis of the principles, algorithms and functions underlying these networks. The article examines the general architecture of generative adversarial networks, examines each of its components (namely, the two main network models – generator and discriminator, their input and output data vectors) and its role in the operation of the algorithm. The author also defined the mathematical principles of generative adversarial networks, focusing on game theory and optimization methods (in particular, special attention is paid to minimax and maximin problems, zero-sum game, saddle points, Nash equilibrium) used in their study. The cost function and the process of deriving it using the Nash equilibrium in a zero-sum game for generative adversarial networks are described, and the learning algorithm using the method of stochastic gradient descent and the mini-batch approach in the form of a pseudocode, its iterations, is visualized network architecture. Finally, the conclusion that generative adversarial networks is an effective tool for creating realistic and believable data samples based on the use of elements of game theory is substantiated. Due to the high quality of generated data, generative adversarial networks can be used in various fields, including: cyber security, medicine, commerce, science, art, etc.
50
Budhijanto, Wiratni, Sholahuddin Al Ayyubi, and Khalid Abdul Latif. "Evaluasi Rangkaian Anaerobic Fluidized Bed Reactor (AFBR) dan Micro Bubble Generator (MBG) untuk Pengolahan Air Lindi Sampah." Jurnal Teknik Kimia Indonesia 18, no. 1 (January 14, 2020): 1. http://dx.doi.org/10.5614/jtki.2019.18.1.1.
Full textAbstract:
Abstrak. Timbulan air lindi adalah masalah serius pada tempat pengolahan sampah akhir (TPA) di Indonesia. Kandungan komponen organik pada sampah Indonesia yang mencapai 70-75% dari total timbulan sampah menyebabkan tingginya produksi lindi sebagai cairan hasil pembusukan. Studi ini bertujuan mengoptimalkan proses pembersihan air lindi dengan rangkaian proses anaerob yang diikuti dengan proses aerob pada skala mini pilot plant. Peruraian anaerobik dijalankan dalam anaerobic fluidized bed reactor (AFBR) dengan media imobilisasi mikroorganisme yang difluidisasi. Tahap selanjutnya adalah proses peruraian secara aerob dengan aerasi menggunakan micro bubble generator (MBG). Pilot plant yang didirikan di tempat pengolahan akhir (TPA) Piyungan di Yogyakarta ini terdiri atas AFBR dengan volume 500 L dan bak aerasi dengan MBG berukuran 500 L. Pengamatan data kualitas air (soluble chemical oxygen demand (sCOD) dan volatile fatty acid (VFA)) pada input/output AFBR dan input/output MBG serta volume biogas yang dihasilkan di AFBR dilakukan secara berkala selama 70 hari start-up di mana reaktor mulai dioperasikan secara kontinu setelah inokulasi secara batch dan 50 hari operasional pada kondisi steady state. Walaupun telah dioperasikan selama lebih dari sebulan, performa AFBR setelah tercapai kondisi steady state belum optimal karena baru mencapai kurang lebih 30% pengurangan kandungan senyawa organik. Performa yang lebih baik teramati pada proses aerob dengan aerasi menggunakan MBG. Proses tersebut berhasil menurunkan sCOD sampai 60%. Studi awal ini menunjukkan bahwa rangkaian AFBR dan MBG berpotensi untuk mengatasi masalah pencemaran air lindi di TPA. Optimalisasi kinerja unit ini terutama ditentukan oleh proses start-up yang dipengaruhi oleh teknik inokulasi. Kata Kunci: fluidisasi, imobilisasi mikrobia, lindi, peruraian aerob, peruraian anaerob, sampah. Abstract. Evaluation of Anaerobic Fluidized Bed Reactor (AFBR) and Micro Bubble Generator (MBG) for Landfill Leachate Treatment. Landfill leachate emission is a very serious problem in Indonesian landfill sites. High organic fraction in Indonesian garbage, which accounts for 70-75% of total municipal solid waste amount, emits high flow rate of leachate as the result of decay process. This study aims to optimize landfill leachate treatment by means of anaerobic process followed by aerobic process. The anaerobic digestion was carried out in AFBR in which microbial immobilization media was fluidized. The next stage was aerobic digestion by applying novel aeration technology using MBG. The pilot plant was installed in Piyungan Landfill Site in Yogyakarta, which consisted of 500 L AFBR and 500 L MBG units. Observation was conducted periodically for 70 days of start-up when the unit was operated continuously after batch inoculation followed by 50 days of steady-state operation. The measurement was taken as soluble chemical oxygen demand (sCOD) and volatile fatty acids (VFA) on the input/output of AFBR and input/output of MBG. The biogas volume production in the AFBR was also measured. AFBR performance was not optimal since even after achieving a steady state condition (for one-month operation), it could only reduce less than 30% organic content. A better performance was observed in the aerobic process where MBG was used for the aeration. It could reduce 60% of sCOD. This preliminary study showed that the coupling of AFBR and MBG units is potential for landfill leachate treatment. Optimization of this unit depended on the inoculation technique during the start-up period. Keywords: aerobic digestion, anaerobic digestion, fluidization, landfill leachate, microbial immobilization, municipal solid waste. Graphical Abstract