Journal articles on the topic 'Scaled gradient descent'
To see the other types of publications on this topic, follow the link: Scaled gradient descent.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Scaled gradient descent.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Farhana Husin, Siti, Mustafa Mamat, Mohd Asrul Hery Ibrahim, and Mohd Rivaie. "A modification of steepest descent method for solving large-scaled unconstrained optimization problems." International Journal of Engineering & Technology 7, no. 3.28 (August 17, 2018): 72. http://dx.doi.org/10.14419/ijet.v7i3.28.20969.
Full textAbstract:
In this paper, we develop a new search direction for Steepest Descent (SD) method by replacing previous search direction from Conjugate Gradient (CG) method, , with gradient from the previous step, for solving large-scale optimization problem. We also used one of the conjugate coefficient as a coefficient for matrix . Under some reasonable assumptions, we prove that the proposed method with exact line search satisfies descent property and possesses the globally convergent. Further, the numerical results on some unconstrained optimization problem show that the proposed algorithm is promising.
2
Okoubi, Firmin Andzembe, and Jonas Koko. "Parallel Nesterov Domain Decomposition Method for Elliptic Partial Differential Equations." Parallel Processing Letters 30, no. 01 (March 2020): 2050004. http://dx.doi.org/10.1142/s0129626420500048.
Full textAbstract:
We study a parallel non-overlapping domain decomposition method, based on the Nesterov accelerated gradient descent, for the numerical approximation of elliptic partial differential equations. The problem is reformulated as a constrained (convex) minimization problem with the interface continuity conditions as constraints. The resulting domain decomposition method is an accelerated projected gradient descent with convergence rate [Formula: see text]. At each iteration, the proposed method needs only one matrix/vector multiplication. Numerical experiments show that significant (standard and scaled) speed-ups can be obtained.
3
Maduranga, Kehelwala D. G., Kyle E. Helfrich, and Qiang Ye. "Complex Unitary Recurrent Neural Networks Using Scaled Cayley Transform." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4528–35. http://dx.doi.org/10.1609/aaai.v33i01.33014528.
Full textAbstract:
Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN) which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the scaling matrix is fixed before training and a hyperparameter is introduced to tune the matrix for each particular task. In this paper, we develop a unitary RNN architecture based on a complex scaled Cayley transform. Unlike the real orthogonal case, the transformation uses a diagonal scaling matrix consisting of entries on the complex unit circle which can be optimized using gradient descent and no longer requires the tuning of a hyperparameter. We also provide an analysis of a potential issue of the modReLU activiation function which is used in our work and several other unitary RNNs. In the experiments conducted, the scaled Cayley unitary recurrent neural network (scuRNN) achieves comparable or better results than scoRNN and other unitary RNNs without fixing the scaling matrix.
4
Bayati. "New Scaled Sufficient Descent Conjugate Gradient Algorithm for Solving Unconstraint Optimization Problems." Journal of Computer Science 6, no. 5 (May 1, 2010): 511–18. http://dx.doi.org/10.3844/jcssp.2010.511.518.
Full text
5
Al-batah, Mohammad Subhi, Mutasem Sh Alkhasawneh, Lea Tien Tay, Umi Kalthum Ngah, Habibah Hj Lateh, and Nor Ashidi Mat Isa. "Landslide Occurrence Prediction Using Trainable Cascade Forward Network and Multilayer Perceptron." Mathematical Problems in Engineering 2015 (2015): 1–9. http://dx.doi.org/10.1155/2015/512158.
Full textAbstract:
Landslides are one of the dangerous natural phenomena that hinder the development in Penang Island, Malaysia. Therefore, finding the reliable method to predict the occurrence of landslides is still the research of interest. In this paper, two models of artificial neural network, namely, Multilayer Perceptron (MLP) and Cascade Forward Neural Network (CFNN), are introduced to predict the landslide hazard map of Penang Island. These two models were tested and compared using eleven machine learning algorithms, that is, Levenberg Marquardt, Broyden Fletcher Goldfarb, Resilient Back Propagation, Scaled Conjugate Gradient, Conjugate Gradient with Beale, Conjugate Gradient with Fletcher Reeves updates, Conjugate Gradient with Polakribiere updates, One Step Secant, Gradient Descent, Gradient Descent with Momentum and Adaptive Learning Rate, and Gradient Descent with Momentum algorithm. Often, the performance of the landslide prediction depends on the input factors beside the prediction method. In this research work, 14 input factors were used. The prediction accuracies of networks were verified using the Area under the Curve method for the Receiver Operating Characteristics. The results indicated that the best prediction accuracy of 82.89% was achieved using the CFNN network with the Levenberg Marquardt learning algorithm for the training data set and 81.62% for the testing data set.
6
Al-Naemi, Ghada M., and Ahmed H. Sheekoo. "New scaled algorithm for non-linear conjugate gradients in unconstrained optimization." Indonesian Journal of Electrical Engineering and Computer Science 24, no. 3 (December 1, 2021): 1589. http://dx.doi.org/10.11591/ijeecs.v24.i3.pp1589-1595.
Full textAbstract:
<p>A new scaled conjugate gradient (SCG) method is proposed throughout this paper, the SCG technique may be a special important generalization conjugate gradient (CG) method, and it is an efficient numerical method for solving nonlinear large scale unconstrained optimization. As a result, we proposed the new SCG method with a strong Wolfe condition (SWC) line search is proposed. The proposed technique's descent property, as well as its global convergence property, are satisfied without the use of any line searches under some suitable assumptions. The proposed technique's efficiency and feasibility are backed up by numerical experiments comparing them to traditional CG techniques.</p>
7
Arthur, C. K., V. A. Temeng, and Y. Y. Ziggah. "Performance Evaluation of Training Algorithms in Backpropagation Neural Network Approach to Blast-Induced Ground Vibration Prediction." Ghana Mining Journal 20, no. 1 (July 7, 2020): 20–33. http://dx.doi.org/10.4314/gm.v20i1.3.
Full textAbstract:
Abstract
Backpropagation Neural Network (BPNN) is an artificial intelligence technique that has seen several applications in many fields of science and engineering. It is well-known that, the critical task in developing an effective and accurate BPNN model depends on an appropriate training algorithm, transfer function, number of hidden layers and number of hidden neurons. Despite the numerous contributing factors for the development of a BPNN model, training algorithm is key in achieving optimum BPNN model performance. This study is focused on evaluating and comparing the performance of 13 training algorithms in BPNN for the prediction of blast-induced ground vibration. The training algorithms considered include: Levenberg-Marquardt, Bayesian Regularisation, Broyden–Fletcher–Goldfarb–Shanno (BFGS) Quasi-Newton, Resilient Backpropagation, Scaled Conjugate Gradient, Conjugate Gradient with Powell/Beale Restarts, Fletcher-Powell Conjugate Gradient, Polak-Ribiére Conjugate Gradient, One Step Secant, Gradient Descent with Adaptive Learning Rate, Gradient Descent with Momentum, Gradient Descent, and Gradient Descent with Momentum and Adaptive Learning Rate. Using ranking values for the performance indicators of Mean Squared Error (MSE), correlation coefficient (R), number of training epoch (iteration) and the duration for convergence, the performance of the various training algorithms used to build the BPNN models were evaluated. The obtained overall ranking results showed that the BFGS Quasi-Newton algorithm outperformed the other training algorithms even though the Levenberg Marquardt algorithm was found to have the best computational speed and utilised the smallest number of epochs.
Keywords: Artificial Intelligence, Blast-induced Ground Vibration, Backpropagation Training Algorithms
8
Abbaspour-Gilandeh, Yousef, Masoud Fazeli, Ali Roshanianfard, Mario Hernández-Hernández, Iván Gallardo-Bernal, and José Luis Hernández-Hernández. "Prediction of Draft Force of a Chisel Cultivator Using Artificial Neural Networks and Its Comparison with Regression Model." Agronomy 10, no. 4 (March 25, 2020): 451. http://dx.doi.org/10.3390/agronomy10040451.
Full textAbstract:
In this study, artificial neural networks (ANNs) were used to predict the draft force of a rigid tine chisel cultivator. The factorial experiment based on the randomized complete block design (RCBD) was used to obtain the required data and to determine the factors affecting the draft force. The draft force of the chisel cultivator was measured using a three-point hitch dynamometer and data were collected using a DT800 datalogger. A recurrent back-propagation multilayer network was selected to predict the draft force of the cultivator. The gradient descent algorithm with momentum, Levenberg–Marquardt algorithm, and scaled conjugate gradient descent algorithm were used for network training. The tangent sigmoid transfer function was the activation functions in the layers. The draft force was predicted based on the tillage depth, soil moisture content, soil cone index, and forward speed. The results showed that the developed ANNs with two hidden layers (24 and 26 neurons in the first and second layers, respectively) with the use of the scaled conjugate gradient descent algorithm outperformed the networks developed with other algorithms. The average simulation accuracy and the correlation coefficient for the prediction of draft force of a chisel cultivator were 99.83% and 0.9445, respectively. The linear regression model had a much lower accuracy and correlation coefficient for predicting the draft force compared to the ANNs.
9
Sra, Suvrit. "On the Matrix Square Root via Geometric Optimization." Electronic Journal of Linear Algebra 31 (February 5, 2016): 433–43. http://dx.doi.org/10.13001/1081-3810.3196.
Full textAbstract:
This paper is triggered by the preprint [P. Jain, C. Jin, S.M. Kakade, and P. Netrapalli. Computing matrix squareroot via non convex local search. Preprint, arXiv:1507.05854, 2015.], which analyzes gradient-descent for computing the square root of a positive definite matrix. Contrary to claims of Jain et al., the authorâs experiments reveal that Newton-like methods compute matrix square roots rapidly and reliably, even for highly ill-conditioned matrices and without requiring com-mutativity. The author observes that gradient-descent converges very slowly primarily due to tiny step-sizes and ill-conditioning. The paper derives an alternative first-order method based on geodesic convexity; this method admits a transparent convergence analysis (< 1 page), attains linear rate, and displays reliable convergence even for rank deficient problems. Though superior to gradient-descent, ultimately this method is also outperformed by a well-known scaled Newton method. Nevertheless, the primary value of the paper is conceptual: it shows that for deriving gradient based methods for the matrix square root, the manifold geometric view of positive definite matrices can be much more advantageous than the Euclidean view.
10
Hamed, Eman T., Rana Z. Al-Kawaz, and Abbas Y. Al-Bayati. "New Investigation for the Liu-Story Scaled Conjugate Gradient Method for Nonlinear Optimization." Journal of Mathematics 2020 (January 25, 2020): 1–12. http://dx.doi.org/10.1155/2020/3615208.
Full textAbstract:
This article considers modified formulas for the standard conjugate gradient (CG) technique that is planned by Li and Fukushima. A new scalar parameter θkNew for this CG technique of unconstrained optimization is planned. The descent condition and global convergent property are established below using strong Wolfe conditions. Our numerical experiments show that the new proposed algorithms are more stable and economic as compared to some well-known standard CG methods.
11
Xu, Kai, and Zhi Xiong. "Nonparametric Tensor Completion Based on Gradient Descent and Nonconvex Penalty." Symmetry 11, no. 12 (December 12, 2019): 1512. http://dx.doi.org/10.3390/sym11121512.
Full textAbstract:
Existing tensor completion methods all require some hyperparameters. However, these hyperparameters determine the performance of each method, and it is difficult to tune them. In this paper, we propose a novel nonparametric tensor completion method, which formulates tensor completion as an unconstrained optimization problem and designs an efficient iterative method to solve it. In each iteration, we not only calculate the missing entries by the aid of data correlation, but consider the low-rank of tensor and the convergence speed of iteration. Our iteration is based on the gradient descent method, and approximates the gradient descent direction with tensor matricization and singular value decomposition. Considering the symmetry of every dimension of a tensor, the optimal unfolding direction in each iteration may be different. So we select the optimal unfolding direction by scaled latent nuclear norm in each iteration. Moreover, we design formula for the iteration step-size based on the nonconvex penalty. During the iterative process, we store the tensor in sparsity and adopt the power method to compute the maximum singular value quickly. The experiments of image inpainting and link prediction show that our method is competitive with six state-of-the-art methods.
12
Sabi’u, Jamilu, Kazeem Olalekan Aremu, Ali Althobaiti, and Abdullah Shah. "Scaled Three-Term Conjugate Gradient Methods for Solving Monotone Equations with Application." Symmetry 14, no. 5 (May 5, 2022): 936. http://dx.doi.org/10.3390/sym14050936.
Full textAbstract:
In this paper, we derived a modified conjugate gradient (CG) parameter by adopting the Birgin and Marti´nez strategy using the descent three-term CG direction and the Newton direction. The proposed CG parameter is applied and suggests a robust algorithm for solving constrained monotone equations with an application to image restoration problems. The global convergence of this algorithm is established using some proper assumptions. Lastly, the numerical comparison with some existing algorithms shows that the proposed algorithm is a robust approach for solving large-scale systems of monotone equations. Additionally, the proposed CG parameter can be used to solve the symmetric system of nonlinear equations as well as other relevant classes of nonlinear equations.
13
Abubakar, Auwal Bala, Kanikar Muangchoo, Abdulkarim Hassan Ibrahim, Sunday Emmanuel Fadugba, Kazeem Olalekan Aremu, and Lateef Olakunle Jolaoso. "A Modified Scaled Spectral-Conjugate Gradient-Based Algorithm for Solving Monotone Operator Equations." Journal of Mathematics 2021 (April 26, 2021): 1–9. http://dx.doi.org/10.1155/2021/5549878.
Full textAbstract:
This paper proposes a modified scaled spectral-conjugate-based algorithm for finding solutions to monotone operator equations. The algorithm is a modification of the work of Li and Zheng in the sense that the uniformly monotone assumption on the operator is relaxed to just monotone. Furthermore, unlike the work of Li and Zheng, the search directions of the proposed algorithm are shown to be descent and bounded independent of the monotonicity assumption. Moreover, the global convergence is established under some appropriate assumptions. Finally, numerical examples on some test problems are provided to show the efficiency of the proposed algorithm compared to that of Li and Zheng.
14
Bojari, S., and M. R. Eslahchi. "Two families of scaled three-term conjugate gradient methods with sufficient descent property for nonconvex optimization." Numerical Algorithms 83, no. 3 (May 2, 2019): 901–33. http://dx.doi.org/10.1007/s11075-019-00709-7.
Full text
15
Althobaiti, Ali, Jamilu Sabi’u, Homan Emadifar, Prem Junsawang, and Soubhagya Kumar Sahoo. "A Scaled Dai–Yuan Projection-Based Conjugate Gradient Method for Solving Monotone Equations with Applications." Symmetry 14, no. 7 (July 7, 2022): 1401. http://dx.doi.org/10.3390/sym14071401.
Full textAbstract:
In this paper, we propose two scaled Dai–Yuan (DY) directions for solving constrained monotone nonlinear systems. The proposed directions satisfy the sufficient descent condition independent of the line search strategy. We also reasonably proposed two different relations for computing the scaling parameter at every iteration. The first relation is proposed by approaching the quasi-Newton direction, and the second one is by taking the advantage of the popular Barzilai–Borwein strategy. Moreover, we propose a robust projection-based algorithm for solving constrained monotone nonlinear equations with applications in signal restoration problems and reconstructing the blurred images. The global convergence of this algorithm is also provided, using some mild assumptions. Finally, a comprehensive numerical comparison with the relevant algorithms shows that the proposed algorithm is efficient.
16
Zhou, Guoling, Yueting Yang, and Mingyuan Cao. "A New Spectral Three-Term Conjugate Gradient Method with Random Parameter Based on Modified Secant Equation and Its Application to Low-Carbon Supply Chain Optimization." Journal of Mathematics 2022 (October 15, 2022): 1–15. http://dx.doi.org/10.1155/2022/8939770.
Full textAbstract:
In this work, considering the advantages of spectral conjugate gradient method and quasi-Newton method, a spectral three-term conjugate gradient method with random parameter is proposed. The parameter in the search direction of the new method is determined by minimizing the Frobenius norm of difference between search direction matrix and self-scaled memoryless BFGS matrix based on modified secant equation. Then, the search direction satisfying the sufficient descent condition is obtained. The global convergence of new method is proved under appropriate assumptions. Numerical experiments show that our method has better performance by comparing with the up-to-date method. Furthermore, the new method has been successfully applied to the optimization of low-carbon supply chain.
17
Garkani-Nejad, Zahra, and Behzad Ahmadi-Roudi. "Investigating the role of weight update functions in developing artificial neural network modeling of retention times of furan and phenol derivatives." Canadian Journal of Chemistry 91, no. 4 (April 2013): 255–62. http://dx.doi.org/10.1139/cjc-2012-0372.
Full textAbstract:
A quantitative structure−retention relationship study has been carried out on the retention times of 63 furan and phenol derivatives using artificial neural networks (ANNs). First, a large number of descriptors were calculated using HyperChem, Mopac, and Dragon softwares. Then, a suitable number of these descriptors were selected using a multiple linear regression technique. This paper focuses on investigating the role of weight update functions in developing ANNs. Therefore, selected descriptors were used as inputs for ANNs with six different weight update functions including the Levenberg−Marquardt back-propagation network, scaled conjugate gradient back-propagation network, conjugate gradient back-propagation with Powell−Beale restarts network, one-step secant back-propagation network, resilient back-propagation network, and gradient descent with momentum back-propagation network. Comparison of the results indicates that the Levenberg−Marquardt back-propagation network has better predictive power than the other methods.
18
Kozlova, L. E., and E. V. Bolovin. "Building the Structure and the Neuroemulator Angular Velocity's Learning Algorithm Selection of the Electric Drive of TVR-IM Type." Applied Mechanics and Materials 792 (September 2015): 44–50. http://dx.doi.org/10.4028/www.scientific.net/amm.792.44.
Full textAbstract:
Today, one of the most common ways to control smooth starting and stopping of the induction motors are soft-start system. To ensure such control method the use of closed-speed asynchronous electric drive of TVR-IM type is required. Using real speed sensors is undesirable due to a number of inconveniences exploitation of the drive. The use of the observer based on a neural network is more convenient than the use of the real sensors. Its advantages are robustness, high generalizing properties, lack of requirements to the motor parameters, the relative ease of creation. This article presents the research and selection of the best learning algorithm of the neuroemulator angular velocity of the electric drive of TVR-IM type. There were investigated such learning algorithms as gradient descent back propagation, gradient descent with momentum back propagation, algorithm of Levenberg – Marquardt, scaled conjugate gradient back propagation (SCG). The input parameters of the neuroemulator were the pre treatment signals from the real sensors the stator current and the stator voltage and their delay, as well as a feedback signal from the estimated speed with delay. A comparative analysis of learning algorithms was performed on a simulation model of asynchronous electric drive implemented in software MATLAB Simulink, when the electric drive was running in dynamic mode. The simulation results demonstrate that the best method of learning is algorithm of Levenberg – Marquardt.
19
Verma, Shilpa, G. T. Thampi, and Madhuri Rao. "ANN based method for improving gold price forecasting accuracy through modified gradient descent methods." IAES International Journal of Artificial Intelligence (IJ-AI) 9, no. 1 (March 1, 2020): 46. http://dx.doi.org/10.11591/ijai.v9.i1.pp46-57.
Full textAbstract:
Forecast of prices of financial assets including gold is of considerable importance for planning the economy. For centuries, people have been holding gold for many important reasons such as smoothening inflation fluctuations, protection from an economic crisis, sound investment etc.. Forecasting of gold prices is therefore an ever important exercise undertaken both by individuals and groups. Various local, global, political, psychological and economic factors make such a forecast a complex problem. Data analysts have been increasingly applying Artificial Intelligence (AI) techniques to make such forecasts. In the present work an inter comparison of gold price forecasting in Indian market is first done by employing a few classical Artificial Neural Network (ANN) techniques, namely Gradient Descent Method (GDM), Resilient Backpropagation method (RP), Scaled Conjugate Gradient method (SCG), Levenberg-Marquardt method (LM), Bayesian Regularization method (BR), One Step Secant method (OSS) and BFGS Quasi Newton method (BFG). Improvement in forecasting accuracy is achieved by proposing and developing a few modified GDM algorithms that incorporate different optimization functions by replacing the standard quadratic error function of classical GDM. Various optimization functions investigated in the present work are Mean median error function (MMD), Cauchy error function (CCY), Minkowski error function (MKW), Log cosh error function (LCH) and Negative logarithmic likelihood function (NLG). Modified algorithms incorporating these optimization functions are referred to here by GDM_MMD, GDM_CCY, GDM_KWK, GDM_LCH and GDM_NLG respectively. Gold price forecasting is then done by employing these algorithms and the results are analysed. The results of our study suggest that the forecasting efficiency improves considerably on applying the modified methods proposed by us.
20
Srivastava, Satyam, Abhai Tiwari, Pankaj Kumar, and Shashikant Sadistap. "A Multispectral Spectroscopic Based Sensing System for Quality Parameters Measurement in Raw Milk Samples." Sensor Letters 18, no. 4 (April 1, 2020): 311–21. http://dx.doi.org/10.1166/sl.2020.4222.
Full textAbstract:
Lactometer is used to monitor milk quality at various dairy centers but this may lead towards incorrect results because it requires human intervention and exact temperature correction as well as overall process is time-consuming. Presented work proposes the multispectral based spectroscopic approach along with the comparative study of different chemometric and artificial neural network (ANN) based techniques to measure different milk quality parameters. A multispectral spectroscopic sensing module has been designed using off the shelf components and further interfaced with 8-bit microcontroller based embedded system to produce three different spectrums of transmittance and scattering at +90 degree and –90 degree over the wavelength range of 340–1030 nm. Data acquisition process has been performed for 150 milk samples (cow, buffalo, and mix) collected from the bulk milk cooling center (BMC), Jaipur. Different statistical modeling techniques such as principle component regression (PCR), multiple linear regression (MLR) and partial least square regression (PLSR) have been implemented to develop correlation models between extracted features and target milk parameters. Implemented techniques have been compared based on the accuracy of their prediction models and it has been observed that PLSR shows better results compared to other two techniques. ANN-based modeling approach also has been explored to improve the accuracy of results. Five different artificial neural networks (ANN) based modeling techniques (LevenbergMarquardt, Bayesian regulation, scaled conjugate gradient, gradient descent and resilient) have been used to predict targeted milk quality parameters. Out of them, Gradient descent modeling technique performs better to predict fat content of the milk (R2 = 0.96198), Bayesian regulation performs better to predict lactose content (R2 = 0.90594) and others (solid non-fat (SNF), protein) are just satisfactory (R2 = 0.76077 for SNF using scaled conjugate gradient, R2 = 0.41935 for protein using Levenberg Marquardt). Produced results are validated with the MilkoScan FT1 system installed at Rajasthan Corporation of Dairy Federation (RCDF), Jaipur and it has been observed that results presented higher order of coefficient of determination as mentioned above (except protein, S.N.F.). A smartphone-based android application also has been developed to acquire data from the embedded system using Bluetooth protocol and transfer to cloud with the location information for further analysis.
21
Sabi'u, Jamilu, Ali Althobaiti, Saad Althobaiti, Soubhagya Kumar Sahoo, and Thongchai Botmart. "A scaled Polak-Ribi$ \grave{e} $re-Polyak conjugate gradient algorithm for constrained nonlinear systems and motion control." AIMS Mathematics 8, no. 2 (2022): 4843–61. http://dx.doi.org/10.3934/math.2023241.
Full textAbstract:
<abstract><p>This paper proposes Polak-Ribi$ \grave{e} $re-Polyak (PRP) conjugate gradient (CG) directions based on two efficient scaling strategies. The first scaling parameter is determined by approaching the quasi-Newton direction, and the second by utilizing the well-known Barzilai-Borwein approach. In addition, we proposed two directions that satisfy the sufficient descent criterion regardless of the line search strategy. The proposed directions lead to a matrix-free algorithm for solving monotone-constrained nonlinear systems. The proposed algorithm's global convergence analysis is presented using some underlying assumptions. Furthermore, a detailed numerical comparison with other existing algorithms revealed that the proposed algorithm is both efficient and effective. Finally, the proposed technique is applied to the motion control problem of a two-joint planar robotic manipulator.</p></abstract>
22
Günen, Mehmet Akif, Umit Haluk Atasever, and Erkan Beşdok. "Analyzing the Contribution of Training Algorithms on Deep Neural Networks for Hyperspectral Image Classification." Photogrammetric Engineering & Remote Sensing 86, no. 9 (September 1, 2020): 581–88. http://dx.doi.org/10.14358/pers.86.9.581.
Full textAbstract:
Autoencoder (<small>AE</small>)-based deep neural networks learn complex problems by generating feature-space conjugates of input data. The learning success of an AE is too sensitive for a training algorithm. The problem of hyperspectral image (<small>HSI</small>) classification by using spectral features of pixels is a highly complex problem due to its multi-dimensional and excessive data nature. In this paper, the contribution of three gradient-based training algorithms (i.e., scaled conjugate gradient (<small>SCG</small>), gradient descent (<small>GD</small>), and resilient backpropagation algorithms (<small>RP</small>)) on the solution of the HSI classification problem by using AE was analyzed. Also, it was investigated how neighborhood component analysis affects classification performance for training algorithms on HSIs. Two hyperspectral image classification benchmark data sets were used in the experimental analysis. Wilcoxon signed-rank test indicates that RB is statistically better than SCG and GD in solving the related image classification problem.
23
Yang, Can, Zhengshun Cheng, Longfei Xiao, Xinliang Tian, Mingyue Liu, and Binrong Wen. "A gradient-descent-based method for design of performance-scaled rotor for floating wind turbine model testing in wave basins." Renewable Energy 187 (March 2022): 144–55. http://dx.doi.org/10.1016/j.renene.2022.01.068.
Full text
24
Babaie–Kafaki, Saman, and Reza Ghanbari. "A class of descent four–term extension of the Dai–Liao conjugate gradient method based on the scaled memoryless BFGS update." Journal of Industrial & Management Optimization 13, no. 2 (2017): 649–58. http://dx.doi.org/10.3934/jimo.2016038.
Full text
25
Mohktar, M. S., F. Ibrahim, and N. A. Ismail. "NON-INVASIVE APPROACH TO PREDICT THE CHOLESTEROL LEVEL IN BLOOD USING BIOIMPEDANCE AND NEURAL NETWORK TECHNIQUES." Biomedical Engineering: Applications, Basis and Communications 25, no. 06 (December 2013): 1350046. http://dx.doi.org/10.4015/s1016237213500464.
Full textAbstract:
This paper presents a new non-invasive approach to predict the status of high total cholesterol (TC) level in blood using bioimpedance and the artificial neural network (ANN) techniques. The input parameters for the ANN model are acquired from a non-invasive bioelectrical impedance analysis (BIA) measurement technique. The measurement data were obtained from 260 volunteered participants. A total of 190 subject's data were used for the ANN training purpose and the remaining 70 subject's data were used for model testing. Six parameters from the BIA parameters were found to be significant predictors for TC level in blood using logistic regression analysis. The six input predictors for the ANN modeling are age, body mass index (BMI), body capacitance, basal metabolic rate, extracellular mass and lean body mass. Four ANN techniques such as the gradient descent with momentum, the resilient, the scaled conjugate gradient and the Levenberg–Marquardt were used and compared for predicting the high TC level in the blood. The finding showed that the resilient method was the best model with prediction accuracy, sensitivity, specificity and area under the curve value obtained from the test data were 82.9%, 85.4%, 79.3% and 0.83%, respectively.
26
Hossain, SK Safdar, Bamidele Victor Ayodele, Zaid Abdulhamid Alhulaybi, and Muhammad Mudassir Ahmad Alwi. "Data-Driven Approach to Modeling Biohydrogen Production from Biodiesel Production Waste: Effect of Activation Functions on Model Configurations." Applied Sciences 12, no. 24 (December 15, 2022): 12914. http://dx.doi.org/10.3390/app122412914.
Full textAbstract:
Biodiesel production often results in the production of a significant amount of waste glycerol. Through various technological processes, waste glycerol can be sustainably utilized for the production of value-added products such as hydrogen. One such process used for waste glycerol conversion is the bioprocess, whereby thermophilic microorganisms are utilized. However, due to the complex mechanism of the bioprocess, it is uncertain how various input parameters are interrelated with biohydrogen production. In this study, a data-driven machine-learning approach is employed to model the prediction of biohydrogen from waste glycerol. Twelve configurations consisting of the multilayer perceptron neural network (MLPNN) and the radial basis function neural network (RBFNN) were investigated. The effect of using different combinations of activation functions such as hyperbolic tangent, identity, and sigmoid on the model’s performance was investigated. Moreover, the effect of two optimization algorithms, scaled conjugate gradient and gradient descent, on the model performance was also investigated. The performance analysis of the models revealed that the manner in which the activation functions are combined in the hidden and outer layers significantly influences the performance of various models. Similarly, the model performance was also influenced by the nature of the optimization algorithms. The MLPNN models displayed better predictive performance compared to the RBFNN models. The RBFNN model with softmax as the hidden layer activation function and identity as the outer layer activation function has the least predictive performance, as indicated by an R2 of 0.403 and a RMSE of 301.55. While the MLPNN configuration with the hyperbolic tangent as the hidden layer activation function and the sigmoid as the outer layer activation function yielded the best performance as indicated by an R2 of 0.978 and a RMSE of 9.91. The gradient descent optimization algorithm was observed to help improve the model’s performance. All the input variables significantly influence the predicted biohydrogen. However, waste glycerol has the most significant effects.
27
Ikpang, Ikpang Nkereuwem, and Ekom-obong Jackson Okon. "Modeling Average Rainfall in Nigeria With Artificial Neural Network (ANN) Models and Seasonal Autoregressive Integrated Moving Average (SARIMA) Models." International Journal of Statistics and Probability 11, no. 4 (June 30, 2022): 53. http://dx.doi.org/10.5539/ijsp.v11n4p53.
Full textAbstract:
Rainfall prediction is one of the most essential and challenging operational obligations undertaken by meteorological services globally. In this article we conduct a comparative study between the ANN models and the traditional SARIMA models to show the most suitable model for predicting rainfall in Nigeria. Average monthly rainfall data in Nigerian for the period Jan. 1991 to Dec.2020 were considered. The ACF and PACF plots clearly identifies the SARIMA (1,0,2)x(1,1,2)12 as an appropriate model for predicting average monthly rainfall. The performance of the trained Neural Network (NN) analysis clearly favours Levenberg-Marquardt (LM) over the Scaled Conjugate Gradient Descent (SCGD) algorithms and Bayesian Regularization (BR) method with Average Absolute Error 0.000525056. The forecasting performance metric using the RSME and MAE, showed that Neural Network trained by Levenberg-Marquadrt algorithm gives better predicted values of Nigerian rainfall than the SARIMA (1,0,2)x(1,1,2)12.
28
Hadad, Ahmad F., Sudjati Rachmat, Tutuka Ariadji, and Kuntjoro A. Sidarto. "The Prediction of Three Key Properties on Coalbed Methane Reservoir Using Artificial Intelligence." Modern Applied Science 11, no. 8 (July 9, 2017): 57. http://dx.doi.org/10.5539/mas.v11n8p57.
Full textAbstract:
This research focuses on creating the prediction tools for the three key properties in coalbed methane (CBM) reservoir; the properties are gas content, Langmuir parameters, and permeability. Basically, their roles are to describe the gas in place and also future dynamic behavior of CBM reservoir. These three key properties are tried to be predicted with open-hole data as the inputs.It uses artificial neural network (ANN) and adaptive neuro fuzzy inference system (ANFIS) to generate the prediction tools. It is started from data preparation and processing, then pattern or function identifications, and finalized by validation and testing. Several training algorithms are applied for ANN such as adaptive gradient descent (ANN_GDX), Levenberg-Marquardt (ANN_LM), resilient backpropagation (ANN_RP), scaled conjugate gradient (ANN_SCG), and Bayesian regularization algorithm (ANN_BR). The first fives employ the early stopping technique for regularization, and the last one does Bayesian regularization. On the other hand, the ANFIS will use only early stopping technique.Based on this research, it is concluded that both ANN and ANFIS are able to identify the patterns or function between open-hole log data and the three key properties (TKP). Furthermore, it can be concluded that ANN_LM, ANFIS, and ANN_BR are the best selected algorithms which resulted the lowest error of TKP’s predictions.
29
Jallal, Mohammed Ali, Abdessalam El Yassini, Samira Chabaa, Abdelouhab Zeroual, and Saida Ibnyaich. "Multi-Target Learning Algorithm for Solar Radiation Components Forecasting Based on the Desired Tilt Angle of a Solar Energy System." Instrumentation Mesure Métrologie 20, no. 4 (August 31, 2021): 187–93. http://dx.doi.org/10.18280/i2m.200402.
Full textAbstract:
Solar radiation components (SRC) forecasting with different tilt angles plays a key role for planning, managing, and controlling the solar energy system production. To overcome the gaps related to the intermittence and to the absence of SRC data, an accurate predictive model needs to be established. The main goal of the present work is to develop for solar system engineers and grid operators a precise predictive approach based on multi-target learning algorithm to forecast the hourly SRC measurements that is related to the city of Marrakesh (latitude 31°37′N, longitude 08°01′W, elevation 466m), Morocco, received by different inclined solar panels’ surfaces. For this purpose, eight training algorithms (Resilient back Propagation (Rp), One step secant (OSS), Levenberg-Marquardt (LM) Algorithm, Fletcher-Reeves algorithm (Cgf), Polak-Ribiere algorithm (Cgp), Powell-Beale algorithm (Cgb), gradient descent (Gd) algorithm and scaled conjugate gradient algorithm (Scg)) are tested to optimize the developed approach’s parameters. The forecasting results were performed based on the angle of inclination desired by the operator and some accessible meteorological measurements that are recorded at each hour, comprising time variables. The achieved performance demonstrates the stability and the accuracy of the established approach to estimate the hourly SRC time series compared to several recent literature studies.
30
Bui, Manh-Ha, Thanh-Luu Pham, and Thanh-Son Dao. "Prediction of cyanobacterial blooms in the Dau Tieng Reservoir using an artificial neural network." Marine and Freshwater Research 68, no. 11 (2017): 2070. http://dx.doi.org/10.1071/mf16327.
Full textAbstract:
An artificial neural network (ANN) model was used to predict the cyanobacteria bloom in the Dau Tieng Reservoir, Vietnam. Eight environmental parameters (pH, dissolved oxygen, temperature, total dissolved solids, total nitrogen (TN), total phosphorus, biochemical oxygen demand and chemical oxygen demand) were introduced as inputs, whereas the cell density of three cyanobacteria genera (Anabaena, Microcystis and Oscillatoria) with microcystin concentrations were introduced as outputs of the three-layer feed-forward back-propagation ANN. Eighty networks covering all combinations of four learning algorithms (Bayesian regularisation (BR), gradient descent with momentum and adaptive learning rate, Levenberg–Mardquart, scaled conjugate gradient) with two transfer functions (tansig, logsig) and 10 numbers of hidden neurons (6–16) were trained and validated to find the best configuration fitting the observed data. The result is a network using the BR learning algorithm, tansig transfer function and nine neurons in the hidden layer, which shows satisfactory predictions with the low values of error (root mean square error=0.108) and high correlation coefficient values (R=0.904) between experimental and predicted values. Sensitivity analysis on the developed ANN indicated that TN and temperature had the most positive and negative effects respectively on microcystin concentrations. These results indicate that ANN modelling can effectively predict the behaviour of the cyanobacteria bloom process.
31
Ozfatura, Emre, Sennur Ulukus, and Deniz Gündüz. "Straggler-Aware Distributed Learning: Communication–Computation Latency Trade-Off." Entropy 22, no. 5 (May 13, 2020): 544. http://dx.doi.org/10.3390/e22050544.
Full textAbstract:
When gradient descent (GD) is scaled to many parallel workers for large-scale machine learning applications, its per-iteration computation time is limited by straggling workers. Straggling workers can be tolerated by assigning redundant computations and/or coding across data and computations, but in most existing schemes, each non-straggling worker transmits one message per iteration to the parameter server (PS) after completing all its computations. Imposing such a limitation results in two drawbacks: over-computation due to inaccurate prediction of the straggling behavior, and under-utilization due to discarding partial computations carried out by stragglers. To overcome these drawbacks, we consider multi-message communication (MMC) by allowing multiple computations to be conveyed from each worker per iteration, and propose novel straggler avoidance techniques for both coded computation and coded communication with MMC. We analyze how the proposed designs can be employed efficiently to seek a balance between the computation and communication latency. Furthermore, we identify the advantages and disadvantages of these designs in different settings through extensive simulations, both model-based and real implementation on Amazon EC2 servers, and demonstrate that proposed schemes with MMC can help improve upon existing straggler avoidance schemes.
32
Chen, Zaiwei, Shancong Mou, and Siva Theja Maguluri. "Stationary Behavior of Constant Stepsize SGD Type Algorithms." ACM SIGMETRICS Performance Evaluation Review 50, no. 1 (June 20, 2022): 109–10. http://dx.doi.org/10.1145/3547353.3522659.
Full textAbstract:
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However, constant stepsize SA algorithms do not converge to the optimal solution, but instead have a stationary distribution, which in general cannot be analytically characterized. In this work, we study the asymptotic behavior of the appropriately scaled stationary distribution, in the limit when the constant stepsize goes to zero. Specifically, we consider the following three settings: (1) SGD algorithm with a smooth and strongly convex objective, (2) linear SA algorithm involving a Hurwitz matrix, and (3) nonlinear SA algorithm involving a contractive operator. When the iterate is scaled by 1/√α, where α is the constant stepsize, we show that the limiting scaled stationary distribution is a solution of an implicit equation. Under a uniqueness assumption (which can be removed in certain settings) on this equation, we further characterize the limiting distribution as a Gaussian distribution whose covariance matrix is the unique solution of an appropriate Lyapunov equation. For SA algorithms beyond these cases, our numerical experiments suggest that unlike central limit theorem type results: (1) the scaling factor need not be 1/√α, and (2) the limiting distribution need not be Gaussian. Based on the numerical study, we come up with a heuristic formula to determine the right scaling factor, and make a connection to the Euler-Maruyama discretization scheme for approximating stochastic differential equations.
33
Chen, Zaiwei, Shancong Mou, and Siva Theja Maguluri. "Stationary Behavior of Constant Stepsize SGD Type Algorithms." Proceedings of the ACM on Measurement and Analysis of Computing Systems 6, no. 1 (February 24, 2022): 1–24. http://dx.doi.org/10.1145/3508039.
Full textAbstract:
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However, constant stepsize SA algorithms do not converge to the optimal solution, but instead have a stationary distribution, which in general cannot be analytically characterized. In this work, we study the asymptotic behavior of the appropriately scaled stationary distribution, in the limit when the constant stepsize goes to zero. Specifically, we consider the following three settings: (1) SGD algorithm with a smooth and strongly convex objective, (2) linear SA algorithm involving a Hurwitz matrix, and (3) nonlinear SA algorithm involving a contractive operator. When the iterate is scaled by 1/α, where α is the constant stepsize, we show that the limiting scaled stationary distribution is a solution of an implicit equation. Under a uniqueness assumption (which can be removed in certain settings) on this equation, we further characterize the limiting distribution as a Gaussian distribution whose covariance matrix is the unique solution of a suitable Lyapunov equation. For SA algorithms beyond these cases, our numerical experiments suggest that unlike central limit theorem type results: (1) the scaling factor need not be 1/α, and (2) the limiting distribution need not be Gaussian. Based on the numerical study, we come up with a heuristic formula to determine the right scaling factor, and make insightful connection to the Euler-Maruyama discretization scheme for approximating stochastic differential equations.
34
Nam, Kihoon, Chanyang Park, Jun-Sik Yoon, Hyeok Yun, Hyundong Jang, Kyeongrae Cho, Ho-Jung Kang, et al. "Optimal Energetic-Trap Distribution of Nano-Scaled Charge Trap Nitride for Wider Vth Window in 3D NAND Flash Using a Machine-Learning Method." Nanomaterials 12, no. 11 (May 25, 2022): 1808. http://dx.doi.org/10.3390/nano12111808.
Full textAbstract:
A machine-learning (ML) technique was used to optimize the energetic-trap distributions of nano-scaled charge trap nitride (CTN) in 3D NAND Flash to widen the threshold voltage (Vth) window, which is crucial for NAND operation. The energetic-trap distribution is a critical material property of the CTN that affects the Vth window between the erase and program Vth. An artificial neural network (ANN) was used to model the relationship between the energetic-trap distributions as an input parameter and the Vth window as an output parameter. A well-trained ANN was used with the gradient-descent method to determine the specific inputs that maximize the outputs. The trap densities (NTD and NTA) and their standard deviations (σTD and σTA) were found to most strongly impact the Vth window. As they increased, the Vth window increased because of the availability of a larger number of trap sites. Finally, when the ML-optimized energetic-trap distributions were simulated, the Vth window increased by 49% compared with the experimental value under the same bias condition. Therefore, the developed ML technique can be applied to optimize cell transistor processes by determining the material properties of the CTN in 3D NAND Flash.
35
Ivanova, Vanya. "Multiple IoT based Network Attacks Discrimination by Multilayer Feedforward Neural Networks." International Journal of Circuits, Systems and Signal Processing 16 (January 17, 2022): 675–85. http://dx.doi.org/10.46300/9106.2022.16.84.
Full textAbstract:
In this paper a new neural model for detection of multiple network IoT-based attacks, such as DDoS TCP, UDP, and HHTP flood, is presented. It consists of feedforward multilayer network with back propagation. A general algorithm for its optimization during training is proposed, leading to proper number of neurons in the hidden layers. The Scaled Gradient Descent algorithm and the Adam optimization are studied with better classification results, obtained by the developed classifiers, using the latter. Tangent hyperbolic function appears to be proper selection for the hidden neurons. Two sets of features, gathered from aggregated records of the network traffic, are tested, containing 8 and 10 components. While more accurate results are obtained for the 10-feature set, the 8-feature set offers twice lower training time and seems applicable for real-world applications. The detection rate for 7 of 10 different network attacks, primarily various types of floods, is higher than 90% and for 3 of them – mainly reconnaissance and keylogging activities with low intensity of the generated traffic, deviates between 57% and 68%. The classifier is considered applicable for industrial implementation.
36
He, Wei, Xiaodong Liang, Lu Deng, Xuan Kong, and Hong Xie. "Axle Configuration and Weight Sensing for Moving Vehicles on Bridges Based on the Clustering and Gradient Method." Remote Sensing 13, no. 17 (September 2, 2021): 3477. http://dx.doi.org/10.3390/rs13173477.
Full textAbstract:
Traffic information, including vehicle weight and axle spacing, is vital for bridge safety. The bridge weigh-in-motion (BWIM) system remotely estimates the axle weights of moving vehicles using the response measured from instrumented bridges. It has been proved more accurate and durable than the traditional pavement-based method. However, the main drawback of conventional BWIM algorithms is that they can only identify the axle weight and the information of axle configuration (the number of axles and axle spacing) is required to be determined using an extra device in advance of the weight identification procedure. Namely, dedicated sensors (pressure-sensitive sensors placed on the deck surface or under the soffit of a bridge) in addition to weighing sensors must be adopted for identifying the axle configuration, which significantly decreases the utility, feasibility, and economic efficiency of BWIM technology. In this study, a new iterative procedure simultaneously identifying axle spacing as well as axle weights and gross weights of vehicles is proposed. The novel method is based on k-means clustering and the gradient descent method. In this method, both the axle weight and the axle location are obtained by using the same global response of bridges; thus the axle detectors are no longer required, which makes it economical and easier to be implemented. Furthermore, the proposed optimization method has good computational efficiency and thus is practical for real-time application. Comprehensive numerical simulations and laboratory experiments based on scaled vehicle and bridge models were conducted to verify the proposed method. The identification results show that the proposed method has good accuracy and high computational efficiency in axle spacing and axle weight identification.
37
Wang, Yi, Zong Woo Geem, and Kohei Nagai. "Bond Strength Assessment of Concrete-Corroded Rebar Interface Using Artificial Neutral Network." Applied Sciences 10, no. 14 (July 9, 2020): 4724. http://dx.doi.org/10.3390/app10144724.
Full textAbstract:
Bond strength assessment is important for reinforced concrete structures with rebar corrosion since the bond degradation can threaten the structural safety. In this study, to assess the bond strength in concrete-corroded rebar interface, one of the machine learning techniques, artificial neutral network (ANN), was utilized for the application. From existing literature, data related to the bond strength of concrete and corroded rebar were collected. The ANN model was applied to understand the factors on bond property degradation. For the input in the ANN model, the following factors were considered the relative bond strength: (1) corrosion level; (2) crack width; (3) cover-to-diameter ratio; and (4) concrete strength. For the cases with confinement (stirrups), (5) the diameter/stirrups spacing ratio was also considered. The assessment was conducted from input with single parameter to multiple parameters. The scaled feed-forward multi-layer perception ANN model with the error back-propagation algorithm of gradient descent and momentum was found to match the experimental and computed results. The correlation of each parameter to the bond strength degradation was clarified. In cases without confinement, the relative importance was (1) > (2) > (4) > (3), while it was (2) > (1) > (3) > (5) > (4) for the cases with confinement.
38
Chao, Kuo-Wei, Nian-Ze Hu, Yi-Chu Chao, Chin-Kai Su, and Wei-Hang Chiu. "Implementation of Artificial Intelligence for Classification of Frogs in Bioacoustics." Symmetry 11, no. 12 (November 26, 2019): 1454. http://dx.doi.org/10.3390/sym11121454.
Full textAbstract:
This research presents the implementation of artificial intelligence (AI) for classification of frogs in symmetry of the bioacoustics spectral by using the feedforward neural network approach (FNNA) and support vector machine (SVM). Recently, the symmetry concept has been applied in physics, and in mathematics to help make mathematical models tractable to achieve the best learning performance. Owing to the symmetry of the bioacoustics spectral, feature extraction can be achieved by integrating the techniques of Mel-scale frequency cepstral coefficient (MFCC) and mentioned machine learning algorithms, such as SVM, neural network, and so on. At the beginning, the raw data information for our experiment is taken from a website which collects many kinds of frog sounds. This in fact saves us collecting the raw data by using a digital signal processing technique. The generally proposed system detects bioacoustic features by using the microphone sensor to record the sounds of different frogs. The data acquisition system uses an embedded controller and a dynamic signal module for making high-accuracy measurements. With regard to bioacoustic features, they are filtered through the MFCC algorithm. As the filtering process is finished, all values from ceptrum signals are collected to form the datasets. For classification and identification of frogs, we adopt the multi-layer FNNA algorithm in machine learning and the results are compared with those obtained by the SVM method at the same time. Additionally, two optimizer functions in neural network include: scaled conjugate gradient (SCG) and gradient descent adaptive learning rate (GDA). Both optimization methods are used to evaluate the classification results from the feature datasets in model training. Also, calculation results from the general central processing unit (CPU) and Nvidia graphics processing unit (GPU) processors are evaluated and discussed. The effectiveness of the experimental system on the filtered feature datasets is classified by using the FNNA and the SVM scheme. The expected experimental results of the identification with respect to different symmetry bioacoustic features of fifteen frogs are obtained and finally distinguished.
39
Khan, Sher Afzal, Yaser Daanial Khan, Shakeel Ahmad, and Khalid H. Allehaibi. "N-MyristoylG-PseAAC: Sequence-based Prediction of N-Myristoyl Glycine Sites in Proteins by Integration of PseAAC and Statistical Moments." Letters in Organic Chemistry 16, no. 3 (February 11, 2019): 226–34. http://dx.doi.org/10.2174/1570178616666181217153958.
Full textAbstract:
N-Myristoylation, an irreversible protein modification, occurs by the covalent attachment of myristate with the N-terminal glycine of the eukaryotic and viral proteins, and is associated with a variety of pathogens and disease-related proteins. Identification of myristoylation sites through experimental mechanisms can be costly, labour associated and time-consuming. Due to the association of N-myristoylation with various diseases, its timely prediction can help in diagnosing and controlling the associated fatal diseases. Herein, we present a method named N-MyristoylG-PseAAC in which we have incorporated PseAAC with statistical moments for the prediction of N-Myristoyl Glycine (NMG) sites. A benchmark dataset of 893 positive and 1093 negative samples was collected and used in this study. For feature vector, various position and composition relative features along with the statistical moments were calculated. Later on, a back propagation neural network was trained using feature vectors and scaled conjugate gradient descent with adaptive learning was used as an optimizer. Selfconsistency testing and 10-fold cross-validation were performed to evaluate the performance of N-MyristoylG-PseAAC, by using accuracy metrics. For self-consistency testing, 99.80% Acc, 99.78% Sp, 99.81% Sn and 0.99 MCC were observed, whereas, for 10-fold cross validation, 97.18% Acc, 98.54% Sp, 96.07% Sn and 0.94 MCC were observed. Thus, it was found that the proposed predictor can help in predicting the myristoylation sites in an efficient and accurate way.
40
Ahmed, Ehtasham, Muhammad Usman, Sibghatallah Anwar, Hafiz Muhammad Ahmad, Muhammad Waqar Nasir, and Muhammad Ali Ijaz Malik. "Application of ANN to predict performance and emissions of SI engine using gasoline-methanol blends." Science Progress 104, no. 1 (January 2021): 003685042110023. http://dx.doi.org/10.1177/00368504211002345.
Full textAbstract:
The deployment of methanol like alternative fuels in engines is a necessity of the present time to comprehend power requirements and environmental pollution. Furthermore, a comprehensive prediction of the impact of the methanol-gasoline blend on engine characteristics is also required in the era of artificial intelligence. The current study analyzes and compares the experimental and Artificial Neural Network (ANN) aided performance and emissions of four-stroke, single-cylinder SI engine using methanol-gasoline blends of 0%, 3%, 6%, 9%, 12%, 15%, and 18%. The experiments were performed at engine speeds of 1300–3700 rpm with constant loads of 20 and 40 psi for seven different fractions of fuels. Further, an ANN model has developed setting fuel blends, speed and load as inputs, and exhaust emissions and performance parameters as the target. The dataset was randomly divided into three groups of training (70%), validation (15%), and testing (15%) using MATLAB. The feedforward algorithm was used with tangent sigmoid transfer active function (tansig) and gradient descent with an adaptive learning method. It was observed that the continuous addition of methanol up to 12% (M12) increased the performance of the engine. However, a reduction in emissions was observed except for NOx emissions. The regression correlation coefficient (R) and the mean relative error (MRE) were in the range of 0.99100–0.99832 and 1.2%–2.4% respectively, while the values of root mean square error were extremely small. The findings depicted that M12 performed better than other fractions. ANN approach was found suitable for accurately predicting the performance and exhaust emissions of small-scaled SI engines.
41
Butt, Marya, and Ander de Keijzer. "Using Transfer Learning to Train a Binary Classifier for Lorrca Ektacytometery Microscopic Images of Sickle Cells and Healthy Red Blood Cells." Data 7, no. 9 (September 5, 2022): 126. http://dx.doi.org/10.3390/data7090126.
Full textAbstract:
Multiple blood images of stressed and sheared cells, taken by a Lorrca Ektacytometery microscope, needed a classification for biomedical researchers to assess several treatment options for blood-related diseases. The study proposes the design of a model capable of classifying these images, with high accuracy, into healthy Red Blood Cells (RBCs) or Sickle Cells (SCs) images. The performances of five Deep Learning (DL) models with two different optimizers, namely Adam and Stochastic Gradient Descent (SGD), were compared. The first three models consisted of 1, 2 and 3 blocks of CNN, respectively, and the last two models used a transfer learning approach to extract features. The dataset was first augmented, scaled, and then trained to develop models. The performance of the models was evaluated by testing on new images and was illustrated by confusion matrices, performance metrics (accuracy, recall, precision and f1 score), a receiver operating characteristic (ROC) curve and the area under the curve (AUC) value. The first, second and third models with the Adam optimizer could not achieve training, validation or testing accuracy above 50%. However, the second and third models with SGD optimizers showed good loss and accuracy scores during training and validation, but the testing accuracy did not exceed 51%. The fourth and fifth models used VGG16 and Resnet50 pre-trained models for feature extraction, respectively. VGG16 performed better than Resnet50, scoring 98% accuracy and an AUC of 0.98 with both optimizers. The study suggests that transfer learning with the VGG16 model helped to extract features from images for the classification of healthy RBCs and SCs, thus making a significant difference in performance comparing the first, second, third and fifth models.
42
Duan, Jinhuan, Xianxian Li, Shiqi Gao, Zili Zhong, and Jinyan Wang. "SSGD: A Safe and Efficient Method of Gradient Descent." Security and Communication Networks 2021 (August 5, 2021): 1–11. http://dx.doi.org/10.1155/2021/5404061.
Full textAbstract:
With the vigorous development of artificial intelligence technology, various engineering technology applications have been implemented one after another. The gradient descent method plays an important role in solving various optimization problems, due to its simple structure, good stability, and easy implementation. However, in multinode machine learning system, the gradients usually need to be shared, which will cause privacy leakage, because attackers can infer training data with the gradient information. In this paper, to prevent gradient leakage while keeping the accuracy of the model, we propose the super stochastic gradient descent approach to update parameters by concealing the modulus length of gradient vectors and converting it or them into a unit vector. Furthermore, we analyze the security of super stochastic gradient descent approach and demonstrate that our algorithm can defend against the attacks on the gradient. Experiment results show that our approach is obviously superior to prevalent gradient descent approaches in terms of accuracy, robustness, and adaptability to large-scale batches. Interestingly, our algorithm can also resist model poisoning attacks to a certain extent.
43
Liu, Lewis, and Kun Zhao. "Asynchronous Stochastic Gradient Descent for Extreme-Scale Recommender Systems." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 1 (May 18, 2021): 328–35. http://dx.doi.org/10.1609/aaai.v35i1.16108.
Full textAbstract:
Recommender systems are influential for many internet applications. As the size of the dataset provided for a recommendation model grows rapidly, how to utilize such amount of data effectively matters a lot. For a typical Click-Through-Rate(CTR) prediction model, the amount of daily samples can probably be up to hundreds of terabytes, which reaches dozens of petabytes at an extreme-scale when we take several days into consideration. Such data makes it essential to train the model parallelly and continuously. Traditional asynchronous stochastic gradient descent (ASGD) and its variants are proved efficient but often suffer from stale gradients. Hence, the model convergence tends to be worse as more workers are used. Moreover, the existing adaptive optimizers, which are friendly to sparse data, stagger in long-term training due to the significant imbalance between new and accumulated gradients. To address the challenges posed by extreme-scale data, we propose: 1) Staleness normalization and data normalization to eliminate the turbulence of stale gradients when training asynchronously in hundreds and thousands of workers; 2) SWAP, a novel framework for adaptive optimizers to balance the new and historical gradients by taking sampling period into consideration. We implement these approaches in TensorFlow and apply them to CTR tasks in real-world e- commerce scenarios. Experiments show that the number of workers in asynchronous training can be extended to 3000 with guaranteed convergence, and the final AUC is improved by more than 5 percentage.
44
Tas, Engin. "Large Scale Ranking Using Stochastic Gradient Descent." Proceedings of the Bulgarian Academy of Sciences 75, no. 10 (October 30, 2022): 1419–27. http://dx.doi.org/10.7546/crabs.2022.10.03.
Full textAbstract:
A system of linear equations can represent any ranking problem that minimizes a pairwise ranking loss. We utilize a fast version of gradient descent algorithm with a near-optimal learning rate and momentum factor to solve this linear equations system iteratively. Tikhonov regularization is also integrated into this framework to avoid overfitting problems where we have very large and high dimensional but sparse data.
45
Aalipour, Mehdi, Bohumil Šťastný, Filip Horký, and Bahman Jabbarian Amiri. "Scaling an Artificial Neural Network-Based Water Quality Index Model from Small to Large Catchments." Water 14, no. 6 (March 15, 2022): 920. http://dx.doi.org/10.3390/w14060920.
Full textAbstract:
Scaling models is one of the challenges for water resource planning and management, with the aim of bringing the developed models into practice by applying them to predict water quality and quantity for catchments that lack sufficient data. For this study, we evaluated artificial neural network (ANN) training algorithms to predict the water quality index in a source catchment. Then, multiple linear regression (MLR) models were developed, using the predicted water quality index of the ANN training algorithms and water quality variables, as dependent and independent variables, respectively. The most appropriate MLR model has been selected on the basis of the Akaike information criterion, sensitivity and uncertainty analyses. The performance of the MLR model was then evaluated by a variable aggregation and disaggregation approach, for upscaling and downscaling proposes, using the data from four very large- and three large-sized catchments and from eight medium-, three small- and seven very small-sized catchments, where they are located in the southern basin of the Caspian Sea. The performance of seven artificial neural network training algorithms, including Quick Propagation, Conjugate Gradient Descent, Quasi-Newton, Limited Memory Quasi-Newton, Levenberg–Marquardt, Online Back Propagation, and Batch Back Propagation, has been evaluated to predict the water quality index. The results show that the highest mean absolute error was observed in the WQI, as predicted by the ANN LM training algorithm; the lowest error values were for the ANN LMQN and CGD training algorithms. Our findings also indicate that for upscaling, the aggregated MLR model could provide reliable performance to predict the water quality index, since the r2 coefficient of the models varies from 0.73 ± 0.2 for large catchments, to 0.85 ± 0.15 for very large catchments, and for downscaling, the r2 coefficient of the disaggregated MLR model ranges from 0.93 ± 0.05 for very large catchments, to 0.97 ± 0.02 for medium catchments. Therefore, scaled models could be applied to catchments that lack sufficient data to perform a rapid assessment of the water quality index in the study area.
46
Guo, Yiyou, and Chao Wei. "Multi-Task Learning Using Gradient Balance and Clipping with an Application in Joint Disparity Estimation and Semantic Segmentation." Electronics 11, no. 8 (April 12, 2022): 1217. http://dx.doi.org/10.3390/electronics11081217.
Full textAbstract:
In this paper, we propose a novel multi-task learning (MTL) strategy from the gradient optimization view which enables automatically learning the optimal gradient from different tasks. In contrast with current multi-task learning methods which rely on careful network architecture adjustment or elaborate loss functions optimization, the proposed gradient-based MTL is simple and flexible. Specifically, we introduce a multi-task stochastic gradient descent optimization (MTSGD) to learn task-specific and shared representation in the deep neural network. In MTSGD, we decompose the total gradient into multiple task-specific sub-gradients and find the optimal sub-gradient via gradient balance and clipping operations. In this way, the learned network can satisfy the performance of specific task optimization while maintaining the shared representation. We take the joint learning of semantic segmentation and disparity estimation tasks as the exemplar to verify the effectiveness of the proposed method. Extensive experimental results on a large-scale dataset show that our proposed algorithm is superior to the baseline methods by a large margin. Meanwhile, we perform a series of ablation studies to have a deep analysis of gradient descent for MTL.
47
Raizada, S., and H. S. S. Sinha. "Some new features of electron density irregularities over SHAR during strong spread F." Annales Geophysicae 18, no. 2 (February 29, 2000): 141–51. http://dx.doi.org/10.1007/s00585-000-0141-8.
Full textAbstract:
Abstract. An RH-560 rocket flight was conducted from Sriharikota rocket range (SHAR) (14°N, 80°E, dip latitude 5.5°N) to study electron density and electric field irregularities during spread F. The rocket was launched at 2130 local time (LT) and it attained an apogee of 348 km. Results of electron density fluctuations are presented here. Two extremely sharp layers of very high electron density were observed at 105 and 130 km. The electron density increase in these layers was by a factor of 50 in a vertical extent of 10 km. Large depletions in electron density were observed around 175 and 238 km. Both sharp layers as well as depletions were observed also during the descent. The presence of sharp layers and depletions during the ascent and the descent of the rocket as well as an order of magnitude less electron density, in 150-300 km region during the descent, indicate the presence of strong large-scale horizontal gradients in the electron density. Some of the valley region irregularities (165-178 km), in the intermediate scale size range, observed during this flight, show spectral peaks at 2 km and can be interpreted in terms of the image striation theory suggested by Vickrey et al. The irregularities at 176 km do not exhibit any peak at kilometer scales and appear to be of new type. The growth rate of intermediate scale size irregularities, produced through generalized Rayleigh Taylor instability, was calculated for the 200-330 km altitude, using observed values of electron density gradients and an assumed vertically downward wind of 20 ms-1. These growth rate calculations suggest that the observed irregularities could be produced by the gradient drift instability.Key words: Ionosphere (equatorial ionosphere; ionospheric irregularities) - Radio science (ionospheric physics)
48
Liu, San-Yang, and Yuan-Yuan Huang. "Several Guaranteed Descent Conjugate Gradient Methods for Unconstrained Optimization." Journal of Applied Mathematics 2014 (2014): 1–14. http://dx.doi.org/10.1155/2014/825958.
Full textAbstract:
This paper investigates a general form of guaranteed descent conjugate gradient methods which satisfies the descent conditiongkTdk≤-1-1/4θkgk2 θk>1/4and which is strongly convergent whenever the weak Wolfe line search is fulfilled. Moreover, we present several specific guaranteed descent conjugate gradient methods and give their numerical results for large-scale unconstrained optimization.
49
Potluru, Vamsi. "Frugal Coordinate Descent for Large-Scale NNLS." Proceedings of the AAAI Conference on Artificial Intelligence 26, no. 1 (September 20, 2021): 2451–52. http://dx.doi.org/10.1609/aaai.v26i1.8432.
Full textAbstract:
The Nonnegative Least Squares (NNLS) formulation arises in many important regression problems. We present a novel coordinate descent method which differs from previous approaches in that we do not explicitly maintain complete gradient information. Empirical evidence shows that our approach outperforms a state-of-the-art NNLS solver in computation time for calculating radiation dosage for cancer treatment problems.
50
Alhawarat, Ahmad, Ghaliah Alhamzi, Ibitsam Masmali, and Zabidin Salleh. "A Descent Four-Term Conjugate Gradient Method with Global Convergence Properties for Large-Scale Unconstrained Optimisation Problems." Mathematical Problems in Engineering 2021 (August 14, 2021): 1–14. http://dx.doi.org/10.1155/2021/6219062.
Full textAbstract:
The conjugate gradient method is a useful method to solve large-scale unconstrained optimisation problems and to be used in some applications in several fields such as engineering, medical science, image restorations, neural network, and many others. The main benefit of the conjugate gradient method is not using the second derivative or its approximation, such as Newton’s method or its approximation. Moreover, the algorithm of the conjugate gradient method is simple and easy to apply. This study proposes a new modified conjugate gradient method that contains four terms depending on popular two- and three-term conjugate gradient methods. The new algorithm satisfies the descent condition. In addition, the new CG algorithm possesses the convergence property. In the numerical results part, we compare the new algorithm with famous methods such as CG-Descent. We conclude from numerical results that the new algorithm is more efficient than other popular CG methods such as CG-Descent 6.8 in terms of number of function evaluations, number of gradient evaluations, number of iterations, and CPU time.