To see the other types of publications on this topic, follow the link: Long Short-term Memory (LSTM) Networks.
Journal articles on the topic 'Long Short-term Memory (LSTM) Networks'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Long Short-term Memory (LSTM) Networks.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Singh, Arjun, Shashi Kant Dargar, Amit Gupta, Ashish Kumar, Atul Kumar Srivastava, Mitali Srivastava, Pradeep Kumar Tiwari, and Mohammad Aman Ullah. "Evolving Long Short-Term Memory Network-Based Text Classification." Computational Intelligence and Neuroscience 2022 (February 21, 2022): 1–11. http://dx.doi.org/10.1155/2022/4725639.
Full textAbstract:
Recently, long short-term memory (LSTM) networks are extensively utilized for text classification. Compared to feed-forward neural networks, it has feedback connections, and thus, it has the ability to learn long-term dependencies. However, the LSTM networks suffer from the parameter tuning problem. Generally, initial and control parameters of LSTM are selected on a trial and error basis. Therefore, in this paper, an evolving LSTM (ELSTM) network is proposed. A multiobjective genetic algorithm (MOGA) is used to optimize the architecture and weights of LSTM. The proposed model is tested on a well-known factory reports dataset. Extensive analyses are performed to evaluate the performance of the proposed ELSTM network. From the comparative analysis, it is found that the LSTM network outperforms the competitive models.
2
Hochreiter, Sepp, and Jürgen Schmidhuber. "Long Short-Term Memory." Neural Computation 9, no. 8 (November 1, 1997): 1735–80. http://dx.doi.org/10.1162/neco.1997.9.8.1735.
Full textAbstract:
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
3
Xu, Wei, Yanan Jiang, Xiaoli Zhang, Yi Li, Run Zhang, and Guangtao Fu. "Using long short-term memory networks for river flow prediction." Hydrology Research 51, no. 6 (October 5, 2020): 1358–76. http://dx.doi.org/10.2166/nh.2020.026.
Full textAbstract:
Abstract Deep learning has made significant advances in methodologies and practical applications in recent years. However, there is a lack of understanding on how the long short-term memory (LSTM) networks perform in river flow prediction. This paper assesses the performance of LSTM networks to understand the impact of network structures and parameters on river flow predictions. Two river basins with different characteristics, i.e., Hun river and Upper Yangtze river basins, are used as case studies for the 10-day average flow predictions and the daily flow predictions, respectively. The use of the fully connected layer with the activation function before the LSTM cell layer can substantially reduce learning efficiency. On the contrary, non-linear transformation following the LSTM cells is required to improve learning efficiency due to the different magnitudes of precipitation and flow. The batch size and the number of LSTM cells are sensitive parameters and should be carefully tuned to achieve a balance between learning efficiency and stability. Compared with several hydrological models, the LSTM network achieves good performance in terms of three evaluation criteria, i.e., coefficient of determination, Nash–Sutcliffe Efficiency and relative error, which demonstrates its powerful capacity in learning non-linear and complex processes in hydrological modelling.
4
Song, Tianyu, Wei Ding, Jian Wu, Haixing Liu, Huicheng Zhou, and Jinggang Chu. "Flash Flood Forecasting Based on Long Short-Term Memory Networks." Water 12, no. 1 (December 29, 2019): 109. http://dx.doi.org/10.3390/w12010109.
Full textAbstract:
Flash floods occur frequently and distribute widely in mountainous areas because of complex geographic and geomorphic conditions and various climate types. Effective flash flood forecasting with useful lead times remains a challenge due to its high burstiness and short response time. Recently, machine learning has led to substantial changes across many areas of study. In hydrology, the advent of novel machine learning methods has started to encourage novel applications or substantially improve old ones. This study aims to establish a discharge forecasting model based on Long Short-Term Memory (LSTM) networks for flash flood forecasting in mountainous catchments. The proposed LSTM flood forecasting (LSTM-FF) model is composed of T multivariate single-step LSTM networks and takes spatial and temporal dynamics information of observed and forecast rainfall and early discharge as inputs. The case study in Anhe revealed that the proposed models can effectively predict flash floods, especially the qualified rates (the ratio of the number of qualified events to the total number of flood events) of large flood events are above 94.7% at 1–5 h lead time and range from 84.2% to 89.5% at 6–10 h lead-time. For the large flood simulation, the small flood events can help the LSTM-FF model to explore a better rainfall-runoff relationship. The impact analysis of weights in the LSTM network structures shows that the discharge input plays a more obvious role in the 1-h LSTM network and the effect decreases with the lead-time. Meanwhile, in the adjacent lead-time, the LSTM networks explored a similar relationship between input and output. The study provides a new approach for flash flood forecasting and the highly accurate forecast contributes to prepare for and mitigate disasters.
5
Shankar, Sonali, P. Vigneswara Ilavarasan, Sushil Punia, and Surya Prakash Singh. "Forecasting container throughput with long short-term memory networks." Industrial Management & Data Systems 120, no. 3 (December 4, 2019): 425–41. http://dx.doi.org/10.1108/imds-07-2019-0370.
Full textAbstract:
Purpose Better forecasting always leads to better management and planning of the operations. The container throughput data are complex and often have multiple seasonality. This makes it difficult to forecast accurately. The purpose of this paper is to forecast container throughput using deep learning methods and benchmark its performance over other traditional time-series methods. Design/methodology/approach In this study, long short-term memory (LSTM) networks are implemented to forecast container throughput. The container throughput data of the Port of Singapore are used for empirical analysis. The forecasting performance of the LSTM model is compared with seven different time-series forecasting methods, namely, autoregressive integrated moving average (ARIMA), simple exponential smoothing, Holt–Winter’s, error-trend-seasonality, trigonometric regressors (TBATS), neural network (NN) and ARIMA + NN. The relative error matrix is used to analyze the performance of the different models with respect to bias, accuracy and uncertainty. Findings The results showed that LSTM outperformed all other benchmark methods. From a statistical perspective, the Diebold–Mariano test is also conducted to further substantiate better forecasting performance of LSTM over other counterpart methods. Originality/value The proposed study is a contribution to the literature on the container throughput forecasting and adds value to the supply chain theory of forecasting. Second, this study explained the architecture of the deep-learning-based LSTM method and discussed in detail the steps to implement it.
6
Nguyen, Sang Thi Thanh, and Bao Duy Tran. "Long Short-Term Memory Based Movie Recommendation." Science & Technology Development Journal - Engineering and Technology 3, SI1 (September 19, 2020): SI1—SI9. http://dx.doi.org/10.32508/stdjet.v3isi1.540.
Full textAbstract:
Recommender systems (RS) have become a fundamental tool for helping users make decisions around millions of different choices nowadays – the era of Big Data. It brings a huge benefit for many business models around the world due to their effectiveness on the target customers. A lot of recommendation models and techniques have been proposed and many accomplished incredible outcomes. Collaborative filtering and content-based filtering methods are common, but these both have some disadvantages. A critical one is that they only focus on a user's long-term static preference while ignoring his or her short-term transactional patterns, which results in missing the user's preference shift through the time. In this case, the user's intent at a certain time point may be easily submerged by his or her historical decision behaviors, which leads to unreliable recommendations. To deal with this issue, a session of user interactions with the items can be considered as a solution. In this study, Long Short-Term Memory (LSTM) networks will be analyzed to be applied to user sessions in a recommender system. The MovieLens dataset is considered as a case study of movie recommender systems. This dataset is preprocessed to extract user-movie sessions for user behavior discovery and making movie recommendations to users. Several experiments have been carried out to evaluate the LSTM-based movie recommender system. In the experiments, the LSTM networks are compared with a similar deep learning method, which is Recurrent Neural Networks (RNN), and a baseline machine learning method, which is the collaborative filtering using item-based nearest neighbors (item-KNN). It has been found that the LSTM networks are able to be improved by optimizing their hyperparameters and outperform the other methods when predicting the next movies interested by users.
7
Tra, Nguyen Ngoc, Ho Phuoc Tien, Nguyen Thanh Dat, and Nguyen Ngoc Vu. "VN-INDEX TREND PREDICTION USING LONG-SHORT TERM MEMORY NEURAL NETWORKS." Journal of Science and Technology: Issue on Information and Communications Technology 17, no. 12.2 (December 9, 2019): 61. http://dx.doi.org/10.31130/ict-ud.2019.94.
Full textAbstract:
The paper attemps to forecast the future trend of Vietnam index (VN-index) by using long-short term memory (LSTM) networks. In particular, an LSTM-based neural network is employed to study the temporal dependence in time-series data of past and present VN index values. Empirical forecasting results show that LSTM-based stock trend prediction offers an accuracy of about 60% which outperforms moving-average-based prediction.
8
Chen Wang, Chen Wang, Bingchun Liu Chen Wang, Jiali Chen Bingchun Liu, and Xiaogang Yu Jiali Chen. "Air Quality Index Prediction Based on a Long Short-Term Memory Artificial Neural Network Model." 電腦學刊 34, no. 2 (April 2023): 069–79. http://dx.doi.org/10.53106/199115992023043402006.
Full textAbstract:
<p>Air pollution has become one of the important challenges restricting the sustainable development of cities. Therefore, it is of great significance to achieve accurate prediction of Air Quality Index (AQI). Long Short Term Memory (LSTM) is a deep learning method suitable for learning time series data. Considering its superiority in processing time series data, this study established an LSTM forecasting model suitable for air quality index forecasting. First, we focus on optimizing the feature metrics of the model input through Information Gain (IG). Second, the prediction results of the LSTM model are compared with other machine learning models. At the same time the time step aspect of the LSTM model is used with selective experiments to ensure that model validation works properly. The results show that compared with other machine learning models, the LSTM model constructed in this paper is more suitable for the prediction of air quality index.</p> <p> </p>
9
Wang, Jianyong, Lei Zhang, Yuanyuan Chen, and Zhang Yi. "A New Delay Connection for Long Short-Term Memory Networks." International Journal of Neural Systems 28, no. 06 (June 24, 2018): 1750061. http://dx.doi.org/10.1142/s0129065717500617.
Full textAbstract:
Connections play a crucial role in neural network (NN) learning because they determine how information flows in NNs. Suitable connection mechanisms may extensively enlarge the learning capability and reduce the negative effect of gradient problems. In this paper, a new delay connection is proposed for Long Short-Term Memory (LSTM) unit to develop a more sophisticated recurrent unit, called Delay Connected LSTM (DCLSTM). The proposed delay connection brings two main merits to DCLSTM with introducing no extra parameters. First, it allows the output of the DCLSTM unit to maintain LSTM, which is absent in the LSTM unit. Second, the proposed delay connection helps to bridge the error signals to previous time steps and allows it to be back-propagated across several layers without vanishing too quickly. To evaluate the performance of the proposed delay connections, the DCLSTM model with and without peephole connections was compared with four state-of-the-art recurrent model on two sequence classification tasks. DCLSTM model outperformed the other models with higher accuracy and F1[Formula: see text]score. Furthermore, the networks with multiple stacked DCLSTM layers and the standard LSTM layer were evaluated on Penn Treebank (PTB) language modeling. The DCLSTM model achieved lower perplexity (PPL)/bit-per-character (BPC) than the standard LSTM model. The experiments demonstrate that the learning of the DCLSTM models is more stable and efficient.
10
Lees, Thomas, Steven Reece, Frederik Kratzert, Daniel Klotz, Martin Gauch, Jens De Bruijn, Reetik Kumar Sahu, Peter Greve, Louise Slater, and Simon J. Dadson. "Hydrological concept formation inside long short-term memory (LSTM) networks." Hydrology and Earth System Sciences 26, no. 12 (June 20, 2022): 3079–101. http://dx.doi.org/10.5194/hess-26-3079-2022.
Full textAbstract:
Abstract. Neural networks have been shown to be extremely effective rainfall-runoff models, where the river discharge is predicted from meteorological inputs. However, the question remains: what have these models learned? Is it possible to extract information about the learned relationships that map inputs to outputs, and do these mappings represent known hydrological concepts? Small-scale experiments have demonstrated that the internal states of long short-term memory networks (LSTMs), a particular neural network architecture predisposed to hydrological modelling, can be interpreted. By extracting the tensors which represent the learned translation from inputs (precipitation, temperature, and potential evapotranspiration) to outputs (discharge), this research seeks to understand what information the LSTM captures about the hydrological system. We assess the hypothesis that the LSTM replicates real-world processes and that we can extract information about these processes from the internal states of the LSTM. We examine the cell-state vector, which represents the memory of the LSTM, and explore the ways in which the LSTM learns to reproduce stores of water, such as soil moisture and snow cover. We use a simple regression approach to map the LSTM state vector to our target stores (soil moisture and snow). Good correlations (R2>0.8) between the probe outputs and the target variables of interest provide evidence that the LSTM contains information that reflects known hydrological processes comparable with the concept of variable-capacity soil moisture stores. The implications of this study are threefold: (1) LSTMs reproduce known hydrological processes. (2) While conceptual models have theoretical assumptions embedded in the model a priori, the LSTM derives these from the data. These learned representations are interpretable by scientists. (3) LSTMs can be used to gain an estimate of intermediate stores of water such as soil moisture. While machine learning interpretability is still a nascent field and our approach reflects a simple technique for exploring what the model has learned, the results are robust to different initial conditions and to a variety of benchmarking experiments. We therefore argue that deep learning approaches can be used to advance our scientific goals as well as our predictive goals.
11
Salman, Umar, Shafiqur Rehman, Basit Alawode, and Luai Alhems. "Short term prediction of wind speed based on long-short term memory networks." FME Transactions 49, no. 3 (2021): 643–52. http://dx.doi.org/10.5937/fme2103643s.
Full textAbstract:
Power utilities, developers, and investors are pushing towards larger penetrations of wind and solar energy-based power generation in their existing energy mix. This study, specifically, looks towards wind power deployment in Saudi Arabia. For profitable development of wind power, accurate knowledge of wind speed both in spatial and time domains is critical. The wind speed is the most fluctuating and intermittent parameter in nature compared to all the meteorological variables. This uncertain nature of wind speed makes wind power more difficult to predict ahead of time. Wind speed is dependent on meteorological factors such as pressure, temperature, and relative humidity and can be predicted using these meteorological parameters. The forecasting of wind speed is critical for grid management, cost of energy, and quality power supply. This study proposes a short-term, multi-dimensional prediction of wind speed based on Long-Short Term Memory Networks (LSTM). Five models are developed by training the networks with measured hourly mean wind speed values from1980 to 2019 including exogenous inputs (temperature and pressure). The study found that LSTM is a powerful tool for a short-term prediction of wind speed. However, the accuracy of LSTM may be compromised with the inclusion of exogenous features in the training sets and the duration of prediction ahead.
12
Min, Huasong, Ziming Chen, Bin Fang, Ziwei Xia, Yixu Song, Zongtao Wang, Quan Zhou, Fuchun Sun, and Chunfang Liu. "Cross-Individual Gesture Recognition Based on Long Short-Term Memory Networks." Scientific Programming 2021 (July 6, 2021): 1–11. http://dx.doi.org/10.1155/2021/6680417.
Full textAbstract:
Gestures recognition based on surface electromyography (sEMG) has been widely used for human-computer interaction. However, there are few research studies on overcoming the influence of physiological factors among different individuals. In this paper, a cross-individual gesture recognition method based on long short-term memory (LSTM) networks is proposed, named cross-individual LSTM (CI-LSTM). CI-LSTM has a dual-network structure, including a gesture recognition module and an individual recognition module. By designing the loss function, the individual information recognition module assists the gesture recognition module to train, which tends to orthogonalize the gesture features and individual features to minimize the impact of individual information differences on gesture recognition. Through cross-individual gesture recognition experiments, it is verified that compared with other selected algorithm models, the recognition accuracy obtained by using the CI-LSTM model can be improved by an average of 9.15%. Compared with other models, CI-LSTM can overcome the influence of individual characteristics and complete the task of cross-individual hand gestures recognition. Based on the proposed model, online control of the prosthetic hand is realized.
13
Kratzert, Frederik, Daniel Klotz, Claire Brenner, Karsten Schulz, and Mathew Herrnegger. "Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks." Hydrology and Earth System Sciences 22, no. 11 (November 22, 2018): 6005–22. http://dx.doi.org/10.5194/hess-22-6005-2018.
Full textAbstract:
Abstract. Rainfall–runoff modelling is one of the key challenges in the field of hydrology. Various approaches exist, ranging from physically based over conceptual to fully data-driven models. In this paper, we propose a novel data-driven approach, using the Long Short-Term Memory (LSTM) network, a special type of recurrent neural network. The advantage of the LSTM is its ability to learn long-term dependencies between the provided input and output of the network, which are essential for modelling storage effects in e.g. catchments with snow influence. We use 241 catchments of the freely available CAMELS data set to test our approach and also compare the results to the well-known Sacramento Soil Moisture Accounting Model (SAC-SMA) coupled with the Snow-17 snow routine. We also show the potential of the LSTM as a regional hydrological model in which one model predicts the discharge for a variety of catchments. In our last experiment, we show the possibility to transfer process understanding, learned at regional scale, to individual catchments and thereby increasing model performance when compared to a LSTM trained only on the data of single catchments. Using this approach, we were able to achieve better model performance as the SAC-SMA + Snow-17, which underlines the potential of the LSTM for hydrological modelling applications.
14
Sun, Lichao, Hang Qin, Krzysztof Przystupa, Michal Majka, and Orest Kochan. "Individualized Short-Term Electric Load Forecasting Using Data-Driven Meta-Heuristic Method Based on LSTM Network." Sensors 22, no. 20 (October 17, 2022): 7900. http://dx.doi.org/10.3390/s22207900.
Full textAbstract:
Short-term load forecasting is viewed as one promising technology for demand prediction under the most critical inputs for the promising arrangement of power plant units. Thus, it is imperative to present new incentive methods to motivate such power system operations for electricity management. This paper proposes an approach for short-term electric load forecasting using long short-term memory networks and an improved sine cosine algorithm called MetaREC. First, using long short-term memory networks for a special kind of recurrent neural network, the dispatching commands have the characteristics of storing and transmitting both long-term and short-term memories. Next, four important parameters are determined using the sine cosine algorithm base on a logistic chaos operator and multilevel modulation factor to overcome the inaccuracy of long short-term memory networks prediction, in terms of the manual selection of parameter values. Moreover, the performance of the MetaREC method outperforms others with regard to convergence accuracy and convergence speed on a variety of test functions. Finally, our analysis is extended to the scenario of the MetaREC_long short-term memory with back propagation neural network, long short-term memory networks with default parameters, long short-term memory networks with the conventional sine-cosine algorithm, and long short-term memory networks with whale optimization for power load forecasting on a real electric load dataset. Simulation results demonstrate that the multiple forecasts with MetaREC_long short-term memory can effectively incentivize the high accuracy and stability for short-term power load forecasting.
15
Alamri, Nawaf Mohammad H., Michael Packianather, and Samuel Bigot. "Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm." Applied Sciences 13, no. 4 (February 16, 2023): 2536. http://dx.doi.org/10.3390/app13042536.
Full textAbstract:
Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.
16
Regan, Kevin, Abolfazl Saghafi, and Zhijun Li. "Splice Junction Identification using Long Short-Term Memory Neural Networks." Current Genomics 22, no. 5 (December 6, 2021): 384–90. http://dx.doi.org/10.2174/1389202922666211011143008.
Full textAbstract:
Background: Splice junctions are the key to going from pre-messenger RNA to mature messenger RNA in many multi-exon genes due to alternative splicing. Since the percentage of multi-exon genes that undergo alternative splicing is very high, identifying splice junctions is an attractive research topic with important implications. Objective: The aim is to develop a deep learning model capable of identifying splice junctions in RNA sequences using 13,666 unique sequences of primate RNA. Method: A Long Short-Term Memory (LSTM) Neural Network model is developed that classifies a given sequence as EI (Exon-Intron splice), IE (Intron-Exon splice), or N (No splice). The model is trained with groups of trinucleotides and its performance is tested using validation and test data to prevent bias. Results: Model performance was measured using accuracy and f-score in test data. The finalized model achieved an average accuracy of 91.34% with an average f-score of 91.36% over 50 runs. Conclusion: Comparisons show a highly competitive model to recent Convolutional Neural Network structures. The proposed LSTM model achieves the highest accuracy and f-score among published alternative LSTM structures.
17
Wei, Xiaolu, Binbin Lei, Hongbing Ouyang, and Qiufeng Wu. "Stock Index Prices Prediction via Temporal Pattern Attention and Long-Short-Term Memory." Advances in Multimedia 2020 (December 10, 2020): 1–7. http://dx.doi.org/10.1155/2020/8831893.
Full textAbstract:
This study attempts to predict stock index prices using multivariate time series analysis. The study’s motivation is based on the notion that datasets of stock index prices involve weak periodic patterns, long-term and short-term information, for which traditional approaches and current neural networks such as Autoregressive models and Support Vector Machine (SVM) may fail. This study applied Temporal Pattern Attention and Long-Short-Term Memory (TPA-LSTM) for prediction to overcome the issue. The results show that stock index prices prediction through the TPA-LSTM algorithm could achieve better prediction performance over traditional deep neural networks, such as recurrent neural network (RNN), convolutional neural network (CNN), and long and short-term time series network (LSTNet).
18
Han, Mingchong, Aiguo Tan, and Jianwei Zhong. "Application of Particle Swarm Optimization Combined with Long and Short-term Memory Networks for Short-term Load Forecasting." Journal of Physics: Conference Series 2203, no. 1 (February 1, 2022): 012047. http://dx.doi.org/10.1088/1742-6596/2203/1/012047.
Full textAbstract:
Abstract In this paper, we apply the Long Short-Term Memory (LSTM) network to short-term load forecasting, and use the TensorFlow deep learning framework to build a Particle Swarm Optimization (PSO) model to optimize the parameters of the LSTM. Optimization (PSO) model to optimize the parameters of LSTM. In this paper, we use the meteorological data and historical load data of a certain place as the input of LSTM before and after optimization, and compare the model with the BP Neural Network before and after optimization, and the results show that the PSO-LSTM model has higher reliability and prediction accuracy.
19
BALOGLU, ULAS BARAN, and ÖZAL YILDIRIM. "CONVOLUTIONAL LONG-SHORT TERM MEMORY NETWORKS MODEL FOR LONG DURATION EEG SIGNAL CLASSIFICATION." Journal of Mechanics in Medicine and Biology 19, no. 01 (February 2019): 1940005. http://dx.doi.org/10.1142/s0219519419400050.
Full textAbstract:
Background and objective: Deep learning structures have recently achieved remarkable success in the field of machine learning. Convolutional neural networks (CNN) in image processing and long-short term memory (LSTM) in the time-series analysis are commonly used deep learning algorithms. Healthcare applications of deep learning algorithms provide important contributions for computer-aided diagnosis research. In this study, convolutional long-short term memory (CLSTM) network was used for automatic classification of EEG signals and automatic seizure detection. Methods: A new nine-layer deep network model consisting of convolutional and LSTM layers was designed. The signals processed in the convolutional layers were given as an input to the LSTM network whose outputs were processed in densely connected neural network layers. The EEG data is appropriate for a model having 1-D convolution layers. A bidirectional model was employed in the LSTM layer. Results: Bonn University EEG database with five different datasets was used for experimental studies. In this database, each dataset contains 23.6[Formula: see text]s duration 100 single channel EEG segments which consist of 4097 dimensional samples (173.61[Formula: see text]Hz). Eight two-class and three three-class clinical scenarios were examined. When the experimental results were evaluated, it was seen that the proposed model had high accuracy on both binary and ternary classification tasks. Conclusions: The proposed end-to-end learning structure showed a good performance without using any hand-crafted feature extraction or shallow classifiers to detect the seizures. The model does not require filtering, and also automatically learns to filter the input as well. As a result, the proposed model can process long duration EEG signals without applying segmentation, and can detect epileptic seizures automatically by using the correlation of ictal and interictal signals of raw data.
20
Zhang, Jun, Xiyao Cao, Jiemin Xie, and Pangao Kou. "An Improved Long Short-Term Memory Model for Dam Displacement Prediction." Mathematical Problems in Engineering 2019 (April 24, 2019): 1–14. http://dx.doi.org/10.1155/2019/6792189.
Full textAbstract:
Displacement plays a vital role in dam safety monitoring data, which adequately responds to security risks such as the flood water pressure, extreme temperature, structure deterioration, and bottom bedrock damage. To make accurate predictions, former researchers established various models. However, these models’ input variables cannot efficiently reflect the delays between the external environment and displacement. Therefore, a long short-term memory (LSTM) model is proposed to make full use of the historical data to reflect the delays. Furthermore, the LSTM model is improved to optimize the performance by making variables more physically reasonable. Finally, a real-world radial displacement dataset is used to compare the performance of LSTM models, multiple linear regression (MLR), multilayer perceptron (MLP) neural networks, support vector machine (SVM), and boosted regression tree (BRT). The results indicate that (1) the LSTM models can efficiently reflect the delays and make the variables selection more convenient and (2) the improved LSTM model achieves the best performance by optimizing the input form and network structure based on a clearer physical meaning.
21
Song, Fei, Yong Li, Wei Cheng, and Limeng Dong. "Learning to Track Multiple Radar Targets with Long Short-Term Memory Networks." Wireless Communications and Mobile Computing 2023 (February 15, 2023): 1–9. http://dx.doi.org/10.1155/2023/1033371.
Full textAbstract:
Radar multitarget tracking in a dense clutter environment remains a complex problem to be solved. Most existing solutions still rely on complex motion models and prior distribution knowledge. In this paper, a new online tracking method based on a long short-term memory (LSTM) network is proposed. It combines state prediction, measurement association, and trajectory management functions in an end-to-end manner. We employ LSTM networks to model target motion and trajectory associations, relying on their strong learning ability to learn target motion properties and long-term dependence of trajectory associations from noisy data. Moreover, to address the problem of missing appearance information of radar targets, we propose an architecture based on the LSTM network to calculate similarity function by extracting long-term motion features. And the similarity is applied to trajectory associations to improve their robustness. Our proposed method is validated in simulation scenarios and achieves good results.
22
Sen, Sanchari, and Anand Raghunathan. "Approximate Computing for Long Short Term Memory (LSTM) Neural Networks." IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, no. 11 (November 2018): 2266–76. http://dx.doi.org/10.1109/tcad.2018.2858362.
Full text
23
Wei, Chih-Chiang. "Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions." Advances in Meteorology 2020 (July 23, 2020): 1–18. http://dx.doi.org/10.1155/2020/5462040.
Full textAbstract:
Taiwan, being located on a path in the west Pacific Ocean where typhoons often strike, is often affected by typhoons. The accompanying strong winds and torrential rains make typhoons particularly damaging in Taiwan. Therefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better preparation to mitigate a typhoon’s toll on life and property. For more accurate wind speed predictions during a typhoon episode, we used cutting-edge machine learning techniques to construct a wind speed prediction model. To ensure model accuracy, we used, as variable input, simulated values from the Weather Research and Forecasting model of the numerical weather prediction system in addition to adopting deeper neural networks that can deepen neural network structures in the construction of estimation models. Our deeper neural networks comprise multilayer perceptron (MLP), deep recurrent neural networks (DRNNs), and stacked long short-term memory (LSTM). These three model-structure types differ by their memory capacity: MLPs are model networks with no memory capacity, whereas DRNNs and stacked LSTM are model networks with memory capacity. A model structure with memory capacity can analyze time-series data and continue memorizing and learning along the time axis. The study area is northeastern Taiwan. Results showed that MLP, DRNN, and stacked LSTM prediction error rates increased with prediction time (1–6 hours). Comparing the three models revealed that model networks with memory capacity (DRNN and stacked LSTM) were more accurate than those without memory capacity. A further comparison of model networks with memory capacity revealed that stacked LSTM yielded slightly more accurate results than did DRNN. Additionally, we determined that in the construction of the wind speed prediction model, the use of numerically simulated values reduced the error rate approximately by 30%. These results indicate that the inclusion of numerically simulated values in wind speed prediction models enhanced their prediction accuracy.
24
Zhou, Chenze. "Long Short-term Memory Applied on Amazon's Stock Prediction." Highlights in Science, Engineering and Technology 34 (February 28, 2023): 71–76. http://dx.doi.org/10.54097/hset.v34i.5380.
Full textAbstract:
More and more investors are paying attention to how to use data mining technology into stock investing decisions as a result of the introduction of big data and the quick expansion of financial markets. Machine learning can automatically apply complex mathematical calculations to big data repeatedly and faster. The machine model can analyze all the factors and indicators affecting stock price and achieve high efficiency. Based on the Amazon stock price published on Kaggle, this paper adopts the Long Short-term Memory (LSTM) method for model training. The Keras package in the Python program is used to normalize the data. The Sequence model in Keras establishes a two-layer LSTM network and a three-layer LSTM network to compare and analyze the fitting effect of the model on stock prices. By calculating RMSE and RMPE, the study found that the stock price prediction accuracy of two-layer LSTM is similar to that of three-layer LSTM. In terms of F-measure and Accuracy, the LSTM model of the three-layer network is significantly better than the LSTM model of the two-layer network layer. In general, the LSTM model can accurately predict stock price. Therefore, investors will know the upward or downward trend of stock prices in advance according to the prediction results of the model to make corresponding decisions.
25
Vlachas, Pantelis R., Wonmin Byeon, Zhong Y. Wan, Themistoklis P. Sapsis, and Petros Koumoutsakos. "Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks." Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474, no. 2213 (May 2018): 20170844. http://dx.doi.org/10.1098/rspa.2017.0844.
Full textAbstract:
We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto–Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM–LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.
26
Santra, Arpita Samanta, and Jun-Lin Lin. "Integrating Long Short-Term Memory and Genetic Algorithm for Short-Term Load Forecasting." Energies 12, no. 11 (May 28, 2019): 2040. http://dx.doi.org/10.3390/en12112040.
Full textAbstract:
Electricity load forecasting is an important task for enhancing energy efficiency and operation reliability of the power system. Forecasting the hourly electricity load of the next day assists in optimizing the resources and minimizing the energy wastage. The main motivation of this study was to improve the robustness of short-term load forecasting (STLF) by utilizing long short- term memory (LSTM) and genetic algorithm (GA). The proposed method is novel: LSTM networks are designed to avoid the problem of long-term dependencies, and GA is used to obtain the optimal LSTM’s parameters, which are then applied to predict the hourly electricity load for the next day. The proposed method was trained using actual load and weather data, and the performance results showed that it yielded small mean absolute percentage error on the test data.
27
Xie, Yuhong, Yuzuru Ueda, and Masakazu Sugiyama. "A Two-Stage Short-Term Load Forecasting Method Using Long Short-Term Memory and Multilayer Perceptron." Energies 14, no. 18 (September 16, 2021): 5873. http://dx.doi.org/10.3390/en14185873.
Full textAbstract:
Load forecasting is an essential task in the operation management of a power system. Electric power companies utilize short-term load forecasting (STLF) technology to make reasonable power generation plans. A forecasting model with low prediction errors helps reduce operating costs and risks for the operators. In recent years, machine learning has become one of the most popular technologies for load forecasting. In this paper, a two-stage STLF model based on long short-term memory (LSTM) and multilayer perceptron (MLP), which improves the forecasting accuracy over the entire time horizon, is proposed. In the first stage, a sequence-to-sequence (seq2seq) architecture, which can handle a multi-sequence of input to extract more features of historical data than that of single sequence, is used to make multistep predictions. In the second stage, the MLP is used for residual modification by perceiving other information that the LSTM cannot. To construct the model, we collected the electrical load, calendar, and meteorological records of Kanto region in Japan for four years. Unlike other LSTM-based hybrid architectures, the proposed model uses two independent neural networks instead of making the neural network deeper by concatenating a series of LSTM cells and convolutional neural networks (CNNs). Therefore, the proposed model is easy to be trained and more interpretable. The seq2seq module performs well in the first few hours of the predictions. The MLP inherits the advantage of the seq2seq module and improves the results by feeding artificially selected features both from historical data and information of the target day. Compared to the LSTM-AM model and single MLP model, the mean absolute percentage error (MAPE) of the proposed model decreases from 2.82% and 2.65% to 2%, respectively. The results demonstrate that the MLP helps improve the prediction accuracy of seq2seq module and the proposed model achieves better performance than other popular models. In addition, this paper also reveals the reason why the MLP achieves the improvement.
28
Hong, Binjie, Zhijie Yan, Yingxi Chen, and Xiaobo-Jin. "Long Memory Gated Recurrent Unit for Time Series Classification." Journal of Physics: Conference Series 2278, no. 1 (May 1, 2022): 012017. http://dx.doi.org/10.1088/1742-6596/2278/1/012017.
Full textAbstract:
Abstract Time series analysis is an important and challenging problem in data mining, where time series is a class of temporal data objects. In the classification task, the label is dependent on the features from the last moments. Due to the time dependency, the recurrent neural networks, as one of the prevalent learning-based architectures, take advantage of the relation among history data. The Long Short-Term Memory Network (LSTM) and Gated Recurrent Unit (GRU) are two popular artificial recurrent neural networks used in the field of deep learning. LSTM designed a gate-like method to control the short and long historical information, and GRU simplified those gates to obtain more efficient training. In our work, we propose a new model called as Long Memory Gated Recurrent Unit (LMGRU) based on such two remarkable models, where the reset gate is introduced to reset the stored value of the cell in Long Short-Term Memory (LSTM) model but the forget gate and the input gate are omitted. The experimental results on several time series benchmarks show that LMGRU achieves better effectiveness and efficiency than LSTM and GRU.
29
Sher, Madiha, Nasru Minallah, Tufail Ahmad, and Waleed Khan. "Hyperparameters analysis of long short-term memory architecture for crop classification." International Journal of Electrical and Computer Engineering (IJECE) 13, no. 4 (August 1, 2023): 4661. http://dx.doi.org/10.11591/ijece.v13i4.pp4661-4670.
Full textAbstract:
<span lang="EN-US">Deep learning (DL) has seen a massive rise in popularity for remote sensing (RS) based applications over the past few years. However, the performance of DL algorithms is dependent on the optimization of various hyperparameters since the hyperparameters have a huge impact on the performance of deep neural networks. The impact of hyperparameters on the accuracy and reliability of DL models is a significant area for investigation. In this study, the grid Search algorithm is used for hyperparameters optimization of long short-term memory (LSTM) network for the RS-based classification. The hyperparameters considered for this study are, optimizer, activation function, batch size, and the number of LSTM layers. In this study, over 1,000 hyperparameter sets are evaluated and the result of all the sets are analyzed to see the effects of various combinations of hyperparameters as well the individual parameter effect on the performance of the LSTM model. The performance of the LSTM model is evaluated using the performance metric of minimum loss and average loss and it was found that classification can be highly affected by the choice of optimizer; however, other parameters such as the number of LSTM layers have less influence.</span>
30
Bukhari, Syed Basit Ali, Khawaja Khalid Mehmood, Abdul Wadood, and Herie Park. "Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks." Energies 14, no. 18 (September 13, 2021): 5762. http://dx.doi.org/10.3390/en14185762.
Full textAbstract:
This paper presents a new intelligent islanding detection scheme (IIDS) based on empirical wavelet transform (EWT) and long short-term memory (LSTM) network to identify islanding events in microgrids. The concept of EWT is extended to extract features from three-phase signals. First, the three-phase voltage signals sampled at the terminal of targeted distributed energy resource (DER) or point of common coupling (PCC) are decomposed into empirical modes/frequency subbands using EWT. Then, instantaneous amplitudes and instantaneous frequencies of the three-phases at different frequency subbands are combined, and various statistical features are calculated. Finally, the EWT-based features along with the three-phase voltage signals are input to the LSTM network to differentiate between non-islanding and islanding events. To assess the efficacy of the proposed IIDS, extensive simulations are performed on an IEC microgrid and an IEEE 34-node system. The simulation results verify the effectiveness of the proposed IIDS in terms of non-detection zone (NDZ), computational time, detection accuracy, and robustness against noisy measurement. Furthermore, comparisons with existing intelligent methods and different LSTM architectures demonstrate that the proposed IIDS offers higher reliability by significantly reducing the NDZ and stands robust against measurements uncertainty.
31
KADARI, REKIA, YU ZHANG, WEINAN ZHANG, and TING LIU. "CCG supertagging with bidirectional long short-term memory networks." Natural Language Engineering 24, no. 1 (September 4, 2017): 77–90. http://dx.doi.org/10.1017/s1351324917000250.
Full textAbstract:
AbstractNeural Network-based approaches have recently produced good performances in Natural language tasks, such as Supertagging. In the supertagging task, a Supertag (Lexical category) is assigned to each word in an input sequence. Combinatory Categorial Grammar Supertagging is a more challenging problem than various sequence-tagging problems, such as part-of-speech (POS) tagging and named entity recognition due to the large number of the lexical categories. Specifically, simple Recurrent Neural Network (RNN) has shown to significantly outperform the previous state-of-the-art feed-forward neural networks. On the other hand, it is well known that Recurrent Networks fail to learn long dependencies. In this paper, we introduce a new neural network architecture based on backward and Bidirectional Long Short-Term Memory (BLSTM) Networks that has the ability to memorize information for long dependencies and benefit from both past and future information. State-of-the-art methods focus on previous information, whereas BLSTM has access to information in both previous and future directions. Our main findings are that bidirectional networks outperform unidirectional ones, and Long Short-Term Memory (LSTM) networks are more precise and successful than both unidirectional and bidirectional standard RNNs. Experiment results reveal the effectiveness of our proposed method on both in-domain and out-of-domain datasets. Experiments show improvements about (1.2 per cent) over standard RNN.
32
S, Gnanasaravanan, Tharani B, and Mona Sahu. "Long Short-Term Memory Recurrent Neural Networks for Plant disease Identification." International Journal of Computer Communication and Informatics 3, no. 2 (October 30, 2021): 51–62. http://dx.doi.org/10.34256/ijcci2125.
Full textAbstract:
Farming profitability is something on which economy profoundly depends. This is the one reason that sickness recognition in plants assumes a critical job in farming field, as having infection in plants are very common. In the event that legitimate consideration isn't taken here, it causes genuine consequences for plants and because of which particular item quality, amount or profitability is influenced. This paper displays an algorithm for image segmentation technique which is utilized for automatic identification and classification plant leaf infections. It additionally covers review on various classification techniques that can be utilized for plant leaf ailment discovery. As the infected regions vary in length it is difficult to develop a feature vector of identical finite length representing all the sequences. A simple method to go around this issue is given by Recurrent Neural Networks (RNN). In this work we separate a feature vector through the use of Long Short-Term Memory (LSTM) recurrent neural networks. The LSTM network recursively repeats and concentrates two limited vectors whose link yields finite length vector portrayal.
33
Hu, Caihong, Qiang Wu, Hui Li, Shengqi Jian, Nan Li, and Zhengzheng Lou. "Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation." Water 10, no. 11 (October 30, 2018): 1543. http://dx.doi.org/10.3390/w10111543.
Full textAbstract:
Considering the high random and non-static property of the rainfall-runoff process, lots of models are being developed in order to learn about such a complex phenomenon. Recently, Machine learning techniques such as the Artificial Neural Network (ANN) and other networks have been extensively used by hydrologists for rainfall-runoff modelling as well as for other fields of hydrology. However, deep learning methods such as the state-of-the-art for LSTM networks are little studied in hydrological sequence time-series predictions. We deployed ANN and LSTM network models for simulating the rainfall-runoff process based on flood events from 1971 to 2013 in Fen River basin monitored through 14 rainfall stations and one hydrologic station in the catchment. The experimental data were from 98 rainfall-runoff events in this period. In between 86 rainfall-runoff events were used as training set, and the rest were used as test set. The results show that the two networks are all suitable for rainfall-runoff models and better than conceptual and physical based models. LSTM models outperform the ANN models with the values of R 2 and N S E beyond 0.9, respectively. Considering different lead time modelling the LSTM model is also more stable than ANN model holding better simulation performance. The special units of forget gate makes LSTM model better simulation and more intelligent than ANN model. In this study, we want to propose new data-driven methods for flood forecasting.
34
Liu, Feiyang, Panke Qin, Junru You, and Yanyan Fu. "Sparrow Search Algorithm-Optimized Long Short-Term Memory Model for Stock Trend Prediction." Computational Intelligence and Neuroscience 2022 (August 12, 2022): 1–11. http://dx.doi.org/10.1155/2022/3680419.
Full textAbstract:
The long short-term memory (LSTM) network is especially suitable for dealing with time series-related problems, which has led to a wide range of applications in analyzing stock market quotations and predicting future price trends. However, the selection of hyperparameters in LSTM networks was often based on subjective experience and existing research. The inability to determine the optimal values of the parameters results in a reduced generalization capability of the model. Therefore, we proposed a sparrow search algorithm-optimized LSTM (SSA-LSTM) model for stock trend prediction. The SSA was used to find the optimal hyperparameters of the LSTM model to adapt the features of the data to the structure of the model, so as to construct a highly accurate stock trend prediction model. With the Shanghai Composite Index stock data in the last decade, the mean absolute percentage error, root mean square error, mean absolute error, and coefficient of determination between stock prices predicted by the SSA-LSTM method and actual prices are 0.0093, 41.9505, 30.5300, and 0.9754. The result indicates that the proposed model possesses higher forecasting precision than other traditional stock forecasting methods and enhances the interpretability of the network model structure and parameters.
35
Arifin, Samsul, AndyanKalmer Wijaya, Rinda Nariswari, I. Gusti Agung Anom Yudistira, Suwarno Suwarno, Faisal Faisal, and Diah Wihardini. "Long Short-Term Memory (LSTM): Trends and Future Research Potential." International Journal of Emerging Technology and Advanced Engineering 13, no. 5 (May 13, 2023): 24–34. http://dx.doi.org/10.46338/ijetae0523_04.
Full textAbstract:
-One of the most widely used machine learning methods, Long Short-Term Memory (LSTM), is particularly useful for time series prediction. In this study, we carried out a bibliometric analysis against publications about LSTMs to identify trends and contributions of researchers in the development of machine learning technology. We collect bibliometric data from the Scopus database and use the bibliometric analysis method to analyze trends and contributions of researchers in publications about LSTM. Results of the bibliometric analysis show that LSTM is a lot used in related machine learning applications with time series data and is one the most popular technique for use in predictions. In addition, the use of LSTM is often combined with other deep learning methods, such as neural networks, to improve accuracy prediction. In addition, the results of the bibliometric analysis also show that the use of LSTM has spread to various fields, such as in handwriting recognition, processing Language experience, and recognition of a face. Implications from the results of this study are that the use of LSTM can provide solutions that are accurate and effective in solving prediction problems in various fields, especially in practical applications such as business, health, and transportation. The results of the LSTM bibliometric analysis can provide a broader view of trends and the contributions of researchers to the development of machine learning technology, as well as identify potential research areas for further development. Therefore, this research provides an important contribution to strengthening the results of previous research and showing that the use of LSTM has great potential in the development of future machine learning technology
36
Kumar, Naresh, Jatin Bindra, Rajat Sharma, and Deepali Gupta. "Air Pollution Prediction Using Recurrent Neural Network, Long Short-Term Memory and Hybrid of Convolutional Neural Network and Long Short-Term Memory Models." Journal of Computational and Theoretical Nanoscience 17, no. 9 (July 1, 2020): 4580–84. http://dx.doi.org/10.1166/jctn.2020.9283.
Full textAbstract:
Air pollution prediction was not an easy task few years back. With the increasing computation power and wide availability of the datasets, air pollution prediction problem is solved to some extend. Inspired by the deep learning models, in this paper three techniques for air pollution prediction have been proposed. The models used includes recurrent neural network (RNN), Long short-term memory (LSTM) and a hybrid combination of Convolutional neural network (CNN) and LSTM models. These models are tested by comparing MSE loss on air pollution test of Belgium. The validation loss on RNN is 0.0045, LSTM is 0.00441 and CNN and LSTM is 0.0049. The loss on testing dataset for these models are 0.00088, 0.00441 and 0.0049 respectively.
37
Muneer, Amgad, Rao Faizan Ali, Ahmed Almaghthawi, Shakirah Mohd Taib, Amal Alghamdi, and Ebrahim Abdulwasea Abdullah Ghaleb. "Short term residential load forecasting using long short-term memory recurrent neural network." International Journal of Electrical and Computer Engineering (IJECE) 12, no. 5 (October 1, 2022): 5589. http://dx.doi.org/10.11591/ijece.v12i5.pp5589-5599.
Full textAbstract:
<span>Load forecasting plays an essential role in power system planning. The efficiency and reliability of the whole power system can be increased with proper planning and organization. Residential load forecasting is indispensable due to its increasing role in the smart grid environment. Nowadays, smart meters can be deployed at the residential level for collecting historical data consumption of residents. Although the employment of smart meters ensures large data availability, the inconsistency of load data makes it challenging and taxing to forecast accurately. Therefore, the traditional forecasting techniques may not suffice the purpose. However, a deep learning forecasting network-based long short-term memory (LSTM) is proposed in this paper. The powerful nonlinear mapping capabilities of RNN in time series make it effective along with the higher learning capabilities of long sequences of LSTM. The proposed method is tested and validated through available real-world data sets. A comparison of LSTM is then made with two traditionally available techniques, exponential smoothing and auto-regressive integrated moving average model (ARIMA). Real data from 12 houses over three months is used to evaluate and validate the performance of load forecasts performed using the three mentioned techniques. LSTM model has achieved the best results due to its higher capability of memorizing large data in time series-based predictions.</span>
38
Wang, Ying, Bo Feng, Qing-Song Hua, and Li Sun. "Short-Term Solar Power Forecasting: A Combined Long Short-Term Memory and Gaussian Process Regression Method." Sustainability 13, no. 7 (March 25, 2021): 3665. http://dx.doi.org/10.3390/su13073665.
Full textAbstract:
Solar power is considered a promising power generation candidate in dealing with climate change. Because of the strong randomness, volatility, and intermittence, its safe integration into the smart grid requires accurate short-term forecasting with the required accuracy. The use of solar power should meet requirements proscribed by environmental law and safety standards applied for consumer protection. First, time-series-based solar power forecasting (SPF) model is developed with the time element and predicted weather information from the local meteorological station. Considering the data correlation, long short-term memory (LSTM) algorithm is utilized for short-term SPF. However, the point prediction provided by LSTM fails in revealing the underlying uncertainty range of the solar power output, which is generally needed in some stochastic optimization frameworks. A novel hybrid strategy combining LSTM and Gaussian process regression (GPR), namely LSTM-GPR, is proposed to obtain a highly accurate point prediction with a reliable interval estimation. The hybrid model is evaluated in comparison with other algorithms in terms of two aspects: Point prediction accuracy and interval forecasting reliability. Numerical investigations confirm the superiority of LSTM algorithm over the conventional neural networks. Furthermore, the performance of the proposed hybrid model is demonstrated to be slightly better than the individual LSTM model and significantly superior to the individual GPR model in both point prediction and interval forecasting, indicating a promising prospect for future SPF applications.
39
Hopp, Daniel. "Economic Nowcasting with Long Short-Term Memory Artificial Neural Networks (LSTM)." Journal of Official Statistics 38, no. 3 (September 1, 2022): 847–73. http://dx.doi.org/10.2478/jos-2022-0037.
Full textAbstract:
Abstract Artificial neural networks (ANNs) have been the catalyst to numerous advances in a variety of fields and disciplines in recent years. Their impact on economics, however, has been comparatively muted. One type of ANN, the long short-term memory network (LSTM), is particularly well-suited to deal with economic time-series. Here, the architecture’s performance and characteristics are evaluated in comparison with the dynamic factor model (DFM), currently a popular choice in the field of economic nowcasting. LSTMs are found to produce superior results to DFMs in the nowcasting of three separate variables; global merchandise export values and volumes, and global services exports. Further advantages include their ability to handle large numbers of input features in a variety of time frequencies. A disadvantage is the stochastic nature of outputs, common to all ANNs. In order to facilitate continued applied research of the methodology by avoiding the need for any knowledge of deep-learning libraries, an accompanying Python (Hopp 2021a) library was developed using PyTorch. The library is also available in R, MATLAB, and Julia.
40
Ayzel, Georgy, Liubov Kurochkina, Dmitriy Abramov, and Sergei Zhuravlev. "Development of a Regional Gridded Runoff Dataset Using Long Short-Term Memory (LSTM) Networks." Hydrology 8, no. 1 (January 8, 2021): 6. http://dx.doi.org/10.3390/hydrology8010006.
Full textAbstract:
Gridded datasets provide spatially and temporally consistent runoff estimates that serve as reliable sources for assessing water resources from regional to global scales. This study presents LSTM-REG, a regional gridded runoff dataset for northwest Russia based on Long Short-Term Memory (LSTM) networks. LSTM-REG covers the period from 1980 to 2016 at a 0.5° spatial and daily temporal resolution. LSTM-REG has been extensively validated and benchmarked against GR4J-REG, a gridded runoff dataset based on a parsimonious regionalization scheme and the GR4J hydrological model. While both datasets provide runoff estimates with reliable prediction efficiency, LSTM-REG outperforms GR4J-REG for most basins in the independent evaluation set. Thus, the results demonstrate a higher generalization capacity of LSTM-REG than GR4J-REG, which can be attributed to the higher efficiency of the proposed LSTM-based regionalization scheme. The developed datasets are freely available in open repositories to foster further regional hydrology research in northwest Russia.
41
Ayzel, Georgy, Liubov Kurochkina, Dmitriy Abramov, and Sergei Zhuravlev. "Development of a Regional Gridded Runoff Dataset Using Long Short-Term Memory (LSTM) Networks." Hydrology 8, no. 1 (January 8, 2021): 6. http://dx.doi.org/10.3390/hydrology8010006.
Full textAbstract:
Gridded datasets provide spatially and temporally consistent runoff estimates that serve as reliable sources for assessing water resources from regional to global scales. This study presents LSTM-REG, a regional gridded runoff dataset for northwest Russia based on Long Short-Term Memory (LSTM) networks. LSTM-REG covers the period from 1980 to 2016 at a 0.5° spatial and daily temporal resolution. LSTM-REG has been extensively validated and benchmarked against GR4J-REG, a gridded runoff dataset based on a parsimonious regionalization scheme and the GR4J hydrological model. While both datasets provide runoff estimates with reliable prediction efficiency, LSTM-REG outperforms GR4J-REG for most basins in the independent evaluation set. Thus, the results demonstrate a higher generalization capacity of LSTM-REG than GR4J-REG, which can be attributed to the higher efficiency of the proposed LSTM-based regionalization scheme. The developed datasets are freely available in open repositories to foster further regional hydrology research in northwest Russia.
42
Rußwurm, M., and M. Körner. "MULTI-TEMPORAL LAND COVER CLASSIFICATION WITH LONG SHORT-TERM MEMORY NEURAL NETWORKS." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-1/W1 (May 31, 2017): 551–58. http://dx.doi.org/10.5194/isprs-archives-xlii-1-w1-551-2017.
Full textAbstract:
<i>Land cover classification (LCC)</i> is a central and wide field of research in earth observation and has already put forth a variety of classification techniques. Many approaches are based on classification techniques considering observation at certain points in time. However, some land cover classes, such as crops, change their spectral characteristics due to environmental influences and can thus not be monitored effectively with classical mono-temporal approaches. Nevertheless, these temporal observations should be utilized to benefit the classification process. After extensive research has been conducted on modeling temporal dynamics by spectro-temporal profiles using vegetation indices, we propose a deep learning approach to utilize these temporal characteristics for classification tasks. In this work, we show how <i>long short-term memory</i> (LSTM) neural networks can be employed for crop identification purposes with SENTINEL 2A observations from large study areas and label information provided by local authorities. We compare these temporal neural network models, <i>i.e.</i>, LSTM and <i>recurrent neural network</i> (RNN), with a classical non-temporal <i>convolutional neural network</i> (CNN) model and an additional <i>support vector machine</i> (SVM) baseline. With our rather straightforward LSTM variant, we exceeded state-of-the-art classification performance, thus opening promising potential for further research.
43
Awad, Asmaa Ahmed, Ahmed Fouad Ali, and Tarek Gaber. "An improved long short term memory network for intrusion detection." PLOS ONE 18, no. 8 (August 1, 2023): e0284795. http://dx.doi.org/10.1371/journal.pone.0284795.
Full textAbstract:
Over the years, intrusion detection system has played a crucial role in network security by discovering attacks from network traffics and generating an alarm signal to be sent to the security team. Machine learning methods, e.g., Support Vector Machine, K Nearest Neighbour, have been used in building intrusion detection systems but such systems still suffer from low accuracy and high false alarm rate. Deep learning models (e.g., Long Short-Term Memory, LSTM) have been employed in designing intrusion detection systems to address this issue. However, LSTM needs a high number of iterations to achieve high performance. In this paper, a novel, and improved version of the Long Short-Term Memory (ILSTM) algorithm was proposed. The ILSTM is based on the novel integration of the chaotic butterfly optimization algorithm (CBOA) and particle swarm optimization (PSO) to improve the accuracy of the LSTM algorithm. The ILSTM was then used to build an efficient intrusion detection system for binary and multi-class classification cases. The proposed algorithm has two phases: phase one involves training a conventional LSTM network to get initial weights, and phase two involves using the hybrid swarm algorithms, CBOA and PSO, to optimize the weights of LSTM to improve the accuracy. The performance of ILSTM and the intrusion detection system were evaluated using two public datasets (NSL-KDD dataset and LITNET-2020) under nine performance metrics. The results showed that the proposed ILSTM algorithm outperformed the original LSTM and other related deep-learning algorithms regarding accuracy and precision. The ILSTM achieved an accuracy of 93.09% and a precision of 96.86% while LSTM gave an accuracy of 82.74% and a precision of 76.49%. Also, the ILSTM performed better than LSTM in both datasets. In addition, the statistical analysis showed that ILSTM is more statistically significant than LSTM. Further, the proposed ISTLM gave better results of multiclassification of intrusion types such as DoS, Prob, and U2R attacks.
44
Balmuri, Kavitha Rani, Srinivas Konda, Wen-Cheng Lai, Parameshachari Bidare Divakarachari, Kavitha Malali Vishveshwarappa Gowda, and Hemalatha Kivudujogappa Lingappa. "A Long Short-Term Memory Network-Based Radio Resource Management for 5G Network." Future Internet 14, no. 6 (June 14, 2022): 184. http://dx.doi.org/10.3390/fi14060184.
Full textAbstract:
Nowadays, the Long-Term Evolution-Advanced system is widely used to provide 5G communication due to its improved network capacity and less delay during communication. The main issues in the 5G network are insufficient user resources and burst errors, because it creates losses in data transmission. In order to overcome this, an effective Radio Resource Management (RRM) is required to be developed in the 5G network. In this paper, the Long Short-Term Memory (LSTM) network is proposed to develop the radio resource management in the 5G network. The proposed LSTM-RRM is used for assigning an adequate power and bandwidth to the desired user equipment of the network. Moreover, the Grid Search Optimization (GSO) is used for identifying the optimal hyperparameter values for LSTM. In radio resource management, a request queue is used to avoid the unwanted resource allocation in the network. Moreover, the losses during transmission are minimized by using frequency interleaving and guard level insertion. The performance of the LSTM-RRM method has been analyzed in terms of throughput, outage percentage, dual connectivity, User Sum Rate (USR), Threshold Sum Rate (TSR), Outdoor Sum Rate (OSR), threshold guaranteed rate, indoor guaranteed rate, and outdoor guaranteed rate. The indoor guaranteed rate of LSTM-RRM for 1400 m of building distance improved up to 75.38% compared to the existing QOC-RRM.
45
Muthukumaran V., Vinothkumar V., Rose Bindu Joseph, Meram Munirathanam, and Balajee Jeyakumar. "Improving Network Security Based on Trust-Aware Routing Protocols Using Long Short-Term Memory-Queuing Segment-Routing Algorithms." International Journal of Information Technology Project Management 12, no. 4 (October 2021): 47–60. http://dx.doi.org/10.4018/ijitpm.2021100105.
Full textAbstract:
Defending all single connection failures for a particular system, segment routing issue, the switch will focus on the problems of selecting a small subset of trust-aware routing to improve the deep learning (DL). In the end, even if there were multiple path failures, these paths may introduce long-term, unnecessary overload in the proposed long short-term memory networks-based queuing routing segmentation (LSTM-QRS) experience of reducing traffic delays and adjusting traffic length by reducing network bandwidth. The critical factor is a novel traffic repair technique used to create a traffic repair path that switches to software-defined network (SDN) using multiple routing and providing additional flexibility in re-routing using long short-term memory networks (LSTM)-based queuing routing segment (LSTM-QRS) algorithms. It reduces the repair path length and recommends replacing the target-based traffic with the connection-based traffic fault detection router to avoid targeted traffic network congestion.
46
Han, Xu, and Shicai Gong. "LST-GCN: Long Short-Term Memory Embedded Graph Convolution Network for Traffic Flow Forecasting." Electronics 11, no. 14 (July 17, 2022): 2230. http://dx.doi.org/10.3390/electronics11142230.
Full textAbstract:
Traffic flow prediction is an important part of the intelligent transportation system. Accurate traffic flow prediction is of great significance for strengthening urban management and facilitating people’s travel. In this paper, we propose a model named LST-GCN to improve the accuracy of current traffic flow predictions. We simulate the spatiotemporal correlations present in traffic flow prediction by optimizing GCN (graph convolutional network) parameters using an LSTM (long short-term memory) network. Specifically, we capture spatial correlations by learning topology through GCN networks and temporal correlations by embedding LSTM networks into the training process of GCN networks. This method improves the traditional method of combining the recurrent neural network and graph neural network in the original spatiotemporal traffic flow prediction, so it can better capture the spatiotemporal features existing in the traffic flow. Extensive experiments conducted on the PEMS dataset illustrate the effectiveness and outperformance of our method compared with other state-of-the-art methods.
47
Mohamed, Takwa, Sabah Sayed, Akram Salah, and Essam H. Houssein. "Long Short-Term Memory Neural Networks for RNA Viruses Mutations Prediction." Mathematical Problems in Engineering 2021 (June 25, 2021): 1–9. http://dx.doi.org/10.1155/2021/9980347.
Full textAbstract:
Viral progress remains a major deterrent in the viability of antiviral drugs. The ability to anticipate this development will provide assistance in the early detection of drug-resistant strains and may encourage antiviral drugs to be the most effective plan. In recent years, a deep learning model called the seq2seq neural network has emerged and has been widely used in natural language processing. In this research, we borrow this approach for predicting next generation sequences using the seq2seq LSTM neural network while considering these sequences as text data. We used hot single vectors to represent the sequences as input to the model; subsequently, it maintains the basic information position of each nucleotide in the sequences. Two RNA viruses sequence datasets are used to evaluate the proposed model which achieved encouraging results. The achieved results illustrate the potential for utilizing the LSTM neural network for DNA and RNA sequences in solving other sequencing issues in bioinformatics.
48
Yu, Yong, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. "A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures." Neural Computation 31, no. 7 (July 2019): 1235–70. http://dx.doi.org/10.1162/neco_a_01199.
Full textAbstract:
Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.
49
Bhandarkar, Tanvi, Vardaan K, Nikhil Satish, S. Sridhar, R. Sivakumar, and Snehasish Ghosh. "Earthquake trend prediction using long short-term memory RNN." International Journal of Electrical and Computer Engineering (IJECE) 9, no. 2 (April 1, 2019): 1304. http://dx.doi.org/10.11591/ijece.v9i2.pp1304-1312.
Full textAbstract:
<p>The prediction of a natural calamity such as earthquakes has been an area of interest for a long time but accurate results in earthquake forecasting have evaded scientists, even leading some to deem it intrinsically impossible to forecast them accurately. In this paper an attempt to forecast earthquakes and trends using a data of a series of past earthquakes. A type of recurrent neural network called Long Short-Term Memory (LSTM) is used to model the sequence of earthquakes. The trained model is then used to predict the future trend of earthquakes. An ordinary Feed Forward Neural Network (FFNN) solution for the same problem was done for comparison. The LSTM neural network was found to outperform the FFNN. The R^2 score of the LSTM is better than the FFNN’s by 59%.</p>
50
Mukhlis, Mukhlis, Aziz Kustiyo, and Aries Suharso. "Peramalan Produksi Pertanian Menggunakan Model Long Short-Term Memory." BINA INSANI ICT JOURNAL 8, no. 1 (June 24, 2021): 22. http://dx.doi.org/10.51211/biict.v8i1.1492.
Full textAbstract:
Abstrak: Masalah yang timbul dalam peramalan hasil produksi pertanian antara lain adalah sulit untuk mendapatkan data yang lengkap dari variabel-variabel yang mempengaruhi hasil pertanian dalam jangka panjang. Kondisi ini akan semakin sulit ketika peramalan mencakup wilayah yang cukup luas. Akibatnya, variabel-variabel tersebut harus diinterpolasi sehingga akan menyebabkan bias terhadap hasil peramalan. (1) Mengetahui gambaran meta analisis penelitian peramalan produk pertanian menggunakan Long Short Term Memory (LSTM), (2) Mengetahui penelitian meta analisis cakupan wilayah, komoditi dan periode data terkait produk pertanian terutama gandum, kedelai jagung dan pisang, (3) Mengetahui praproses data antara lain menghilangkan data yang tidak sesuai, menangani data yang kosong, serta memilih variabel tertentu. Sebagai solusi dari masalah tersebut, peramalan hasil produksi pertanian dilakukan berdasarkan data historis hasil produksi pertanian. Salah model peramalan yang saat ini banyak dikembangkan adalah model jaringan syaraf LSTM yang merupakan pengembangan dari model jaringan syaraf recurrent (RNN). Tulisan ini merupakan hasil kajian literatur pengembangan model-model LSTM untuk peramalan hasil produksi pertanian meliputi gandum, kedelai, jagung dan pisang. Perbaikan kinerja model LSTM dilakukan mulai dari praproses, tuning hyperparameter, sampai dengan penggabungan dengan metode lain. Berdasarkan kajian tersebut, model-model LSTM memiliki kinerja yang lebih baik dibandingkan dengan model benchmark.
Kata kunci: jaringan syaraf, LSTM, peramalan, produksi pertanian, RNN.
Abstract: Problems that arise in forecasting agricultural products include the difficulty of obtaining complete data on the variables that affect agricultural yields in the long term. This condition will be more difficult when the forecast covers a large area. As a result, these variables must be interpolated so that it will cause a bias towards the forecasting results. (1) Knowing the description of research maps for forecasting agricultural products using Long short term memory (LSTM), (2) Knowing Research Coverage areas, commodities, and data periods related to agricultural products, especially Wheat, Soybeans, corn, and bananas, (3) Knowing Preprocessing data between others remove inappropriate data, handle blank data, and select certain variables. This paper is the result of a literature review on the development of LSTM models for crop yields forecasting including wheat, soybeans, corn, and bananas. The Performance Improvements of the LSTM models were carried out by preprocessing data, hyperparameter tuning, and combining LSTM with other methods. Based on this study, LSTM models have better performance compared to the benchmark model.
Keywords: neural network, LSTM, forecasting, crop yield, RNN.