To see the other types of publications on this topic, follow the link: Bidirectional LSTM (BiLSTM).
Journal articles on the topic 'Bidirectional LSTM (BiLSTM)'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Bidirectional LSTM (BiLSTM).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Kiperwasser, Eliyahu, and Yoav Goldberg. "Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations." Transactions of the Association for Computational Linguistics 4 (December 2016): 313–27. http://dx.doi.org/10.1162/tacl_a_00101.
Full textAbstract:
We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very simple architectures, and match or surpass the state-of-the-art accuracies on English and Chinese.
2
Zhai, Yujia, Yan Wan, and Xiaoxiao Wang. "Optimization of Traffic Congestion Management in Smart Cities under Bidirectional Long and Short-Term Memory Model." Journal of Advanced Transportation 2022 (April 1, 2022): 1–8. http://dx.doi.org/10.1155/2022/3305400.
Full textAbstract:
To solve the increasingly serious traffic congestion and reduce traffic pressure, the bidirectional long and short-term memory (BiLSTM) algorithm is adopted to the traffic flow prediction. Firstly, a BiLSTM-based urban road short-term traffic state algorithm network is established based on the collected road traffic flow data, and then the internal memory unit structure of the network is optimized. After training and optimization, it becomes a high-quality prediction model. Then, the experimental simulation verification and prediction performance evaluation are performed. Finally, the data predicted by the BiLSTM algorithm model are compared with the actual data and the data predicted by the long short-term memory (LSTM) algorithm model. Simulation comparison shows that the prediction results of LSTM and BiLSTM are consistent with the actual traffic flow trend, but the data of LSTM deviate greatly from the real situation, and the error is more serious during peak periods. BiLSTM is in good agreement with the real situation during the stationary period and the low peak period, and it is slightly different from the real situation during the peak period, but it can still be used as a reference. In general, the prediction accuracy of the BiLSTM algorithm for traffic flow is relatively high. The comparison of evaluation indicators shows that the coefficient of determination value of BiLSTM is 0.795746 greater than that of LSTM (0.778742), indicating that BiLSTM shows a higher degree of fitting than the LSTM algorithm, that is, the prediction of BiLSTM is more accurate. The mean absolute percentage error (MAPE) value of BiLSTM is 9.718624%, which is less than 9.722147% of LSTM, indicating that the trend predicted by the BiLSTM is more consistent with the actual trend than that of LSTM. The mean absolute error (MAE) value of BiLSTM (105.087415) is smaller than that of LSTM (106.156847), indicating that its actual prediction error is smaller than LSTM. Generally speaking, BiLSTM shows advantages in traffic flow prediction over LSTM. Results of this study play a reliable reference role in the dynamic control, monitoring, and guidance of urban traffic, and congestion management.
3
Abduljabbar, Rusul L., Hussein Dia, and Pei-Wei Tsai. "Unidirectional and Bidirectional LSTM Models for Short-Term Traffic Prediction." Journal of Advanced Transportation 2021 (March 26, 2021): 1–16. http://dx.doi.org/10.1155/2021/5589075.
Full textAbstract:
This paper presents the development and evaluation of short-term traffic prediction models using unidirectional and bidirectional deep learning long short-term memory (LSTM) neural networks. The unidirectional LSTM (Uni-LSTM) model provides high performance through its ability to recognize longer sequences of traffic time series data. In this work, Uni-LSTM is extended to bidirectional LSTM (BiLSTM) networks which train the input data twice through forward and backward directions. The paper presents a comparative evaluation of the two models for short-term speed and traffic flow prediction using a common dataset of field observations collected from multiple freeways in Australia. The results showed BiLSTM performed better for variable prediction horizons for both speed and flow. Stacked and mixed Uni-LSTM and BiLSTM models were also investigated for 15-minute prediction horizons resulting in improved accuracy when using 4-layer BiLSTM networks. The optimized 4-layer BiLSTM model was then calibrated and validated for multiple prediction horizons using data from three different freeways. The validation results showed a high degree of prediction accuracy exceeding 90% for speeds up to 60-minute prediction horizons. For flow, the model achieved accuracies above 90% for 5- and 10-minute prediction horizons and more than 80% accuracy for 15- and 30-minute prediction horizons. These findings extend the set of AI models available for road operators and provide them with confidence in applying robust models that have been tested and evaluated on different freeways in Australia.
4
Casabianca, Pietro, Yu Zhang, Miguel Martínez-García, and Jiafu Wan. "Vehicle Destination Prediction Using Bidirectional LSTM with Attention Mechanism." Sensors 21, no. 24 (December 17, 2021): 8443. http://dx.doi.org/10.3390/s21248443.
Full textAbstract:
Satellite navigation has become ubiquitous to plan and track travelling. Having access to a vehicle’s position enables the prediction of its destination. This opens the possibility to various benefits, such as early warnings of potential hazards, route diversions to pass traffic congestion, and optimizing fuel consumption for hybrid vehicles. Thus, reliably predicting destinations can bring benefits to the transportation industry. This paper investigates using deep learning methods for predicting a vehicle’s destination based on its journey history. With this aim, Dense Neural Networks (DNNs), Long Short-Term Memory (LSTM) networks, Bidirectional LSTM (BiLSTM), and networks with and without attention mechanisms are tested. Especially, LSTM and BiLSTM models with attention mechanism are commonly used for natural language processing and text-classification-related applications. On the other hand, this paper demonstrates the viability of these techniques in the automotive and associated industrial domain, aimed at generating industrial impact. The results of using satellite navigation data show that the BiLSTM with an attention mechanism exhibits better prediction performance destination, achieving an average accuracy of 96% against the test set (4% higher than the average accuracy of the standard BiLSTM) and consistently outperforming the other models by maintaining robustness and stability during forecasting.
5
Liu, Lingfeng, Baodan Bai, Xinrong Chen, and Qin Xia. "Semantic Segmentation of QRS Complex in Single Channel ECG with Bidirectional LSTM Networks." Journal of Medical Imaging and Health Informatics 10, no. 3 (March 1, 2020): 758–62. http://dx.doi.org/10.1166/jmihi.2020.2929.
Full textAbstract:
In this paper, bidirectional Long Short-Term Memory (BiLSTM) networks are designed to realize the semantic segmentation of QRS complex in single channel electrocardiogram (ECG) for the tasks of R peak detection and heart rate estimation. Three types of seq2seq BiLSTM networks are introduced, including the densely connected BiLSTM with attention model, the BiLSTM U-Net, and the BiLSTM U-Net++. To alleviate the sparse problem of the QRS labels, symmetric label expansion is applied by extending the single R peak into a time interval of fixed length. Linear ensemble method is introduced that averages the outputs of different BiLSTM networks. The cross-validation results show significant increase of the accuracy and decrease of discontinuous gaps in the QRS interval prediction by the ensembling over singular neural networks.
6
Gao, Yunqing, Juping Zhu, and Hongbo Gao. "The surrounding vehicles behavior prediction for intelligent vehicle based on Attention-BiLSTM." JUSTC 52 (2022): 1. http://dx.doi.org/10.52396/justc-2021-0115.
Full textAbstract:
A surrounding vehicles behavior prediction method was presented for intelligent vehicles. The surrounding vehicles’ behavior is hard to predict since the significant uncertainty of vehicle driving and environmental changes. This method adopts bidirectional long short-term memory (BiLSTM) model combined with an encoder to ensure the memory of long-time series training. By constructing an attention mechanism based on BiLSTM, we consider the importance of different information which could guarantee the encoder’ memory under long sequence. The designed attention-bidirectional LSTM (Att-BiLSTM) model is adopted to ensure the surrounding vehicles’ prediction accuracy and effectiveness.
7
Rahman, Md Mostafizer, Yutaka Watanobe, and Keita Nakamura. "A Bidirectional LSTM Language Model for Code Evaluation and Repair." Symmetry 13, no. 2 (February 1, 2021): 247. http://dx.doi.org/10.3390/sym13020247.
Full textAbstract:
Programming is a vital skill in computer science and engineering-related disciplines. However, developing source code is an error-prone task. Logical errors in code are particularly hard to identify for both students and professionals, and a single error is unexpected to end-users. At present, conventional compilers have difficulty identifying many of the errors (especially logical errors) that can occur in code. To mitigate this problem, we propose a language model for evaluating source codes using a bidirectional long short-term memory (BiLSTM) neural network. We trained the BiLSTM model with a large number of source codes with tuning various hyperparameters. We then used the model to evaluate incorrect code and assessed the model’s performance in three principal areas: source code error detection, suggestions for incorrect code repair, and erroneous code classification. Experimental results showed that the proposed BiLSTM model achieved 50.88% correctness in identifying errors and providing suggestions. Moreover, the model achieved an F-score of approximately 97%, outperforming other state-of-the-art models (recurrent neural networks (RNNs) and long short-term memory (LSTM)).
8
Xu, Chuanjie, Feng Yuan, and Shouqiang Chen. "BJBN： BERT-JOIN-BiLSTM Networks for Medical Auxiliary Diagnostic." Journal of Healthcare Engineering 2022 (January 11, 2022): 1–7. http://dx.doi.org/10.1155/2022/3496810.
Full textAbstract:
This study proposed a medicine auxiliary diagnosis model based on neural network. The model combines a bidirectional long short-term memory（Bi-LSTM）network and bidirectional encoder representations from transformers (BERT), which can well complete the extraction of local features of Chinese medicine texts. BERT can learn the global information of the text, so use BERT to get the global representation of medical text and then use Bi-LSTM to extract local features. We conducted a large number of comparative experiments on datasets. The results show that the proposed model has significant advantages over the state-of-the-art baseline model. The accuracy of the proposed model is 0.75.
9
Zhen, Hao, Dongxiao Niu, Min Yu, Keke Wang, Yi Liang, and Xiaomin Xu. "A Hybrid Deep Learning Model and Comparison for Wind Power Forecasting Considering Temporal-Spatial Feature Extraction." Sustainability 12, no. 22 (November 15, 2020): 9490. http://dx.doi.org/10.3390/su12229490.
Full textAbstract:
The inherent intermittency and uncertainty of wind power have brought challenges in accurate wind power output forecasting, which also cause tricky problems in the integration of wind power to the grid. In this paper, a hybrid deep learning model bidirectional long short term memory-convolutional neural network (BiLSTM-CNN) is proposed for short-term wind power forecasting. First, the grey correlation analysis is utilized to select the inputs for forecasting model; Then, the proposed hybrid model extracts multi-dimension features of inputs to predict the wind power from the temporal-spatial perspective, where the Bi-LSTM model is utilized to mine the bidirectional temporal characteristics while the convolution and pooling operations of CNN are utilized to extract the spatial characteristics from multiple input time series. Lastly, a case study is conducted to verify the superiority of the proposed model. Other deep learning models (Bi-LSTM, LSTM, CNN, LSTM-CNN, CNN-BiLSTM, CNN-LSTM) are also simulated to conduct comparison from three aspects. The results show that the BiLSTM-CNN model has the best accuracy with the lowest RMSE of 2.5492, MSE of 6.4984, MAE of 1.7344 and highest R2 of 0.9929. CNN has the fastest speed with an average computational time of 0.0741s. The hybrid model that mines the spatial feature based on the extracted temporal feature has a better performance than the model mines the temporal feature based on the extracted spatial feature.
10
Namgung, Juhong, Siwoon Son, and Yang-Sae Moon. "Efficient Deep Learning Models for DGA Domain Detection." Security and Communication Networks 2021 (January 18, 2021): 1–15. http://dx.doi.org/10.1155/2021/8887881.
Full textAbstract:
In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.
11
Yu, Wanting, Hongyi Yu, Ding Wang, Jianping Du, and Mengli Zhang. "SL-BiLSTM: A Signal-Based Bidirectional LSTM Network for Over-the-Horizon Target Localization." Mathematical Problems in Engineering 2021 (July 13, 2021): 1–9. http://dx.doi.org/10.1155/2021/9992120.
Full textAbstract:
Deep learning technology provides novel solutions for localization in complex scenarios. Conventional methods generally suffer from performance loss in the long-distance over-the-horizon (OTH) scenario due to uncertain ionospheric conditions. To overcome the adverse effects of the unknown and complex ionosphere on positioning, we propose a deep learning positioning method based on multistation received signals and bidirectional long short-term memory (BiLSTM) network framework (SL-BiLSTM), which refines position information from signal data. Specifically, we first obtain the form of the network input by constructing the received signal model. Second, the proposed method is developed to predict target positions using an SL-BiLSTM network, consisting of three BiLSTM layers, a maxout layer, a fully connected layer, and a regression layer. Then, we discuss two regularization techniques of dropout and randomization which are mainly adopted to prevent network overfitting. Simulations of OTH localization are conducted to examine the performance. The parameters of the network have been trained properly according to the scenario. Finally, the experimental results show that the proposed method can significantly improve the accuracy of OTH positioning at low SNR. When the number of training locations increases to 200, the positioning result of SL-BiLSTM is closest to CRLB at high SNR.
12
Hsu, Fu-Shun, Shang-Ran Huang, Chien-Wen Huang, Chao-Jung Huang, Yuan-Ren Cheng, Chun-Chieh Chen, Jack Hsiao, et al. "Benchmarking of eight recurrent neural network variants for breath phase and adventitious sound detection on a self-developed open-access lung sound database—HF_Lung_V1." PLOS ONE 16, no. 7 (July 1, 2021): e0254134. http://dx.doi.org/10.1371/journal.pone.0254134.
Full textAbstract:
A reliable, remote, and continuous real-time respiratory sound monitor with automated respiratory sound analysis ability is urgently required in many clinical scenarios—such as in monitoring disease progression of coronavirus disease 2019—to replace conventional auscultation with a handheld stethoscope. However, a robust computerized respiratory sound analysis algorithm for breath phase detection and adventitious sound detection at the recording level has not yet been validated in practical applications. In this study, we developed a lung sound database (HF_Lung_V1) comprising 9,765 audio files of lung sounds (duration of 15 s each), 34,095 inhalation labels, 18,349 exhalation labels, 13,883 continuous adventitious sound (CAS) labels (comprising 8,457 wheeze labels, 686 stridor labels, and 4,740 rhonchus labels), and 15,606 discontinuous adventitious sound labels (all crackles). We conducted benchmark tests using long short-term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), bidirectional GRU (BiGRU), convolutional neural network (CNN)-LSTM, CNN-GRU, CNN-BiLSTM, and CNN-BiGRU models for breath phase detection and adventitious sound detection. We also conducted a performance comparison between the LSTM-based and GRU-based models, between unidirectional and bidirectional models, and between models with and without a CNN. The results revealed that these models exhibited adequate performance in lung sound analysis. The GRU-based models outperformed, in terms of F1 scores and areas under the receiver operating characteristic curves, the LSTM-based models in most of the defined tasks. Furthermore, all bidirectional models outperformed their unidirectional counterparts. Finally, the addition of a CNN improved the accuracy of lung sound analysis, especially in the CAS detection tasks.
13
Rhanoui, Maryem, Mounia Mikram, Siham Yousfi, and Soukaina Barzali. "A CNN-BiLSTM Model for Document-Level Sentiment Analysis." Machine Learning and Knowledge Extraction 1, no. 3 (July 25, 2019): 832–47. http://dx.doi.org/10.3390/make1030048.
Full textAbstract:
Document-level sentiment analysis is a challenging task given the large size of the text, which leads to an abundance of words and opinions, at times contradictory, in the same document. This analysis is particularly useful in analyzing press articles and blog posts about a particular product or company, and it requires a high concentration, especially when the topic being discussed is sensitive. Nevertheless, most existing models and techniques are designed to process short text from social networks and collaborative platforms. In this paper, we propose a combination of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) models, with Doc2vec embedding, suitable for opinion analysis in long texts. The CNN-BiLSTM model is compared with CNN, LSTM, BiLSTM and CNN-LSTM models with Word2vec/Doc2vec embeddings. The Doc2vec with CNN-BiLSTM model was applied on French newspapers articles and outperformed the other models with 90.66% accuracy.
14
Hilmawan, Muhammad David. "Deteksi Sarkasme Pada Judul Berita Berbahasa Inggris Menggunakan Algoritme Bidirectional LSTM." Journal of Dinda : Data Science, Information Technology, and Data Analytics 2, no. 1 (February 23, 2022): 46–51. http://dx.doi.org/10.20895/dinda.v2i1.331.
Full textAbstract:
Sarkasme adalah penggunaan kata-kata pedas untuk menyakiti hati orang lain, berupa cemoohan atau ejekan kasar. Kata sarkasme diturunkan dari kata Yunani sarkasmos yang berarti “merobek-robek daging seperti anjing”, “menggigit bibir karena marah”, atau ”berbicara dengan kepahitan”. Sarkasme dapat bersifat ironis, atau tidak, tetapi yang pasti adalah bahwa gaya bahasa ini selalu akan menyakiti hati dan kurang enak didengar. Pada penelitian ini akan dibuat model klasifikasi untuk memprediksi sarkasme pada judul berita berbahasa inggris dikarenakan judul berita menggunakan kata baku dan tidak ada salah pengejaan kata, menjadikan judul berita sebuah dataset yang tepat untuk dilakukan deteksi sarkasme. Algoritme Bidirectional Long Short-Term Memory (BiLSTM) yang merupakan salah satu algoritme deep learning digunakan pada penelitian untuk membuat model klasifikasi. Model ini lalu dibandingkan dengan model algoritme Long Short-Term Memory (LSTM) untuk memvalidasi keunggulan dari algoritme BiLSTM daripada algoritme LSTM dasar. Didapatkan akurasi validasi dari model sebesar 82,55%, precision validasi sebesar 82,36%, recall validasi sebesar 79,53%, dan f1 score validasi sebesar 80,92%.
15
Ye, Jing, Hui Wang, MeiJie Li, and Ning Wang. "IoT-Based Wearable Sensors and Bidirectional LSTM Network for Action Recognition of Aerobics Athletes." Journal of Healthcare Engineering 2021 (July 26, 2021): 1–7. http://dx.doi.org/10.1155/2021/9601420.
Full textAbstract:
Aerobics is the fusion of gymnastics, dance, and music; it is a body of a sports project, along with the development of the society. The growing demand for aerobics inevitably increases the demand for aerobics coach and teacher and has opened elective aerobics class which is an effective way of cultivating professional talents relevant to aerobics. Aerobics has extended fixed teaching mode and cannot conform to the development of the times. The motion prediction of aerobics athletes is a new set of teaching aid. In this paper, a motion prediction model of aerobics athletes is built based on the wearable inertial sensor of the Internet of Things and the bidirectional long short term memory (BiLSTM) network. Firstly, a wireless sensor network based on ZigBee was designed and implemented to collect the posture data of aerobics athletes. The inertial sensors were used for data collection and transmission of the data to the cloud platform through Ethernet. Then, the movement of aerobics athletes is recognized and predicted by the BiLSTM network. Based on the BiLSTM network and the attention mechanism, this paper proposes to solve the problem of low classification accuracy caused by the traditional method of directly summing and averaging the updated output vectors corresponding to each moment of the BiLSTM layer. The simulation experiment is also carried out in this paper. The experimental results show that the proposed model can recognize aerobics effectively.
16
Wu, Xiao, and Qingge Ji. "TBRNet: Two-Stream BiLSTM Residual Network for Video Action Recognition." Algorithms 13, no. 7 (July 15, 2020): 169. http://dx.doi.org/10.3390/a13070169.
Full textAbstract:
Modeling spatiotemporal representations is one of the most essential yet challenging issues in video action recognition. Existing methods lack the capacity to accurately model either the correlations between spatial and temporal features or the global temporal dependencies. Inspired by the two-stream network for video action recognition, we propose an encoder–decoder framework named Two-Stream Bidirectional Long Short-Term Memory (LSTM) Residual Network (TBRNet) which takes advantage of the interaction between spatiotemporal representations and global temporal dependencies. In the encoding phase, the two-stream architecture, based on the proposed Residual Convolutional 3D (Res-C3D) network, extracts features with residual connections inserted between the two pathways, and then the features are fused to become the short-term spatiotemporal features of the encoder. In the decoding phase, those short-term spatiotemporal features are first fed into a temporal attention-based bidirectional LSTM (BiLSTM) network to obtain long-term bidirectional attention-pooling dependencies. Subsequently, those temporal dependencies are integrated with short-term spatiotemporal features to obtain global spatiotemporal relationships. On two benchmark datasets, UCF101 and HMDB51, we verified the effectiveness of our proposed TBRNet by a series of experiments, and it achieved competitive or even better results compared with existing state-of-the-art approaches.
17
Wu, Jizhou, Hongmin Zhang, and Xuanhao Gao. "Radar High-Resolution Range Profile Target Recognition by the Dual Parallel Sequence Network Model." International Journal of Antennas and Propagation 2021 (December 20, 2021): 1–9. http://dx.doi.org/10.1155/2021/4699373.
Full textAbstract:
Using traditional neural network algorithms to adapt to high-resolution range profile (HRRP) target recognition is a complex problem in the current radar target recognition field. Under the premise of in-depth analysis of the long short-term memory (LSTM) network structure and algorithm, this study uses an attention model to extract data from the sequence. We build a dual parallel sequence network model for rapid classification and recognition and to effectively improve the initial LSTM network structure while reducing network layers. Through demonstration by designing control experiments, the target recognition performance of HRRP is demonstrated. The experimental results show that the bidirectional long short-term memory (BiLSTM) algorithm has obvious advantages over the template matching method and initial LSTM networks. The improved BiLSTM algorithm proposed in this study has significantly improved the radar HRRP target recognition accuracy, which enhanced the effectiveness of the improved algorithm.
18
K A, Shirien, Neethu George, and Surekha Mariam Varghese. "Descriptive Answer Script Grading System using CNN-BiLSTM Network." International Journal of Recent Technology and Engineering 9, no. 5 (January 30, 2021): 139–44. http://dx.doi.org/10.35940/ijrte.e5212.019521.
Full textAbstract:
Descriptive answer script assessment and rating program is an automated framework to evaluate the answer scripts correctly. There are several classification schemes in which a piece of text is evaluated on the basis of spelling, semantics and meaning. But, lots of these aren’t successful. Some of the models available to rate the response scripts include Simple Long Short Term Memory (LSTM), Deep LSTM. In addition to that Convolution Neural Network and Bi-directional LSTM is considered here to refine the result. The model uses convolutional neural networks and bidirectional LSTM networks to learn local information of words and capture long-term dependency information of contexts on the Tensorflow and Keras deep learning framework. The embedding semantic representation of texts can be used for computing semantic similarities between pieces of texts and to grade them based on the similarity score. The experiment used methods for data optimization, such as data normalization and dropout, and tested the model on an Automated Student Evaluation Short Response Scoring, a commonly used public dataset. By comparing with the existing systems, the proposed model has achieved the state-of-the-art performance and achieves better results in the accuracy of the test dataset.
19
Meng, Yao, and Long Liu. "A Deep Learning Approach for a Source Code Detection Model Using Self-Attention." Complexity 2020 (September 16, 2020): 1–15. http://dx.doi.org/10.1155/2020/5027198.
Full textAbstract:
With the development of deep learning, many approaches based on neural networks are proposed for code clone. In this paper, we propose a novel source code detection model At-biLSTM based on a bidirectional LSTM network with a self-attention layer. At-biLSTM is composed of a representation model and a discriminative model. The representation model firstly transforms the source code into an abstract syntactic tree and splits it into a sequence of statement trees; then, it encodes each of the statement trees with a deep-first traversal algorithm. Finally, the representation model encodes the sequence of statement vectors via a bidirectional LSTM network, which is a classical deep learning framework, with a self-attention layer and outputs a vector representing the given source code. The discriminative model identifies the code clone depending on the vectors generated by the presentation model. Our proposed model retains both the syntactics and semantics of the source code in the process of encoding, and the self-attention algorithm makes the classifier concentrate on the effect of key statements and improves the classification performance. The contrast experiments on the benchmarks OJClone and BigCloneBench indicate that At-LSTM is effective and outperforms the state-of-art approaches in source code clone detection.
20
Zhang, Chen, Qingxu Li, and Xue Cheng. "Text Sentiment Classification Based on Feature Fusion." Revue d'Intelligence Artificielle 34, no. 4 (September 30, 2020): 515–20. http://dx.doi.org/10.18280/ria.340418.
Full textAbstract:
The convolutional neural network (CNN) and long short-term memory (LSTM) network are adept at extracting local and global features, respectively. Both can achieve excellent classification effects. However, the CNN performs poorly in extracting the global contextual information of the text, while LSTM often overlooks the features hidden between words. For text sentiment classification, this paper combines the CNN with bidirectional LSTM (BiLSTM) into a parallel hybrid model called CNN_BiLSTM. Firstly, the CNN was adopted to extract the local features of the text quickly. Next, the BiLSTM was employed to obtain the global text features containing contextual semantics. After that, the features extracted by the two neural networks (NNs) were fused, and processed by Softmax classifier for text sentiment classification. To verify its performance, the CNN_BiLSTM was compared with single NNs like CNN and LSTM, as well as other deep learning (DL) NNs through experiments. The experimental results show that the proposed parallel hybrid model outperformed the contrastive methods in F1-score and accuracy. Therefore, our model can solve text sentiment classification tasks effectively, and boast better practical value than other NNs.
21
Chaudhari, Minal, Hrutik Mayekar, Abhishek Mishra, and Diksha Bhave. "Performance Analysis of Stock Price Prediction Model using LSTM and BiLSTM." International Journal for Research in Applied Science and Engineering Technology 10, no. 4 (April 30, 2022): 810–12. http://dx.doi.org/10.22214/ijraset.2022.41362.
Full textAbstract:
Abstract: In the world of market price prediction, price prediction plays an important role in terms of providing investors with an opportunity to reduce risk while making a profit. A successful prediction model has the potential to have a significant impact on industry. For prediction, researchers use a variety of methods. We used two recurrent neural network models in our research: long short term memory and bidirectional long short term memory network model. This model can assist in determining the first accurate stock price with high accuracy. Keywords: LSTM, Bi-LSTM, RNN, Deep learning, Stock price prediction
22
Venkatesh, S., and Dr M. Jeyakarthic. "Adagrad Optimizer with Elephant Herding Optimization based Hyper Parameter Tuned Bidirectional LSTM for Customer Churn Prediction in IoT Enabled Cloud Environment." Webology 17, no. 2 (December 21, 2020): 631–51. http://dx.doi.org/10.14704/web/v17i2/web17057.
Full textAbstract:
At recent times, customer churn is an important activity in quickly developing industries such as telecom, banking, e-commerce, etc. Earlier studies revealed that the cost of getting a new customer is considerably higher than the cost of retaining the existing ones. Therefore, it becomes essential to predict the nature of customer churn for retaining the customers to a greater extent. The advent of deep learning (DL) models have begun to be employed for efficient CCP. This paper presents a new Adagrad Optimizer with Elephant Herding Optimization (EHO) based Hyper-parameter Tuned Bidirectional Long Short Term Memory (AG-EHO-BiLSTM) for CCP in Internet of Things (IoT) enabled Cloud Environment. The proposed AG-EHO-BiLSTM model initially acquires the customer data using its devices like smart phones, laptop, smart watch, etc. Next, the gathered data will be classified by the use of Bi-LSTM model, which determines the customers as churner or non-churner. The efficiency of the Bi-LSTM model can be increased through hyper parameter tuning techniques, namely Adagrad optimizer and EHO algorithm to optimally select the parameter values namely learning rate, number of hidden layer and epochs. The performance validation of the AG-EHO-BiLSTM model takes place on benchmark dataset and the simulation outcome reported the supremacy of the AG-EHO-BiLSTM model over the comparative methods.
23
Peng, Yi, Qi Han, Fei Su, Xingwei He, and Xiaohu Feng. "Meteorological Satellite Operation Prediction Using a BiLSTM Deep Learning Model." Security and Communication Networks 2021 (June 21, 2021): 1–9. http://dx.doi.org/10.1155/2021/9916461.
Full textAbstract:
The current satellite management system mainly relies on manual work. If small faults cannot be found in time, it may cause systematic fault problems and then affect the accuracy of satellite data and the service quality of meteorological satellite. If the operation trend of satellite will be predicted, the fault can be avoided. However, the satellite system is complex, and the telemetry signal is unstable, nonlinear, and time-related. It is difficult to predict through a certain model. Based on these, this paper proposes a bidirectional long short-term memory (BiLSTM) deep leaning model to predict the operation trend of meteorological satellite. In the method, the layer number of the model is designed to be two, and the prediction results, which are forecasted by LSTM network as the future trend data and historical data, are both taken as the input of BiLSTM model. The dataset for the research is generated and transmitted from Advanced Geostationary Radiation Imager (AGRI), which is the load of FY4A meteorological satellite. In order to demonstrate the superiority of the BiLSTM prediction model, it is compared with LSTM based on the same dataset in the experiment. The result shows that the BiLSTM method reports a state-of-the-art performance on satellite telemetry data.
24
Kim, Kyutae, and Jongpil Jeong. "Real-Time Monitoring for Hydraulic States Based on Convolutional Bidirectional LSTM with Attention Mechanism." Sensors 20, no. 24 (December 11, 2020): 7099. http://dx.doi.org/10.3390/s20247099.
Full textAbstract:
By monitoring a hydraulic system using artificial intelligence, we can detect anomalous data in a manufacturing workshop. In addition, by analyzing the anomalous data, we can diagnose faults and prevent failures. However, artificial intelligence, especially deep learning, needs to learn much data, and it is often difficult to get enough data at the real manufacturing site. In this paper, we apply augmentation to increase the amount of data. In addition, we propose real-time monitoring based on a deep-learning model that uses convergence of a convolutional neural network (CNN), a bidirectional long short-term memory network (BiLSTM), and an attention mechanism. CNN extracts features from input data, and BiLSTM learns feature information. The learned information is then fed to the sigmoid classifier to find out if it is normal or abnormal. Experimental results show that the proposed model works better than other deep-learning models, such as CNN or long short-term memory (LSTM).
25
Lu, Yuanyao, and Jie Yan. "Automatic Lip Reading Using Convolution Neural Network and Bidirectional Long Short-term Memory." International Journal of Pattern Recognition and Artificial Intelligence 34, no. 01 (May 24, 2019): 2054003. http://dx.doi.org/10.1142/s0218001420540038.
Full textAbstract:
Traditional automatic lip-reading systems generally consist of two stages: feature extraction and recognition, while the handcrafted features are empirical and cannot learn the relevance of lip movement sequence sufficiently. Recently, deep learning approaches have attracted increasing attention, especially the significant improvements of convolution neural network (CNN) applied to image classification and long short-term memory (LSTM) used in speech recognition, video processing and text analysis. In this paper, we propose a hybrid neural network architecture, which integrates CNN and bidirectional LSTM (BiLSTM) for lip reading. First, we extract key frames from each isolated video clip and use five key points to locate mouth region. Then, features are extracted from raw mouth images using an eight-layer CNN. The extracted features have the characteristics of stronger robustness and fault-tolerant capability. Finally, we use BiLSTM to capture the correlation of sequential information among frame features in two directions and the softmax function to predict final recognition result. The proposed method is capable of extracting local features through convolution operations and finding hidden correlation in temporal information from lip image sequences. The evaluation results of lip-reading recognition experiments demonstrate that our proposed method outperforms conventional approaches such as active contour model (ACM) and hidden Markov model (HMM).
26
Wang, Haiyao, Jianxuan Wang, Lihui Cao, Yifan Li, Qiuhong Sun, and Jingyang Wang. "A Stock Closing Price Prediction Model Based on CNN-BiSLSTM." Complexity 2021 (September 21, 2021): 1–12. http://dx.doi.org/10.1155/2021/5360828.
Full textAbstract:
As the stock market is an important part of the national economy, more and more investors have begun to pay attention to the methods to improve the return on investment and effectively avoid certain risks. Many factors affect the trend of the stock market, and the relevant information has the nature of time series. This paper proposes a composite model CNN-BiSLSTM to predict the closing price of the stock. Bidirectional special long short-term memory (BiSLSTM) improved on bidirectional long short-term memory (BiLSTM) adds 1 − tanh(x) function in the output gate which makes the model better predict the stock price. The model extracts advanced features that influence stock price through convolutional neural network (CNN), and predicts the stock closing price through BiSLSTM after the data processed by CNN. To verify the effectiveness of the model, the historical data of the Shenzhen Component Index from July 1, 1991, to October 30, 2020, are used to train and test the CNN-BiSLSTM. CNN-BiSLSTM is compared with multilayer perceptron (MLP), recurrent neural network (RNN), long short-term memory (LSTM), BiLSTM, CNN-LSTM, and CNN-BiLSTM. The experimental results show that the mean absolute error (MAE), root-mean-squared error (RMSE), and R-square (R2) evaluation indicators of the CNN-BiSLSTM are all optimal. Therefore, CNN-BiSLSTM can accurately predict the closing price of the Shenzhen Component Index of the next trading day, which can be used as a reference for the majority of investors to effectively avoid certain risks.
27
Wang, Shaoxiu, Yonghua Zhu, Wenjing Gao, Meng Cao, and Mengyao Li. "Emotion-Semantic-Enhanced Bidirectional LSTM with Multi-Head Attention Mechanism for Microblog Sentiment Analysis." Information 11, no. 5 (May 22, 2020): 280. http://dx.doi.org/10.3390/info11050280.
Full textAbstract:
The sentiment analysis of microblog text has always been a challenging research field due to the limited and complex contextual information. However, most of the existing sentiment analysis methods for microblogs focus on classifying the polarity of emotional keywords while ignoring the transition or progressive impact of words in different positions in the Chinese syntactic structure on global sentiment, as well as the utilization of emojis. To this end, we propose the emotion-semantic-enhanced bidirectional long short-term memory (BiLSTM) network with the multi-head attention mechanism model (EBILSTM-MH) for sentiment analysis. This model uses BiLSTM to learn feature representation of input texts, given the word embedding. Subsequently, the attention mechanism is used to assign the attentive weights of each words to the sentiment analysis based on the impact of emojis. The attentive weights can be combined with the output of the hidden layer to obtain the feature representation of posts. Finally, the sentiment polarity of microblog can be obtained through the dense connection layer. The experimental results show the feasibility of our proposed model on microblog sentiment analysis when compared with other baseline models.
28
Chen, Kai, Rabea Jamil Mahfoud, Yonghui Sun, Dongliang Nan, Kaike Wang, Hassan Haes Alhelou, and Pierluigi Siano. "Defect Texts Mining of Secondary Device in Smart Substation with GloVe and Attention-Based Bidirectional LSTM." Energies 13, no. 17 (September 1, 2020): 4522. http://dx.doi.org/10.3390/en13174522.
Full textAbstract:
In the process of the operation and maintenance of secondary devices in smart substation, a wealth of defect texts containing the state information of the equipment is generated. Aiming to overcome the low efficiency and low accuracy problems of artificial power text classification and mining, combined with the characteristics of power equipment defect texts, a defect texts mining method for a secondary device in a smart substation is proposed, which integrates global vectors for word representation (GloVe) method and attention-based bidirectional long short-term memory (BiLSTM-Attention) method in one model. First, the characteristics of the defect texts are analyzed and preprocessed to improve the quality of the defect texts. Then, defect texts are segmented into words, and the words are mapped to the high-dimensional feature space based on the global vectors for word representation (GloVe) model to form distributed word vectors. Finally, a text classification model based on BiLSTM-Attention was proposed to classify the defect texts of a secondary device. Precision, Recall and F1-score are selected as evaluation indicators, and compared with traditional machine learning and deep learning models. The analysis of a case study shows that the BiLSTM-Attention model has better performance and can achieve the intelligent, accurate and efficient classification of secondary device defect texts. It can assist the operation and maintenance personnel to make scientific maintenance decisions on a secondary device and improve the level of intelligent management of equipment.
29
Chandio, Bilal Ahmed, Ali Shariq Imran, Maheen Bakhtyar, Sher Muhammad Daudpota, and Junaid Baber. "Attention-Based RU-BiLSTM Sentiment Analysis Model for Roman Urdu." Applied Sciences 12, no. 7 (April 4, 2022): 3641. http://dx.doi.org/10.3390/app12073641.
Full textAbstract:
Deep neural networks have emerged as a leading approach towards handling many natural language processing (NLP) tasks. Deep networks initially conquered the problems of computer vision. However, dealing with sequential data such as text and sound was a nightmare for such networks as traditional deep networks are not reliable in preserving contextual information. This may not harm the results in the case of image processing where we do not care about the sequence, but when we consider the data collected from text for processing, such networks may trigger disastrous results. Moreover, establishing sentence semantics in a colloquial text such as Roman Urdu is a challenge. Additionally, the sparsity and high dimensionality of data in such informal text have encountered a significant challenge for building sentence semantics. To overcome this problem, we propose a deep recurrent architecture RU-BiLSTM based on bidirectional LSTM (BiLSTM) coupled with word embedding and an attention mechanism for sentiment analysis of Roman Urdu. Our proposed model uses the bidirectional LSTM to preserve the context in both directions and the attention mechanism to concentrate on more important features. Eventually, the last dense softmax output layer is used to acquire the binary and ternary classification results. We empirically evaluated our model on two available datasets of Roman Urdu, i.e., RUECD and RUSA-19. Our proposed model outperformed the baseline models on many grounds, and a significant improvement of 6% to 8% is achieved over baseline models.
30
Setiawan, Esther Irawati, and Ika Lestari. "Stance Classification Pada Berita Berbahasa Indonesia Berbasis Bidirectional LSTM." Journal of Intelligent System and Computation 3, no. 1 (April 1, 2021): 41–48. http://dx.doi.org/10.52985/insyst.v3i1.148.
Full textAbstract:
Berita palsu masih menjadi masalah yang harus mendapat perhatian khusus. Media sosial, termasuk Facebook menjadi salah satu sarana yang mudah dan murah untuk menyebarkan suatu informasi yang bahkan belum tentu kebenarannya. Informasi tentang kesehatan menjadi salah satu topik berita palsu yang banyak tersebar ke masyarakat. Cara yang berbeda untuk mendeteksi berita palsu yaitu dengan menggunakan deteksi sikap (stance detection). Tujuan utama dari penelitian ini adalah merancang model yang memiliki kemampuan terbaik untuk melakukan tugas stance classification pada konteks bahasa Indonesia. Model ini diharapkan dapat digunakan untuk berkontribusi dalam menanggulangi masalah penyebaran berita palsu, khususnya di Indonesia. Metode BiLSTM dan GRU diusulkan untuk digunakan dalam melakukan stance classification terhadap headline berita dengan kelas for (mendukung), against (menentang), dan observing (netral). Stance classification pada penelitian ini menggunakan data sebanyak 3.941 headline berita yang terdiri dari 563 klaim dengan 7 tanggapan. Dataset dikumpulkan dari artikel-artikel berita kesehatan berbahasa Indonesia yang diposting pada laman Facebook. Model pada penelitian ini mampu menghasilkan akurasi F1-score paling tinggi sebesar 64% dengan FastText embedding. Metode GRU dapat menjadi salah satu pilihan tepat untuk melakukan stance classification dengan komputasinya yang lebih sederhana. Kinerja FastText jauh lebih unggul dibandingkan dengan Word2Vec dalam melakukan pembentukan vektor kata karena mampu mengatasi masalah out-of-vocabulary (OOV).
31
Wang, Qinghua, Yuexiao Yu, Hosameldin O. A. Ahmed, Mohamed Darwish, and Asoke K. Nandi. "Open-Circuit Fault Detection and Classification of Modular Multilevel Converters in High Voltage Direct Current Systems (MMC-HVDC) with Long Short-Term Memory (LSTM) Method." Sensors 21, no. 12 (June 17, 2021): 4159. http://dx.doi.org/10.3390/s21124159.
Full textAbstract:
Fault detection and classification are two of the challenging tasks in Modular Multilevel Converters in High Voltage Direct Current (MMC-HVDC) systems. To directly classify the raw sensor data without certain feature extraction and classifier design, a long short-term memory (LSTM) neural network is proposed and used for seven states of the MMC-HVDC transmission power system simulated by Power Systems Computer Aided Design/Electromagnetic Transients including DC (PSCAD/EMTDC). It is observed that the LSTM method can detect faults with 100% accuracy and classify different faults as well as provide promising fault classification performance. Compared with a bidirectional LSTM (BiLSTM), the LSTM can get similar classification accuracy, requiring less training time and testing time. Compared with Convolutional Neural Networks (CNN) and AutoEncoder-based deep neural networks (AE-based DNN), the LSTM method can get better classification accuracy around the middle of the testing data proportion, but it needs more training time.
32
Zhu, Chenhao, Sheng Cai, Yifan Yang, Wei Xu, Honghai Shen, and Hairong Chu. "A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments." Sensors 21, no. 4 (February 8, 2021): 1181. http://dx.doi.org/10.3390/s21041181.
Full textAbstract:
In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.
33
Elfaik, Hanane, and El Habib Nfaoui. "Deep Bidirectional LSTM Network Learning-Based Sentiment Analysis for Arabic Text." Journal of Intelligent Systems 30, no. 1 (December 31, 2020): 395–412. http://dx.doi.org/10.1515/jisys-2020-0021.
Full textAbstract:
Abstract Sentiment analysis aims to predict sentiment polarities (positive, negative or neutral) of a given piece of text. It lies at the intersection of many fields such as Natural Language Processing (NLP), Computational Linguistics, and Data Mining. Sentiments can be expressed explicitly or implicitly. Arabic Sentiment Analysis presents a challenge undertaking due to its complexity, ambiguity, various dialects, the scarcity of resources, the morphological richness of the language, the absence of contextual information, and the absence of explicit sentiment words in an implicit piece of text. Recently, deep learning has obviously shown a great success in the field of sentiment analysis and is considered as the state-of-the-art model in Arabic Sentiment Analysis. However, the state-of-the-art accuracy for Arabic sentiment analysis still needs improvements regarding contextual information and implicit sentiment expressed in different real cases. In this paper, an efficient Bidirectional LSTM Network (BiLSTM) is investigated to enhance Arabic Sentiment Analysis, by applying Forward-Backward encapsulate contextual information from Arabic feature sequences. The experimental results on six benchmark sentiment analysis datasets demonstrate that our model achieves significant improvements over the state-of-art deep learning models and the baseline traditional machine learning methods.
34
Yang, Biao, Yinshuang Wang, and Yuedong Zhan. "Lithium Battery State-of-Charge Estimation Based on a Bayesian Optimization Bidirectional Long Short-Term Memory Neural Network." Energies 15, no. 13 (June 25, 2022): 4670. http://dx.doi.org/10.3390/en15134670.
Full textAbstract:
State of charge (SOC) is the most important parameter in battery management systems (BMSs), but since the SOC is not a directly measurable state quantity, it is particularly important to use advanced strategies for accurate SOC estimation. In this paper, we first propose a bidirectional long short-term memory (BiLSTM) neural network, which enhances the comprehensiveness of information by acquiring both forward and reverse battery information compared to the general one-way recurrent neural network (RNN). Then, the parameters of this network are optimized by introducing a Bayesian optimization algorithm to match the data characteristics of lithium batteries with the network topology. Finally, two sets of lithium battery public data sets are used to carry out experiments under different constant temperature and variable temperature environments. The experimental results show that the proposed model can effectively fit the actual measurement curve. Compared with traditional long short-term memory network (LSTM) and BiLSTM models, the prediction accuracy of the Bayes-BiLSTM model is the best, with a root mean square error (RMSE) within 1%, achieving a better ability for capturing long-term dependencies. Overall, the model exhibits high accuracy, adaptability, and generalization for the SOC estimation of batteries with different chemical compositions.
35
Abduallah, Yasser, Vania K. Jordanova, Hao Liu, Qin Li, Jason T. L. Wang, and Haimin Wang. "Predicting Solar Energetic Particles Using SDO/HMI Vector Magnetic Data Products and a Bidirectional LSTM Network." Astrophysical Journal Supplement Series 260, no. 1 (May 1, 2022): 16. http://dx.doi.org/10.3847/1538-4365/ac5f56.
Full textAbstract:
Abstract Solar energetic particles (SEPs) are an essential source of space radiation, and are hazardous for humans in space, spacecraft, and technology in general. In this paper, we propose a deep-learning method, specifically a bidirectional long short-term memory (biLSTM) network, to predict if an active region (AR) would produce an SEP event given that (i) the AR will produce an M- or X-class flare and a coronal mass ejection (CME) associated with the flare, or (ii) the AR will produce an M- or X-class flare regardless of whether or not the flare is associated with a CME. The data samples used in this study are collected from the Geostationary Operational Environmental Satellite's X-ray flare catalogs provided by the National Centers for Environmental Information. We select M- and X-class flares with identified ARs in the catalogs for the period between 2010 and 2021, and find the associations of flares, CMEs, and SEPs in the Space Weather Database of Notifications, Knowledge, Information during the same period. Each data sample contains physical parameters collected from the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory. Experimental results based on different performance metrics demonstrate that the proposed biLSTM network is better than related machine-learning algorithms for the two SEP prediction tasks studied here. We also discuss extensions of our approach for probabilistic forecasting and calibration with empirical evaluation.
36
Choi, Sang, and Brian Kim. "Applying PCA to Deep Learning Forecasting Models for Predicting PM2.5." Sustainability 13, no. 7 (March 26, 2021): 3726. http://dx.doi.org/10.3390/su13073726.
Full textAbstract:
Fine particulate matter (PM2.5) is one of the main air pollution problems that occur in major cities around the world. A country’s PM2.5 can be affected not only by country factors but also by the neighboring country’s air quality factors. Therefore, forecasting PM2.5 requires collecting data from outside the country as well as from within which is necessary for policies and plans. The data set of many variables with a relatively small number of observations can cause a dimensionality problem and limit the performance of the deep learning model. This study used daily data for five years in predicting PM2.5 concentrations in eight Korean cities through deep learning models. PM2.5 data of China were collected and used as input variables to solve the dimensionality problem using principal components analysis (PCA). The deep learning models used were a recurrent neural network (RNN), long short-term memory (LSTM), and bidirectional LSTM (BiLSTM). The performance of the models with and without PCA was compared using root-mean-square error (RMSE) and mean absolute error (MAE). As a result, the application of PCA in LSTM and BiLSTM, excluding the RNN, showed better performance: decreases of up to 16.6% and 33.3% in RMSE and MAE values. The results indicated that applying PCA in deep learning time series prediction can contribute to practical performance improvements, even with a small number of observations. It also provides a more accurate basis for the establishment of PM2.5 reduction policy in the country.
37
Lu, Huimin, Rui Yang, Zhenrong Deng, Yonglin Zhang, Guangwei Gao, and Rushi Lan. "Chinese Image Captioning via Fuzzy Attention-based DenseNet-BiLSTM." ACM Transactions on Multimedia Computing, Communications, and Applications 17, no. 1s (March 31, 2021): 1–18. http://dx.doi.org/10.1145/3422668.
Full textAbstract:
Chinese image description generation tasks usually have some challenges, such as single-feature extraction, lack of global information, and lack of detailed description of the image content. To address these limitations, we propose a fuzzy attention-based DenseNet-BiLSTM Chinese image captioning method in this article. In the proposed method, we first improve the densely connected network to extract features of the image at different scales and to enhance the model’s ability to capture the weak features. At the same time, a bidirectional LSTM is used as the decoder to enhance the use of context information. The introduction of an improved fuzzy attention mechanism effectively improves the problem of correspondence between image features and contextual information. We conduct experiments on the AI Challenger dataset to evaluate the performance of the model. The results show that compared with other models, our proposed model achieves higher scores in objective quantitative evaluation indicators, including BLEU , BLEU , METEOR, ROUGEl, and CIDEr. The generated description sentence can accurately express the image content.
38
Li, Dezhi, Dongfang Yang, Liwei Li, Licheng Wang, and Kai Wang. "Electrochemical Impedance Spectroscopy Based on the State of Health Estimation for Lithium-Ion Batteries." Energies 15, no. 18 (September 13, 2022): 6665. http://dx.doi.org/10.3390/en15186665.
Full textAbstract:
The state of health (SOH) is critical to the efficient and reliable use of lithium-ion batteries (LIBs). Recently, the SOH estimation method based on electrochemical impedance spectroscopy (EIS) has been proven effective. In response to different practical applications, two models for SOH estimation are proposed in this paper. Aiming at based on the equivalent circuit model (ECM) method, a variety of ECMs are proposed. Used EIS to predict the ECM, the results show that the improved method ensures the correctness of the ECM and improves the estimation results of SOH. Aiming at a data-driven algorithm, proposes a convolution neural network (CNN) to process EIS data which can not only extract the key points but also simplifies the complexity of manual feature extraction. The bidirectional long short-term memory (BiLSTM) model was used for serial regression prediction. Moreover, the improved Particle Swarm Optimization (IPSO) algorithm is proposed to optimize the model. Comparing the improved model (IPSO-CNN-BiLSTM) with the traditional PSO-CNN-BiLSTM, CNN-BiLSTM and LSTM models, the prediction results are improved by 13.6%, 93.75% and 94.8%, respectively. Besides that, the two proposed methods are 27% and 35% better than the existing gaussion process regression (GPR) model, which indicates that the proposed improved methods are more flexible for SOH estimation with higher precision.
39
Chen, Che-Wen, Shih-Pang Tseng, Ta-Wen Kuan, and Jhing-Fa Wang. "Outpatient Text Classification Using Attention-Based Bidirectional LSTM for Robot-Assisted Servicing in Hospital." Information 11, no. 2 (February 16, 2020): 106. http://dx.doi.org/10.3390/info11020106.
Full textAbstract:
In general, patients who are unwell do not know with which outpatient department they should register, and can only get advice after they are diagnosed by a family doctor. This may cause a waste of time and medical resources. In this paper, we propose an attention-based bidirectional long short-term memory (Att-BiLSTM) model for service robots, which has the ability to classify outpatient categories according to textual content. With the outpatient text classification system, users can talk about their situation to a service robot and the robot can tell them which clinic they should register with. In the implementation of the proposed method, dialog text of users in the Taiwan E Hospital were collected as the training data set. Through natural language processing (NLP), the information in the dialog text was extracted, sorted, and converted to train the long-short term memory (LSTM) deep learning model. Experimental results verify the ability of the robot to respond to questions autonomously through acquired casual knowledge.
40
Zhang, Jingren, Fang’ai Liu, Weizhi Xu, and Hui Yu. "Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism." Future Internet 11, no. 11 (November 12, 2019): 237. http://dx.doi.org/10.3390/fi11110237.
Full textAbstract:
Convolutional neural networks (CNN) and long short-term memory (LSTM) have gained wide recognition in the field of natural language processing. However, due to the pre- and post-dependence of natural language structure, relying solely on CNN to implement text categorization will ignore the contextual meaning of words and bidirectional long short-term memory (BiLSTM). The feature fusion model is divided into a multiple attention (MATT) CNN model and a bi-directional gated recurrent unit (BiGRU) model. The CNN model inputs the word vector (word vector attention, part of speech attention, position attention) that has been labeled by the attention mechanism into our multi-attention mechanism CNN model. Obtaining the influence intensity of the target keyword on the sentiment polarity of the sentence, and forming the first dimension of the sentiment classification, the BiGRU model replaces the original BiLSTM and extracts the global semantic features of the sentence level to form the second dimension of sentiment classification. Then, using PCA to reduce the dimension of the two-dimensional fusion vector, we finally obtain a classification result combining two dimensions of keywords and sentences. The experimental results show that the proposed MATT-CNN+BiGRU fusion model has 5.94% and 11.01% higher classification accuracy on the MRD and SemEval2016 datasets, respectively, than the mainstream CNN+BiLSTM method.
41
Pratiwi, Riszki Wijayatun, Yunita Sari, and Yohanes Suyanto. "Attention-Based BiLSTM for Negation Handling in Sentimen Analysis." IJCCS (Indonesian Journal of Computing and Cybernetics Systems) 14, no. 4 (October 31, 2020): 397. http://dx.doi.org/10.22146/ijccs.60733.
Full textAbstract:
Research on sentiment analysis in recent years has increased. However, in sentiment analysis research there are still few ideas about the handling of negation, one of which is in the Indonesian sentence. This results in sentences that contain elements of the word negation have not found the exact polarity.The purpose of this research is to analyze the effect of the negation word in Indonesian. Based on positive, neutral and negative classes, using attention-based Long Short Term Memory and word2vec feature extraction method with continuous bag-of-word (CBOW) architecture. The dataset used is data from Twitter. Model performance is seen in the accuracy value.The use of word2vec with CBOW architecture and the addition of layer attention to the Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BiLSTM) methods obtained an accuracy of 78.16% and for BiLSTM resulted in an accuracy of 79.68%. whereas in the FSW algorithm is 73.50% and FWL 73.79%. It can be concluded that attention based BiLSTM has the highest accuracy, but the addition of layer attention in the Long Short Term Memory method is not too significant for negation handling. because the addition of the attention layer cannot determine the words that you want to pay attention to.
42
Chaweewanchon, Apichat, and Rujira Chaysiri. "Markowitz Mean-Variance Portfolio Optimization with Predictive Stock Selection Using Machine Learning." International Journal of Financial Studies 10, no. 3 (August 8, 2022): 64. http://dx.doi.org/10.3390/ijfs10030064.
Full textAbstract:
With the advances in time-series prediction, several recent developments in machine learning have shown that integrating prediction methods into portfolio selection is a great opportunity. In this paper, we propose a novel approach to portfolio formation strategy based on a hybrid machine learning model that combines convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) with robust input features obtained from Huber’s location for stock prediction and the Markowitz mean-variance (MV) model for optimal portfolio construction. Specifically, this study first applies a prediction method for stock preselection to ensure high-quality stock inputs for portfolio formation. Then, the predicted results are integrated into the MV model. To comprehensively demonstrate the superiority of the proposed model, we used two portfolio models, the MV model and the equal-weight portfolio (1/N) model, with LSTM, BiLSTM, and CNN-BiLSTM, and employed them as benchmarks. Between January 2015 and December 2020, historical data from the Stock Exchange of Thailand 50 Index (SET50) were collected for the study. The experiment shows that integrating preselection of stocks can improve MV performance, and the results of the proposed method show that they outperform comparison models in terms of Sharpe ratio, mean return, and risk.
43
Adil, Mohd, Jei-Zheng Wu, Ripon K. Chakrabortty, Ahmad Alahmadi, Mohd Faizan Ansari, and Michael J. Ryan. "Attention-Based STL-BiLSTM Network to Forecast Tourist Arrival." Processes 9, no. 10 (September 30, 2021): 1759. http://dx.doi.org/10.3390/pr9101759.
Full textAbstract:
Tourism makes a significant contribution to the economy of almost every country, so accurate demand forecasting can help in better planning for the government and a range of stakeholders involved in the tourism industry and can aid economic sustainability. Machine learning models, and in particular, deep neural networks, can perform better than traditional forecasting models which depend mainly on past observations (e.g., past data) to forecast future tourist arrivals. However, search intensities indices (SII) indicators have recently been included as a forecasting model, which significantly enhances forecasting accuracy. In this study, we propose a bidirectional long short-term memory (BiLSTM) neural network to forecast the arrival of tourists along with SII indicators. The proposed BiLSTM network can remember information from left to right and right to left, which further adds more context for forecasting in memory as compared to a simple long short- term memory (LSTM) network that can remember information only from left to right. A seasonal and trend decomposition using the Loess (STL) approach is utilized to decompose time series tourist arrival data suggested by previous studies. The resultant approach, called STL-BiLSTM, decomposes time series into trend, seasonality, and residual. The trend provides the general direction of the overall data. Seasonality is a regular and predictable pattern which re-occurs at fixed time intervals, and residual is a random fluctuation that is something which cannot be forecast. The proposed BiLSTM network achieves better accuracy than the other methods considered under the current study.
44
Li, Yuxuan, Chunjie Yang, and Youxian Sun. "Sintering Quality Prediction Model Based on Semi-Supervised Dynamic Time Feature Extraction Framework." Sensors 22, no. 15 (August 5, 2022): 5861. http://dx.doi.org/10.3390/s22155861.
Full textAbstract:
In the sintering process, it is difficult to obtain the key quality variables in real time, so there is lack of real-time information to guide the production process. Furthermore, these labeled data are too few, resulting in poor performance of conventional soft sensor models. Therefore, a novel semi-supervised dynamic feature extraction framework (SS-DTFEE) based on sequence pre-training and fine-tuning is proposed in this paper. Firstly, based on the DTFEE model, the time features of the sequences are extended and extracted. Secondly, a novel weighted bidirectional LSTM unit (BiLSTM) is designed to extract the latent variables of original sequence data. Based on improved BiLSTM, an encoder-decoder model is designed as a pre-training model with unsupervised learning to obtain the hidden information in the process. Next, through model migration and fine-tuning strategy, the prediction performance of labeled datasets is improved. The proposed method is applied in the actual sintering process to estimate the FeO content, which shows a significant improvement of the prediction accuracy, compared to traditional methods.
45
Pang, Geng, and Duyan Geng. "Research on Heartbeat Detection Method of Ballistocardiogram Based on Bidirectional Long Short-term Memory Network." WSEAS TRANSACTIONS ON BIOLOGY AND BIOMEDICINE 19 (June 28, 2022): 151–57. http://dx.doi.org/10.37394/23208.2022.19.16.
Full textAbstract:
In order to improve the accuracy and generalization ability of extracting successive heartbeat cycle based on ballistocardiogram (BCG), this paper proposed a general method for detecting J peak of BCG signals by using bidirectional long short-term memory network. First, the clustering method is used to establish the sequence feature set of BCG signals in different sleeping positions, and the data set used contains a variety of different forms of BCG signals. Then, according to the Bidirectional LSTM (BiLSTM) many-to-many recognition model, the number of J peaks in the output sequence is counted to achieve real-time heartbeat detection. The results showed that the deviation rate of BCG heart rate detection was 0.27%, and there was no significant difference between BCG and ECG in the detection of heartbeat interval. Compared with other methods, this method has higher robustness and accuracy in detection effect, which provides a new idea for realizing high-precision unconstrained heartbeat detection.
46
Essai Ali, Mohamed Hassan, and Ibrahim B. M. Taha. "Channel state information estimation for 5G wireless communication systems: recurrent neural networks approach." PeerJ Computer Science 7 (August 26, 2021): e682. http://dx.doi.org/10.7717/peerj-cs.682.
Full textAbstract:
In this study, a deep learning bidirectional long short-term memory (BiLSTM) recurrent neural network-based channel state information estimator is proposed for 5G orthogonal frequency-division multiplexing systems. The proposed estimator is a pilot-dependent estimator and follows the online learning approach in the training phase and the offline approach in the practical implementation phase. The estimator does not deal with complete a priori certainty for channels’ statistics and attains superior performance in the presence of a limited number of pilots. A comparative study is conducted using three classification layers that use loss functions: mean absolute error, cross entropy function for kth mutually exclusive classes and sum of squared of the errors. The Adam, RMSProp, SGdm, and Adadelat optimisation algorithms are used to evaluate the performance of the proposed estimator using each classification layer. In terms of symbol error rate and accuracy metrics, the proposed estimator outperforms long short-term memory (LSTM) neural network-based channel state information, least squares and minimum mean square error estimators under different simulation conditions. The computational and training time complexities for deep learning BiLSTM- and LSTM-based estimators are provided. Given that the proposed estimator relies on the deep learning neural network approach, where it can analyse massive data, recognise statistical dependencies and characteristics, develop relationships between features and generalise the accrued knowledge for new datasets that it has not seen before, the approach is promising for any 5G and beyond communication system.
47
Adytia, Didit, Deni Saepudin, Sri Redjeki Pudjaprasetya, Semeidi Husrin, and Ardhasena Sopaheluwakan. "A Deep Learning Approach for Wave Forecasting Based on a Spatially Correlated Wind Feature, with a Case Study in the Java Sea, Indonesia." Fluids 7, no. 1 (January 17, 2022): 39. http://dx.doi.org/10.3390/fluids7010039.
Full textAbstract:
For safety and survival at sea and on the shore, wave predictions are essential for marine-related activities, such as harbor operations, naval navigation, and other coastal and offshore activities. In general, wave height predictions rely heavily on numerical simulations. The computational cost of such a simulation can be very high (and it can be time-consuming), especially when considering a complex coastal area, since these simulations require high-resolution grids. This study utilized a deep learning technique called bidirectional long short-term memory (BiLSTM) for wave forecasting to save computing time and to produce accurate predictions. The deep learning method was trained using wave data obtained by a continuous numerical wave simulation using the SWAN wave model over a 20-year period with ECMWF ERA-5 wind data. We utilized highly spatially correlated wind as input for the deep learning method to select the best feature for wave forecasting. We chose an area with a complex geometry as the study case, an area in Indonesia’s Java Sea. We also compared the results of wave prediction using BiLSTM with those of other methods, i.e., LSTM, support vector regression (SVR), and a generalized regression neural network (GRNN). The forecasting results using the BiLSTM were the best, with a correlation coefficient of 0.96 and an RMSE value of 0.06.
48
Visweswaran, Shyam, Jason B. Colditz, Patrick O’Halloran, Na-Rae Han, Sanya B. Taneja, Joel Welling, Kar-Hai Chu, Jaime E. Sidani, and Brian A. Primack. "Machine Learning Classifiers for Twitter Surveillance of Vaping: Comparative Machine Learning Study." Journal of Medical Internet Research 22, no. 8 (August 12, 2020): e17478. http://dx.doi.org/10.2196/17478.
Full textAbstract:
Background Twitter presents a valuable and relevant social media platform to study the prevalence of information and sentiment on vaping that may be useful for public health surveillance. Machine learning classifiers that identify vaping-relevant tweets and characterize sentiments in them can underpin a Twitter-based vaping surveillance system. Compared with traditional machine learning classifiers that are reliant on annotations that are expensive to obtain, deep learning classifiers offer the advantage of requiring fewer annotated tweets by leveraging the large numbers of readily available unannotated tweets. Objective This study aims to derive and evaluate traditional and deep learning classifiers that can identify tweets relevant to vaping, tweets of a commercial nature, and tweets with provape sentiments. Methods We continuously collected tweets that matched vaping-related keywords over 2 months from August 2018 to October 2018. From this data set of tweets, a set of 4000 tweets was selected, and each tweet was manually annotated for relevance (vape relevant or not), commercial nature (commercial or not), and sentiment (provape or not). Using the annotated data, we derived traditional classifiers that included logistic regression, random forest, linear support vector machine, and multinomial naive Bayes. In addition, using the annotated data set and a larger unannotated data set of tweets, we derived deep learning classifiers that included a convolutional neural network (CNN), long short-term memory (LSTM) network, LSTM-CNN network, and bidirectional LSTM (BiLSTM) network. The unannotated tweet data were used to derive word vectors that deep learning classifiers can leverage to improve performance. Results LSTM-CNN performed the best with the highest area under the receiver operating characteristic curve (AUC) of 0.96 (95% CI 0.93-0.98) for relevance, all deep learning classifiers including LSTM-CNN performed better than the traditional classifiers with an AUC of 0.99 (95% CI 0.98-0.99) for distinguishing commercial from noncommercial tweets, and BiLSTM performed the best with an AUC of 0.83 (95% CI 0.78-0.89) for provape sentiment. Overall, LSTM-CNN performed the best across all 3 classification tasks. Conclusions We derived and evaluated traditional machine learning and deep learning classifiers to identify vaping-related relevant, commercial, and provape tweets. Overall, deep learning classifiers such as LSTM-CNN had superior performance and had the added advantage of requiring no preprocessing. The performance of these classifiers supports the development of a vaping surveillance system.
49
Chen, Xie, Yuan, Huang, and Li. "Research on a Real-Time Monitoring Method for the Wear State of a Tool Based on a Convolutional Bidirectional LSTM Model." Symmetry 11, no. 10 (October 2, 2019): 1233. http://dx.doi.org/10.3390/sym11101233.
Full textAbstract:
To monitor the tool wear state of computerized numerical control (CNC) machining equipment in real time in a manufacturing workshop, this paper proposes a real-time monitoring method based on a fusion of a convolutional neural network (CNN) and a bidirectional long short-term memory (BiLSTM) network with an attention mechanism (CABLSTM). In this method, the CNN is used to extract deep features from the time-series signal as an input, and then the BiLSTM network with a symmetric structure is constructed to learn the time-series information between the feature vectors. The attention mechanism is introduced to self-adaptively perceive the network weights associated with the classification results of the wear state and distribute the weights reasonably. Finally, the signal features of different weights are sent to a Softmax classifier to classify the tool wear state. In addition, a data acquisition experiment platform is developed with a high-precision CNC milling machine and an acceleration sensor to collect the vibration signals generated during tool processing in real time. The original data are directly fed into the depth neural network of the model for analysis, which avoids the complexity and limitations caused by a manual feature extraction. The experimental results show that, compared with other deep learning neural networks and traditional machine learning network models, the model can predict the tool wear state accurately in real time from original data collected by sensors, and the recognition accuracy and generalization have been improved to a certain extent.
50
Contractor, Danish, Barun Patra, Mausam, and Parag Singla. "Constrained BERT BiLSTM CRF for understanding multi-sentence entity-seeking questions." Natural Language Engineering 27, no. 1 (February 13, 2020): 65–87. http://dx.doi.org/10.1017/s1351324920000017.
Full textAbstract:
AbstractWe present the novel task of understanding multi-sentence entity-seeking questions (MSEQs), that is, the questions that may be expressed in multiple sentences, and that expect one or more entities as an answer. We formulate the problem of understanding MSEQs as a semantic labeling task over an open representation that makes minimal assumptions about schema or ontology-specific semantic vocabulary. At the core of our model, we use a BiLSTM (bidirectional LSTM) conditional random field (CRF), and to overcome the challenges of operating with low training data, we supplement it by using BERT embeddings, hand-designed features, as well as hard and soft constraints spanning multiple sentences. We find that this results in a 12–15 points gain over a vanilla BiLSTM CRF. We demonstrate the strengths of our work using the novel task of answering real-world entity-seeking questions from the tourism domain. The use of our labels helps answer 36% more questions with 35% more (relative) accuracy as compared to baselines. We also demonstrate how our framework can rapidly enable the parsing of MSEQs in an entirely new domain with small amounts of training data and little change in the semantic representation.