Добірка наукової літератури з теми "Bidirectional LSTM (BiLSTM)"

We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very simple architectures, and match or surpass the state-of-the-art accuracies on English and Chinese.

To solve the increasingly serious traffic congestion and reduce traffic pressure, the bidirectional long and short-term memory (BiLSTM) algorithm is adopted to the traffic flow prediction. Firstly, a BiLSTM-based urban road short-term traffic state algorithm network is established based on the collected road traffic flow data, and then the internal memory unit structure of the network is optimized. After training and optimization, it becomes a high-quality prediction model. Then, the experimental simulation verification and prediction performance evaluation are performed. Finally, the data predicted by the BiLSTM algorithm model are compared with the actual data and the data predicted by the long short-term memory (LSTM) algorithm model. Simulation comparison shows that the prediction results of LSTM and BiLSTM are consistent with the actual traffic flow trend, but the data of LSTM deviate greatly from the real situation, and the error is more serious during peak periods. BiLSTM is in good agreement with the real situation during the stationary period and the low peak period, and it is slightly different from the real situation during the peak period, but it can still be used as a reference. In general, the prediction accuracy of the BiLSTM algorithm for traffic flow is relatively high. The comparison of evaluation indicators shows that the coefficient of determination value of BiLSTM is 0.795746 greater than that of LSTM (0.778742), indicating that BiLSTM shows a higher degree of fitting than the LSTM algorithm, that is, the prediction of BiLSTM is more accurate. The mean absolute percentage error (MAPE) value of BiLSTM is 9.718624%, which is less than 9.722147% of LSTM, indicating that the trend predicted by the BiLSTM is more consistent with the actual trend than that of LSTM. The mean absolute error (MAE) value of BiLSTM (105.087415) is smaller than that of LSTM (106.156847), indicating that its actual prediction error is smaller than LSTM. Generally speaking, BiLSTM shows advantages in traffic flow prediction over LSTM. Results of this study play a reliable reference role in the dynamic control, monitoring, and guidance of urban traffic, and congestion management.

This paper presents the development and evaluation of short-term traffic prediction models using unidirectional and bidirectional deep learning long short-term memory (LSTM) neural networks. The unidirectional LSTM (Uni-LSTM) model provides high performance through its ability to recognize longer sequences of traffic time series data. In this work, Uni-LSTM is extended to bidirectional LSTM (BiLSTM) networks which train the input data twice through forward and backward directions. The paper presents a comparative evaluation of the two models for short-term speed and traffic flow prediction using a common dataset of field observations collected from multiple freeways in Australia. The results showed BiLSTM performed better for variable prediction horizons for both speed and flow. Stacked and mixed Uni-LSTM and BiLSTM models were also investigated for 15-minute prediction horizons resulting in improved accuracy when using 4-layer BiLSTM networks. The optimized 4-layer BiLSTM model was then calibrated and validated for multiple prediction horizons using data from three different freeways. The validation results showed a high degree of prediction accuracy exceeding 90% for speeds up to 60-minute prediction horizons. For flow, the model achieved accuracies above 90% for 5- and 10-minute prediction horizons and more than 80% accuracy for 15- and 30-minute prediction horizons. These findings extend the set of AI models available for road operators and provide them with confidence in applying robust models that have been tested and evaluated on different freeways in Australia.

Satellite navigation has become ubiquitous to plan and track travelling. Having access to a vehicle’s position enables the prediction of its destination. This opens the possibility to various benefits, such as early warnings of potential hazards, route diversions to pass traffic congestion, and optimizing fuel consumption for hybrid vehicles. Thus, reliably predicting destinations can bring benefits to the transportation industry. This paper investigates using deep learning methods for predicting a vehicle’s destination based on its journey history. With this aim, Dense Neural Networks (DNNs), Long Short-Term Memory (LSTM) networks, Bidirectional LSTM (BiLSTM), and networks with and without attention mechanisms are tested. Especially, LSTM and BiLSTM models with attention mechanism are commonly used for natural language processing and text-classification-related applications. On the other hand, this paper demonstrates the viability of these techniques in the automotive and associated industrial domain, aimed at generating industrial impact. The results of using satellite navigation data show that the BiLSTM with an attention mechanism exhibits better prediction performance destination, achieving an average accuracy of 96% against the test set (4% higher than the average accuracy of the standard BiLSTM) and consistently outperforming the other models by maintaining robustness and stability during forecasting.

In this paper, bidirectional Long Short-Term Memory (BiLSTM) networks are designed to realize the semantic segmentation of QRS complex in single channel electrocardiogram (ECG) for the tasks of R peak detection and heart rate estimation. Three types of seq2seq BiLSTM networks are introduced, including the densely connected BiLSTM with attention model, the BiLSTM U-Net, and the BiLSTM U-Net++. To alleviate the sparse problem of the QRS labels, symmetric label expansion is applied by extending the single R peak into a time interval of fixed length. Linear ensemble method is introduced that averages the outputs of different BiLSTM networks. The cross-validation results show significant increase of the accuracy and decrease of discontinuous gaps in the QRS interval prediction by the ensembling over singular neural networks.

A surrounding vehicles behavior prediction method was presented for intelligent vehicles. The surrounding vehicles’ behavior is hard to predict since the significant uncertainty of vehicle driving and environmental changes. This method adopts bidirectional long short-term memory (BiLSTM) model combined with an encoder to ensure the memory of long-time series training. By constructing an attention mechanism based on BiLSTM, we consider the importance of different information which could guarantee the encoder’ memory under long sequence. The designed attention-bidirectional LSTM (Att-BiLSTM) model is adopted to ensure the surrounding vehicles’ prediction accuracy and effectiveness.

Programming is a vital skill in computer science and engineering-related disciplines. However, developing source code is an error-prone task. Logical errors in code are particularly hard to identify for both students and professionals, and a single error is unexpected to end-users. At present, conventional compilers have difficulty identifying many of the errors (especially logical errors) that can occur in code. To mitigate this problem, we propose a language model for evaluating source codes using a bidirectional long short-term memory (BiLSTM) neural network. We trained the BiLSTM model with a large number of source codes with tuning various hyperparameters. We then used the model to evaluate incorrect code and assessed the model’s performance in three principal areas: source code error detection, suggestions for incorrect code repair, and erroneous code classification. Experimental results showed that the proposed BiLSTM model achieved 50.88% correctness in identifying errors and providing suggestions. Moreover, the model achieved an F-score of approximately 97%, outperforming other state-of-the-art models (recurrent neural networks (RNNs) and long short-term memory (LSTM)).

This study proposed a medicine auxiliary diagnosis model based on neural network. The model combines a bidirectional long short-term memory（Bi-LSTM）network and bidirectional encoder representations from transformers (BERT), which can well complete the extraction of local features of Chinese medicine texts. BERT can learn the global information of the text, so use BERT to get the global representation of medical text and then use Bi-LSTM to extract local features. We conducted a large number of comparative experiments on datasets. The results show that the proposed model has significant advantages over the state-of-the-art baseline model. The accuracy of the proposed model is 0.75.

The inherent intermittency and uncertainty of wind power have brought challenges in accurate wind power output forecasting, which also cause tricky problems in the integration of wind power to the grid. In this paper, a hybrid deep learning model bidirectional long short term memory-convolutional neural network (BiLSTM-CNN) is proposed for short-term wind power forecasting. First, the grey correlation analysis is utilized to select the inputs for forecasting model; Then, the proposed hybrid model extracts multi-dimension features of inputs to predict the wind power from the temporal-spatial perspective, where the Bi-LSTM model is utilized to mine the bidirectional temporal characteristics while the convolution and pooling operations of CNN are utilized to extract the spatial characteristics from multiple input time series. Lastly, a case study is conducted to verify the superiority of the proposed model. Other deep learning models (Bi-LSTM, LSTM, CNN, LSTM-CNN, CNN-BiLSTM, CNN-LSTM) are also simulated to conduct comparison from three aspects. The results show that the BiLSTM-CNN model has the best accuracy with the lowest RMSE of 2.5492, MSE of 6.4984, MAE of 1.7344 and highest R2 of 0.9929. CNN has the fastest speed with an average computational time of 0.0741s. The hybrid model that mines the spatial feature based on the extracted temporal feature has a better performance than the model mines the temporal feature based on the extracted spatial feature.

In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.

L'estimation de l'état de charge (SOC) est un point crucial pour la sécurité des performances et la durée de vie des batteries lithium-ion (Li-ion) utilisées pour alimenter les VE.Dans cette thèse, la précision de l'estimation de l'état de charge est étudiée à l'aide d'algorithmes de réseaux neuronaux récurrents profonds (DRNN). Pour ce faire, pour une cellule d’une batterie Li-ion, trois nouvelles méthodes sont proposées : une mémoire bidirectionnelle à long et court terme (BiLSTM), une mémoire robuste à long et court terme (RoLSTM) et une technique d'unités récurrentes à grille (GRU).En utilisant ces techniques, on ne dépend pas de modèles précis de la batterie et on peut éviter les méthodes mathématiques complexes, en particulier dans un bloc de batterie. En outre, ces modèles sont capables d'estimer précisément le SOC à des températures variables. En outre, contrairement au réseau de neurones récursif traditionnel dont le contenu est réécrit à chaque fois, ces réseaux peuvent décider de préserver la mémoire actuelle grâce aux passerelles proposées. Dans ce cas, il peut facilement transférer l'information sur de longs chemins pour recevoir et maintenir des dépendances à long terme.La comparaison des résultats indique que le réseau BiLSTM a de meilleures performances que les deux autres méthodes. De plus, le modèle BiLSTM peut travailler avec des séquences plus longues provenant de deux directions, le passé et le futur, sans problème de disparition du gradient. Cette caractéristique permet de sélectionner une longueur de séquence équivalente à une période de décharge dans un cycle de conduite, et d'obtenir une plus grande précision dans l'estimation. En outre, ce modèle s'est bien comporté face à une valeur initiale incorrecte du SOC.Enfin, une nouvelle méthode BiLSTM a été introduite pour estimer le SOC d'un pack de batteries dans un EV. Le logiciel IPG Carmaker a été utilisé pour collecter les données et tester le modèle en simulation. Les résultats ont montré que l'algorithme proposé peut fournir une bonne estimation du SOC sans utilisation de filtre dans le système de gestion de la batterie (BMS)
The State Of Charge (SOC) estimation is a significant issue for safe performance and the lifespan of Lithium-ion (Li-ion) batteries, which is used to power the Electric Vehicles (EVs). In this thesis, the accuracy of SOC estimation is investigated using Deep Recurrent Neural Network (DRNN) algorithms. To do this, for a one cell Li-ion battery, three new SOC estimator based on different DRNN algorithms are proposed: a Bidirectional LSTM (BiLSTM) method, Robust Long-Short Term Memory (RoLSTM) algorithm, and a Gated Recurrent Units (GRUs) technique. Using these, one is not dependent on precise battery models and can avoid complicated mathematical methods especially in a battery pack. In addition, these models are able to precisely estimate the SOC at varying temperature. Also, unlike the traditional recursive neural network where content is re-written at each time, these networks can decide on preserving the current memory through the proposed gateways. In such case, it can easily transfer the information over long paths to receive and maintain long-term dependencies. Comparing the results indicates the BiLSTM network has a better performance than the other two. Moreover, the BiLSTM model can work with longer sequences from two direction, the past and the future, without gradient vanishing problem. This feature helps to select a sequence length as much as a discharge period in one drive cycle, and to have more accuracy in the estimation. Also, this model well behaved against the incorrect initial value of SOC. Finally, a new BiLSTM method introduced to estimate the SOC of a pack of batteries in an Ev. IPG Carmaker software was used to collect data and test the model in the simulation. The results showed that the suggested algorithm can provide a good SOC estimation without using any filter in the Battery Management System (BMS)

In this paper, we tackle an intent detection problem for the Lithuanian language with the real supervised data. Our main focus is on the enhancement of the Natural Language Understanding (NLU) module, responsible for the comprehension of user’s questions. The NLU model is trained with a properly selected word vectorization type and Deep Neural Network (DNN) classifier. During our experiments, we have experimentally investigated fastText and BERT embeddings. Besides, we have automatically optimized different architectures and hyper-parameters of the following DNN approaches: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM) and Convolutional Neural Network (CNN). The highest accuracy=∼0.715 (∼0.675 and ∼0.625 over random and majority baselines, respectively) was achieved with the CNN classifier applied on a top of BERT embeddings. The detailed error analysis revealed that prediction accuracies degrade for the least covered intents and due to intent ambiguities; therefore, in the future, we are planning to make necessary adjustments to boost the intent detection accuracy for the Lithuanian language even more.

Learning a 3D shape representation from a collection of its rendered 2D images has been extensively studied. However, existing view-based techniques have not yet fully exploited the information among all the views of projections. In this paper, by employing recurrent neural network to efficiently capture features across different views, we propose a siamese CNN-BiLSTM network for 3D shape representation learning. The proposed method minimizes a discriminative loss function to learn a deep nonlinear transformation, mapping 3D shapes from the original space into a nonlinear feature space. In the transformed space, the distance of 3D shapes with the same label is minimized, otherwise the distance is maximized to a large margin. Specifically, the 3D shapes are first projected into a group of 2D images from different views. Then convolutional neural network (CNN) is adopted to extract features from different view images, followed by a bidirectional long short-term memory (LSTM) to aggregate information across different views. Finally, we construct the whole CNN-BiLSTM network into a siamese structure with contrastive loss function. Our proposed method is evaluated on two benchmarks, ModelNet40 and SHREC 2014, demonstrating superiority over the state-of-the-art methods.

Transformer Models have taken over most of the Natural language Inference tasks. In recent times they have proved to beat several benchmarks. Chunking means splitting the sentences into tokens and then grouping them in a meaningful way. Chunking is a task that has gradually moved from POS tag-based statistical models to neural nets using Language models such as LSTM, Bidirectional LSTMs, attention models, etc. Deep neural net Models are deployed indirectly for classifying tokens as different tags defined under Named Recognition Tasks. Later these tags are used in conjunction with pointer frameworks for the final chunking task. In our paper, we propose an Ensemble Model using a fine-tuned Transformer Model and a recurrent neural network model together to predict tags and chunk substructures of a sentence. We analyzed the shortcomings of the transformer models in predicting different tags and then trained the BILSTM+CNN accordingly to compensate for the same.