To see the other types of publications on this topic, follow the link: Gated Recurrent Units (GRUs).
Journal articles on the topic 'Gated Recurrent Units (GRUs)'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Gated Recurrent Units (GRUs).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Dangovski, Rumen, Li Jing, Preslav Nakov, Mićo Tatalović, and Marin Soljačić. "Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications." Transactions of the Association for Computational Linguistics 7 (November 2019): 121–38. http://dx.doi.org/10.1162/tacl_a_00258.
Full textAbstract:
Stacking long short-term memory (LSTM) cells or gated recurrent units (GRUs) as part of a recurrent neural network (RNN) has become a standard approach to solving a number of tasks ranging from language modeling to text summarization. Although LSTMs and GRUs were designed to model long-range dependencies more accurately than conventional RNNs, they nevertheless have problems copying or recalling information from the long distant past. Here, we derive a phase-coded representation of the memory state, Rotational Unit of Memory (RUM), that unifies the concepts of unitary learning and associative memory. We show experimentally that RNNs based on RUMs can solve basic sequential tasks such as memory copying and memory recall much better than LSTMs/GRUs. We further demonstrate that by replacing LSTM/GRU with RUM units we can apply neural networks to real-world problems such as language modeling and text summarization, yielding results comparable to the state of the art.
2
Khadka, Shauharda, Jen Jen Chung, and Kagan Tumer. "Neuroevolution of a Modular Memory-Augmented Neural Network for Deep Memory Problems." Evolutionary Computation 27, no. 4 (December 2019): 639–64. http://dx.doi.org/10.1162/evco_a_00239.
Full textAbstract:
We present Modular Memory Units (MMUs), a new class of memory-augmented neural network. MMU builds on the gated neural architecture of Gated Recurrent Units (GRUs) and Long Short Term Memory (LSTMs), to incorporate an external memory block, similar to a Neural Turing Machine (NTM). MMU interacts with the memory block using independent read and write gates that serve to decouple the memory from the central feedforward operation. This allows for regimented memory access and update, giving our network the ability to choose when to read from memory, update it, or simply ignore it. This capacity to act in detachment allows the network to shield the memory from noise and other distractions, while simultaneously using it to effectively retain and propagate information over an extended period of time. We train MMU using both neuroevolution and gradient descent, and perform experiments on two deep memory benchmarks. Results demonstrate that MMU performs significantly faster and more accurately than traditional LSTM-based methods, and is robust to dramatic increases in the sequence depth of these memory benchmarks.
3
Akpudo, Ugochukwu Ejike, and Jang-Wook Hur. "A CEEMDAN-Assisted Deep Learning Model for the RUL Estimation of Solenoid Pumps." Electronics 10, no. 17 (August 25, 2021): 2054. http://dx.doi.org/10.3390/electronics10172054.
Full textAbstract:
This paper develops a data-driven remaining useful life prediction model for solenoid pumps. The model extracts high-level features using stacked autoencoders from decomposed pressure signals (using complementary ensemble empirical mode decomposition with adaptive noise (CEEMDAN) algorithm). These high-level features are then received by a recurrent neural network-gated recurrent units (GRUs) for the RUL estimation. The case study presented demonstrates the robustness of the proposed RUL estimation model with extensive empirical validations. Results support the validity of using the CEEMDAN for non-stationary signal decomposition and the accuracy, ease-of-use, and superiority of the proposed DL-based model for solenoid pump failure prognostics.
4
Shen, Wenjuan, and Xiaoling Li. "Facial expression recognition based on bidirectional gated recurrent units within deep residual network." International Journal of Intelligent Computing and Cybernetics 13, no. 4 (October 12, 2020): 527–43. http://dx.doi.org/10.1108/ijicc-07-2020-0088.
Full textAbstract:
Purposerecent years, facial expression recognition has been widely used in human machine interaction, clinical medicine and safe driving. However, there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approachTo solve such limitation, this paper proposes a novel model based on bidirectional gated recurrent unit networks (Bi-GRUs) with two-way propagations, and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network. Since the Inception-V3 network model for spatial feature extraction has too many parameters, it is prone to overfitting during training. This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters, so as to obtain an Inception-W network with better generalization.FindingsFinally, the proposed model is pretrained to determine the best settings and selections. Then, the pretrained model is experimented on two facial expression data sets of CK+ and Oulu- CASIA, and the recognition performance and efficiency are compared with the existing methods. The highest recognition rate is 99.6%, which shows that the method has good recognition accuracy in a certain range.Originality/valueBy using the proposed model for the applications of facial expression, the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.
5
Ding, Chen, Zhouyi Zheng, Sirui Zheng, Xuke Wang, Xiaoyan Xie, Dushi Wen, Lei Zhang, and Yanning Zhang. "Accurate Air-Quality Prediction Using Genetic-Optimized Gated-Recurrent-Unit Architecture." Information 13, no. 5 (April 26, 2022): 223. http://dx.doi.org/10.3390/info13050223.
Full textAbstract:
Air pollution is becoming a serious concern with the development of society and urban expansion, and predicting air quality is the most pressing problem for human beings. Recently, more and more machine-learning-based methods are being used to solve the air-quality-prediction problem, and gated recurrent units (GRUs) are a representative method because of their advantage for processing time-series data. However, in the same air-quality-prediction task, different researchers have always designed different structures of the GRU due to their different experiences. Data-adaptively designing a GRU structure has thus become a problem. In this paper, we propose an adaptive GRU to address this problem, and the adaptive GRU structures are determined by the dataset, which mainly contributes with three steps. Firstly, an encoding method for the GRU structure is proposed for representing the network structure in a fixed-length binary string; secondly, we define the reciprocal of the sum of the loss of each individual as the fitness function for the iteration computation; thirdly, the genetic algorithm is used for computing the data-adaptive GRU network structure, which can enhance the air-quality-prediction result. The experiment results from three real datasets in Xi’an show that the proposed method achieves better effectiveness in RMSE and SAMPE than the existing LSTM-, SVM-, and RNN-based methods.
6
Ding, Chen, Zhouyi Zheng, Sirui Zheng, Xuke Wang, Xiaoyan Xie, Dushi Wen, Lei Zhang, and Yanning Zhang. "Accurate Air-Quality Prediction Using Genetic-Optimized Gated-Recurrent-Unit Architecture." Information 13, no. 5 (April 26, 2022): 223. http://dx.doi.org/10.3390/info13050223.
Full textAbstract:
Air pollution is becoming a serious concern with the development of society and urban expansion, and predicting air quality is the most pressing problem for human beings. Recently, more and more machine-learning-based methods are being used to solve the air-quality-prediction problem, and gated recurrent units (GRUs) are a representative method because of their advantage for processing time-series data. However, in the same air-quality-prediction task, different researchers have always designed different structures of the GRU due to their different experiences. Data-adaptively designing a GRU structure has thus become a problem. In this paper, we propose an adaptive GRU to address this problem, and the adaptive GRU structures are determined by the dataset, which mainly contributes with three steps. Firstly, an encoding method for the GRU structure is proposed for representing the network structure in a fixed-length binary string; secondly, we define the reciprocal of the sum of the loss of each individual as the fitness function for the iteration computation; thirdly, the genetic algorithm is used for computing the data-adaptive GRU network structure, which can enhance the air-quality-prediction result. The experiment results from three real datasets in Xi’an show that the proposed method achieves better effectiveness in RMSE and SAMPE than the existing LSTM-, SVM-, and RNN-based methods.
7
ArunKumar, K. E., Dinesh V. Kalaga, Ch Mohan Sai Kumar, Masahiro Kawaji, and Timothy M. Brenza. "Forecasting of COVID-19 using deep layer Recurrent Neural Networks (RNNs) with Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM) cells." Chaos, Solitons & Fractals 146 (May 2021): 110861. http://dx.doi.org/10.1016/j.chaos.2021.110861.
Full text
8
Oliveira, Pedro, Bruno Fernandes, Cesar Analide, and Paulo Novais. "Forecasting Energy Consumption of Wastewater Treatment Plants with a Transfer Learning Approach for Sustainable Cities." Electronics 10, no. 10 (May 12, 2021): 1149. http://dx.doi.org/10.3390/electronics10101149.
Full textAbstract:
A major challenge of today’s society is to make large urban centres more sustainable. Improving the energy efficiency of the various infrastructures that make up cities is one aspect being considered when improving their sustainability, with Wastewater Treatment Plants (WWTPs) being one of them. Consequently, this study aims to conceive, tune, and evaluate a set of candidate deep learning models with the goal being to forecast the energy consumption of a WWTP, following a recursive multi-step approach. Three distinct types of models were experimented, in particular, Long Short-Term Memory networks (LSTMs), Gated Recurrent Units (GRUs), and uni-dimensional Convolutional Neural Networks (CNNs). Uni- and multi-variate settings were evaluated, as well as different methods for handling outliers. Promising forecasting results were obtained by CNN-based models, being this difference statistically significant when compared to LSTMs and GRUs, with the best model presenting an approximate overall error of 630 kWh when on a multi-variate setting. Finally, to overcome the problem of data scarcity in WWTPs, transfer learning processes were implemented, with promising results being achieved when using a pre-trained uni-variate CNN model, with the overall error reducing to 325 kWh.
9
Fang, Weiguang, Yu Guo, Wenhe Liao, Shaohua Huang, Nengjun Yang, and Jinshan Liu. "A Parallel Gated Recurrent Units (P-GRUs) network for the shifting lateness bottleneck prediction in make-to-order production system." Computers & Industrial Engineering 140 (February 2020): 106246. http://dx.doi.org/10.1016/j.cie.2019.106246.
Full text
10
Fang, Qiang, and Xavier Maldague. "A Method of Defect Depth Estimation for Simulated Infrared Thermography Data with Deep Learning." Applied Sciences 10, no. 19 (September 29, 2020): 6819. http://dx.doi.org/10.3390/app10196819.
Full textAbstract:
Infrared thermography has already been proven to be a significant method in non-destructive evaluation since it gives information with immediacy, rapidity, and low cost. However, the thorniest issue for the wider application of IRT is quantification. In this work, we proposed a specific depth quantifying technique by employing the Gated Recurrent Units (GRUs) in composite material samples via pulsed thermography (PT). Finite Element Method (FEM) modeling provides the economic examination of the response pulsed thermography. In this work, Carbon Fiber Reinforced Polymer (CFRP) specimens embedded with flat bottom holes are stimulated by a FEM modeling (COMSOL) with precisely controlled depth and geometrics of the defects. The GRU model automatically quantified the depth of defects presented in the stimulated CFRP material. The proposed method evaluated the accuracy and performance of synthetic CFRP data from FEM for defect depth predictions.
11
Chui, Kwok Tai, Brij B. Gupta, Ryan Wen Liu, Xinyu Zhang, Pandian Vasant, and J. Joshua Thomas. "Extended-Range Prediction Model Using NSGA-III Optimized RNN-GRU-LSTM for Driver Stress and Drowsiness." Sensors 21, no. 19 (September 25, 2021): 6412. http://dx.doi.org/10.3390/s21196412.
Full textAbstract:
Road traffic accidents have been listed in the top 10 global causes of death for many decades. Traditional measures such as education and legislation have contributed to limited improvements in terms of reducing accidents due to people driving in undesirable statuses, such as when suffering from stress or drowsiness. Attention is drawn to predicting drivers’ future status so that precautions can be taken in advance as effective preventative measures. Common prediction algorithms include recurrent neural networks (RNNs), gated recurrent units (GRUs), and long short-term memory (LSTM) networks. To benefit from the advantages of each algorithm, nondominated sorting genetic algorithm-III (NSGA-III) can be applied to merge the three algorithms. This is named NSGA-III-optimized RNN-GRU-LSTM. An analysis can be made to compare the proposed prediction algorithm with the individual RNN, GRU, and LSTM algorithms. Our proposed model improves the overall accuracy by 11.2–13.6% and 10.2–12.2% in driver stress prediction and driver drowsiness prediction, respectively. Likewise, it improves the overall accuracy by 6.9–12.7% and 6.9–8.9%, respectively, compared with boosting learning with multiple RNNs, multiple GRUs, and multiple LSTMs algorithms. Compared with existing works, this proposal offers to enhance performance by taking some key factors into account—namely, using a real-world driving dataset, a greater sample size, hybrid algorithms, and cross-validation. Future research directions have been suggested for further exploration and performance enhancement.
12
Noh, Seol-Hyun. "Analysis of Gradient Vanishing of RNNs and Performance Comparison." Information 12, no. 11 (October 25, 2021): 442. http://dx.doi.org/10.3390/info12110442.
Full textAbstract:
A recurrent neural network (RNN) combines variable-length input data with a hidden state that depends on previous time steps to generate output data. RNNs have been widely used in time-series data analysis, and various RNN algorithms have been proposed, such as the standard RNN, long short-term memory (LSTM), and gated recurrent units (GRUs). In particular, it has been experimentally proven that LSTM and GRU have higher validation accuracy and prediction accuracy than the standard RNN. The learning ability is a measure of the effectiveness of gradient of error information that would be backpropagated. This study provided a theoretical and experimental basis for the result that LSTM and GRU have more efficient gradient descent than the standard RNN by analyzing and experimenting the gradient vanishing of the standard RNN, LSTM, and GRU. As a result, LSTM and GRU are robust to the degradation of gradient descent even when LSTM and GRU learn long-range input data, which means that the learning ability of LSTM and GRU is greater than standard RNN when learning long-range input data. Therefore, LSTM and GRU have higher validation accuracy and prediction accuracy than the standard RNN. In addition, it was verified whether the experimental results of river-level prediction models, solar power generation prediction models, and speech signal models using the standard RNN, LSTM, and GRUs are consistent with the analysis results of gradient vanishing.
13
Jiao, Wenxiang, Michael Lyu, and Irwin King. "Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Network." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (April 3, 2020): 8002–9. http://dx.doi.org/10.1609/aaai.v34i05.6309.
Full textAbstract:
Real-time emotion recognition (RTER) in conversations is significant for developing emotionally intelligent chatting machines. Without the future context in RTER, it becomes critical to build the memory bank carefully for capturing historical context and summarize the memories appropriately to retrieve relevant information. We propose an Attention Gated Hierarchical Memory Network (AGHMN) to address the problems of prior work: (1) Commonly used convolutional neural networks (CNNs) for utterance feature extraction are less compatible in the memory modules; (2) Unidirectional gated recurrent units (GRUs) only allow each historical utterance to have context before it, preventing information propagation in the opposite direction; (3) The Soft Attention for summarizing loses the positional and ordering information of memories, regardless of how the memory bank is built. Particularly, we propose a Hierarchical Memory Network (HMN) with a bidirectional GRU (BiGRU) as the utterance reader and a BiGRU fusion layer for the interaction between historical utterances. For memory summarizing, we propose an Attention GRU (AGRU) where we utilize the attention weights to update the internal state of GRU. We further promote the AGRU to a bidirectional variant (BiAGRU) to balance the contextual information from recent memories and that from distant memories. We conduct experiments on two emotion conversation datasets with extensive analysis, demonstrating the efficacy of our AGHMN models.
14
Sattari, Mohammad Taghi, Halit Apaydin, and Shahaboddin Shamshirband. "Performance Evaluation of Deep Learning-Based Gated Recurrent Units (GRUs) and Tree-Based Models for Estimating ETo by Using Limited Meteorological Variables." Mathematics 8, no. 6 (June 13, 2020): 972. http://dx.doi.org/10.3390/math8060972.
Full textAbstract:
The amount of water allocated to irrigation systems is significantly greater than the amount allocated to other sectors. Thus, irrigation water demand management is at the center of the attention of the Ministry of Agriculture and Forestry in Turkey. To plan more effective irrigation systems in agriculture, it is necessary to accurately calculate plant water requirements. In this study, daily reference evapotranspiration (ETo) values were estimated using tree-based regression and deep learning-based gated recurrent unit (GRU) models. For this purpose, 15 input scenarios, consisting of meteorological variables including maximum and minimum temperature, wind speed, maximum and minimum relative humidity, dew point temperature, and sunshine duration, were considered. ETo values calculated according to the United Nations Food and Agriculture Organization (FAO) Penman-Monteith method were considered as model outputs. The results indicate that the random forest model, with a correlation coefficient of 0.9926, is better than the other tree-based models. In addition, the GRU model, with R = 0.9837, presents good performance relative to the other models. In this study, it was found that maximum temperature was more effective in estimating ETo than other variables.
15
Aldallal, Ammar. "Toward Efficient Intrusion Detection System Using Hybrid Deep Learning Approach." Symmetry 14, no. 9 (September 13, 2022): 1916. http://dx.doi.org/10.3390/sym14091916.
Full textAbstract:
The increased adoption of cloud computing resources produces major loopholes in cloud computing for cybersecurity attacks. An intrusion detection system (IDS) is one of the vital defenses against threats and attacks to cloud computing. Current IDSs encounter two challenges, namely, low accuracy and a high false alarm rate. Due to these challenges, additional efforts are required by network experts to respond to abnormal traffic alerts. To improve IDS efficiency in detecting abnormal network traffic, this work develops an IDS using a recurrent neural network based on gated recurrent units (GRUs) and improved long short-term memory (LSTM) through a computing unit to form Cu-LSTMGRU. The proposed system efficiently classifies the network flow instances as benign or malevolent. This system is examined using the most up-to-date dataset CICIDS2018. To further optimize computational complexity, the dataset is optimized through the Pearson correlation feature selection algorithm. The proposed model is evaluated using several metrics. The results show that the proposed model remarkably outperforms benchmarks by up to 12.045%. Therefore, the Cu-LSTMGRU model provides a high level of symmetry between cloud computing security and the detection of intrusions and malicious attacks.
16
Gim, Juhui, Wansik Choi, and Changsun Ahn. "Design of Unscented Kalman Filter with Gated Recurrent Units-based Battery Model for SOC Estimation." Transaction of the Korean Society of Automotive Engineers 30, no. 1 (January 1, 2022): 61–68. http://dx.doi.org/10.7467/ksae.2022.30.1.061.
Full text
17
Khan, Muhammad Almas, Muazzam A. Khan, Sana Ullah Jan, Jawad Ahmad, Sajjad Shaukat Jamal, Awais Aziz Shah, Nikolaos Pitropakis, and William J. Buchanan. "A Deep Learning-Based Intrusion Detection System for MQTT Enabled IoT." Sensors 21, no. 21 (October 22, 2021): 7016. http://dx.doi.org/10.3390/s21217016.
Full textAbstract:
A large number of smart devices in Internet of Things (IoT) environments communicate via different messaging protocols. Message Queuing Telemetry Transport (MQTT) is a widely used publish–subscribe-based protocol for the communication of sensor or event data. The publish–subscribe strategy makes it more attractive for intruders and thus increases the number of possible attacks over MQTT. In this paper, we proposed a Deep Neural Network (DNN) for intrusion detection in the MQTT-based protocol and also compared its performance with other traditional machine learning (ML) algorithms, such as a Naive Bayes (NB), Random Forest (RF), k-Nearest Neighbour (kNN), Decision Tree (DT), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRUs). The performance is proved using two different publicly available datasets, including (1) MQTT-IoT-IDS2020 and (2) a dataset with three different types of attacks, such as Man in the Middle (MitM), Intrusion in the network, and Denial of Services (DoS). The MQTT-IoT-IDS2020 contains three abstract-level features, including Uni-Flow, Bi-Flow, and Packet-Flow. The results for the first dataset and binary classification show that the DNN-based model achieved 99.92%, 99.75%, and 94.94% accuracies for Uni-flow, Bi-flow, and Packet-flow, respectively. However, in the case of multi-label classification, these accuracies reduced to 97.08%, 98.12%, and 90.79%, respectively. On the other hand, the proposed DNN model attains the highest accuracy of 97.13% against LSTM and GRUs for the second dataset.
18
Aslam, Muhammad, Jae-Myeong Lee, Hyung-Seung Kim, Seung-Jae Lee, and Sugwon Hong. "Deep Learning Models for Long-Term Solar Radiation Forecasting Considering Microgrid Installation: A Comparative Study." Energies 13, no. 1 (December 27, 2019): 147. http://dx.doi.org/10.3390/en13010147.
Full textAbstract:
Microgrid is becoming an essential part of the power grid regarding reliability, economy, and environment. Renewable energies are main sources of energy in microgrids. Long-term solar generation forecasting is an important issue in microgrid planning and design from an engineering point of view. Solar generation forecasting mainly depends on solar radiation forecasting. Long-term solar radiation forecasting can also be used for estimating the degradation-rate-influenced energy potentials of photovoltaic (PV) panel. In this paper, a comparative study of different deep learning approaches is carried out for forecasting one year ahead hourly and daily solar radiation. In the proposed method, state of the art deep learning and machine learning architectures like gated recurrent units (GRUs), long short term memory (LSTM), recurrent neural network (RNN), feed forward neural network (FFNN), and support vector regression (SVR) models are compared. The proposed method uses historical solar radiation data and clear sky global horizontal irradiance (GHI). Even though all the models performed well, GRU performed relatively better compared to the other models. The proposed models are also compared with traditional state of the art methods for long-term solar radiation forecasting, i.e., random forest regression (RFR). The proposed models outperformed the traditional method, hence proving their efficiency.
19
Gupta, Manish, and Puneet Agrawal. "Compression of Deep Learning Models for Text: A Survey." ACM Transactions on Knowledge Discovery from Data 16, no. 4 (August 31, 2022): 1–55. http://dx.doi.org/10.1145/3487045.
Full textAbstract:
In recent years, the fields of natural language processing (NLP) and information retrieval (IR) have made tremendous progress thanks to deep learning models like Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) networks, and Transformer [ 121 ] based models like Bidirectional Encoder Representations from Transformers (BERT) [ 24 ], Generative Pre-training Transformer (GPT-2) [ 95 ], Multi-task Deep Neural Network (MT-DNN) [ 74 ], Extra-Long Network (XLNet) [ 135 ], Text-to-text transfer transformer (T5) [ 96 ], T-NLG [ 99 ], and GShard [ 64 ]. But these models are humongous in size. On the other hand, real-world applications demand small model size, low response times, and low computational power wattage. In this survey, we discuss six different types of methods (Pruning, Quantization, Knowledge Distillation (KD), Parameter Sharing, Tensor Decomposition, and Sub-quadratic Transformer-based methods) for compression of such models to enable their deployment in real industry NLP projects. Given the critical need of building applications with efficient and small models, and the large amount of recently published work in this area, we believe that this survey organizes the plethora of work done by the “deep learning for NLP” community in the past few years and presents it as a coherent story.
20
Choi, Edward, Andy Schuetz, Walter F. Stewart, and Jimeng Sun. "Using recurrent neural network models for early detection of heart failure onset." Journal of the American Medical Informatics Association 24, no. 2 (August 13, 2016): 361–70. http://dx.doi.org/10.1093/jamia/ocw112.
Full textAbstract:
Objective: We explored whether use of deep learning to model temporal relations among events in electronic health records (EHRs) would improve model performance in predicting initial diagnosis of heart failure (HF) compared to conventional methods that ignore temporality. Materials and Methods: Data were from a health system’s EHR on 3884 incident HF cases and 28 903 controls, identified as primary care patients, between May 16, 2000, and May 23, 2013. Recurrent neural network (RNN) models using gated recurrent units (GRUs) were adapted to detect relations among time-stamped events (eg, disease diagnosis, medication orders, procedure orders, etc.) with a 12- to 18-month observation window of cases and controls. Model performance metrics were compared to regularized logistic regression, neural network, support vector machine, and K-nearest neighbor classifier approaches. Results: Using a 12-month observation window, the area under the curve (AUC) for the RNN model was 0.777, compared to AUCs for logistic regression (0.747), multilayer perceptron (MLP) with 1 hidden layer (0.765), support vector machine (SVM) (0.743), and K-nearest neighbor (KNN) (0.730). When using an 18-month observation window, the AUC for the RNN model increased to 0.883 and was significantly higher than the 0.834 AUC for the best of the baseline methods (MLP). Conclusion: Deep learning models adapted to leverage temporal relations appear to improve performance of models for detection of incident heart failure with a short observation window of 12–18 months.
21
Cowton, Jake, Ilias Kyriazakis, Thomas Plötz, and Jaume Bacardit. "A Combined Deep Learning GRU-Autoencoder for the Early Detection of Respiratory Disease in Pigs Using Multiple Environmental Sensors." Sensors 18, no. 8 (August 2, 2018): 2521. http://dx.doi.org/10.3390/s18082521.
Full textAbstract:
We designed and evaluated an assumption-free, deep learning-based methodology for animal health monitoring, specifically for the early detection of respiratory disease in growing pigs based on environmental sensor data. Two recurrent neural networks (RNNs), each comprising gated recurrent units (GRUs), were used to create an autoencoder (GRU-AE) into which environmental data, collected from a variety of sensors, was processed to detect anomalies. An autoencoder is a type of network trained to reconstruct the patterns it is fed as input. By training the GRU-AE using environmental data that did not lead to an occurrence of respiratory disease, data that did not fit the pattern of “healthy environmental data” had a greater reconstruction error. All reconstruction errors were labelled as either normal or anomalous using threshold-based anomaly detection optimised with particle swarm optimisation (PSO), from which alerts are raised. The results from the GRU-AE method outperformed state-of-the-art techniques, raising alerts when such predictions deviated from the actual observations. The results show that a change in the environment can result in occurrences of pigs showing symptoms of respiratory disease within 1–7 days, meaning that there is a period of time during which their keepers can act to mitigate the negative effect of respiratory diseases, such as porcine reproductive and respiratory syndrome (PRRS), a common and destructive disease endemic in pigs.
22
Lanera, Corrado, Ileana Baldi, Andrea Francavilla, Elisa Barbieri, Lara Tramontan, Antonio Scamarcia, Luigi Cantarutti, Carlo Giaquinto, and Dario Gregori. "A Deep Learning Approach to Estimate the Incidence of Infectious Disease Cases for Routinely Collected Ambulatory Records: The Example of Varicella-Zoster." International Journal of Environmental Research and Public Health 19, no. 10 (May 13, 2022): 5959. http://dx.doi.org/10.3390/ijerph19105959.
Full textAbstract:
The burden of infectious diseases is crucial for both epidemiological surveillance and prompt public health response. A variety of data, including textual sources, can be fruitfully exploited. Dealing with unstructured data necessitates the use of methods for automatic data-driven variable construction and machine learning techniques (MLT) show promising results. In this framework, varicella-zoster virus (VZV) infection was chosen to perform an automatic case identification with MLT. Pedianet, an Italian pediatric primary care database, was used to train a series of models to identify whether a child was diagnosed with VZV infection between 2004 and 2014 in the Veneto region, starting from free text fields. Given the nature of the task, a recurrent neural network (RNN) with bidirectional gated recurrent units (GRUs) was chosen; the same models were then used to predict the children’s status for the following years. A gold standard produced by manual extraction for the same interval was available for comparison. RNN-GRU improved its performance over time, reaching the maximum value of area under the ROC curve (AUC-ROC) of 95.30% at the end of the period. The absolute bias in estimates of VZV infection was below 1.5% in the last five years analyzed. The findings in this study could assist the large-scale use of EHRs for clinical outcome predictive modeling and help establish high-performance systems in other medical domains.
23
Meng, Zhaorui, and Xianze Xu. "A Hybrid Short-Term Load Forecasting Framework with an Attention-Based Encoder–Decoder Network Based on Seasonal and Trend Adjustment." Energies 12, no. 24 (December 4, 2019): 4612. http://dx.doi.org/10.3390/en12244612.
Full textAbstract:
Accurate electrical load forecasting plays an important role in power system operation. An effective load forecasting approach can improve the operation efficiency of a power system. This paper proposes the seasonal and trend adjustment attention encoder–decoder (STA–AED), a hybrid short-term load forecasting approach based on a multi-head attention encoder–decoder module with seasonal and trend adjustment. A seasonal and trend decomposing technique is used to preprocess the original electrical load data. Each decomposed datum is regressed to predict the future electric load value by utilizing the encoder–decoder network with the multi-head attention mechanism. With the multi-head attention mechanism, STA–AED can interpret the prediction results more effectively. A large number of experiments and extensive comparisons have been carried out with a load forecasting dataset from the United States. The proposed hybrid STA–AED model is superior to the other five counterpart models such as random forest, gradient boosting decision tree (GBDT), gated recurrent units (GRUs), Encoder–Decoder, and Encoder–Decoder with multi-head attention. The proposed hybrid model shows the best prediction accuracy in 14 out of 15 zones in terms of both root mean square error (RMSE) and mean absolute percentage error (MAPE).
24
Wei, Minghua, and Feng Lin. "A novel multi-dimensional features fusion algorithm for the EEG signal recognition of brain's sensorimotor region activated tasks." International Journal of Intelligent Computing and Cybernetics 13, no. 2 (June 8, 2020): 239–60. http://dx.doi.org/10.1108/ijicc-02-2020-0019.
Full textAbstract:
PurposeAiming at the shortcomings of EEG signals generated by brain's sensorimotor region activated tasks, such as poor performance, low efficiency and weak robustness, this paper proposes an EEG signals classification method based on multi-dimensional fusion features.Design/methodology/approachFirst, the improved Morlet wavelet is used to extract the spectrum feature maps from EEG signals. Then, the spatial-frequency features are extracted from the PSD maps by using the three-dimensional convolutional neural networks (3DCNNs) model. Finally, the spatial-frequency features are incorporated to the bidirectional gated recurrent units (Bi-GRUs) models to extract the spatial-frequency-sequential multi-dimensional fusion features for recognition of brain's sensorimotor region activated task.FindingsIn the comparative experiments, the data sets of motor imagery (MI)/action observation (AO)/action execution (AE) tasks are selected to test the classification performance and robustness of the proposed algorithm. In addition, the impact of extracted features on the sensorimotor region and the impact on the classification processing are also analyzed by visualization during experiments.Originality/valueThe experimental results show that the proposed algorithm extracts the corresponding brain activation features for different action related tasks, so as to achieve more stable classification performance in dealing with AO/MI/AE tasks, and has the best robustness on EEG signals of different subjects.
25
Lv, Yafei, Xiaohan Zhang, Wei Xiong, Yaqi Cui, and Mi Cai. "An End-to-End Local-Global-Fusion Feature Extraction Network for Remote Sensing Image Scene Classification." Remote Sensing 11, no. 24 (December 13, 2019): 3006. http://dx.doi.org/10.3390/rs11243006.
Full textAbstract:
Remote sensing image scene classification (RSISC) is an active task in the remote sensing community and has attracted great attention due to its wide applications. Recently, the deep convolutional neural networks (CNNs)-based methods have witnessed a remarkable breakthrough in performance of remote sensing image scene classification. However, the problem that the feature representation is not discriminative enough still exists, which is mainly caused by the characteristic of inter-class similarity and intra-class diversity. In this paper, we propose an efficient end-to-end local-global-fusion feature extraction (LGFFE) network for a more discriminative feature representation. Specifically, global and local features are extracted from channel and spatial dimensions respectively, based on a high-level feature map from deep CNNs. For the local features, a novel recurrent neural network (RNN)-based attention module is first proposed to capture the spatial layout information and context information across different regions. Gated recurrent units (GRUs) is then exploited to generate the important weight of each region by taking a sequence of features from image patches as input. A reweighed regional feature representation can be obtained by focusing on the key region. Then, the final feature representation can be acquired by fusing the local and global features. The whole process of feature extraction and feature fusion can be trained in an end-to-end manner. Finally, extensive experiments have been conducted on four public and widely used datasets and experimental results show that our method LGFFE outperforms baseline methods and achieves state-of-the-art results.
26
Mohsenimanesh, Ahmad, Evgueniy Entchev, and Filip Bosnjak. "Hybrid Model Based on an SD Selection, CEEMDAN, and Deep Learning for Short-Term Load Forecasting of an Electric Vehicle Fleet." Applied Sciences 12, no. 18 (September 16, 2022): 9288. http://dx.doi.org/10.3390/app12189288.
Full textAbstract:
Forecasting the aggregate charging load of a fleet of electric vehicles (EVs) plays an important role in the energy management of the future power system. Therefore, accurate charging load forecasting is necessary for reliable and efficient power system operation. A hybrid method that is a combination of the similar day (SD) selection, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), and deep neural networks is proposed and explored in this paper. For the SD selection, an extreme gradient boosting (XGB)-based weighted k-means method is chosen and applied to evaluate the similarity between the prediction and historical days. The CEEMDAN algorithm, which is an advanced method of empirical mode decomposition (EMD), is used to decompose original data, to acquire intrinsic mode functions (IMFs) and residuals, and to improve the noise reduction effect. Three popular deep neural networks that have been utilized for load predictions are gated recurrent units (GRUs), long short-term memory (LSTM), and bidirectional long short-term memory (BiLSTM). The developed models were assessed on a real-life charging load dataset that was collected from 1000 EVs in nine provinces in Canada from 2017 to 2019. The obtained numerical results of six predictive combination models show that the proposed hybrid SD-CEEMDAN-BiLSTM model outperformed the single and other hybrid models with the smallest forecasting mean absolute percentage error (MAPE) of 2.63% Canada-wide.
27
Harrou, Fouzi, Abdelkader Dairi, Abdelhafid Zeroual, and Ying Sun. "Forecasting of Bicycle and Pedestrian Traffic Using Flexible and Efficient Hybrid Deep Learning Approach." Applied Sciences 12, no. 9 (April 28, 2022): 4482. http://dx.doi.org/10.3390/app12094482.
Full textAbstract:
Recently, increasing interest in managing pedestrian and bicycle flows has been demonstrated by cities and transportation professionals aiming to reach community goals related to health, safety, and the environment. Precise forecasting of pedestrian and bicycle traffic flow is crucial for identifying the potential use of bicycle and pedestrian infrastructure and improving bicyclists’ safety and comfort. Advances in sensory technology enable collecting massive traffic flow data, including road traffic, bicycle, and pedestrian traffic flow. This paper introduces a novel deep hybrid learning model with a fully guided-attention mechanism to improve bicycles and pedestrians’ traffic flow forecasting. Notably, the proposed approach extends the modeling capability of the Variational Autoencoder (VAE) by merging a long short-term memory (LSTM) model with the VAE’s decoder and using a self-attention mechanism at multi-stage of the VAE model (i.e., decoder and before data resampling). Specifically, LSTM improves the VAE decoder’s capacity in learning temporal dependencies, and the guided-attention units enable selecting relevant features based on the self-attention mechanism. This proposed deep hybrid learning model with a multi-stage guided-attention mechanism is called GAHD-VAE. Proposed methods were validated with traffic measurements from six publicly available pedestrian and bicycle traffic flow datasets. The proposed method provides promising forecasting results but requires no assumptions that the data are drawn from a given distribution. Results revealed that the GAHD-VAE methodology can efficiently enhance the traffic forecasting accuracy and achieved better performance than the deep learning methods VAE, LSTM, gated recurrent units (GRUs), bidirectional LSTM, bidirectional GRU, convolutional neural network (CNN), and convolutional LSTM (ConvLSTM), and four shallow methods, linear regression, lasso regression, ridge regression, and support vector regression.
28
Ahanger, Tariq Ahamed, Abdulaziz Aldaej, Mohammed Atiquzzaman, Imdad Ullah, and Muhammad Yousufudin. "Federated Learning-Inspired Technique for Attack Classification in IoT Networks." Mathematics 10, no. 12 (June 20, 2022): 2141. http://dx.doi.org/10.3390/math10122141.
Full textAbstract:
More than 10-billion physical items are being linked to the internet to conduct activities more independently and with less human involvement owing to the Internet of Things (IoT) technology. IoT networks are considered a source of identifiable data for vicious attackers to carry out criminal actions using automated processes. Machine learning (ML)-assisted methods for IoT security have gained much attention in recent years. However, the ML-training procedure incorporates large data which is transferable to the central server since data are created continually by IoT devices at the edge. In other words, conventional ML relies on a single server to store all of its data, which makes it a less desirable option for domains concerned about user privacy. The Federated Learning (FL)-based anomaly detection technique, which utilizes decentralized on-device data to identify IoT network intrusions, represents the proposed solution to the aforementioned problem. By exchanging updated weights with the centralized FL-server, the data are kept on local IoT devices while federating training cycles over GRUs (Gated Recurrent Units) models. The ensemble module of the technique assesses updates from several sources for improving the accuracy of the global ML technique. Experiments have shown that the proposed method surpasses the state-of-the-art techniques in protecting user data by registering enhanced performance measures of Statistical Analysis, Energy Efficiency, Memory Utilization, Attack Classification, and Client Accuracy Analysis for the identification of attacks.
29
Reich, Thilo, David Hulbert, and Marcin Budka. "A Model Architecture for Public Transport Networks Using a Combination of a Recurrent Neural Network Encoder Library and a Attention Mechanism." Algorithms 15, no. 9 (September 14, 2022): 328. http://dx.doi.org/10.3390/a15090328.
Full textAbstract:
This study presents a working concept of a model architecture allowing to leverage the state of an entire transport network to make estimated arrival time (ETA) and next-step location predictions. To this end, a combination of an attention mechanism with a dynamically changing recurrent neural network (RNN)-based encoder library is used. To achieve this, an attention mechanism was employed that incorporates the states of other vehicles in the network by encoding their positions using gated recurrent units (GRUs) of the individual bus line to encode their current state. By muting specific parts of the imputed information, their impact on prediction accuracy can be estimated on a subset of the available data. The results of the experimental investigation show that the full model with access to all the network data performed better in some scenarios. However, a model limited to vehicles of the same line ahead of the target was the best performing model, suggesting that the incorporation of additional data can have a negative impact on the prediction accuracy if they do not add any useful information. This could be caused by poor data quality but also by a lack of interaction between the included lines and the target line. The technical aspects of this study are challenging and resulted in a very inefficient training procedure. We highlight several areas where improvements to our presented method are required to make it a viable alternative to current methods. The findings in this study should be considered as a possible and promising avenue for further research into this novel architecture. As such, it is a stepping stone for future research to improve public transport predictions if network operators provide high-quality datasets.
30
Reza, Selim, Marta Campos Ferreira, José J. M. Machado, and João Manuel R. S. Tavares. "Traffic State Prediction Using One-Dimensional Convolution Neural Networks and Long Short-Term Memory." Applied Sciences 12, no. 10 (May 19, 2022): 5149. http://dx.doi.org/10.3390/app12105149.
Full textAbstract:
Traffic prediction is a vitally important keystone of an intelligent transportation system (ITS). It aims to improve travel route selection, reduce overall carbon emissions, mitigate congestion, and enhance safety. However, efficiently modelling traffic flow is challenging due to its dynamic and non-linear behaviour. With the availability of a vast number of data samples, deep neural network-based models are best suited to solve these challenges. However, conventional network-based models lack robustness and accuracy because of their incapability to capture traffic’s spatial and temporal correlations. Besides, they usually require data from adjacent roads to achieve accurate predictions. Hence, this article presents a one-dimensional (1D) convolution neural network (CNN) and long short-term memory (LSTM)-based traffic state prediction model, which was evaluated using the Zenodo and PeMS datasets. The model used three stacked layers of 1D CNN, and LSTM with a logarithmic hyperbolic cosine loss function. The 1D CNN layers extract the features from the data, and the goodness of the LSTM is used to remember the past events to leverage them for the learnt features for traffic state prediction. A comparative performance analysis of the proposed model against support vector regression, standard LSTM, gated recurrent units (GRUs), and CNN and GRU-based models under the same conditions is also presented. The results demonstrate very encouraging performance of the proposed model, improving the mean absolute error, root mean squared error, mean percentage absolute error, and coefficient of determination scores by a mean of 16.97%, 52.1%, 54.15%, and 7.87%, respectively, relative to the baselines under comparison.
31
Chen, Zengshun, Chenfeng Yuan, Haofan Wu, Likai Zhang, Ke Li, Xuanyi Xue, and Lei Wu. "An Improved Method Based on EEMD-LSTM to Predict Missing Measured Data of Structural Sensors." Applied Sciences 12, no. 18 (September 8, 2022): 9027. http://dx.doi.org/10.3390/app12189027.
Full textAbstract:
Time history testing using a shaking table is one of the most widely used methods for assessing the dynamic response of structures. In shaking-table experiments and on-site monitoring, acceleration sensors are facing problems of missing data due to the fact of measurement point failures, affecting the validity and accuracy of assessing the structural dynamic response. The original measured signals are decomposed by ensemble empirical mode decomposition (EEMD), and the widely used deep neural networks (DNNs), gated recurrent units (GRUs), and long short-term memory networks (LSTMs) are used to predict the subseries of the decomposed original measured signal data to help model and recover the irregular, periodic variations in the measured signal data. The raw acceleration data of a liquefied natural gas (LNG) storage tank in shaking-table experiments were used as an example to compare and discuss the method’s performance for the complementation of missing measured signal data. The results of the measured signal data recovery showed that the hybrid method (EEMD based) proposed in this paper had a higher complementary performance compared with the traditional deep learning methods, while the EEMD-LSTM exhibited the best missing data complementary accuracy among all models. In addition, the effect of the number of prediction steps on the prediction accuracy of the EEMD-LSTM model is also discussed. This study not only provides a method to fuse EEMD and deep learning models to predict measured signal’ missing data but also provides suggestions for the use of EEMD-LSTM models under different conditions.
32
Chen, Yuren, Yu Chen, and Bo Yu. "Speed Distribution Prediction of Freight Vehicles on Mountainous Freeway Using Deep Learning Methods." Journal of Advanced Transportation 2020 (January 10, 2020): 1–14. http://dx.doi.org/10.1155/2020/8953182.
Full textAbstract:
Driving speed is one of the most critical indicators in safety evaluation and network monitoring in freight transportation. Speed prediction model serves as the most efficient method to obtain the data of driving speed. Current speed prediction models mostly focus on operating speed, which is hard to reveal the overall condition of driving speed on the road section. Meanwhile, the models were mostly developed based on the regression method, which is inconsistent with natural driving process. Recurrent neural network (RNN) is a distinctive type of deep learning method to capture the temporary dependency in behavioral research. The aim of this paper is to apply the deep learning method to predict the general condition of driving speed in consideration of the road geometry and the temporal evolutions. 3D mobile mapping was applied to obtain road geometry information with high precision, and driving simulation experiment was then conducted with the help of the road geometry data. Driving speed was characterized by the bimodal Gauss mixture model. RNN and its variants including long short-term memory (LSTM) and RNN and gated recurrent units (GRUs) were utilized to predict speed distribution in a spatial-temporal dimension with KL divergence being the loss function. The result proved the applicability of the model in speed distribution prediction of freight vehicles, while LSTM holds the best performance with the length of input sequence being 400 m. The result can be related to the threshold of drivers’ information processing on mountainous freeway. Multiple linear regression models were constructed to be a contrast with the LSTM model, and the results showed that LSTM was superior to regression models in terms of the model accuracy and interpretability of the driving process and the formation of vehicle speed. This study may help to understand speed change behavior of freight vehicles on mountainous freeways, while providing the feasible method for safety evaluation or network efficiency analysis.
33
Ravanelli, Mirco, Philemon Brakel, Maurizio Omologo, and Yoshua Bengio. "Light Gated Recurrent Units for Speech Recognition." IEEE Transactions on Emerging Topics in Computational Intelligence 2, no. 2 (April 2018): 92–102. http://dx.doi.org/10.1109/tetci.2017.2762739.
Full text
34
Zhang, Yaquan, Qi Wu, Nanbo Peng, Min Dai, Jing Zhang, and Hu Wang. "Memory-Gated Recurrent Networks." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 12 (May 18, 2021): 10956–63. http://dx.doi.org/10.1609/aaai.v35i12.17308.
Full textAbstract:
The essence of multivariate sequential learning is all about how to extract dependencies in data. These data sets, such as hourly medical records in intensive care units and multi-frequency phonetic time series, often time exhibit not only strong serial dependencies in the individual components (the "marginal" memory) but also non-negligible memories in the cross-sectional dependencies (the "joint" memory). Because of the multivariate complexity in the evolution of the joint distribution that underlies the data generating process, we take a data-driven approach and construct a novel recurrent network architecture, termed Memory-Gated Recurrent Networks (mGRN), with gates explicitly regulating two distinct types of memories: the marginal memory and the joint memory. Through a combination of comprehensive simulation studies and empirical experiments on a range of public datasets, we show that our proposed mGRN architecture consistently outperforms state-of-the-art architectures targeting multivariate time series.
35
Mateus, Balduíno César, Mateus Mendes, José Torres Farinha, Rui Assis, and António Marques Cardoso. "Comparing LSTM and GRU Models to Predict the Condition of a Pulp Paper Press." Energies 14, no. 21 (October 22, 2021): 6958. http://dx.doi.org/10.3390/en14216958.
Full textAbstract:
The accuracy of a predictive system is critical for predictive maintenance and to support the right decisions at the right times. Statistical models, such as ARIMA and SARIMA, are unable to describe the stochastic nature of the data. Neural networks, such as long short-term memory (LSTM) and the gated recurrent unit (GRU), are good predictors for univariate and multivariate data. The present paper describes a case study where the performances of long short-term memory and gated recurrent units are compared, based on different hyperparameters. In general, gated recurrent units exhibit better performance, based on a case study on pulp paper presses. The final result demonstrates that, to maximize the equipment availability, gated recurrent units, as demonstrated in the paper, are the best options.
36
Li, Xuelong, Aihong Yuan, and Xiaoqiang Lu. "Multi-modal gated recurrent units for image description." Multimedia Tools and Applications 77, no. 22 (March 15, 2018): 29847–69. http://dx.doi.org/10.1007/s11042-018-5856-1.
Full text
37
Jing, Li, Caglar Gulcehre, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljacic, and Yoshua Bengio. "Gated Orthogonal Recurrent Units: On Learning to Forget." Neural Computation 31, no. 4 (April 2019): 765–83. http://dx.doi.org/10.1162/neco_a_01174.
Full textAbstract:
We present a novel recurrent neural network (RNN)–based model that combines the remembering ability of unitary evolution RNNs with the ability of gated RNNs to effectively forget redundant or irrelevant information in its memory. We achieve this by extending restricted orthogonal evolution RNNs with a gating mechanism similar to gated recurrent unit RNNs with a reset gate and an update gate. Our model is able to outperform long short-term memory, gated recurrent units, and vanilla unitary or orthogonal RNNs on several long-term-dependency benchmark tasks. We empirically show that both orthogonal and unitary RNNs lack the ability to forget. This ability plays an important role in RNNs. We provide competitive results along with an analysis of our model on many natural sequential tasks, including question answering, speech spectrum prediction, character-level language modeling, and synthetic tasks that involve long-term dependencies such as algorithmic, denoising, and copying tasks.
38
PARDEDE, JASMAN, and MUHAMMAD FAUZAN RASPATI. "Gated Recurrent Units dalam Mendeteksi Obstructive Sleep Apnea." MIND Journal 6, no. 2 (December 12, 2021): 221–35. http://dx.doi.org/10.26760/mindjournal.v6i2.221-235.
Full textAbstract:
AbstrakDalam melakukan penelitian obstructive sleep apnea (OSA), polysomnography (PSG) digunakan untuk diagnosis. Namun subjek diharuskan menginap dilaboratorium selama beberapa malam untuk melakukan tes dengan PSG dan karena banyaknya alat yang harus dikenakan pada tubuh dapat membuat tidak nyaman saat pengambilan data. Belakangan ini, beberapa peneliti mengunakan single-lead ECG untuk melakukan deteksi OSA. Untuk menghasilkan model terbaik, akan dilakukan eksperimen training, dengan batch normalization dan dropout yang berbeda. Pada penelitian ini apnea-ecg dataset digunakan, RR-Interval dan amplitudo QRS complex dari released set berjumlah 35 data akan disegmentasi permenit untuk digunakan sebagai input dari arsitektur yang diajukan adalah gated recurrent unit (GRU). Lalu withheld set berjumlah 35 data akan digunakan untuk pengujian per-segment dan per-recording. Kinerja sistem diukur berdasarkan accuracy, sensitifity, dan specificity dengan pengujian per-segment mendapat hasil accuracy 83.92%, sensitifity 81.28%, dan specificity 85.55%, dan pengujian per-recording mendapat hasil accuracy 97.14%, sensitifity 95.65% dan specificity 100%.Kata kunci: Obstructive sleep apnea, GRU, ECG, RR-Interval, QRS complex.AbstractIn conducting obstructive sleep apnea (OSA) studies, polysomnography (PSG) was used for the diagnosis. However, the subject was required to stay in the laboratory for several nights to carry out tests with the PSG and because of the many devices that had to be worn on the body, it could be uncomfortable to collect data. Recently, several researchers have used single-lead ECG to detect OSA. To produce the best model, training experiments will be conducted, with different batch normalization and dropout. In this study, the apnea-ecg dataset is used, the RR-Interval and the QRS complex amplitude from the released set totaling 35 data will be segmented per minute to be used as input for the proposed architecture is the gated recurrent unit (GRU). Then the withheld set of 35 data will be used for per-segment and per-recording testing. System performance was measured based on accuracy, sensitivity, and specificity with per-segment testing getting 83.92% accuracy, 81.28% sensitivity, and 85.55% specificity, and per-recording testing got 97.14% accuracy, 95.65% sensitivity and 100% specificity.Keywords: Obstructive sleep apnea, GRU, ECG, RR-Interval, QRS complex.
39
Tan, Yi-Fei, Xiaoning Guo, and Soon-Chang Poh. "Time series activity classification using gated recurrent units." International Journal of Electrical and Computer Engineering (IJECE) 11, no. 4 (August 1, 2021): 3551. http://dx.doi.org/10.11591/ijece.v11i4.pp3551-3558.
Full textAbstract:
<span>The population of elderly is growing and is projected to outnumber the youth in the future. Many researches on elderly assisted living technology were carried out. One of the focus areas is activity monitoring of the elderly. AReM dataset is a time series activity recognition dataset for seven different types of activities, which are bending 1, bending 2, cycling, lying, sitting, standing and walking. In the original paper, the author used a many-to-many Recurrent Neural Network for activity recognition. Here, we introduced a time series classification method where Gated Recurrent Units with many-to-one architecture were used for activity classification. The experimental results obtained showed an excellent accuracy of 97.14%.</span>
40
Onyekpe, Uche, Vasile Palade, Stratis Kanarachos, and Stavros-Richard G. Christopoulos. "A Quaternion Gated Recurrent Unit Neural Network for Sensor Fusion." Information 12, no. 3 (March 9, 2021): 117. http://dx.doi.org/10.3390/info12030117.
Full textAbstract:
Recurrent Neural Networks (RNNs) are known for their ability to learn relationships within temporal sequences. Gated Recurrent Unit (GRU) networks have found use in challenging time-dependent applications such as Natural Language Processing (NLP), financial analysis and sensor fusion due to their capability to cope with the vanishing gradient problem. GRUs are also known to be more computationally efficient than their variant, the Long Short-Term Memory neural network (LSTM), due to their less complex structure and as such, are more suitable for applications requiring more efficient management of computational resources. Many of such applications require a stronger mapping of their features to further enhance the prediction accuracy. A novel Quaternion Gated Recurrent Unit (QGRU) is proposed in this paper, which leverages the internal and external dependencies within the quaternion algebra to map correlations within and across multidimensional features. The QGRU can be used to efficiently capture the inter- and intra-dependencies within multidimensional features unlike the GRU, which only captures the dependencies within the sequence. Furthermore, the performance of the proposed method is evaluated on a sensor fusion problem involving navigation in Global Navigation Satellite System (GNSS) deprived environments as well as a human activity recognition problem. The results obtained show that the QGRU produces competitive results with almost 3.7 times fewer parameters compared to the GRU. The QGRU code is available at.
41
Hosseini, Majid, Satya Katragadda, Jessica Wojtkiewicz, Raju Gottumukkala, Anthony Maida, and Terrence Lynn Chambers. "Direct Normal Irradiance Forecasting Using Multivariate Gated Recurrent Units." Energies 13, no. 15 (July 31, 2020): 3914. http://dx.doi.org/10.3390/en13153914.
Full textAbstract:
Power grid operators rely on solar irradiance forecasts to manage uncertainty and variability associated with solar power. Meteorological factors such as cloud cover, wind direction, and wind speed affect irradiance and are associated with a high degree of variability and uncertainty. Statistical models fail to accurately capture the dependence between these factors and irradiance. In this paper, we introduce the idea of applying multivariate Gated Recurrent Units (GRU) to forecast Direct Normal Irradiance (DNI) hourly. The proposed GRU-based forecasting method is evaluated against traditional Long Short-Term Memory (LSTM) using historical irradiance data (i.e., weather variables that include cloud cover, wind direction, and wind speed) to forecast irradiance forecasting over intra-hour and inter-hour intervals. Our evaluation on one of the sites from Measurement and Instrumentation Data Center indicate that both GRU and LSTM improved DNI forecasting performance when evaluated under different conditions. Moreover, including wind direction and wind speed can have substantial improvement in the accuracy of DNI forecasts. Besides, the forecasting model can accurately forecast irradiance values over multiple forecasting horizons.
42
Zeeshan Ansari, Mohd, Tanvir Ahmad, Mirza Mohd Sufyan Beg, and Faiyaz Ahmad. "Hindi to English transliteration using multilayer gated recurrent units." Indonesian Journal of Electrical Engineering and Computer Science 27, no. 2 (August 1, 2022): 1083. http://dx.doi.org/10.11591/ijeecs.v27.i2.pp1083-1090.
Full textAbstract:
Transliteration is <span lang="EN-US">the task of translating text from source script to target script provided that the language of the text remains the same. In this work, we perform transliteration on less explored Devanagari to Roman Hindi transliteration and its back transliteration. The neural transliteration model in this work is based on a sequence-to-sequence neural network that is composed of two major components, an encoder that transforms source language words into a meaningful representation and the decoder that is responsible for decoding the target language words. We utilize gated recurrent units (GRU) to design the multilayer encoder and decoder network. Among the several models, the multilayer model shows the best performance in terms of coupon equivalent rate (CER) and word error rate (WER). The method generates quite satisfactory predictions in Hindi-English bilingual machine transliteration with WER of 64.8% and CER of 20.1% which is a significant improvement over existing methods.</span>
43
Wojtkiewicz, Jessica, Matin Hosseini, Raju Gottumukkala, and Terrence Lynn Chambers. "Hour-Ahead Solar Irradiance Forecasting Using Multivariate Gated Recurrent Units." Energies 12, no. 21 (October 24, 2019): 4055. http://dx.doi.org/10.3390/en12214055.
Full textAbstract:
Variation in solar irradiance causes power generation fluctuations in solar power plants. Power grid operators need accurate irradiance forecasts to manage this variability. Many factors affect irradiance, including the time of year, weather and time of day. Cloud cover is one of the most important variables that affects solar power generation, but is also characterized by a high degree of variability and uncertainty. Deep learning methods have the ability to learn long-term dependencies within sequential data. We investigate the application of Gated Recurrent Units (GRU) to forecast solar irradiance and present the results of applying multivariate GRU to forecast hourly solar irradiance in Phoenix, Arizona. We compare and evaluate the performance of GRU against Long Short-Term Memory (LSTM) using strictly historical solar irradiance data as well as the addition of exogenous weather variables and cloud cover data. Based on our results, we found that the addition of exogenous weather variables and cloud cover data in both GRU and LSTM significantly improved forecasting accuracy, performing better than univariate and statistical models.
44
Bonassi, Fabio, Marcello Farina, and Riccardo Scattolini. "On the stability properties of Gated Recurrent Units neural networks." Systems & Control Letters 157 (November 2021): 105049. http://dx.doi.org/10.1016/j.sysconle.2021.105049.
Full text
45
Lobacheva, Ekaterina, Nadezhda Chirkova, Alexander Markovich, and Dmitry Vetrov. "Structured Sparsification of Gated Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 4989–96. http://dx.doi.org/10.1609/aaai.v34i04.5938.
Full textAbstract:
One of the most popular approaches for neural network compression is sparsification — learning sparse weight matrices. In structured sparsification, weights are set to zero by groups corresponding to structure units, e. g. neurons. We further develop the structured sparsification approach for the gated recurrent neural networks, e. g. Long Short-Term Memory (LSTM). Specifically, in addition to the sparsification of individual weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies an LSTM structure. We test our approach on the text classification and language modeling tasks. Our method improves the neuron-wise compression of the model in most of the tasks. We also observe that the resulting structure of gate sparsity depends on the task and connect the learned structures to the specifics of the particular tasks.
46
Jangir, Mahendra Kumar, and Karan Singh. "HARGRURNN: Human activity recognition using inertial body sensor gated recurrent units recurrent neural network." Journal of Discrete Mathematical Sciences and Cryptography 22, no. 8 (November 17, 2019): 1577–87. http://dx.doi.org/10.1080/09720529.2019.1696552.
Full text
47
Liu, Juntao, Caihua Wu, and Junwei Wang. "Gated recurrent units based neural network for time heterogeneous feedback recommendation." Information Sciences 423 (January 2018): 50–65. http://dx.doi.org/10.1016/j.ins.2017.09.048.
Full text
48
do Carmo Nogueira, Tiago, Cássio Dener Noronha Vinhal, Gélson da Cruz Júnior, and Matheus Rudolfo Diedrich Ullmann. "Reference-based model using multimodal gated recurrent units for image captioning." Multimedia Tools and Applications 79, no. 41-42 (August 15, 2020): 30615–35. http://dx.doi.org/10.1007/s11042-020-09539-5.
Full text
49
Rehmer, Alexander, and Andreas Kroll. "On the vanishing and exploding gradient problem in Gated Recurrent Units." IFAC-PapersOnLine 53, no. 2 (2020): 1243–48. http://dx.doi.org/10.1016/j.ifacol.2020.12.1342.
Full text
50
Soliman, Hatem, Izhar Ahmed Khan, and Yasir Hussain. "Learning to transfer knowledge from RDF Graphs with gated recurrent units." Intelligent Data Analysis 26, no. 3 (April 18, 2022): 679–94. http://dx.doi.org/10.3233/ida-215919.
Full textAbstract:
The Internet is a vital part of today’s ecosystem. The speedy evolution of the Internet has brought up practical issues such as the problem of information retrieval. Several methods have been proposed to solve this issue. Such approaches retrieve the information by using SPARQL queries over the Resource Description Framework (RDF) content which requires a precise match concerning the query structure and the RDF content. In this work, we propose a transfer learning-based neural learning method that helps to search RDF graphs to provide probabilistic reasoning between the queries and their results. The problem is formulated as a classification task where RDF graphs are preprocessed to abstract the N-Triples, then encode the abstracted N-triples into a transitional state that is suitable for neural transfer learning. Next, we fine-tune the neural learner to learn the semantic relationships between the N-triples. To validate the proposed approach, we employ ten-fold cross-validation. The results have shown that the anticipated approach is accurate by acquiring the average accuracy, recall, precision, and f-measure. The achieved scores are 97.52%, 96.31%, 98.45%, and 97.37%, respectively, and outperforms the baseline approaches.