To see the other types of publications on this topic, follow the link: Recurrent Neural Network architecture.
Journal articles on the topic 'Recurrent Neural Network architecture'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Recurrent Neural Network architecture.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Back, Andrew D., and Ah Chung Tsoi. "A Low-Sensitivity Recurrent Neural Network." Neural Computation 10, no. 1 (January 1, 1998): 165–88. http://dx.doi.org/10.1162/089976698300017935.
Full textAbstract:
The problem of high sensitivity in modeling is well known. Small perturbations in the model parameters may result in large, undesired changes in the model behavior. A number of authors have considered the issue of sensitivity in feedforward neural networks from a probabilistic perspective. Less attention has been given to such issues in recurrent neural networks. In this article, we present a new recurrent neural network architecture, that is capable of significantly improved parameter sensitivity properties compared to existing recurrent neural networks. The new recurrent neural network generalizes previous architectures by employing alternative discrete-time operators in place of the shift operator normally used. An analysis of the model demonstrates the existence of parameter sensitivity in recurrent neural networks and supports the proposed architecture. The new architecture performs significantly better than previous recurrent neural networks, as shown by a series of simple numerical experiments.
2
Паршин, А. И., М. Н. Аралов, В. Ф. Барабанов, and Н. И. Гребенникова. "RANDOM MULTI-MODAL DEEP LEARNING IN THE PROBLEM OF IMAGE RECOGNITION." ВЕСТНИК ВОРОНЕЖСКОГО ГОСУДАРСТВЕННОГО ТЕХНИЧЕСКОГО УНИВЕРСИТЕТА, no. 4 (October 20, 2021): 21–26. http://dx.doi.org/10.36622/vstu.2021.17.4.003.
Full textAbstract:
Задача распознавания изображений - одна из самых сложных в машинном обучении, требующая от исследователя как глубоких знаний, так и больших временных и вычислительных ресурсов. В случае использования нелинейных и сложных данных применяются различные архитектуры глубоких нейронных сетей, но при этом сложным вопросом остается проблема выбора нейронной сети. Основными архитектурами, используемыми повсеместно, являются свёрточные нейронные сети (CNN), рекуррентные нейронные сети (RNN), глубокие нейронные сети (DNN). На основе рекуррентных нейронных сетей (RNN) были разработаны сети с долгой краткосрочной памятью (LSTM) и сети с управляемыми реккурентными блоками (GRU). Каждая архитектура нейронной сети имеет свою структуру, свои настраиваемые и обучаемые параметры, обладает своими достоинствами и недостатками. Комбинируя различные виды нейронных сетей, можно существенно улучшить качество предсказания в различных задачах машинного обучения. Учитывая, что выбор оптимальной архитектуры сети и ее параметров является крайне трудной задачей, рассматривается один из методов построения архитектуры нейронных сетей на основе комбинации свёрточных, рекуррентных и глубоких нейронных сетей. Показано, что такие архитектуры превосходят классические алгоритмы машинного обучения The image recognition task is one of the most difficult in machine learning, requiring both deep knowledge and large time and computational resources from the researcher. In the case of using nonlinear and complex data, various architectures of deep neural networks are used but the problem of choosing a neural network remains a difficult issue. The main architectures used everywhere are convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN). Based on recurrent neural networks (RNNs), Long Short Term Memory Networks (LSTMs) and Controlled Recurrent Unit Networks (GRUs) were developed. Each neural network architecture has its own structure, customizable and trainable parameters, and advantages and disadvantages. By combining different types of neural networks, you can significantly improve the quality of prediction in various machine learning problems. Considering that the choice of the optimal network architecture and its parameters is an extremely difficult task, one of the methods for constructing the architecture of neural networks based on a combination of convolutional, recurrent and deep neural networks is considered. We showed that such architectures are superior to classical machine learning algorithms
3
Gallicchio, Claudio, and Alessio Micheli. "Fast and Deep Graph Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 3898–905. http://dx.doi.org/10.1609/aaai.v34i04.5803.
Full textAbstract:
We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification.
4
VASSILIADIS, STAMATIS, GERALD G. PECHANEK, and JOSÉ G. DELGADO-FRIAS. "SPIN: THE SEQUENTIAL PIPELINED NEUROEMULATOR." International Journal on Artificial Intelligence Tools 02, no. 01 (March 1993): 117–32. http://dx.doi.org/10.1142/s0218213093000084.
Full textAbstract:
This paper proposes a novel digital neural network architecture referred to as the Sequential PIpelined Neuroemulator or Neurocomputer (SPIN). The SPIN processor emulates neural networks producing high performance with minimum hardware by sequentially processing each neuron in the modeled completely connected network with a pipelined physical neuron structure. In addition to describing SPIN, performance equations are estimated for the ring systolic, the recurrent systolic array, and the neuromimetic neurocomputer architectures, three previously reported schemes for the emulation of neural networks, and a comparison with the SPIN architecture is reported.
5
Uzdiaev, M. Yu, R. N. Iakovlev, D. M. Dudarenko, and A. D. Zhebrun. "Identification of a Person by Gait in a Video Stream." Proceedings of the Southwest State University 24, no. 4 (February 4, 2021): 57–75. http://dx.doi.org/10.21869/2223-1560-2020-24-4-57-75.
Full textAbstract:
Purpose of research. The given paper considers the problem of identifying a person by gait through the use of neural network recognition models focused on working with RGB images. The main advantage of using neural network models over existing methods of motor activity analysis is obtaining images from the video stream without frames preprocessing, which increases the analysis time. Methods. The present paper presents an approach to identifying a person by gait. The approach is based upon the idea of multi-class classification on video sequences. The quality of the developed approach operation was evaluated on the basis of CASIA Gait Database data set, which includes more than 15,000 video sequences. As classifiers, 5 neural network architectures have been tested: the three-dimensional convolutional neural network I3D, as well as 4 architectures representing convolutional-recurrent networks, such as unidirectional and bidirectional LTSM, unidirectional and bidirectional GRU, combined with the convolutional neural network of ResNet architecture being used in these architectures as a visual feature extractor. Results. According to the results of the conducted testing, the developed approach makes it possible to identify a person in a video stream in real-time mode without the use of specialized equipment. According to the results of its testing and through the use of the neural network models under consideration, the accuracy of human identification was more than 80% for convolutional-recurrent models and 79% for the I3D model. Conclusion. The suggested models based on I3D architecture and convolutional-recurrent architectures have shown higher accuracy for solving the problem of identifying a person by gait than existing methods. Due to the possibility of frame-by-frame video processing, the most preferred classifier for the developed approach is the use of convolutional-recurrent architectures based on unidirectional LSTM or GRU models, respectively.
6
Kalinin, Maxim, Vasiliy Krundyshev, and Evgeny Zubkov. "Estimation of applicability of modern neural network methods for preventing cyberthreats to self-organizing network infrastructures of digital economy platforms,." SHS Web of Conferences 44 (2018): 00044. http://dx.doi.org/10.1051/shsconf/20184400044.
Full textAbstract:
The problems of applying neural network methods for solving problems of preventing cyberthreats to flexible self-organizing network infrastructures of digital economy platforms: vehicle adhoc networks, wireless sensor networks, industrial IoT, “smart buildings” and “smart cities” are considered. The applicability of the classic perceptron neural network, recurrent, deep, LSTM neural networks and neural networks ensembles in the restricting conditions of fast training and big data processing are estimated. The use of neural networks with a complex architecture– recurrent and LSTM neural networks – is experimentally justified for building a system of intrusion detection for self-organizing network infrastructures.
7
Caniago, Afif Ilham, Wilis Kaswidjanti, and Juwairiah Juwairiah. "Recurrent Neural Network With Gate Recurrent Unit For Stock Price Prediction." Telematika 18, no. 3 (October 31, 2021): 345. http://dx.doi.org/10.31315/telematika.v18i3.6650.
Full textAbstract:
Stock price prediction is a solution to reduce the risk of loss from investing in stocks go public. Although stock prices can be analyzed by stock experts, this analysis is analytical bias. Recurrent Neural Network (RNN) is a machine learning algorithm that can predict a time series data, non-linear data and non-stationary. However, RNNs have a vanishing gradient problem when dealing with long memory dependencies. The Gate Recurrent Unit (GRU) has the ability to handle long memory dependency data. In this study, researchers will evaluate the parameters of the RNN-GRU architecture that affect predictions with MAE, RMSE, DA, and MAPE as benchmarks. The architectural parameters tested are the number of units/neurons, hidden layers (Shallow and Stacked) and input data (Chartist and TA). The best number of units/neurons is not the same in all predicted cases. The best architecture of RNN-GRU is Stacked. The best input data is TA. Stock price predictions with RNN-GRU have different performance depending on how far the model predicts and the company's liquidity. The error value in this study (MAE, RMSE, MAPE) constantly increases as the label range increases. In this study, there are six data on stock prices with different companies. Liquid companies have a lower error value than non-liquid companies.
8
TELMOUDI, ACHRAF JABEUR, HATEM TLIJANI, LOTFI NABLI, MAARUF ALI, and RADHI M'HIRI. "A NEW RBF NEURAL NETWORK FOR PREDICTION IN INDUSTRIAL CONTROL." International Journal of Information Technology & Decision Making 11, no. 04 (July 2012): 749–75. http://dx.doi.org/10.1142/s0219622012500198.
Full textAbstract:
A novel neural architecture for prediction in industrial control: the 'Double Recurrent Radial Basis Function network' (R2RBF) is introduced for dynamic monitoring and prognosis of industrial processes. Three applications of the R2RBF network on the prediction values confirmed that the proposed architecture minimizes the prediction error. The proposed R2RBF is excited by the recurrence of the output looped neurons on the input layer which produces a dynamic memory on both the input and output layers. Given the learning complexity of neural networks with the use of the back-propagation training method, a simple architecture is proposed consisting of two simple Recurrent Radial Basis Function networks (RRBF). Each RRBF only has the input layer with looped neurons using the sigmoid activation function. The output of the first RRBF also presents an additional input for the second RRBF. An unsupervised learning algorithm is proposed to determine the parameters of the Radial Basis Function (RBF) nodes. The K-means unsupervised learning algorithm used for the hidden layer is enhanced by the initialization of these input parameters by the output parameters of the RCE algorithm.
9
Ziemke, Tom. "Radar Image Segmentation Using Self-Adapting Recurrent Networks." International Journal of Neural Systems 08, no. 01 (February 1997): 47–54. http://dx.doi.org/10.1142/s0129065797000070.
Full textAbstract:
This paper presents a novel approach to the segmentation and integration of (radar) images using a second-order recurrent artificial neural network architecture consisting of two sub-networks: a function network that classifies radar measurements into four different categories of objects in sea environments (water, oil spills, land and boats), and a context network that dynamically computes the function network's input weights. It is shown that in experiments (using simulated radar images) this mechanism outperforms conventional artificial neural networks since it allows the network to learn to solve the task through a dynamic adaptation of its classification function based on its internal state closely reflecting the current context.
10
K, Karthika, Tejashree K, Naveen Rajan M., and Namita R. "Towards Strong AI with Analog Neural Chips." International Journal of Innovative Research in Advanced Engineering 10, no. 06 (June 23, 2023): 394–99. http://dx.doi.org/10.26562/ijirae.2023.v1006.28.
Full textAbstract:
Applied AI chips with neural networks fail to capture and scale different forms of human intelligence. In this study, the definition of a strong AI system in hardware and architecture for building neuro memristive strong AI chips is proposed. The architecture unit consists of loop and hoop networks that are built on recurrent and feed forward information propagation concepts. Applying the principle that ’every brain is different’; we build a strong network that can take different structural and functional forms. The strong networks are building using hybrids of loop and hoop networks having generalization abilities, with higher levels of randomness incorporated to introduce greater flexibility in creating different neural architectures.
11
Vinayakumar, R., K. P. Soman, and Prabaharan Poornachandran. "Evaluation of Recurrent Neural Network and its Variants for Intrusion Detection System (IDS)." International Journal of Information System Modeling and Design 8, no. 3 (July 2017): 43–63. http://dx.doi.org/10.4018/ijismd.2017070103.
Full textAbstract:
This article describes how sequential data modeling is a relevant task in Cybersecurity. Sequences are attributed temporal characteristics either explicitly or implicitly. Recurrent neural networks (RNNs) are a subset of artificial neural networks (ANNs) which have appeared as a powerful, principle approach to learn dynamic temporal behaviors in an arbitrary length of large-scale sequence data. Furthermore, stacked recurrent neural networks (S-RNNs) have the potential to learn complex temporal behaviors quickly, including sparse representations. To leverage this, the authors model network traffic as a time series, particularly transmission control protocol / internet protocol (TCP/IP) packets in a predefined time range with a supervised learning method, using millions of known good and bad network connections. To find out the best architecture, the authors complete a comprehensive review of various RNN architectures with its network parameters and network structures. Ideally, as a test bed, they use the existing benchmark Defense Advanced Research Projects Agency / Knowledge Discovery and Data Mining (DARPA) / (KDD) Cup ‘99' intrusion detection (ID) contest data set to show the efficacy of these various RNN architectures. All the experiments of deep learning architectures are run up to 1000 epochs with a learning rate in the range [0.01-0.5] on a GPU-enabled TensorFlow and experiments of traditional machine learning algorithms are done using Scikit-learn. Experiments of families of RNN architecture achieved a low false positive rate in comparison to the traditional machine learning classifiers. The primary reason is that RNN architectures are able to store information for long-term dependencies over time-lags and to adjust with successive connection sequence information. In addition, the effectiveness of RNN architectures are shown for the UNSW-NB15 data set.
12
Jeon, DaeHyeon, and Min-Suk Kim. "Deep-Learning-Based Sequence Causal Long-Term Recurrent Convolutional Network for Data Fusion Using Video Data." Electronics 12, no. 5 (February 24, 2023): 1115. http://dx.doi.org/10.3390/electronics12051115.
Full textAbstract:
The purpose of AI-Based schemes in intelligent systems is to advance and optimize system performance. Most intelligent systems adopt sequential data types derived from such systems. Realtime video data, for example, are continuously updated as a sequence to make necessary predictions for efficient system performance. The majority of deep-learning-based network architectures such as long short-term memory (LSTM), data fusion, two streams, and temporal convolutional network (TCN) for sequence data fusion are generally used to enhance robust system efficiency. In this paper, we propose a deep-learning-based neural network architecture for non-fix data that uses both a causal convolutional neural network (CNN) and a long-term recurrent convolutional network (LRCN). Causal CNNs and LRCNs use incorporated convolutional layers for feature extraction, so both architectures are capable of processing sequential data such as time series or video data that can be used in a variety of applications. Both architectures also have extracted features from the input sequence data to reduce the dimensionality of the data and capture the important information, and learn hierarchical representations for effective sequence processing tasks. We have also adopted a concept of series compact convolutional recurrent neural network (SCCRNN), which is a type of neural network architecture designed for processing sequential data combined by both convolutional and recurrent layers compactly, reducing the number of parameters and memory usage to maintain high accuracy. The architecture is challenge-able and suitable for continuously incoming sequence video data, and doing so allowed us to bring advantages to both LSTM-based networks and CNNbased networks. To verify this method, we evaluated it through a sequence learning model with network parameters and memory that are required in real environments based on the UCF-101 dataset, which is an action recognition data set of realistic action videos, collected from YouTube with 101 action categories. The results show that the proposed model in a sequence causal long-term recurrent convolutional network (SCLRCN) provides a performance improvement of at least 12% approximately or more to be compared with the existing models (LRCN and TCN).
13
Wu, Yan, Aoming Liu, Zhiwu Huang, Siwei Zhang, and Luc Van Gool. "Neural Architecture Search as Sparse Supernet." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 12 (May 18, 2021): 10379–87. http://dx.doi.org/10.1609/aaai.v35i12.17243.
Full textAbstract:
This paper aims at enlarging the problem of Neural Architecture Search (NAS) from Single-Path and Multi-Path Search to automated Mixed-Path Search. In particular, we model the NAS problem as a sparse supernet using a new continuous architecture representation with a mixture of sparsity constraints. The sparse supernet enables us to automatically achieve sparsely-mixed paths upon a compact set of nodes. To optimize the proposed sparse supernet, we exploit a hierarchical accelerated proximal gradient algorithm within a bi-level optimization framework. Extensive experiments on Convolutional Neural Network and Recurrent Neural Network search demonstrate that the proposed method is capable of searching for compact, general and powerful neural architectures.
14
Mohammed, Ahmed Salahuddin, Amin Salih Mohammed, and Shahab Wahhab Kareem. "Deep Learning and Neural Network-Based Wind Speed Prediction Model." International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 30, no. 03 (June 2022): 403–25. http://dx.doi.org/10.1142/s021848852240013x.
Full textAbstract:
This paper aims to develop a wind speed prediction model by utilizing deep learning and neural networks. The analysis of weather data using a neural network architecture has been completed. The Long Short-Term Memory (LSTM) architecture is a type of artificial Recurrent Neural Network (RNN) used in deep learning is the first method plots the predicting Wind Speed based on the dataset and predicts the future spread. A dataset from a real-time weather station is used in the implementation model. The dataset consists of information from the weather station implements of the recurrent neural network model that plots the past spread and predicts the future stretch of the weather. The performance of the recurrent neural network model is presented and compared with Adaline neural network, Autoregressive Neural Network (NAR), and Group Method of Data Handling (GMDH). The NAR used three hidden layers. The performance of the model is analyzed by presenting the Wind Speeds of Erbil city. The dataset consists of the Wind Speed of (1992-2020) years, and each year consist of twelve months (from January to December).
15
Han, Bing, Cheng Wang, and Kaushik Roy. "Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 6 (June 28, 2022): 6838–46. http://dx.doi.org/10.1609/aaai.v36i6.20640.
Full textAbstract:
Tremendous progress has been made in sequential processing with the recent advances in recurrent neural networks. However, recurrent architectures face the challenge of exploding/vanishing gradients during training, and require significant computational resources to execute back-propagation through time. Moreover, large models are typically needed for executing complex sequential tasks. To address these challenges, we propose a novel neuron model that has cosine activation with a time varying component for sequential processing. The proposed neuron provides an efficient building block for projecting sequential inputs into spectral domain, which helps to retain long-term dependencies with minimal extra model parameters and computation. A new type of recurrent network architecture, named Oscillatory Fourier Neural Network, based on the proposed neuron is presented and applied to various types of sequential tasks. We demonstrate that recurrent neural network with the proposed neuron model is mathematically equivalent to a simplified form of discrete Fourier transform applied onto periodical activation. In particular, the computationally intensive back-propagation through time in training is eliminated, leading to faster training while achieving the state of the art inference accuracy in a diverse group of sequential tasks. For instance, applying the proposed model to sentiment analysis on IMDB review dataset reaches 89.4% test accuracy within 5 epochs, accompanied by over 35x reduction in the model size compared to LSTM. The proposed novel RNN architecture is well poised for intelligent sequential processing in resource constrained hardware.
16
Azpiazu, Ion Madrazo, and Maria Soledad Pera. "Multiattentive Recurrent Neural Network Architecture for Multilingual Readability Assessment." Transactions of the Association for Computational Linguistics 7 (November 2019): 421–36. http://dx.doi.org/10.1162/tacl_a_00278.
Full textAbstract:
We present a multiattentive recurrent neural network architecture for automatic multilingual readability assessment. This architecture considers raw words as its main input, but internally captures text structure and informs its word attention process using other syntax- and morphology-related datapoints, known to be of great importance to readability. This is achieved by a multiattentive strategy that allows the neural network to focus on specific parts of a text for predicting its reading level. We conducted an exhaustive evaluation using data sets targeting multiple languages and prediction task types, to compare the proposed model with traditional, state-of-the-art, and other neural network strategies.
17
Alashban, Adal A., Mustafa A. Qamhan, Ali H. Meftah, and Yousef A. Alotaibi. "Spoken Language Identification System Using Convolutional Recurrent Neural Network." Applied Sciences 12, no. 18 (September 13, 2022): 9181. http://dx.doi.org/10.3390/app12189181.
Full textAbstract:
Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.
18
Maduranga, Kehelwala D. G., Kyle E. Helfrich, and Qiang Ye. "Complex Unitary Recurrent Neural Networks Using Scaled Cayley Transform." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4528–35. http://dx.doi.org/10.1609/aaai.v33i01.33014528.
Full textAbstract:
Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN) which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the scaling matrix is fixed before training and a hyperparameter is introduced to tune the matrix for each particular task. In this paper, we develop a unitary RNN architecture based on a complex scaled Cayley transform. Unlike the real orthogonal case, the transformation uses a diagonal scaling matrix consisting of entries on the complex unit circle which can be optimized using gradient descent and no longer requires the tuning of a hyperparameter. We also provide an analysis of a potential issue of the modReLU activiation function which is used in our work and several other unitary RNNs. In the experiments conducted, the scaled Cayley unitary recurrent neural network (scuRNN) achieves comparable or better results than scoRNN and other unitary RNNs without fixing the scaling matrix.
19
Kabildjanov, A. S., Ch Z. Okhunboboeva, and S. Yo Ismailov. "Intelligent forecasting of growth and development of fruit trees by deep learning recurrent neural networks." IOP Conference Series: Earth and Environmental Science 1206, no. 1 (June 1, 2023): 012015. http://dx.doi.org/10.1088/1755-1315/1206/1/012015.
Full textAbstract:
Abstract The questions of intellectual forecasting of dynamic processes of growth and development of fruit trees are considered. The average growth rate of shoots of apple trees of the «Renet Simirenko» variety was predicted. Forecasting was carried out using a deep learning recurrent neural network LSTM in relation to a one-dimensional time series, with which the specified parameter was described. The implementation of the recurrent neural network LSTM was carried out in the MATLAB 2021 environment. When defining the architecture and training of the LSTM recurrent neural network, the Deep Network Designer application was used, which is included in the MATLAB 2021 extensions and allows you to create, visualize, edit and train deep learning networks. The recurrent neural network LSTM was trained using the Adam method. The results obtained in the course of predicting the average growth rate of apple shoots using a trained LSTM recurrent neural network were evaluated by the root-mean-square error RMSE and the loss function LOSS.
20
Pal, Subarno, Soumadip Ghosh, and Amitava Nag. "Sentiment Analysis in the Light of LSTM Recurrent Neural Networks." International Journal of Synthetic Emotions 9, no. 1 (January 2018): 33–39. http://dx.doi.org/10.4018/ijse.2018010103.
Full textAbstract:
Long short-term memory (LSTM) is a special type of recurrent neural network (RNN) architecture that was designed over simple RNNs for modeling temporal sequences and their long-range dependencies more accurately. In this article, the authors work with different types of LSTM architectures for sentiment analysis of movie reviews. It has been showed that LSTM RNNs are more effective than deep neural networks and conventional RNNs for sentiment analysis. Here, the authors explore different architectures associated with LSTM models to study their relative performance on sentiment analysis. A simple LSTM is first constructed and its performance is studied. On subsequent stages, the LSTM layer is stacked one upon another which shows an increase in accuracy. Later the LSTM layers were made bidirectional to convey data both forward and backward in the network. The authors hereby show that a layered deep LSTM with bidirectional connections has better performance in terms of accuracy compared to the simpler versions of LSTM used here.
21
Minin, Alexey, Alois Knoll, and Hans-Georg Zimmermann. "Complex Valued Recurrent Neural Network: From Architecture to Training." Journal of Signal and Information Processing 03, no. 02 (2012): 192–97. http://dx.doi.org/10.4236/jsip.2012.32026.
Full text
22
Camero, Andrés, Jamal Toutouh, and Enrique Alba. "Random error sampling-based recurrent neural network architecture optimization." Engineering Applications of Artificial Intelligence 96 (November 2020): 103946. http://dx.doi.org/10.1016/j.engappai.2020.103946.
Full text
23
Lyu, Shengfei, and Jiaqi Liu. "Convolutional Recurrent Neural Networks for Text Classification." Journal of Database Management 32, no. 4 (October 2021): 65–82. http://dx.doi.org/10.4018/jdm.2021100105.
Full textAbstract:
Recurrent neural network (RNN) and convolutional neural network (CNN) are two prevailing architectures used in text classification. Traditional approaches combine the strengths of these two networks by straightly streamlining them or linking features extracted from them. In this article, a novel approach is proposed to maintain the strengths of RNN and CNN to a great extent. In the proposed approach, a bi-directional RNN encodes each word into forward and backward hidden states. Then, a neural tensor layer is used to fuse bi-directional hidden states to get word representations. Meanwhile, a convolutional neural network is utilized to learn the importance of each word for text classification. Empirical experiments are conducted on several datasets for text classification. The superior performance of the proposed approach confirms its effectiveness.
24
Nguyen, Viet-Hung, Minh-Tuan Nguyen, Jeongsik Choi, and Yong-Hwa Kim. "NLOS Identification in WLANs Using Deep LSTM with CNN Features." Sensors 18, no. 11 (November 20, 2018): 4057. http://dx.doi.org/10.3390/s18114057.
Full textAbstract:
Identifying channel states as line-of-sight or non-line-of-sight helps to optimize location-based services in wireless communications. The received signal strength identification and channel state information are used to estimate channel conditions for orthogonal frequency division multiplexing systems in indoor wireless local area networks. This paper proposes a joint convolutional neural network and recurrent neural network architecture to classify channel conditions. Convolutional neural networks extract the feature from frequency-domain characteristics of channel state information data and recurrent neural networks extract the feature from time-varying characteristics of received signal strength identification and channel state information between packet transmissions. The performance of the proposed methods is verified under indoor propagation environments. Experimental results show that the proposed method has a 2% improvement in classification performance over the conventional recurrent neural network model.
25
FREAN, MARCUS, MATT LILLEY, and PHILLIP BOYLE. "IMPLEMENTING GAUSSIAN PROCESS INFERENCE WITH NEURAL NETWORKS." International Journal of Neural Systems 16, no. 05 (October 2006): 321–27. http://dx.doi.org/10.1142/s012906570600072x.
Full textAbstract:
Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.
26
Abudu, Prince M. "CommNets: Communicating Neural Network Architectures for Resource Constrained Systems." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 9909–10. http://dx.doi.org/10.1609/aaai.v33i01.33019909.
Full textAbstract:
Applications that require heterogeneous sensor deployments continue to face practical challenges owing to resource constraints within their operating environments (i.e. energy efficiency, computational power and reliability). This has motivated the need for effective ways of selecting a sensing strategy that maximizes detection accuracy for events of interest using available resources and data-driven approaches. Inspired by those limitations, we ask a fundamental question: whether state-of-the-art Recurrent Neural Networks can observe different series of data and communicate their hidden states to collectively solve an objective in a distributed fashion. We realize our answer by conducting a series of systematic analyses of a Communicating Recurrent Neural Network architecture on varying time-steps, objective functions and number of nodes. The experimental setup we employ models tasks synonymous with those in Wireless Sensor Networks. Our contributions show that Recurrent Neural Networks can communicate through their hidden states and we achieve promising results.
27
Wang, Xiaohui. "Design of English Translation Model Based on Recurrent Neural Network." Mathematical Problems in Engineering 2022 (August 25, 2022): 1–7. http://dx.doi.org/10.1155/2022/5177069.
Full textAbstract:
In order to improve the accuracy and stability of English translation, this paper proposes an English translation model based on recurrent neural network. Based on the end-to-end encoder-decoder architecture, a recursive neural network (RNN) English machine translation model is designed to promote machine autonomous learning features, transform the distributed corpus data into word vectors, and directly map the source language and target language through the recurrent neural network. Selecting semantic errors to construct the objective function during training can well balance the influence of each part of the semantics and fully consider the alignment information, providing a strong guidance for the training of deep recurrent neural networks. The experimental results show that the English translation model based on recurrent neural network has high effectiveness and stability. Compared with the baseline system, it has improved about 1.51–1.86 BLEU scores. Conclusion. The model improves the performance and quality of English machine translation model, and the translation effect is better.
28
FRASCONI, PAOLO, MARCO GORI, and GIOVANNI SODA. "DAPHNE: DATA PARALLELISM NEURAL NETWORK SIMULATOR." International Journal of Modern Physics C 04, no. 01 (February 1993): 17–28. http://dx.doi.org/10.1142/s0129183193000045.
Full textAbstract:
In this paper we describe the guideline of Daphne, a parallel simulator for supervised recurrent neural networks trained by Backpropagation through time. The simulator has a modular structure, based on a parallel training kernel running on the CM-2 Connection Machine. The training kernel is written in CM Fortran in order to exploit some advantages of the slicewise execution model. The other modules are written in serial C code. They are used for designing and testing the network, and for interfacing with the training data. A dedicated language is available for defining the network architecture, which allows the use of linked modules. The implementation of the learning procedures is based on training example parallelism. This dimension of parallelism has been found to be effective for learning static patterns using feedforward networks. We extend training example parallelism for learning sequences with full recurrent networks. Daphne is mainly conceived for applications in the field of Automatic Speech Recognition, though it can also serve for simulating feedforward networks.
29
Bhargava, Rupal, Shivangi Arora, and Yashvardhan Sharma. "Neural Network-Based Architecture for Sentiment Analysis in Indian Languages." Journal of Intelligent Systems 28, no. 3 (July 26, 2019): 361–75. http://dx.doi.org/10.1515/jisys-2017-0398.
Full textAbstract:
Abstract Sentiment analysis refers to determining the polarity of the opinions represented by text. The paper proposes an approach to determine the sentiments of tweets in one of the Indian languages (Hindi, Bengali, and Tamil). Thirty-nine sequential models have been created using three different neural network layers [recurrent neural networks (RNNs), long short-term memory (LSTM), convolutional neural network (CNN)] with optimum parameter settings (to avoid over-fitting and error accumulation). These sequential models have been investigated for each of the three languages. The proposed sequential models are experimented to identify how the hidden layers affect the overall performance of the approach. A comparison has also been performed with existing approaches to find out if neural networks have an added advantage over traditional machine learning techniques.
30
Sibruk, Leonid, and Ihor Zakutynskyi. "Recurrent Neural Networks for Time Series Forecasting. Choosing the best Architecture for Passenger Traffic Data." Electronics and Control Systems 2, no. 72 (September 23, 2022): 38–44. http://dx.doi.org/10.18372/1990-5548.72.16941.
Full textAbstract:
Accurately predicting the urban traffic passenger flow is of great importance for transportation resource scheduling, planning, public safety, and risk assessment. Traditional statistical approaches for forecasting time series are not effective in practice. They often require either strict or weak data stationarity, which is almost impossible to obtain with real data. An alternative method is time series forecasting using neural networks. By their nature, neural networks are non-linear and learn based on input and output data. With this approach, increasing the efficiency of the network is reduced to increasing the amount of data of the initial sample. Today, the class of recurrent neural networks is mainly used for forecasting time series. Another important stage is the choice of neural network architecture. In this article the use of long short term memory and gated recurrent units architecture is considered and also is compared their performance for passenger flow forecasting.
31
Wang, Jie, Jun Wang, Wen Fang, and Hongli Niu. "Financial Time Series Prediction Using Elman Recurrent Random Neural Networks." Computational Intelligence and Neuroscience 2016 (2016): 1–14. http://dx.doi.org/10.1155/2016/4742515.
Full textAbstract:
In recent years, financial market dynamics forecasting has been a focus of economic research. To predict the price indices of stock markets, we developed an architecture which combined Elman recurrent neural networks with stochastic time effective function. By analyzing the proposed model with the linear regression, complexity invariant distance (CID), and multiscale CID (MCID) analysis methods and taking the model compared with different models such as the backpropagation neural network (BPNN), the stochastic time effective neural network (STNN), and the Elman recurrent neural network (ERNN), the empirical results show that the proposed neural network displays the best performance among these neural networks in financial time series forecasting. Further, the empirical research is performed in testing the predictive effects of SSE, TWSE, KOSPI, and Nikkei225 with the established model, and the corresponding statistical comparisons of the above market indices are also exhibited. The experimental results show that this approach gives good performance in predicting the values from the stock market indices.
32
Back, A. D., and A. C. Tsoi. "FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling." Neural Computation 3, no. 3 (September 1991): 375–85. http://dx.doi.org/10.1162/neco.1991.3.3.375.
Full textAbstract:
A new neural network architecture involving either local feedforward global feedforward, and/or local recurrent global feedforward structure is proposed. A learning rule minimizing a mean square error criterion is derived. The performance of this algorithm (local recurrent global feedforward architecture) is compared with a local-feedforward global-feedforward architecture. It is shown that the local-recurrent global-feedforward model performs better than the local-feedforward global-feedforward model.
33
Kilic, Ergin, and Melik Dolen. "Prediction of slip in cable-drum systems using structured neural networks." Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 228, no. 3 (April 26, 2013): 441–56. http://dx.doi.org/10.1177/0954406213487471.
Full textAbstract:
This study focuses on the slip prediction in a cable-drum system using artificial neural networks for the prospect of developing linear motion sensing scheme for such mechanisms. Both feed-forward and recurrent-type artificial neural network architectures are considered to capture the slip dynamics of cable-drum mechanisms. In the article, the network development is presented in a progressive (step-by-step) fashion for the purpose of not only making the design process transparent to the readers but also highlighting the corresponding challenges associated with the design phase (i.e. selection of architecture, network size, training process parameters, etc.). Prediction performances of the devised networks are evaluated rigorously via an experimental study. Finally, a structured neural network, which embodies the network with the best prediction performance, is further developed to overcome the drift observed at low velocity. The study illustrates that the resulting structured neural network could predict the slip in the mechanism within an error band of 100 µm when an absolute reference is utilized.
34
Lalapura, Varsha S., J. Amudha, and Hariramn Selvamuruga Satheesh. "Recurrent Neural Networks for Edge Intelligence." ACM Computing Surveys 54, no. 4 (May 2021): 1–38. http://dx.doi.org/10.1145/3448974.
Full textAbstract:
Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence applications such as speech recognition, predictive healthcare, creative art, and so on. Although they provide accurate superior solutions, they pose a massive challenge “training havoc.” Current expansion of IoT demands intelligent models to be deployed at the edge. This is precisely to handle increasing model sizes and complex network architectures. Design efforts to meet these for greater performance have had inverse effects on portability on edge devices with real-time constraints of memory, latency, and energy. This article provides a detailed insight into various compression techniques widely disseminated in the deep learning regime. They have become key in mapping powerful RNNs onto resource-constrained devices. While compression of RNNs is the main focus of the survey, it also highlights challenges encountered while training. The training procedure directly influences model performance and compression alongside. Recent advancements to overcome the training challenges with their strengths and drawbacks are discussed. In short, the survey covers the three-step process, namely, architecture selection, efficient training process, and suitable compression technique applicable to a resource-constrained environment. It is thus one of the comprehensive survey guides a developer can adapt for a time-series problem context and an RNN solution for the edge.
35
Andreoli, Louis, Xavier Porte, Stéphane Chrétien, Maxime Jacquot, Laurent Larger, and Daniel Brunner. "Boolean learning under noise-perturbations in hardware neural networks." Nanophotonics 9, no. 13 (June 24, 2020): 4139–47. http://dx.doi.org/10.1515/nanoph-2020-0171.
Full textAbstract:
AbstractA high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we experimentally and theoretically investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system’s path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.
36
Obrubov, M., and S. Kirillova. "USING LSTM NETWORK FOR SOLVING THE MULTIDIMENTIONAL TIME SERIES FORECASTING PROBLEM." National Association of Scientists 2, no. 68 (July 1, 2021): 43–48. http://dx.doi.org/10.31618/nas.2413-5291.2021.2.68.450.
Full textAbstract:
The article discusses using of the recurrent neural networks technology to the multidimensional time series prediction problem. There is an experimental determination of the neural network architecture and its main hyperparameters carried out to achieve the minimum error. The revealed network structure going to be used further to detect anomalies in multidimensional time series.
37
Duarte Soares, Lucas, Altamira de Souza Queiroz, Gloria P. López, Edgar M. Carreño-Franco, Jesús M. López-Lezama, and Nicolás Muñoz-Galeano. "BiGRU-CNN Neural Network Applied to Electric Energy Theft Detection." Electronics 11, no. 5 (February 24, 2022): 693. http://dx.doi.org/10.3390/electronics11050693.
Full textAbstract:
This paper presents an assessment of the potential behind the BiGRU-CNN artificial neural network to be used as an electric power theft detection tool. The network is based on different architecture layers of the bidirectional gated recurrent unit and convolutional neural network. The use of such a tool with this classification model can help energy sector companies to make decisions regarding theft detection. The BiGRU-CNN artificial neural network singles out consumer units suspected of fraud for later manual inspections. The proposed artificial neural network was programmed in python, using the keras package. The best detection model was that of the BiGRU-CNN artificial neural network when compared to multilayer perceptron, recurrent neural network, gated recurrent unit, and long short-term memory networks. Several tests were carried out using data of an actual electricity supplier, showing the effectiveness of the proposed approach. The metric values assigned to their classifications were 0.929 for accuracy, 0.885 for precision, 0.801 for recall, 0.841 for F1-Score, and 0.966 for area under the receiver operating characteristic curve.
38
Ahmad, Zeeshan, Adnan Shahid Khan, Kashif Nisar, Iram Haider, Rosilah Hassan, Muhammad Reazul Haque, Seleviawati Tarmizi, and Joel J. P. C. Rodrigues. "Anomaly Detection Using Deep Neural Network for IoT Architecture." Applied Sciences 11, no. 15 (July 30, 2021): 7050. http://dx.doi.org/10.3390/app11157050.
Full textAbstract:
The revolutionary idea of the internet of things (IoT) architecture has gained enormous popularity over the last decade, resulting in an exponential growth in the IoT networks, connected devices, and the data processed therein. Since IoT devices generate and exchange sensitive data over the traditional internet, security has become a prime concern due to the generation of zero-day cyberattacks. A network-based intrusion detection system (NIDS) can provide the much-needed efficient security solution to the IoT network by protecting the network entry points through constant network traffic monitoring. Recent NIDS have a high false alarm rate (FAR) in detecting the anomalies, including the novel and zero-day anomalies. This paper proposes an efficient anomaly detection mechanism using mutual information (MI), considering a deep neural network (DNN) for an IoT network. A comparative analysis of different deep-learning models such as DNN, Convolutional Neural Network, Recurrent Neural Network, and its different variants, such as Gated Recurrent Unit and Long Short-term Memory is performed considering the IoT-Botnet 2020 dataset. Experimental results show the improvement of 0.57–2.6% in terms of the model’s accuracy, while at the same time reducing the FAR by 0.23–7.98% to show the effectiveness of the DNN-based NIDS model compared to the well-known deep learning models. It was also observed that using only the 16–35 best numerical features selected using MI instead of 80 features of the dataset result in almost negligible degradation in the model’s performance but helped in decreasing the overall model’s complexity. In addition, the overall accuracy of the DL-based models is further improved by almost 0.99–3.45% in terms of the detection accuracy considering only the top five categorical and numerical features.
39
Rafi, Quazi Ghulam, Mohammed Noman, Sadia Zahin Prodhan, Sabrina Alam, and Dip Nandi. "Comparative Analysis of Three Improved Deep Learning Architectures for Music Genre Classification." International Journal of Information Technology and Computer Science 13, no. 2 (April 8, 2021): 1–14. http://dx.doi.org/10.5815/ijitcs.2021.02.01.
Full textAbstract:
Among the many music information retrieval (MIR) tasks, music genre classification is noteworthy. The categorization of music into different groups that came to existence through a complex interplay of cultures, musicians, and various market forces to characterize similarities between compositions and organize collections is known as a music genre. The past researchers extracted various hand-crafted features and developed classifiers based on them. But the major drawback of this approach was the requirement of field expertise. However, in recent times researchers, because of the remarkable classification accuracy of deep learning models, have used similar models for MIR tasks. Convolutional Neural Net- work (CNN), Recurrent Neural Network (RNN), and the hybrid model, Convolutional - Recurrent Neural Network (CRNN), are such prominently used deep learning models for music genre classification along with other MIR tasks and various architectures of these models have achieved state-of-the-art results. In this study, we review and discuss three such architectures of deep learning models, already used for music genre classification of music tracks of length of 29-30 seconds. In particular, we analyze improved CNN, RNN, and CRNN architectures named Bottom-up Broadcast Neural Network (BBNN) [1], Independent Recurrent Neural Network (IndRNN) [2] and CRNN in Time and Frequency dimensions (CRNN- TF) [3] respectively, almost all of the architectures achieved the highest classification accuracy among the variants of their base deep learning model. Hence, this study holds a comparative analysis of the three most impressive architectural variants of the main deep learning models that are prominently used to classify music genre and presents the three architecture, hence the models (CNN, RNN, and CRNN) in one study. We also propose two ways that can improve the performances of the RNN (IndRNN) and CRNN (CRNN-TF) architectures.
40
Eigel, Martin, Marvin Haase, and Johannes Neumann. "Topology Optimisation under Uncertainties with Neural Networks." Algorithms 15, no. 7 (July 12, 2022): 241. http://dx.doi.org/10.3390/a15070241.
Full textAbstract:
Topology optimisation is a mathematical approach relevant to different engineering problems where the distribution of material in a defined domain is distributed in some optimal way, subject to a predefined cost function representing desired (e.g., mechanical) properties and constraints. The computation of such an optimal distribution depends on the numerical solution of some physical model (in our case linear elasticity) and robustness is achieved by introducing uncertainties into the model data, namely the forces acting on the structure and variations of the material stiffness, rendering the task high-dimensional and computationally expensive. To alleviate this computational burden, we develop two neural network architectures (NN) that are capable of predicting the gradient step of the optimisation procedure. Since state-of-the-art methods use adaptive mesh refinement, the neural networks are designed to use a sufficiently fine reference mesh such that only one training phase of the neural network suffices. As a first architecture, a convolutional neural network is adapted to the task. To include sequential information of the optimisation process, a recurrent neural network is constructed as a second architecture. A common 2D bridge benchmark is used to illustrate the performance of the proposed architectures. It is observed that the NN prediction of the gradient step clearly outperforms the classical optimisation method, in particular since larger iteration steps become viable.
41
Huang, Fangwan, Shijie Zhuang, Zhiyong Yu, Yuzhong Chen, and Kun Guo. "Adaptive Modularized Recurrent Neural Networks for Electric Load Forecasting." Journal of Database Management 34, no. 1 (May 18, 2023): 1–18. http://dx.doi.org/10.4018/jdm.323436.
Full textAbstract:
In order to provide more efficient and reliable power services than the traditional grid, it is necessary for the smart grid to accurately predict the electric load. Recently, recurrent neural networks (RNNs) have attracted increasing attention in this task because it can discover the temporal correlation between current load data and those long-ago through the self-connection of the hidden layer. Unfortunately, the traditional RNN is prone to the vanishing or exploding gradient problem with the increase of memory depth, which leads to the degradation of predictive accuracy. Many RNN architectures address this problem at the expense of complex internal structures and increased network parameters. Motivated by this, this article proposes two adaptive modularized RNNs to tackle the challenge, which can not only solve the gradient problem effectively with a simple architecture, but also achieve better performance with fewer parameters than other popular RNNs.
42
Seong-Whan Lee and Hee-Heon Song. "A new recurrent neural-network architecture for visual pattern recognition." IEEE Transactions on Neural Networks 8, no. 2 (March 1997): 331–40. http://dx.doi.org/10.1109/72.557671.
Full text
43
Mittal, Nikita, and Akash Saxena. "Layer Recurrent Neural Network based Power System Load Forecasting." TELKOMNIKA Indonesian Journal of Electrical Engineering 16, no. 3 (December 1, 2015): 423. http://dx.doi.org/10.11591/tijee.v16i3.1632.
Full textAbstract:
This paper presents a straight forward application of Layer Recurrent Neural Network (LRNN) to predict the load of a large distribution network. Short term load forecasting provides important information about the system’s load pattern, which is a premier requirement in planning periodical operations and facility expansion. Approximation of data patterns for forecasting is not an easy task to perform. In past, various approaches have been applied for forecasting. In this work application of LRNN is explored. The results of proposed architecture are compared with other conventional topologies of neural networks on the basis of Root Mean Square of Error (RMSE), Mean Absolute Percentage Error (MAPE) and Mean Absolute Error (MAE). It is observed that the results obtained from LRNN are comparatively more significant.
44
Voevoda, Aleksander, and Victor Shipagin. "Structural transformations of a neural network controller with a recurrent network type." Transaction of Scientific Papers of the Novosibirsk State Technical University, no. 3 (November 18, 2020): 7–16. http://dx.doi.org/10.17212/2307-6879-2020-3-7-16.
Full textAbstract:
The complexity of the objects of regulation, as well as the increase in the requirements for the productivity of the applied regulators, leads to the complexity of the applied neural network regulators. One of the complications is the appearance of feedback loops in the regulator. That is, the transition from direct distribution networks to re-current ones. One of the problems when using them is setting up weight coefficients using methods based on gradient calculation (for example, the error propagation method, the Levenberg-Marquardt method, etc.). It manifests itself in a suddenly "dis-appearing" or "exploding" gradient, which means that the learning process of the net-work stops. The purpose of this article is to develop proposals for solving some problems of con-figuring the weight coefficients of a recurrent neural network. As methods for achieving this goal, structural transformations of the architecture of a recurrent neural network are used to bring it to the form of a direct distribution net-work. At the same time, there is a slight increase in the complexity of its architecture. For networks of direct distribution methods based on the computation of the inverse gradient can be used without modification. In the future, it is planned to increase the performance of regulating the system with the help of a converted neuro-regulator, namely, to reduce the over-regulation of the system and, after some complications of the structure, use it to regulate a nonlinear object.
45
Paliwal, Shekhar, and Shivang Sharma. "Stock Prediction using Neural Networks and Evolution Algorithm." International Journal for Research in Applied Science and Engineering Technology 10, no. 4 (April 30, 2022): 661–71. http://dx.doi.org/10.22214/ijraset.2022.41331.
Full textAbstract:
Abstract: Various researches and studies have shown that machine learning techniques like neural network have the ability to learn and predict the general trend of stock market. Artificial intelligence and different machine learning techniques have been widely implemented in the field of forecasting stock prices for a long time. However, selecting the best model and best hyperparameters for these models is highly necessary for better accuracy in prediction. Given the huge number of architecture types and hyper-parameters for each model, it is not practical to find the best combination for best accuracy. Therefore, in this research we used evolution algorithm to optimize model architecture and hyper-parameters. Promising results are found in stock prediction. Keywords: Neural Network, Long Short-Term Memory, Recurrent Neural Network, Dense Neural Network, Gated Recurrent Unit, Stock Prediction, Evolution Algorithm.
46
Park, Dong-Chul. "Multiresolution-based bilinear recurrent neural network." Knowledge and Information Systems 19, no. 2 (September 4, 2008): 235–48. http://dx.doi.org/10.1007/s10115-008-0155-1.
Full text
47
Darmawahyuni, Annisa, Siti Nurmaini, Sukemi, Wahyu Caesarendra, Vicko Bhayyu, M. Naufal Rachmatullah, and Firdaus. "Deep Learning with a Recurrent Network Structure in the Sequence Modeling of Imbalanced Data for ECG-Rhythm Classifier." Algorithms 12, no. 6 (June 7, 2019): 118. http://dx.doi.org/10.3390/a12060118.
Full textAbstract:
The interpretation of Myocardial Infarction (MI) via electrocardiogram (ECG) signal is a challenging task. ECG signals’ morphological view show significant variation in different patients under different physical conditions. Several learning algorithms have been studied to interpret MI. However, the drawback of machine learning is the use of heuristic features with shallow feature learning architectures. To overcome this problem, a deep learning approach is used for learning features automatically, without conventional handcrafted features. This paper presents sequence modeling based on deep learning with recurrent network for ECG-rhythm signal classification. The recurrent network architecture such as a Recurrent Neural Network (RNN) is proposed to automatically interpret MI via ECG signal. The performance of the proposed method is compared to the other recurrent network classifiers such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The objective is to obtain the best sequence model for ECG signal processing. This paper also aims to study a proper data partitioning ratio for the training and testing sets of imbalanced data. The large imbalanced data are obtained from MI and healthy control of PhysioNet: The PTB Diagnostic ECG Database 15-lead ECG signals. According to the comparison result, the LSTM architecture shows better performance than standard RNN and GRU architecture with identical hyper-parameters. The LSTM architecture also shows better classification compared to standard recurrent networks and GRU with sensitivity, specificity, precision, F1-score, BACC, and MCC is 98.49%, 97.97%, 95.67%, 96.32%, 97.56%, and 95.32%, respectively. Apparently, deep learning with the LSTM technique is a potential method for classifying sequential data that implements time steps in the ECG signal.
48
Akdeniz, Esra, Erol Egrioglu, Eren Bas, and Ufuk Yolcu. "An ARMA Type Pi-Sigma Artificial Neural Network for Nonlinear Time Series Forecasting." Journal of Artificial Intelligence and Soft Computing Research 8, no. 2 (April 1, 2018): 121–32. http://dx.doi.org/10.1515/jaiscr-2018-0009.
Full textAbstract:
Abstract Real-life time series have complex and non-linear structures. Artificial Neural Networks have been frequently used in the literature to analyze non-linear time series. High order artificial neural networks, in view of other artificial neural network types, are more adaptable to the data because of their expandable model order. In this paper, a new recurrent architecture for Pi-Sigma artificial neural networks is proposed. A learning algorithm based on particle swarm optimization is also used as a tool for the training of the proposed neural network. The proposed new high order artificial neural network is applied to three real life time series data and also a simulation study is performed for Istanbul Stock Exchange data set.
49
Zhu, Zhenshu, Yuming Bo, and Changhui Jiang. "A MEMS Gyroscope Noise Suppressing Method Using Neural Architecture Search Neural Network." Mathematical Problems in Engineering 2019 (November 21, 2019): 1–9. http://dx.doi.org/10.1155/2019/5491243.
Full textAbstract:
Inertial measurement unit (IMU) (an IMU usually contains three gyroscopes and accelerometers) is the key sensor to construct a self-contained inertial navigation system (INS). IMU manufactured through the Micromechanics Electronics Manufacturing System (MEMS) technology becomes more popular, due to its smaller column, lower cost, and gradually improved accuracy. However, limited by the manufacturing technology, the MEMS IMU raw measurement signals experience complicated noises, which cause the INS navigation solution errors diverge dramatically over time. For addressing this problem, an advanced Neural Architecture Search Recurrent Neural Network (NAS-RNN) was employed in the MEMS gyroscope noise suppressing. NAS-RNN was the recently invented artificial intelligence method for time series problems in data science community. Different from conventional method, NAS-RNN was able to search a more feasible architecture for selected application. In this paper, a popular MEMS IMU STIM300 was employed in the testing experiment, and the sampling frequency was 125 Hz. The experiment results showed that the NAS-RNN was effective for MEMS gyroscope denoising; the standard deviation values of denoised three-axis gyroscope measurements decreased by 44.0%, 34.1%, and 39.3%, respectively. Compared with the Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), the NAS-RNN obtained further decreases by 28.6%, 3.7%, and 8.8% in standard deviation (STD) values of the signals. In addition, the attitude errors decreased by 26.5%, 20.8%, and 16.4% while substituting the LSTM-RNN with the NAS-RNN.
50
Graves, Daniel, and Witold Pedrycz. "Fuzzy prediction architecture using recurrent neural networks." Neurocomputing 72, no. 7-9 (March 2009): 1668–78. http://dx.doi.org/10.1016/j.neucom.2008.07.009.
Full text