Дисертації з теми "Recurrent Neural Network architecture"
Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями
Ознайомтеся з топ-50 дисертацій для дослідження на тему "Recurrent Neural Network architecture".
Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.
Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.
Переглядайте дисертації для різних дисциплін та оформлюйте правильно вашу бібліографію.
Bopaiah, Jeevith. "A recurrent neural network architecture for biomedical event trigger classification." UKnowledge, 2018. https://uknowledge.uky.edu/cs_etds/73.
Повний текст джерелаPan, YaDung. "Fuzzy adaptive recurrent counterpropagation neural networks: A neural network architecture for qualitative modeling and real-time simulation of dynamic processes." Diss., The University of Arizona, 1995. http://hdl.handle.net/10150/187101.
Повний текст джерелаHanson, Jack. "Protein Structure Prediction by Recurrent and Convolutional Deep Neural Network Architectures." Thesis, Griffith University, 2018. http://hdl.handle.net/10072/382722.
Повний текст джерелаThesis (PhD Doctorate)
Doctor of Philosophy (PhD)
School of Eng & Built Env
Science, Environment, Engineering and Technology
Full Text
Silfa, Franyell. "Energy-efficient architectures for recurrent neural networks." Doctoral thesis, Universitat Politècnica de Catalunya, 2021. http://hdl.handle.net/10803/671448.
Повний текст джерелаLos algoritmos de aprendizaje profundo han tenido un éxito notable en aplicaciones como el reconocimiento automático de voz y la traducción automática. Por ende, estas aplicaciones son omnipresentes en nuestras vidas y se encuentran en una gran cantidad de dispositivos. Estos algoritmos se componen de Redes Neuronales Profundas (DNN), tales como las Redes Neuronales Convolucionales y Redes Neuronales Recurrentes (RNN), las cuales tienen un gran número de parámetros y cálculos. Por esto implementar DNNs en dispositivos móviles y servidores es un reto debido a los requisitos de memoria y energía. Las RNN se usan para resolver problemas de secuencia a secuencia tales como traducción automática. Estas contienen dependencias de datos entre las ejecuciones de cada time-step, por ello la cantidad de paralelismo es limitado. Por eso la evaluación de RNNs de forma energéticamente eficiente es un reto. En esta tesis se estudian RNNs para mejorar su eficiencia energética en arquitecturas especializadas. Para esto, proponemos técnicas de ahorro energético y arquitecturas de alta eficiencia adaptadas a la evaluación de RNN. Primero, caracterizamos un conjunto de RNN ejecutándose en un SoC. Luego identificamos que acceder a la memoria para leer los pesos es la mayor fuente de consumo energético el cual llega hasta un 80%. Por ende, creamos E-PUR: una unidad de procesamiento para RNN. E-PUR logra una aceleración de 6.8x y mejora el consumo energético en 88x en comparación con el SoC. Esas mejoras se deben a la maximización de la ubicación temporal de los pesos. En E-PUR, la lectura de los pesos representa el mayor consumo energético. Por ende, nos enfocamos en reducir los accesos a la memoria y creamos un esquema que reutiliza resultados calculados previamente. La observación es que al evaluar las secuencias de entrada de un RNN, la salida de una neurona dada tiende a cambiar ligeramente entre evaluaciones consecutivas, por lo que ideamos un esquema que almacena en caché las salidas de las neuronas y las reutiliza cada vez que detecta un cambio pequeño entre el valor de salida actual y el valor previo, lo que evita leer los pesos. Para decidir cuándo usar un cálculo anterior utilizamos una Red Neuronal Binaria (BNN) como predictor de reutilización, dado que su salida está altamente correlacionada con la salida de la RNN. Esta propuesta evita más del 24.2% de los cálculos y reduce el consumo energético promedio en 18.5%. El tamaño de la memoria de los modelos RNN suele reducirse utilizando baja precisión para la evaluación y el almacenamiento de los pesos. En este caso, la precisión mínima utilizada se identifica de forma estática y se establece de manera que la RNN mantenga su exactitud. Normalmente, este método utiliza la misma precisión para todo los cálculos. Sin embargo, observamos que algunos cálculos se pueden evaluar con una precisión menor sin afectar la exactitud. Por eso, ideamos una técnica que selecciona dinámicamente la precisión utilizada para calcular cada time-step. Un reto de esta propuesta es como elegir una precisión menor. Abordamos este problema reconociendo que el resultado de una evaluación previa se puede emplear para determinar la precisión requerida en el time-step actual. Nuestro esquema evalúa el 57% de los cálculos con una precisión menor que la precisión fija empleada por los métodos estáticos. Por último, la evaluación en E-PUR muestra una aceleración de 1.46x con un ahorro de energía promedio de 19.2%
Мельникова, І. К. "Інтелектуальна технологія прогнозу курсу криптовалют методом рекурентних нейромереж". Master's thesis, Сумський державний університет, 2019. http://essuir.sumdu.edu.ua/handle/123456789/76753.
Повний текст джерелаTekin, Mim Kemal. "Vehicle Path Prediction Using Recurrent Neural Network." Thesis, Linköpings universitet, Statistik och maskininlärning, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-166134.
Повний текст джерелаSivakumar, Shyamala C. "Architectures and algorithms for stable and constructive learning in discrete time recurrent neural networks." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 1997. http://www.collectionscanada.ca/obj/s4/f2/dsk2/ftp03/NQ31533.pdf.
Повний текст джерелаWen, Tsung-Hsien. "Recurrent neural network language generation for dialogue systems." Thesis, University of Cambridge, 2018. https://www.repository.cam.ac.uk/handle/1810/275648.
Повний текст джерелаMelidis, Christos. "Adaptive neural architectures for intuitive robot control." Thesis, University of Plymouth, 2017. http://hdl.handle.net/10026.1/9998.
Повний текст джерелаHe, Jian. "Adaptive power system stabilizer based on recurrent neural network." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 1999. http://www.collectionscanada.ca/obj/s4/f2/dsk1/tape8/PQDD_0008/NQ38471.pdf.
Повний текст джерелаGangireddy, Siva Reddy. "Recurrent neural network language models for automatic speech recognition." Thesis, University of Edinburgh, 2017. http://hdl.handle.net/1842/28990.
Повний текст джерелаAmartur, Sundar C. "Competitive recurrent neural network model for clustering of multispectral data." Case Western Reserve University School of Graduate Studies / OhioLINK, 1995. http://rave.ohiolink.edu/etdc/view?acc_num=case1058445974.
Повний текст джерелаLjungehed, Jesper. "Predicting Customer Churn Using Recurrent Neural Networks." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210670.
Повний текст джерелаIllojalitet prediktering används för att identifiera kunder som är påväg att bli mindre lojala och är ett hjälpsamt verktyg för att ett företag ska kunna driva en konkurrenskraftig verksamhet. I detaljhandel behöves en dynamisk definition av illojalitet för att korrekt kunna identifera illojala kunder. Kundens livstidsvärde är ett mått på monetärt värde av en kundrelation. En avstannad förändring av detta värde indikerar en minskning av kundens lojalitet. Denna rapport föreslår en ny metod för att utföra illojalitet prediktering. Den föreslagna metoden består av ett återkommande neuralt nätverk som används för att identifiera illojalitet hos kunder genom att prediktera kunders livstidsvärde. Resultaten visar att den föreslagna modellen presterar bättre jämfört med slumpmässig metod. Rapporten undersöker också användningen av en k-medelvärdesalgoritm som ett substitut för en regelextraktionsalgoritm. K-medelsalgoritm bidrog till en mer omfattande analys av illojalitet predikteringen.
Zhao, Lichen. "Random pulse artificial neural network architecture." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 1998. http://www.collectionscanada.ca/obj/s4/f2/dsk2/tape17/PQDD_0006/MQ36758.pdf.
Повний текст джерелаDimopoulos, Konstantinos Panagiotis. "Non-linear control strategies using input-state network models." Thesis, University of Reading, 2000. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.340027.
Повний текст джерелаPoormehdi, Ghaemmaghami Masoumeh. "Tracking of Humans in Video Stream Using LSTM Recurrent Neural Network." Thesis, KTH, Teoretisk datalogi, TCS, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-217495.
Повний текст джерелаI detta examensarbete undersöks problemet att spåra människor i videoströmmar genom att använda deep learning. Spårningen utförs genom att använda ett recurrent convolutional neural network. Input till nätverket består av visuella features extraherade med hjälp av ett convolutional neural network, samt av detektionsresultat från tidigare frames. Vi väljer att använda oss av historiska detektioner för att skapa en metod som är robust mot olika utmanande situationer, som t.ex. snabba rörelser, rörelseoskärpa och ocklusion. Long Short- Term Memory (LSTM) är ett recurrent convolutional neural network som är användbart för detta ändamål. Istället för att använda binära klassificering, vilket är vanligt i många deep learning-baserade tracking-metoder, så använder vi oss av regression för att direkt förutse positionen av de spårade subjekten. Vårt syfte är att testa vår metod på videor som spelats in med hjälp av en huvudmonterad kamera. På grund av begränsningar i våra träningsdataset som är spatiellt oblanserade har vi problem att spåra människor som befinner sig i utkanten av bildområdet, men i andra utmanande fall lyckades spårningen bra.
Gonzalez, Juan. "Spacecraft Formation Control| Adaptive PID-Extended Memory Recurrent Neural Network Controller." Thesis, California State University, Long Beach, 2019. http://pqdtopen.proquest.com/#viewpdf?dispub=10978237.
Повний текст джерелаIn today’s space industry, satellite formation flying has become a cost-efficient alternative solution for science, on-orbit repair and military time-critical missions. While in orbit, the satellites are exposed to the space environment and unpredictable spacecraft on-board disturbances that negatively affect the attitude control system’s ability to reduce relative position and velocity error. Satellites utilizing a PID or adaptive controller are typically tune to reduce the error induced by space environment disturbances. However, in the case of an unforeseen spacecraft disturbance, such as a fault in an IMU, the PID based attitude control system effectiveness will deteriorate and will not be able to reduce the error to an acceptable magnitude.
In order to address the shortcomings a PID-Extended Memory RNN (EMRNN) adaptive controller is proposed. A PID-EMRNN with a short memory of multiple time steps is capable of producing a control input that improves the translational position and velocity error transient response compared to a PID. The results demonstrate the PID-EMRNN controller ability to generate a faster settling and rise time for control signal curves. The PID-EMRNN also produced similar results for an altitude range of 400 km to 1000 km and inclination range of 40 to 65 degrees angles of inclination. The proposed PID-EMRNN adaptive controller has demonstrated the capability of yielding a faster position error and control signal transient response in satellite formation flying scenario.
Moradi, Mahdi. "TIME SERIES FORECASTING USING DUAL-STAGE ATTENTION-BASED RECURRENT NEURAL NETWORK." OpenSIUC, 2020. https://opensiuc.lib.siu.edu/theses/2701.
Повний текст джерелаWang, Yuchen. "Detection of Opioid Addicts via Attention-based bidirectional Recurrent Neural Network." Case Western Reserve University School of Graduate Studies / OhioLINK, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=case1592255095863388.
Повний текст джерелаCorell, Simon. "A Recurrent Neural Network For Battery Capacity Estimations In Electrical Vehicles." Thesis, Linköpings universitet, Medie- och Informationsteknik, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-160536.
Повний текст джерелаCunanan, Kevin. "Developing a Recurrent Neural Network with High Accuracy for Binary Sentiment Analysis." Scholarship @ Claremont, 2018. http://scholarship.claremont.edu/cmc_theses/1835.
Повний текст джерелаBudik, Daniel Borisovich. "A resource-efficient localized recurrent neural network architecture and learning algorithm." 2006. http://etd.utk.edu/2006/BudikDaniel.pdf.
Повний текст джерелаOguntala, George A., Yim Fun Hu, Ali A. S. Alabdullah, Raed A. Abd-Alhameed, Muhammad Ali, and D. K. Luong. "Passive RFID Module with LSTM Recurrent Neural Network Activity Classification Algorithm for Ambient Assisted Living." 2021. http://hdl.handle.net/10454/18418.
Повний текст джерелаIEEE Human activity recognition from sensor data is a critical research topic to achieve remote health monitoring and ambient assisted living (AAL). In AAL, sensors are integrated into conventional objects aimed to support targets capabilities through digital environments that are sensitive, responsive and adaptive to human activities. Emerging technological paradigms to support AAL within the home or community setting offers people the prospect of a more individually focused care and improved quality of living. In the present work, an ambient human activity classification framework that augments information from the received signal strength indicator (RSSI) of passive RFID tags to obtain detailed activity profiling is proposed. Key indices of position, orientation, mobility, and degree of activities which are critical to guide reliable clinical management decisions using 4 volunteers are employed to simulate the research objective. A two-layer, fully connected sequence long short-term memory recurrent neural network model (LSTM RNN) is employed. The LSTM RNN model extracts the feature of RSS from the sensor data and classifies the sampled activities using SoftMax. The performance of the LSTM model is evaluated for different data size and the hyper-parameters of the RNN are adjusted to optimal states, which results in an accuracy of 98.18%. The proposed framework suits well for smart health and smart homes which offers pervasive sensing environment for the elderly, persons with disability and chronic illness.
Lopes, Ana Patrícia Ribeiro. "Study of Deep Neural Network architectures for medical image segmentation." Master's thesis, 2020. http://hdl.handle.net/1822/69850.
Повний текст джерелаMedical image segmentation plays a crucial role in the medical field, since it allows performing quantitative analyses used for screening, monitoring and planning the treatment of numerous pathologies. Manual segmentation is time-consuming and prone to inter-rater variability. Thus, several automatic approaches have been proposed for medical image segmentation and most are based on Deep Learning. These approaches became specially relevant after the development of the Fully Convolutional Network. In this method, the fully-connected layers were eliminated and upsampling layers were incorporated, allowing one image to be segmented at once. Nowadays, the developed architectures are based on the FCN, being U-Net one of the most popular. The aim of this dissertation is to study Deep Learning architectures for medical image segmentation. Two challenging and very distinct tasks were selected, namely, retinal vessel segmentation from retinal fundus images and brain tumor segmentation from MRI images. The architectures studied in this work are based on the U-Net, due to high performances obtained in multiple medical segmentation tasks. The models developed for retinal vessel and brain tumor segmentation were tested in publicly available databases, DRIVE and BRATS 2017, respectively. Several studies were performed for the first segmentation task, namely, comparison of downsampling operations, replacement of a downsampling step with dilated convolutions, incorporation of a RNN-based layer and application of test time data augmentation techniques. In the second segmentation task, three modifications were evaluated, specifically, the incorporation of long skip connections, the substitution of standard convolutions with dilated convolutions and the replacement of a downsampling step with dilated convolutions. Regarding retinal vessel segmentation, the final approach achieved accuracy, sensitivity and AUC of 0.9575, 0.7938 and 0.9804, respectively. This approach consists on a U-Net, containing one strided convolution as downsampling step and dilated convolutions with dilation rate of 3, followed by a test time data augmentation technique, performed by a ConvLSTM. Regarding brain tumor segmentation, the proposed approach achieved Dice of 0.8944, 0.8051 and 0.7353 and HD95 of 6.79, 8.34 and 4.76 for complete, core and enhanced regions, respectively. The final method consists on a DLA architecture with a long skip connection and dilated convolutions with dilation rate of 2. For both tasks, the proposed approach is competitive with state-of-the-art methods.
A segmentação de imagens médicas desempenha um papel fundamental na área médica, pois permite realizar análises quantitativas usadas no rastreio, monitorização e planeamento do tratamento de inúmeras patologias. A segmentação manual é demorada e varia consoante o técnico. Assim, diversas abordagens automáticas têm sido propostas para a segmentação de imagens médicas e a maioria é baseada em Deep Learning. Estas abordagens tornaram-se especialmente relevantes após o desenvolvimento da Fully Convolutional Network. Neste método, as camadas totalmente ligadas foram eliminadas e foram incorporadas camadas de upsampling, permitindo que uma imagem seja segmentada de uma só vez. Atualmente, as arquiteturas desenvolvidas baseiam-se na FCN, sendo a U-Net uma das mais populares. O objetivo desta dissertação é estudar arquiteturas de Deep Learning para a segmentação de imagens médicas. Foram selecionadas duas tarefas desafiantes e muito distintas, a segmentação de vasos retinianos a partir de imagens do fundo da retina e a segmentação de tumores cerebrais a partir de imagens de MRI. As arquiteturas estudadas neste trabalho são baseadas na U-Net, devido às elevadas performances que esta obteve em diversas tarefas de segmentação médica. Os modelos desenvolvidos para segmentação de vasos retinianos e de tumores cerebrais foram testados em bases de dados públicas, DRIVE and BRATS 2017, respetivamente. Vários estudos foram realizados para a primeira tarefa, nomeadamente, comparação de operações de downsampling, substituição de uma camada de downsampling por convoluções dilatadas, incorporação de uma camada composta por RNNs e aplicação de técnicas de aumento de dados na fase de teste. Na segunda tarefa, três modificações foram avaliadas, a incorporação de long skip connections, a substituição de convoluções standard por convoluções dilatadas e a substituição de uma camada de downsampling por convoluções dilatadas. Quanto à segmentação de vasos retinianos, a abordagem final obteve accuracy, sensibilidade e AUC de 0.9575, 0.7938 e 0.9804, respetivamente. Esta abordagem consiste numa U-Net, que contém uma convolução strided como operação de downsampling e convoluções dilatadas com dilation rate de 3, seguida de uma técnica de aumento de dados em fase de teste, executada por uma ConvLSTM. Em relação à segmentação de tumores cerebrais, a bordagem proposta obteve Dice de 0.8944, 0.8051 e 0.7353 e HD95 de 6.79, 8.34 e 4.76 para o tumor completo, região central e região contrastante, respetivamente. O método final consiste numa arquitetura DLA com uma long skip connection e convoluções dilatadas com dilation rate de 2. As duas abordagens são competitivas com os métodos do estado da arte.
(9178400), Sanchari Sen. "Efficient and Robust Deep Learning through Approximate Computing." Thesis, 2020.
Знайти повний текст джерелаDeep Neural Networks (DNNs) have greatly advanced the state-of-the-art in a wide range of machine learning tasks involving image, video, speech and text analytics, and are deployed in numerous widely-used products and services. Improvements in the capabilities of hardware platforms such as Graphics Processing Units (GPUs) and specialized accelerators have been instrumental in enabling these advances as they have allowed more complex and accurate networks to be trained and deployed. However, the enormous computational and memory demands of DNNs continue to increase with growing data size and network complexity, posing a continuing challenge to computing system designers. For instance, state-of-the-art image recognition DNNs require hundreds of millions of parameters and hundreds of billions of multiply-accumulate operations while state-of-the-art language models require hundreds of billions of parameters and several trillion operations to process a single input instance. Another major obstacle in the adoption of DNNs, despite their impressive accuracies on a range of datasets, has been their lack of robustness. Specifically, recent efforts have demonstrated that small, carefully-introduced input perturbations can force a DNN to behave in unexpected and erroneous ways, which can have to severe consequences in several safety-critical DNN applications like healthcare and autonomous vehicles. In this dissertation, we explore approximate computing as an avenue to improve the speed and energy efficiency of DNNs, as well as their robustness to input perturbations.
Approximate computing involves executing selected computations of an application in an approximate manner, while generating favorable trade-offs between computational efficiency and output quality. The intrinsic error resilience of machine learning applications makes them excellent candidates for approximate computing, allowing us to achieve execution time and energy reductions with minimal effect on the quality of outputs. This dissertation performs a comprehensive analysis of different approximate computing techniques for improving the execution efficiency of DNNs. Complementary to generic approximation techniques like quantization, it identifies approximation opportunities based on the specific characteristics of three popular classes of networks - Feed-forward Neural Networks (FFNNs), Recurrent Neural Networks (RNNs) and Spiking Neural Networks (SNNs), which vary considerably in their network structure and computational patterns.
First, in the context of feed-forward neural networks, we identify sparsity, or the presence of zero values in the data structures (activations, weights, gradients and errors), to be a major source of redundancy and therefore, an easy target for approximations. We develop lightweight micro-architectural and instruction set extensions to a general-purpose processor core that enable it to dynamically detect zero values when they are loaded and skip future instructions that are rendered redundant by them. Next, we explore LSTMs (the most widely used class of RNNs), which map sequences from an input space to an output space. We propose hardware-agnostic approximations that dynamically skip redundant symbols in the input sequence and discard redundant elements in the state vector to achieve execution time benefits. Following that, we consider SNNs, which are an emerging class of neural networks that represent and process information in the form of sequences of binary spikes. Observing that spike-triggered updates along synaptic connections are the dominant operation in SNNs, we propose hardware and software techniques to identify connections that can be minimally impact the output quality and deactivate them dynamically, skipping any associated updates.
The dissertation also delves into the efficacy of combining multiple approximate computing techniques to improve the execution efficiency of DNNs. In particular, we focus on the combination of quantization, which reduces the precision of DNN data-structures, and pruning, which introduces sparsity in them. We observe that the ability of pruning to reduce the memory demands of quantized DNNs decreases with precision as the overhead of storing non-zero locations alongside the values starts to dominate in different sparse encoding schemes. We analyze this overhead and the overall compression of three different sparse formats across a range of sparsity and precision values and propose a hybrid compression scheme that identifies that optimal sparse format for a pruned low-precision DNN.
Along with improved execution efficiency of DNNs, the dissertation explores an additional advantage of approximate computing in the form of improved robustness. We propose ensembles of quantized DNN models with different numerical precisions as a new approach to increase robustness against adversarial attacks. It is based on the observation that quantized neural networks often demonstrate much higher robustness to adversarial attacks than full precision networks, but at the cost of a substantial loss in accuracy on the original (unperturbed) inputs. We overcome this limitation to achieve the best of both worlds, i.e., the higher unperturbed accuracies of the full precision models combined with the higher robustness of the low precision models, by composing them in an ensemble.
In summary, this dissertation establishes approximate computing as a promising direction to improve the performance, energy efficiency and robustness of neural networks.
Sarvadevabhatla, Ravi Kiran. "Deep Learning for Hand-drawn Sketches: Analysis, Synthesis and Cognitive Process Models." Thesis, 2018. https://etd.iisc.ac.in/handle/2005/5351.
Повний текст джерелаKrueger, David. "Designing Regularizers and Architectures for Recurrent Neural Networks." Thèse, 2016. http://hdl.handle.net/1866/14019.
Повний текст джерелаLin, Ming Jang, and 林明璋. "Research on Dynamic Recurrent Neural Network." Thesis, 1994. http://ndltd.ncl.edu.tw/handle/70522525556782624102.
Повний текст джерела國立政治大學
應用數學研究所
82
Our task in this paper is to discuss the Recurrent Neural Network. We construct a singal layer neural network and apply three different learning rules to simulate circular trajectory and figure eight. Also, we present the proof of convergence.
Agrawal, Harish. "Novel Neural Architectures based on Recurrent Connections and Symmetric Filters for Visual Processing." Thesis, 2022. https://etd.iisc.ac.in/handle/2005/6022.
Повний текст джерелаCHEN, HUNG-PEI, and 陳虹霈. "Integrating Convolutional Neural Network and Recurrent Neural Network for Automatic Text Classification." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/4jqh8z.
Повний текст джерела東吳大學
數學系
108
With the rapid development of huge data research area, the demand for processing textual information is increasing. Text classification is still a hot research in the field of natural language processing. In the traditional text mining process, we often use the "Bag-of-Words" model, which discards the order of the words in the sentence, mainly concerned with the frequency of occurrence of the words. TF-IDF (term frequency–inverse document frequency) is one of the techniques for feature extraction commonly used in text exploration and classification. Therefore, we combine convolutional neural network and recurrent neural network to consider the semantics and order of the words in the sentence for text classification. We apply 20Newsgroups news group as our test dataset. The performance of the result achieves an accuracy of 86.3% on the test set and improves about 3% comparing with the traditional model.
Yang, Neng-Jie, and 楊能傑. "An Optimal Recurrent Fuzzy Neural Network Controller." Thesis, 2002. http://ndltd.ncl.edu.tw/handle/22893053061456487124.
Повний текст джерела中原大學
電機工程研究所
90
In this thesis, an optimal recurrent fuzzy neural network controller is by an adaptive genetic algorithm. The recurrent fuzzy neural network has recurrent connections representing memory elements and uses a generalized dynamic backpropagation algoruthm to adjust fuzzy parameters on-line. Usually, the learning rate and the initial parameter values are chosen randomly or by experience, therefore is human resources consuming and inefficient. An adaptive genetic algorithm is used instead to optimize them. The adaptive genetic algorithm adjust the probability of crossover and mutation adaptively according to fitness values, therefore can avoid falling into local optimum and speed up convergence. The optimal recurrent fuzzy neural network controller is applied to the simulation of a second-ordeer linear system, a nonlinear system, a highly nonlinear system with instantaneous loads. The simulation results show that the learning rate as well as other fuzzy parameters are important factor for the optimal design. Certainly, with the optimal design, every simulation achieve the lowest sum of squared error and the design process done automatically by computer programs.
LIN, CHENG-YANG, and 林政陽. "Recurrent Neural Network-based Microphone Howling Suppression." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/hd839v.
Повний текст джерела國立臺北科技大學
電子工程系
107
When using the karaoke system to sing, it is often too close the microphone and power of the amplified speaker is too large, causing a positive feedback and howling making the singer and the listener to be uncomfortable. Generally, to solve the microphone howling, often using a frequency shift to interrupt the resonance, or using a band-stop filter to remedy afterwards. But both may cause sound quality damage. Therefore, we want to use the adaptive feedback cancellation algorithm. Using the input source of the amplified speaker as the reference signal to automatically estimate the feedback signals that may record in different signal-to-noise. And eliminate the signal gain before howling occurs directly from the source. Based on the above ideas, in this paper, the howling elimination algorithm of normalized least mean square (NLMS) is realized, especially considering the nonlinear distortion of the sound amplification system, and the advanced algorithm based on recurrent neural network (RNN) is proposed. And in the experiment, test the time-domain or frequency-domain processing separately, and use NLMS or RNN, a total of four different combinations, the convergence speed and computational demand of different algorithms under different temperament and different environmental spatial response situations and howling suppression effect. The experimental results show that: (1) the convergence in the time domain is faster, (2) Stable effect in the frequency domain (3) Time domain RNN is best at eliminating effects, but there are too large calculations.
Abdolzadeh, Vida. "Efficient Implementation of Recurrent Neural Network Accelerators." Tesi di dottorato, 2020. http://www.fedoa.unina.it/13225/1/Abdolzadeh_Vida_32.pdf.
Повний текст джерелаKurach, Karol. "Deep Neural Architectures for Algorithms and Sequential Data." Doctoral thesis, 2016. https://depotuw.ceon.pl/handle/item/1860.
Повний текст джерелаPierwsza część pracy przedstawia dwie głębokie architektury neuronowe wykorzystujące pamięć zewnętrzną: Neural Random-Access Machine (NRAM) oraz Hierarchical Attentive Memory (HAM). Pomysł na architekturę NRAM jest inspirowany Neuronowymi Maszynami Turinga (NTM). NRAM, w przeciwieństwie do NTM, posiada mechanizmy umożliwiające wykorzystanie wskaźników do pamięci. To sprawia, że NRAM jest w stanie nauczyć się pojęć wymagających użycia wskaźników, takich jak „lista jednokierunkowa” albo „drzewo binarne”. Architektura HAM bazuje na pełnym drzewie binarnym, w którym liście odpowiadają elementom pamięci. Umożliwia to wykonywanie operacji na pamięci w czasie Θ(log n), co jest znaczącą poprawą względem dostępu w czasie Θ(n), standardowo używanym w implementacji mechanizmu „skupienia uwagi” (ang. attention) w sieciach rekurencyjnych. Pokazujemy, że sieć LSTM połączona z HAM jest w stanie rozwiązać wymagające zadania o charakterze algorytmicznym. W szczególności, jest to pierwsza architektura, która mając dane jedynie pary wejście/poprawne wyjście potrafi się nauczyć sortowania elementów działającego w złożoności Θ(n log n) i dobrze generalizującego się do dłuższych ciągów. Pokazujemy również, że HAM jest ogólną architekturą, która może zostać wytrenowana aby działała jak standardowe struktury danych, takie jak stos, kolejka lub kolejka priorytetowa. Druga część pracy przedstawia trzy nowatorskie systemy bazujące na głębokich sieciach neuronowych. Pierwszy z nich to system do znajdowania wydajnych obliczeniowo formuł matematycznych. Przy wykorzystaniu sieci rekursywnej system jest w stanie efektywnie przeszukiwać przestrzeń stanów i szybko znajdować tożsame formułyo istotnie lepszej złożoności asymptotycznej (przykładowo, Θ(n^2) zamiast złożoności wykładniczej). Następnie, prezentujemy oparty na rekurencyjnej sieci neuronowej system do przewidywania niebezpiecznych zdarzeń z wielowymiarowych, niestacjonarnych szeregów czasowych. Nasza metoda osiągnęła bardzo dobre wyniki w dwóch konkursach uczenia maszynowego. Jako ostatni opisany został Smart Reply – system do sugerowania automatycznych odpowiedzi na e-maile. Smart Reply został zaimplementowany w Google Inbox i codziennie przetwarza setki milionów wiadomości. Aktualnie, 10% wiadomości wysłanych z urządzeń mobilnych jest generowana przez ten system.
Hong, Frank Shihong. "Structural knowledge in simple recurrent network?" 1999. https://scholarworks.umass.edu/theses/2348.
Повний текст джерелаWang, Hui-Hua, and 王慧華. "Adaptive Learning Rates in Diagonal Recurrent Neural Network." Thesis, 1996. http://ndltd.ncl.edu.tw/handle/50105668211095009187.
Повний текст джерела大同工學院
機械工程學系
84
In this paper, the ideal best adaptive learning rates arederived out for diagonal recurrent neural network. The adaptivelearning rates are chosen for fitting error convergence requirements.And the convergence requirements are discussed then modified for a practical control system. Finally the simulation results are shownin diagonal recurrent neural network based control system with the modified adaptive learning rates.
Liao, Yuan-Fu, and 廖元甫. "Isolated Mandarin Speech Recognition Using Recurrent Neural Network." Thesis, 1993. http://ndltd.ncl.edu.tw/handle/68290588901248152864.
Повний текст джерелаThirion, Jan Willem Frederik. "Recurrent neural network-enhanced HMM speech recognition systems." Diss., 2002. http://hdl.handle.net/2263/29149.
Повний текст джерелаDissertation (MEng (Electronic Engineering))--University of Pretoria, 2006.
Electrical, Electronic and Computer Engineering
unrestricted
Tsai, Yao-Cheng, and 蔡曜丞. "Acoustic Echo Cancellation Based on Recurrent Neural Network." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/jgk3ea.
Повний текст джерела國立中央大學
通訊工程學系
107
Acoustic echo cancellation is a common problem in speech and signal processing until now. Application scenarios such as telephone conference, hands-free handsets and mobile communications. In the past we used adaptive filters to deal with acoustic echo cancellation, and today we can use deep learning to solve complex problems in acoustic echo cancellation. The method proposed in this work is to consider acoustic echo cancellation as a problem of speech separation, instead of the traditional adaptive filter to estimate acoustic echo. And use the recurrent neural network architecture in deep learning to train the model. Since the recurrent neural network has a good ability to simulate time-varying functions, it can play a role in solving the problem of acoustic echo cancellation. We train a bidirectional long short-term memory network and a bidirectional gated recurrent unit. Features are extracted from single-talk speech and double-talk speech. Adjust weights to control the ratio between double-talk speech and single-talk speech, and estimate the ideal ratio mask. This way to separate the signal, in order to achieve the purpose of removing the echo. The experimental results show that the method has good effect in echo cancellation.
Hu, Hsiao-Chun, and 胡筱君. "Recurrent Neural Network based Collaborative Filtering Recommender System." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/ytva33.
Повний текст джерела國立臺灣科技大學
資訊工程系
107
As the rapid development of e-commerce, Collaborative Filtering Recommender System has been widely applied to major network platforms. Predict customers’ preferences accurately through recommender system could solve the problem of information overload for users and reinforce their dependence on the network platform. Since the recommender system based on collaborative filtering has the ability to recommend products that are abstract or difficult to describe in words, research related to collaborative filtering has attracted more and more attention. In this paper, we propose a deep learning model framework for collaborative filtering recommender system. We use Recurrent Neural Network as the most important part of this framework which makes our model have the ability to consider the timestamp of implicit feedbacks from each user. This ability then significantly improve the performance of our models when making personalization item recommendations. In addition, we also propose a training data format for Recurrent Neural Network. This format makes our recommender system became the first Recurrent Neural Network model that can consider both positive and negative implicit feedback instance during the training process. Through conducted experiments on the two real-world datasets, MovieLens-1m and Pinterest, we verify that our model can finish the training process during a shorter time and have better recommendation performance than the current deep learning based Collaborative Filtering model.
Chiu, Yi-Feng, and 邱一峰. "STUDY ON SELF-CONSTRUCTING FUZZY NEURAL NETWORK CONTROLLER USING RECURRENT NEURAL NETWORK LEARNING STRATEGY." Thesis, 2013. http://ndltd.ncl.edu.tw/handle/38808034711756082416.
Повний текст джерела大同大學
電機工程學系(所)
101
In this thesis, the self-constructing fuzzy neural network controller (SCFNN) using recurrent neural network (RNN) learning strategy is proposed. For back-propagation (BP) algorithm of the SCFNN controller, the exact calculation of the Jacobian of the system cannot be determined. In this thesis, the RNN learning strategy is proposed to replace the error term of SCFNN controller. After the training of the RNN learning strategy, that will receive the relation between controlling signal and result of the nonlinear of the plant completely. Moreover, the structure and the parameter-learning phases are preformed concurrently and on-line in the SCRFNN. The SCFNN controller is designed to achieve the tracking control of an electronic throttle. The proposed controller, there are two processes that one is structure learning phase and another is parameter learning phase. The structure learning is based on the partition of input space, and the parameter learning is based on the supervised gradient-decent method using BP algorithm. Mahalanobis distance (M-distance) method in this thesis is employed as the criterion to identify the Gaussian function will be generated / eliminated or not. Finally, the simulation results of the electronic throttle valve are provided to demonstrate the performance and effectiveness of the proposed controller.
Wang, Chung-Hao, and 王仲豪. "STUDY ON SELF-CONSTRUCTING FUZZY NEURAL NETWORK CONTROLLER USING RECURRENT WAVELET NEURAL NETWORK LEARNING STRATEGY." Thesis, 2014. http://ndltd.ncl.edu.tw/handle/66373384738532600320.
Повний текст джерела大同大學
電機工程學系(所)
102
In this thesis, the self-constructing fuzzy neural network controller (SCFNN) using recurrent wavelet neural network (RWNN) learning strategy is proposed. SCFNN has been proven over the years to simulate the relationship between input and output of the nonlinear dynamic system. Nevertheless, there are still has the drawback of training retard in this control method. The RWNN approach with a widely similar range of nature since the formation of wavelet transform through the dilation and translation of mother wavelet, it has capability to resolve time domain and scaled and very suitable to describe the function of the nonlinear phenomenon. Importing the adaptable of RWNN learning strategy can improve the learning capability for SCFNN controller. The proposed controller has two learning phase, that is structure learning and parameter learning. In the former, Mahalanobis distance method is used as the basis for identify the function of Gaussian is generated or eliminated. The latter is based on the gradient-decent method to update parameters; the both learning phases are synchronized and real-time executed in parallel. In this study, the electronic throttle system as a control plant of nonlinear dynamic in order to achieve the throttle angle control, the simulation shows that the proposed control method has good capability of identification system and accuracy.
Chang, Chun-Hung, and 張俊弘. "Pricing Euro Currency Options—Comparison of Back-Propagation Neural Network Modeland Recurrent Neural Network Model." Thesis, 2004. http://ndltd.ncl.edu.tw/handle/08966045306928572228.
Повний текст джерела中原大學
企業管理研究所
92
During the past four decades, Options have become one of the most popular derivatives products in the financial market. The accuracy of pricing option has been an interesting topic since Black and Scholes’ model in 1973. The target of this investigation is Euro currency option. The study uses two artificial neural network models (i.e., back-propagation neural network and recurrent neural network ) and employs four volatility variables (i.e., historical volatility, implied volatility, GARCH volatility and non-volatility) in order to compare the pricing performance of all kinds of association, and to analyze the valuation abilities of these two artificial neural network models and the applicability of volatility variables. Furthermore, this work verifies that whether the volatility is the key input under the learning mechanism of the artificial neural network models. The empirical results show that there are some limitations to forecast the accurate valuation for the long-term period on both neural network models. After reducing the length of forecast periods, the implied volatility variable in both artificial neural network models produced the smallest error, while non-volatility variable resulted in the largest error of four volatility variables. Regarding the other two volatility variables, this study finds that, under the back-propagation neural network model, GARCH volatility is just inferior to implied volatility, but the performance of historical volatility is better than GARCH volatility under the recurrent neural network model. In summary, this work suggests that different volatilities chosen will cause various impacts. Therefore, appropriate volatility used seems to be more important than the adoption of which artificial neural network models.
Li, Jyun-Hong, and 李俊宏. "Object Mask and Boundary Guided Recurrent Convolution Neural Network." Thesis, 2016. http://ndltd.ncl.edu.tw/handle/cz2j2t.
Повний текст джерела國立中央大學
資訊工程學系
104
Convolution neural network (CNN) has outstanding performance on recognition, CNN not only enhance the effectiveness of the whole-image classification, but also makes the identification of local task upgrade. The Full convolution neural network (FCN) also makes the improvement on semantic image segmentation, compared to the traditional way using region proposal combined super vector machine, and significantly improved the accuracy of semantic segmentation. In our paper, we combined two network to improve accuracy. One produces mask, and the other one classifies label of pixel. One of our proposed is that, we change the joint images of domain transform in DT-EdgeNet [19]. Due to the joint images of DT-EdgeNet are edges. These edges include the edges of object, which do not belong to the training set. So we guess that result of [19] after domain transform mind be influence by these edges. Our mask net can produce score map of background, object and boundary. These results do not include object belong to the training set. Therefore, we can reduce the influence of non-class object. Our mask net can also produce mask to optimize spatial information. Our other proposal is that we concatenate different pixel stride of OBG-FCN [18]. By adding this concatenate layer to train net, we can enhance the accuracy of object of boundary. In the end, we tested our proposed architecture on Pascal VOC2012, and got 6.6% higher than baseline on mean IOU.
Huang, Bo-Yuan, and 黃柏元. "The Composite Design of Recurrent Neural Network H∞ - Compensator." Thesis, 2005. http://ndltd.ncl.edu.tw/handle/35654951335458184154.
Повний текст джерела國立成功大學
系統及船舶機電工程學系碩博士班
93
In this study, a composite design of Recurrent Neural Network (RNN) H∞-Compensator is proposed for tracking the desired input. The composite control system is composed of an H∞ compensator, which is proposed by Hwang 【3】 and Doyle 【6】, and a back-propagation RNN compensator. In order to make the controlled system robust, the H∞ control law is relatively conservative in the solution process. To speed up the convergence of tracking errors and match the prescribed performance, the recurrent neural network with self-learning algorithm is used to improve the performance of the H∞-compensator. The back-propagation algorithm in the proposed RNN-H∞ compensator is applied to minimize the calculating time of the predicting parameters. Computer simulation results show that the desired performance can easily be achieved by using the proposed RNN-H∞ compensator under the presence of disturbances.
Hau-Lung, Huang, and 黃浩倫. "Real Time Learning Recurrent Neural Network for Flow Estimation." Thesis, 1999. http://ndltd.ncl.edu.tw/handle/90765984108789147121.
Повний текст джерела國立臺灣大學
農業工程學研究所
87
This research presents an alternative approach of the Artificial Neural Network (ANN) model to estimate streamflow. The architecture of Recurrent Neural Network (RNN) that we used provides a representation of dynamic internal feedback loops in the system to store information for later use. The Real-Time Recurrent Learning (RTRL) algorithm is implanted to enhance the learning efficiency. The main feature of the RTRL is that it doesn''t need a lot of historical examples for training. Combining the RNN and RTRL to model watershed rainfall-runoff process will complement traditional techniques in the streamflow estimation.
Peng, Chung-chi, and 彭中麒. "Recurrent Neural Network Control for a Synchronous Reluctance Motor." Thesis, 2013. http://ndltd.ncl.edu.tw/handle/21986022062786916763.
Повний текст джерела國立雲林科技大學
電機工程系碩士班
101
This thesis develops a digital signal processor (dSPACE inc. DS1104) based synchronous reluctance motor (SynRM) drive system. Elman neural network and modified Elman neural network controller are proposed in the SynRM when the SynRM has parameters variations and external disturbances. Recurrent Neural Network (RNN) and Elman neural network (ENN) are compared which ENN has faster convergence for special recurrent structure. The on-line parameters learning of the neural network used the back-propagation (BP) algorithm. We use the discrete-type Lyapunov function to guarantee the output error convergence. Finally, the proposed controller algorithms are shown in experimental results effectively.
Lu, Tsai-Wei, and 盧采威. "Tikhonov regularization for deep recurrent neural network acoustic modeling." Thesis, 2014. http://ndltd.ncl.edu.tw/handle/70636533678066549649.
Повний текст джерела國立交通大學
電信工程研究所
102
Deep learning has been widely demonstrated to achieve high performance in many classification tasks. Deep neural network is now a new trend in the areas of automatic speech recognition. In this dissertation, we deal with the issue of model regularization in deep recurrent neural network and develop the deep acoustic models for speech recognition in noisy environments. Our idea is to compensate the variations of input speech data in the restricted Boltzmann machine (RBM) which is applied as a pre-training stage for feature learning and acoustic modeling. We implement the Tikhonov regularization in pre-training procedure and build the invariance properties in acoustic neural network model. The regularization based on weight decay is further combined with Tikhonov regularization to increase the mixing rate of the alternating Gibbs Markov chain so that the contrastive divergence training tends to approximate the maximum likelihood learning. In addition, the backpropagation through time (BPTT) algorithm is developed in modified truncated minibatch training for recurrent neural network. This algorithm is not implemented in the recurrent weights but also in the weights between previous layer and recurrent layer. In the experiments, we carry out the proposed methods using the open-source Kaldi toolkit. The experimental results using the speech corpora of Resource Management (RM) and Aurora4 show that the ideas of hybrid regularization and BPTT training do improve the performance of deep neural network acoustic model for robust speech recognition.
CHEN, JYUN-HE, and 陳均禾. "System Identification and Classification Using Elman Recurrent Neural Network." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/825p2n.
Повний текст джерела國立雲林科技大學
電機工程系
107
In recent years, the fast development of Artificial Intelligence has promoted the technological progress. That the three major technologies, Machine Learning, Deep Learning, and Natural Language Processing. Machine Learning is the largest part. The use of software programming through artificial neural networks allows computers to emulate learning abilities like the human brain. In this thesis, in order to understand the learning effect of artificial neural networks on classification problems and nonlinear system identification, an Elman neural network with self-feedback factor is used. In this thesis, in order to study the classification problem and system identification problem, six algorithms, i.e., RTRL, GA, PSO, BBO, IWO and Hybrid IWO/BBO methods, are utilized to learn the weight of Elman neural network. To explore the effectiveness of algorithms and neural network architectures, four classification problems are used, Breast Cancer Data Set, Parkinsons Data Set, SPECT Heart Data Set, and Lung Cancer Data Set. Three nonlinear system identification problems are used, Nonlinear plant, Henon system and Mackey-Glass time series. Finally, the MSE, STD and the Classification rate, are used in the experimental classification problem. The MSE, STD and NDEI, are used to compare and analyze the system identification problem.
CHEN, SHEN-CHI, and 陳順麒. "On the Recurrent Neural Network Based Intrusion Detection System." Thesis, 2019. http://ndltd.ncl.edu.tw/handle/75tb39.
Повний текст джерела逢甲大學
資訊工程學系
107
With the advancement of modern science and technology, numerous applications of the Internet of Things are developing faster and faster. Smart grid is one of the examples which provides full communication, monitor, and control abilities to the components in the power systems in order to meet the increasing demands of reliable energy. In such systems, many components can be monitored and controlled remotely. As a result, they could be vulnerable to malicious cyber-attacks if there exist exploitable loopholes. In the power system, the disturbances caused by cyber-attacks are mixed with those caused by natural events. It is crucial for the intrusion detection systems in the smart grid to classify the types of disturbances and pinpoint the attacks with high accuracy. The amount of information in a smart grid system is much larger than before, and the amount of computation of the big data increases accordingly. Many analyzing techniques have been proposed to extract useful information in these data and deep learning is one of them. It can be applied to “learn” a model from a large set of training data and classify unknown events from subsequent data. In this paper, we apply the methods of recurrent neural network (RNN) algorithm as well as two other variants to train models for intrusion detection in smart grid. Our experiment results showed that RNN can achieves high accuracy and precision on a set of real data collected from an experimental power system network.