Thèses sur le sujet « Artificial Neural Networks and Recurrent Neutral Networks »
Créez une référence correcte selon les styles APA, MLA, Chicago, Harvard et plusieurs autres
Consultez les 50 meilleures thèses pour votre recherche sur le sujet « Artificial Neural Networks and Recurrent Neutral Networks ».
À côté de chaque source dans la liste de références il y a un bouton « Ajouter à la bibliographie ». Cliquez sur ce bouton, et nous générerons automatiquement la référence bibliographique pour la source choisie selon votre style de citation préféré : APA, MLA, Harvard, Vancouver, Chicago, etc.
Vous pouvez aussi télécharger le texte intégral de la publication scolaire au format pdf et consulter son résumé en ligne lorsque ces informations sont inclues dans les métadonnées.
Parcourez les thèses sur diverses disciplines et organisez correctement votre bibliographie.
Kolen, John F. « Exploring the computational capabilities of recurrent neural networks / ». The Ohio State University, 1994. http://rave.ohiolink.edu/etdc/view?acc_num=osu1487853913100192.
Texte intégralShao, Yuanlong. « Learning Sparse Recurrent Neural Networks in Language Modeling ». The Ohio State University, 2014. http://rave.ohiolink.edu/etdc/view?acc_num=osu1398942373.
Texte intégralGudjonsson, Ludvik. « Comparison of two methods for evolving recurrent artificial neural networks for ». Thesis, University of Skövde, University of Skövde, 1998. http://urn.kb.se/resolve?urn=urn:nbn:se:his:diva-155.
Texte intégraln this dissertation a comparison of two evolutionary methods for evolving ANNs for robot control is made. The methods compared are SANE with enforced sub-population and delta-coding, and marker-based encoding. In an attempt to speed up evolution, marker-based encoding is extended with delta-coding. The task selected for comparison is the hunter-prey task. This task requires the robot controller to posess some form of memory as the prey can move out of sensor range. Incremental evolution is used to evolve the complex behaviour that is required to successfully handle this task. The comparison is based on computational power needed for evolution, and complexity, robustness, and generalisation of the resulting ANNs. The results show that marker-based encoding is the most efficient method tested and does not need delta-coding to increase the speed of evolution process. Additionally the results indicate that delta-coding does not increase the speed of evolution with marker-based encoding.
Parfitt, Shan Helen. « Explorations in anaphora resolution in artificial neural networks : implications for nativism ». Thesis, Imperial College London, 1997. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.267247.
Texte intégralNAPOLI, CHRISTIAN. « A-I : Artificial intelligence ». Doctoral thesis, Università degli studi di Catania, 2016. http://hdl.handle.net/20.500.11769/490996.
Texte intégralKramer, Gregory Robert. « An analysis of neutral drift's effect on the evolution of a CTRNN locomotion controller with noisy fitness evaluation ». Wright State University / OhioLINK, 2007. http://rave.ohiolink.edu/etdc/view?acc_num=wright1182196651.
Texte intégralRallabandi, Pavan Kumar. « Processing hidden Markov models using recurrent neural networks for biological applications ». Thesis, University of the Western Cape, 2013. http://hdl.handle.net/11394/4525.
Texte intégralIn this thesis, we present a novel hybrid architecture by combining the most popular sequence recognition models such as Recurrent Neural Networks (RNNs) and Hidden Markov Models (HMMs). Though sequence recognition problems could be potentially modelled through well trained HMMs, they could not provide a reasonable solution to the complicated recognition problems. In contrast, the ability of RNNs to recognize the complex sequence recognition problems is known to be exceptionally good. It should be noted that in the past, methods for applying HMMs into RNNs have been developed by other researchers. However, to the best of our knowledge, no algorithm for processing HMMs through learning has been given. Taking advantage of the structural similarities of the architectural dynamics of the RNNs and HMMs, in this work we analyze the combination of these two systems into the hybrid architecture. To this end, the main objective of this study is to improve the sequence recognition/classi_cation performance by applying a hybrid neural/symbolic approach. In particular, trained HMMs are used as the initial symbolic domain theory and directly encoded into appropriate RNN architecture, meaning that the prior knowledge is processed through the training of RNNs. Proposed algorithm is then implemented on sample test beds and other real time biological applications.
Salihoglu, Utku. « Toward a brain-like memory with recurrent neural networks ». Doctoral thesis, Universite Libre de Bruxelles, 2009. http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/210221.
Texte intégral
Based on these assumptions, this thesis provides a computer model of neural network simulation of a brain-like memory. It first shows experimentally that the more information is to be stored in robust cyclic attractors, the more chaos appears as a regime in the background, erratically itinerating among brief appearances of these attractors. Chaos does not appear to be the cause, but the consequence of the learning. However, it appears as an helpful consequence that widens the network’s encoding capacity. To learn the information to be stored, two supervised iterative Hebbian learning algorithm are proposed. One leaves the semantics of the attractors to be associated with the feeding data unprescribed, while the other defines it a priori. Both algorithms show good results, even though the first one is more robust and has a greater storing capacity. Using these promising results, a biologically plausible alternative to these algorithms is proposed using cell assemblies as substrate for information. Even though this is not new, the mechanisms underlying their formation are poorly understood and, so far, there are no biologically plausible algorithms that can explain how external stimuli can be online stored in cell assemblies. This thesis provide such a solution combining a fast Hebbian/anti-Hebbian learning of the network's recurrent connections for the creation of new cell assemblies, and a slower feedback signal which stabilizes the cell assemblies by learning the feed forward input connections. This last mechanism is inspired by the retroaxonal hypothesis.
Doctorat en Sciences
info:eu-repo/semantics/nonPublished
Yang, Jidong. « Road crack condition performance modeling using recurrent Markov chains and artificial neural networks ». [Tampa, Fla.] : University of South Florida, 2004. http://purl.fcla.edu/fcla/etd/SFE0000567.
Texte intégralWillmott, Devin. « Recurrent Neural Networks and Their Applications to RNA Secondary Structure Inference ». UKnowledge, 2018. https://uknowledge.uky.edu/math_etds/58.
Texte intégralNapoli, Christian. « A-I : Artificial intelligence ». Doctoral thesis, Università di Catania, 2016. http://hdl.handle.net/10761/3974.
Texte intégralVikström, Filip. « A recurrent neural network approach to quantification of risks surrounding the Swedish property market ». Thesis, Umeå universitet, Institutionen för matematik och matematisk statistik, 2016. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-126192.
Texte intégralCondarcure, Thomas A. 1952. « A learning automaton approach to trajectory learning and control system design using dynamic recurrent neural networks ». Thesis, The University of Arizona, 1993. http://hdl.handle.net/10150/291987.
Texte intégralGattoni, Giacomo. « Improving the reliability of recurrent neural networks while dealing with bad data ». Master's thesis, Alma Mater Studiorum - Università di Bologna, 2021.
Trouver le texte intégralGrose, Mitchell. « Forecasting Atmospheric Turbulence Conditions From Prior Environmental Parameters Using Artificial Neural Networks : An Ensemble Study ». University of Dayton / OhioLINK, 2021. http://rave.ohiolink.edu/etdc/view?acc_num=dayton1619632748733788.
Texte intégralLindell, Adam. « Pulse Repetition Interval Time Series Modeling for Radar Waves using Long Short-Term Memory Artificial Recurrent Neural Networks ». Thesis, Uppsala universitet, Avdelningen för beräkningsvetenskap, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-377865.
Texte intégralSvebrant, Henrik. « Latent variable neural click models for web search ». Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-232311.
Texte intégralKlickmodellering av användare i söksystem görs vanligtvis med hjälp av probabilistiska modeller. På grund av maskininlärningens framgångar inom andra områden är det intressant att undersöka hur dessa tekniker kan appliceras för klickmodellering. Detta examensarbete undersöker klickmodellering med hjälp av recurrent neural networks tränade på en distribuerad representation av en populär och välpresterande klickmodell benämnd user browsing model (UBM). Det undersöks vidare hur utökandet av denna representation med statistiska variabler som enkelt kan utvinnas från klickloggar, kan påverka denna modells prestanda. Resultaten visar att grundrepresentationen inte presterar särskilt bra. Däremot har användningen av simpla variabler visats medföra drastiska prestandaökningar när det kommer till att förutspå en användares klick. I detta syfte lyckas modellerna prestera bättre än de två baselinemodeller som valts, vilka redan är välpresterande för syftet. De har även lyckats förbättra modellernas förmåga att förutspå relevans, fastän skillnaderna inte är lika drastiska. Relevans utgör inte en lika jämn jämförelse gentemot baselinemodellerna, då dessa kräver mycket större datamängder för att nå verklig prestanda. Det är däremot fördelaktigt att de neurala modellerna når relativt god prestanda för datamängden som använts. Det vore intressant att undersöka hur dessa modeller skulle prestera när de tränas på mycket större datamängder än vad som använts i detta projekt. Även att skräddarsy modellerna för LSTM, vilket borde kunna öka prestandan ytterligare. Att evaluera andra representationer än den som användes i detta projekt är också av intresse, då den använda representationen inte presterade märkvärdigt i sin grundform.
Chancan, Leon Marvin Aldo. « The role of motion-and-visual perception in robot place learning and navigation ». Thesis, Queensland University of Technology, 2022. https://eprints.qut.edu.au/229769/8/Marvin%20Aldo_Chancan%20Leon_Thesis.pdf.
Texte intégralCanaday, Daniel M. « Modeling and Control of Dynamical Systems with Reservoir Computing ». The Ohio State University, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=osu157469471458874.
Texte intégralMax, Lindblad. « The impact of parsing methods on recurrent neural networks applied to event-based vehicular signal data ». Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-223966.
Texte intégralDenna avhandling jämför två olika tillvägagångssätt vad gäller parsningen av händelsebaserad signaldata från fordon för att producera indata till en förutsägelsemodell i form av ett neuronnät, nämligen händelseparsning, där datan förblir ojämnt fördelad över tidsdomänen, och skivparsning, där datan är omgjord till att istället vara jämnt fördelad över tidsdomänen. Det dataset som används för dessa experiment är ett antal signalloggar från fordon som kommer från Scania. Jämförelser mellan parsningsmetoderna gjordes genom att först träna ett lång korttidsminne (LSTM) återkommande neuronnät (RNN) på vardera av de skapade dataseten för att sedan mäta utmatningsfelet och resurskostnader för varje modell efter att de validerats på en delad uppsättning av valideringsdata. Resultaten från dessa tester visar tydligt på att skivparsning står sig väl mot händelseparsning.
Mohammadisohrabi, Ali. « Design and implementation of a Recurrent Neural Network for Remaining Useful Life prediction ». Master's thesis, Alma Mater Studiorum - Università di Bologna, 2020.
Trouver le texte intégralBahceci, Oktay. « Deep Neural Networks for Context Aware Personalized Music Recommendation : A Vector of Curation ». Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210252.
Texte intégralInformationsfiltrering och rekommendationssystem har använts och implementeratspå flera olika sätt från olika enheter sedan gryningen avInternet, och moderna tillvägagångssätt beror påMaskininlärrning samtDjupinlärningför att kunna skapa precisa och personliga rekommendationerför användare i en given kontext. Dessa modeller kräver data i storamängder med en varians av kännetecken såsom tid, plats och användardataför att kunna hitta korrelationer samt mönster som klassiska modellersåsom matris faktorisering samt samverkande filtrering inte kan. Dettaexamensarbete forskar, implementerar och jämför en mängd av modellermed fokus påMaskininlärning samt Djupinlärning för musikrekommendationoch gör det med succé genom att representera rekommendationsproblemetsom ett extremt multi-klass klassifikationsproblem med 100000 unika klasser att välja utav. Genom att jämföra fjorton olika experiment,så lär alla modeller sig kännetäcken såsomtid, plats, användarkänneteckenoch lyssningshistorik för att kunna skapa kontextberoendepersonaliserade musikprediktioner, och löser kallstartsproblemet genomanvändning av användares demografiska kännetäcken, där den bästa modellenklarar av att fånga målklassen i sin rekommendationslista medlängd 100 för mer än 1/3 av det osedda datat under en offline evaluering,när slumpmässigt valda exempel från den osedda kommande veckanevalueras.
Forslund, John, et Jesper Fahlén. « Predicting customer purchase behavior within Telecom : How Artificial Intelligence can be collaborated into marketing efforts ». Thesis, KTH, Skolan för industriell teknik och management (ITM), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-279575.
Texte intégralDenna studie undersöker implementeringen av en AI-modell som förutspår kunders köp, inom telekombranschen. Studien syftar även till att påvisa hur en sådan AI-modell kan understödja beslutsfattande i marknadsföringsstrategier. Genom att designa AI-modellen med en Recurrent Neural Network (RNN) arkitektur med ett Long Short-Term Memory (LSTM) lager, drar studien slutsatsen att en sådan design möjliggör en framgångsrik implementering med tillfredsställande modellprestation. Instruktioner erhålls stegvis för att konstruera modellen i studiens metodikavsnitt. RNN-LSTM-modellen kan med fördel användas som ett hjälpande verktyg till marknadsförare för att bedöma hur en kunds beteendemönster på en hemsida påverkar deras köpbeteende över tiden, på ett kvantitativt sätt - genom att observera det ramverk som författarna kallar för Kundköpbenägenhetsresan, på engelska Customer Purchase Propensity Journey (CPPJ). Den empiriska grunden av CPPJ kan hjälpa organisationer att förbättra allokeringen av marknadsföringsresurser, samt gynna deras digitala närvaro genom att möjliggöra mer relevant personalisering i kundupplevelsen.
Howard, Shaun Michael. « Deep Learning for Sensor Fusion ». Case Western Reserve University School of Graduate Studies / OhioLINK, 2017. http://rave.ohiolink.edu/etdc/view?acc_num=case1495751146601099.
Texte intégralKišš, Martin. « Rozpoznávání historických textů pomocí hlubokých neuronových sítí ». Master's thesis, Vysoké učení technické v Brně. Fakulta informačních technologií, 2018. http://www.nusl.cz/ntk/nusl-385912.
Texte intégralKřepský, Jan. « Rekurentní neuronové sítě v počítačovém vidění ». Master's thesis, Vysoké učení technické v Brně. Fakulta informačních technologií, 2011. http://www.nusl.cz/ntk/nusl-237029.
Texte intégralEtienne, Caroline. « Apprentissage profond appliqué à la reconnaissance des émotions dans la voix ». Thesis, Université Paris-Saclay (ComUE), 2019. http://www.theses.fr/2019SACLS517.
Texte intégralThis thesis deals with the application of artificial intelligence to the automatic classification of audio sequences according to the emotional state of the customer during a commercial phone call. The goal is to improve on existing data preprocessing and machine learning models, and to suggest a model that is as efficient as possible on the reference IEMOCAP audio dataset. We draw from previous work on deep neural networks for automatic speech recognition, and extend it to the speech emotion recognition task. We are therefore interested in End-to-End neural architectures to perform the classification task including an autonomous extraction of acoustic features from the audio signal. Traditionally, the audio signal is preprocessed using paralinguistic features, as part of an expert approach. We choose a naive approach for data preprocessing that does not rely on specialized paralinguistic knowledge, and compare it with the expert approach. In this approach, the raw audio signal is transformed into a time-frequency spectrogram by using a short-term Fourier transform. In order to apply a neural network to a prediction task, a number of aspects need to be considered. On the one hand, the best possible hyperparameters must be identified. On the other hand, biases present in the database should be minimized (non-discrimination), for example by adding data and taking into account the characteristics of the chosen dataset. We study these aspects in order to develop an End-to-End neural architecture that combines convolutional layers specialized in the modeling of visual information with recurrent layers specialized in the modeling of temporal information. We propose a deep supervised learning model, competitive with the current state-of-the-art when trained on the IEMOCAP dataset, justifying its use for the rest of the experiments. This classification model consists of a four-layer convolutional neural networks and a bidirectional long short-term memory recurrent neural network (BLSTM). Our model is evaluated on two English audio databases proposed by the scientific community: IEMOCAP and MSP-IMPROV. A first contribution is to show that, with a deep neural network, we obtain high performances on IEMOCAP, and that the results are promising on MSP-IMPROV. Another contribution of this thesis is a comparative study of the output values of the layers of the convolutional module and the recurrent module according to the data preprocessing method used: spectrograms (naive approach) or paralinguistic indices (expert approach). We analyze the data according to their emotion class using the Euclidean distance, a deterministic proximity measure. We try to understand the characteristics of the emotional information extracted autonomously by the network. The idea is to contribute to research focused on the understanding of deep neural networks used in speech emotion recognition and to bring more transparency and explainability to these systems, whose decision-making mechanism is still largely misunderstood
Schäfer, Anton Maximilian. « Reinforcement Learning with Recurrent Neural Networks ». Doctoral thesis, 2008. https://repositorium.ub.uni-osnabrueck.de/handle/urn:nbn:de:gbv:700-2008112111.
Texte intégralRodríguez, Sotelo José Manuel. « Speech synthesis using recurrent neural networks ». Thèse, 2016. http://hdl.handle.net/1866/19111.
Texte intégralChung, Junyoung. « On Deep Multiscale Recurrent Neural Networks ». Thèse, 2018. http://hdl.handle.net/1866/21588.
Texte intégralRossi, Alberto. « Siamese and Recurrent neural networks for Medical Image Processing ». Doctoral thesis, 2021. http://hdl.handle.net/2158/1238384.
Texte intégralLin, Wen-chung, et 林文中. « Qualitative Modeling of Genetic Regulatory Networks via Recurrent Artificial Neural Network ». Thesis, 2002. http://ndltd.ncl.edu.tw/handle/76416405983136620066.
Texte intégral長庚大學
資訊管理研究所
90
According to the statistical abstract from Department of Health, Taiwan, R.O.C., 2001, cancer is still in the fist place in the cause of the death. Because of this reason, the therapy of cancer is widely emphasized on. Clinically, we can not tell the specific difference between the normal cell and cancer cell. This is one of the barriers to develop the therapy of cancer. Owing to these, a proposed qualitative model in order to help gene-related-disease workers to understand and reason the effect of toxic chemicals and medicines that are capable of activating or inactivating certain genes in the treatment of gene-related diseases. In this paper, we propose to model gene regulation networks qualitatively via recurrent artificial neural network. In this model, we assume the gene regulation network is definitive. Such a computational and representational model can reason about the interactions among related genes effectively and intuitively. It can help trace snapshots of gene regulatory dynamics at any two consecutive time steps concurrently along the discrete time line and it can help to produce what-if scenario when certain genes are activated or inactivated purposely as needed. Hence, it can serve as an auxiliary tool for gene-related-disease workers.
Krueger, David. « Designing Regularizers and Architectures for Recurrent Neural Networks ». Thèse, 2016. http://hdl.handle.net/1866/14019.
Texte intégralPeterson, Cole. « Generating rhyming poetry using LSTM recurrent neural networks ». Thesis, 2019. http://hdl.handle.net/1828/10801.
Texte intégralGraduate
Anbil, Parthipan Sarath Chandar. « On challenges in training recurrent neural networks ». Thèse, 2019. http://hdl.handle.net/1866/23435.
Texte intégralIn a multi-step prediction problem, the prediction at each time step can depend on the input at any of the previous time steps far in the past. Modelling such long-term dependencies is one of the fundamental problems in machine learning. In theory, Recurrent Neural Networks (RNNs) can model any long-term dependency. In practice, they can only model short-term dependencies due to the problem of vanishing and exploding gradients. This thesis explores the problem of vanishing gradient in recurrent neural networks and proposes novel solutions for the same. Chapter 3 explores the idea of using external memory to store the hidden states of a Long Short Term Memory (LSTM) network. By making the read and write operations of the external memory discrete, the proposed architecture reduces the rate of gradients vanishing in an LSTM. These discrete operations also enable the network to create dynamic skip connections across time. Chapter 4 attempts to characterize all the sources of vanishing gradients in a recurrent neural network and proposes a new recurrent architecture which has significantly better gradient flow than state-of-the-art recurrent architectures. The proposed Non-saturating Recurrent Units (NRUs) have no saturating activation functions and use additive cell updates instead of multiplicative cell updates. Chapter 5 discusses the challenges of using recurrent neural networks in the context of lifelong learning. In the lifelong learning setting, the network is expected to learn a series of tasks over its lifetime. The dependencies in lifelong learning are not just within a task, but also across the tasks. This chapter discusses the two fundamental problems in lifelong learning: (i) catastrophic forgetting of old tasks, and (ii) network capacity saturation. Further, it proposes a solution to solve both these problems while training a recurrent neural network.
Zhu, Yuqing. « Nonlinear system identification using a genetic algorithm and recurrent artificial neural networks ». Thesis, 2006. http://spectrum.library.concordia.ca/9060/1/MR20771.pdf.
Texte intégralGhazi-Zahedi, Keyan Mahmoud. « Self-Regulating Neurons. A model for synaptic plasticity in artificial recurrent neural networks ». Doctoral thesis, 2009. https://repositorium.ub.uni-osnabrueck.de/handle/urn:nbn:de:gbv:700-2009020616.
Texte intégralBoulanger-Lewandowski, Nicolas. « Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks ». Thèse, 2014. http://hdl.handle.net/1866/11181.
Texte intégralThis thesis studies models of high-dimensional sequences based on recurrent neural networks (RNNs) and their application to music and speech. While in principle RNNs can represent the long-term dependencies and complex temporal dynamics present in real-world sequences such as video, audio and natural language, they have not been used to their full potential since their introduction by Rumelhart et al. (1986a) due to the difficulty to train them efficiently by gradient-based optimization. In recent years, the successful application of Hessian-free optimization and other advanced training techniques motivated an increase of their use in many state-of-the-art systems. The work of this thesis is part of this development. The main idea is to exploit the power of RNNs to learn a probabilistic description of sequences of symbols, i.e. high-level information associated with observed signals, that in turn can be used as a prior to improve the accuracy of information retrieval. For example, by modeling the evolution of note patterns in polyphonic music, chords in a harmonic progression, phones in a spoken utterance, or individual sources in an audio mixture, we can improve significantly the accuracy of polyphonic transcription, chord recognition, speech recognition and audio source separation respectively. The practical application of our models to these tasks is detailed in the last four articles presented in this thesis. In the first article, we replace the output layer of an RNN with conditional restricted Boltzmann machines to describe much richer multimodal output distributions. In the second article, we review and develop advanced techniques to train RNNs. In the last four articles, we explore various ways to combine our symbolic models with deep networks and non-negative matrix factorization algorithms, namely using products of experts, input/output architectures, and generative frameworks that generalize hidden Markov models. We also propose and analyze efficient inference procedures for those models, such as greedy chronological search, high-dimensional beam search, dynamic programming-like pruned beam search and gradient descent. Finally, we explore issues such as label bias, teacher forcing, temporal smoothing, regularization and pre-training.
Kanuparthi, Bhargav. « Towards better understanding and improving optimization in recurrent neural networks ». Thesis, 2020. http://hdl.handle.net/1866/24319.
Texte intégralLes réseaux de neurones récurrents (RNN) sont connus pour leur problème de gradient d'explosion et de disparition notoire (EVGP). Ce problème devient plus évident dans les tâches où les informations nécessaires pour les résoudre correctement existent sur de longues échelles de temps, car il empêche les composants de gradient importants de se propager correctement sur un grand nombre d'étapes. Les articles écrits dans ce travail formalise la propagation du gradient dans les RNN paramétriques et semi-paramétriques pour mieux comprendre la source de ce problème. Le premier article présente un algorithme stochastique simple (h-detach) spécifique à l'optimisation LSTM et visant à résoudre le problème EVGP. En utilisant cela, nous montrons des améliorations significatives par rapport au LSTM vanille en termes de vitesse de convergence, de robustesse au taux d'amorçage et d'apprentissage, et de généralisation sur divers ensembles de données de référence. Le prochain article se concentre sur les RNN semi-paramétriques et les réseaux auto-attentifs. L'auto-attention fournit un moyen par lequel un système peut accéder dynamiquement aux états passés (stockés en mémoire), ce qui aide à atténuer la disparition des gradients. Bien qu'utile, il est difficile à mettre à l'échelle car la taille du graphe de calcul augmente de manière quadratique avec le nombre de pas de temps impliqués. Dans l'article, nous décrivons un mécanisme de criblage de pertinence, inspiré par le processus cognitif de consolidation de la mémoire, qui permet une utilisation évolutive de l'auto-attention clairsemée avec récurrence tout en assurant une bonne propagation du gradient.
Mehri, Soroush. « Sequential modeling, generative recurrent neural networks, and their applications to audio ». Thèse, 2016. http://hdl.handle.net/1866/18762.
Texte intégralAgrawal, Harish. « Novel Neural Architectures based on Recurrent Connections and Symmetric Filters for Visual Processing ». Thesis, 2022. https://etd.iisc.ac.in/handle/2005/6022.
Texte intégralGhazi-Zahedi, Keyan Mahmoud [Verfasser]. « Self-regulating neurons : a model for synaptic plasticity in artificial recurrent neural networks / Keyan Mahmoud Ghazi-Zahedi ». 2009. http://d-nb.info/992767202/34.
Texte intégralSainath, Pravish. « Modeling functional brain activity of human working memory using deep recurrent neural networks ». Thesis, 2020. http://hdl.handle.net/1866/25468.
Texte intégralIn cognitive systems, the role of working memory is crucial for visual reasoning and decision making. Tremendous progress has been made in understanding the mechanisms of the human/animal working memory, as well as in formulating different frameworks of memory augmented artificial neural networks. The overall objective of our project is to train artificial neural network models that are capable of consolidating memory over a short period of time to solve a memory task and relate them to the brain activity of humans who solved the same task. The project is of interdisciplinary nature in trying to bridge aspects of Artificial Intelligence (deep learning) and Neuroscience. The cognitive task used is the N-back task, a very popular one in Cognitive Neuroscience in which the subjects are presented with a sequence of images, each of which needs to be identified as to whether it was already seen or not. The functional imaging (fMRI) dataset used has been collected as a part of the Courtois Neurmod Project. We study multiple variants of recurrent neural network models that learn to remember input images across timesteps. These trained neural networks optimized for the memory task are ultimately used to generate feature representations for the stimuli images seen by the human subjects during their recordings while solving the task. The representations derived from these neural networks are then to create an encoding model to predict the fMRI BOLD activity of the subjects. We then understand the relationship between the neural network model and brain activity by analyzing this predictive ability of the model in different areas of the brain that are involved in working memory. This work presents a way of using artificial neural networks to model the behavior and information processing of the working memory of the brain and to use brain imaging data captured from human subjects during the N-back task to potentially understand some memory mechanisms of the brain in relation to these artificial neural network models.
Ting, Chien-Chung, et 丁建中. « Robust Stabilization Analysis and Estimator Design for Uncertain Neutral Recurrent Neural Networks with Interval Time-varying Discrete and Distributed Delays ». Thesis, 2010. http://ndltd.ncl.edu.tw/handle/58694883170618759753.
Texte intégral國立彰化師範大學
工業教育與技術學系
98
This thesis presents the complete study of stability analysis and state estimators design. The system is focused on neutral neural networks with both interval discrete and distributed time-varying delays, where the time-varying delays are in a given range. In a stability analysis problem, the purpose is to develop globally robust delay-dependent stability for neutral uncertain neural networks with both discrete and distributed delays. The activation functions are supposed to be bounded and globally Lipschitz continuous. By using a Lyapunov function approach and linear matrix inequality (LMI) techniques, the stability criteria for the neutral uncertain neural networks with both discrete and distributed delays are established in the form of LMIs, which can be readily verified by using standard numerical software. In an estimator design problem, the estimation for neutral neural network with both discrete and distributed interval time-varying delays is investigated. By using the Lyapunov-Krasovskii method, a linear matrix inequality (LMI) approach is developed to construct sufficient conditions for the existence of admissible state estimators such that the error-state system is globally asymptotically stable. Then, we show that both the existence conditions and the explicit expression of the desired estimator can be characterized in terms of the solution to an LMI. Finally, some illustrative examples have been presented to demonstrate the effectiveness of the proposed approach.
Laurent, César. « Advances in parameterisation, optimisation and pruning of neural networks ». Thesis, 2020. http://hdl.handle.net/1866/25592.
Texte intégralNeural networks are a family of Machine Learning models able to learn complex tasks directly from the data. Although already producing impressive results in many areas such as speech recognition, computer vision or machine translation, there are still a lot of challenges in both training and deployment of neural networks. In particular, training neural networks typically requires huge amounts of computational resources, and trained models are often too big or too computationally expensive to be deployed on resource-limited devices, such as smartphones or low-power chips. The articles presented in this thesis investigate solutions to these different issues. The first couple of articles focus on improving the training of Recurrent Neural Networks (RNNs), networks specially designed to process sequential data. RNNs are notoriously hard to train, so we propose to improve their parameterisation by upgrading them with Batch Normalisation (BN), a very effective parameterisation which was hitherto used only in feed-forward networks. In the first article, we apply BN to the input-to-hidden connections of the RNNs, thereby reducing internal covariate shift between layers. In the second article, we show how to apply it to both input-to-hidden and hidden-to-hidden connections of the Long Short-Term Memory (LSTM), a popular RNN architecture, thus also reducing internal covariate shift between time steps. Our experiments show that these proposed parameterisations allow for faster and better training of RNNs on several benchmarks. In the third article, we propose a new optimiser to accelerate the training of neural networks. Traditional diagonal optimisers, such as RMSProp, operate in parameters coordinates, which is not optimal when several parameters are updated at the same time. Instead, we propose to apply such optimisers in a basis in which the diagonal approximation is likely to be more effective. We leverage the same approximation used in Kronecker-factored Approximate Curvature (K-FAC) to efficiently build this Kronecker-factored Eigenbasis (KFE). Our experiments show improvements over K-FAC in training speed for several deep network architectures. The last article focuses on network pruning, the action of removing parameters from the network, in order to reduce its memory footprint and computational cost. Typical pruning methods rely on first or second order Taylor approximations of the loss landscape to identify which parameters can be discarded. We propose to study the impact of the assumptions behind such approximations. Moreover, we systematically compare methods based on first and second order approximations with Magnitude Pruning (MP), showing how they perform both before and after a fine-tuning phase. Our experiments show that better preserving the original network function does not necessarily transfer to better performing networks after fine-tuning, suggesting that only considering the impact of pruning on the loss might not be a sufficient objective to design good pruning criteria.
Leszko, Dominika. « Time series forecasting for a call center in a Warsaw holding company ». Master's thesis, 2020. http://hdl.handle.net/10362/102939.
Texte intégralIn recent years, artificial intelligence and cognitive technologies are actively being adopted in industries that use conversational marketing. Workforce managers face the constant challenge of balancing the priorities of service levels and related service costs. This problem is especially common when inaccurate forecasts lead to inefficient scheduling decisions and in turn result in dramatic impact on the customer engagement and experience and thus call center’s profitability. The main trigger of this project development was the Company X’s struggle to estimate the number of inbound phone calls expected in the upcoming 40 days. Accurate phone call volume forecast could significantly improve consultants’ time management, as well as, service quality. Keeping this goal in mind, the main focus of this internship is to conduct a set of experiments with various types of predictive models and identify the best performing for the analyzed use case. After a thorough review of literature covering work related to time series analysis, the empirical part of the internship follows which describes the process of developing both, univariate and multivariate statistical models. The methods used in the report also include two types of recurrent neural networks which are commonly used for time series prediction. The exogenous variables used in multivariate models are derived from the Media Planning department of the company which stores information about the ads being published in the newspapers. The outcome of the research shows that statistical models outperformed the neural networks in this specific application. This report covers the overview of statistical and neural network models used. After that, a comparative study of all tested models is conducted and one best performing model is selected. Evidently, the experiments showed that SARIMAX model yields best predictions for the analyzed use-case and thus it is recommended for the company to be used for a better staff management driving a more pleasant customer experience of the call center.
(11048391), Hao Sha. « SOLVING PREDICTION PROBLEMS FROM TEMPORAL EVENT DATA ON NETWORKS ». Thesis, 2021.
Trouver le texte intégralMany complex processes can be viewed as sequential events on a network. In this thesis, we study the interplay between a network and the event sequences on it. We first focus on predicting events on a known network. Examples of such include: modeling retweet cascades, forecasting earthquakes, and tracing the source of a pandemic. In specific, given the network structure, we solve two types of problems - (1) forecasting future events based on the historical events, and (2) identifying the initial event(s) based on some later observations of the dynamics. The inverse problem of inferring the unknown network topology or links, based on the events, is also of great important. Examples along this line include: constructing influence networks among Twitter users from their tweets, soliciting new members to join an event based on their participation history, and recommending positions for job seekers according to their work experience. Following this direction, we study two types of problems - (1) recovering influence networks, and (2) predicting links between a node and a group of nodes, from event sequences.
Savard, François. « Réseaux de neurones à relaxation entraînés par critère d'autoencodeur débruitant ». Thèse, 2011. http://hdl.handle.net/1866/6176.
Texte intégralMachine learning is a vast field where we seek to learn parameters for models from concrete data. The goal will be to execute various tasks requiring abilities normally associated more with human intelligence than with a computer program, such as the ability to process high dimensional data containing a lot of variations. Artificial neural networks are a large class of such models. In some neural networks said to be deep, we can observe that high level (or "abstract") concepts are automatically learned. The work we present here takes its inspiration from deep neural networks, from recurrent networks and also from neuroscience of the visual system. Our test tasks are classification and denoising for near binary images. We aim to take advantage of a feedback mechanism through which high-level representations, that is to say relatively abstract concepts, can influence lower-level representations. This influence will happen during what we call relaxation, which is iterations where the different levels (or layers) of the model can influence each other. We will present two families of architectures based on this mechanism. One, the fully connected architecture, can in principle accept generic data. The other, the convolutional one, is specifically made for images. Both were trained on images, though, and mostly images of written characters. In one type of experiment, we want to reconstruct data that has been corrupted. In these tasks, we have observed the feedback influence phenomenon previously described by comparing the results we obtained with and without relaxation. We also note some numerical and visual improvement in terms of reconstruction performance when we add upper layers’ influence. In another type of task, classification, little gain has been noted. Still, in one setting where we tried to classify noisy data with a representation trained without prior class information, relaxation did seem to improve results significantly. The convolutional architecture, a bit more risky at first, was shown to produce numerical and visual results in reconstruction that are near those obtained with the fully connected version, even though the connectivity is much more constrained.
Straková, Jana. « Rozpoznávání pojmenovaných entit pomocí neuronových sítí ». Doctoral thesis, 2017. http://www.nusl.cz/ntk/nusl-368176.
Texte intégralDutil, Francis. « Prédiction et génération de données structurées à l'aide de réseaux de neurones et de décisions discrètes ». Thèse, 2018. http://hdl.handle.net/1866/22124.
Texte intégral