Dissertations / Theses: 'Supervised neural networks'

1

Sporea, Ioana. "Supervised learning in multilayer spiking neural networks." Thesis, University of Surrey, 2012. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.576119.

Full text

Abstract:

In this thesis, a new supervised learning algorithm for multilayer spik- ing neural networks is presented. Gradient descent learning algo- rithms have led traditional neural networks with multiple layers to be one of the most powerful and flexible computational models derived from artificial neural networks. However, more recent experimental evidence suggests that biological neural systems use the exact time of single action potentials to encode information. These findings have led to a new way of simulating neural networks based on temporal en- coding with single spikes. Analytical demonstrations show that these types of neural networks are computationally more powerful than net- works of rate neurons. Conversely, the existing learning algorithms no longer apply to spik- ing neural networks. Supervised learning algorithms based on gradient descent, such as SpikeProp and its extensions, have been developed for spiking neural networks with multiple layers, but these ate limited to a specific model of neurons, with only the first spike being consid- ered. Another learning algorithm, ReSuMe, for single layer networks is based on spike-timing dependent plasticity ~STDP) and uses the computational power of multiple spikes; moreover, this algorithm is not limited to a specific neuron model. The algorithm presented here is based on the gradient descent method, while making use of STDP and can be applied to networks with multi- ple layers. Furthermore, the algorithm is not limited to neurons firing single spikes or specific neuron models. Results on classic benchmarks, such as the XOR problem and the Iris data set, show that the algo- rithm is capable of non-linear transformations. Complex classification tasks have also been applied with fast convergence times. The results of the simulations show that the new learning rule is as efficient as SpikeProp while having all the advantages of STDP. The supervised learning algorithm for spiking neurons is compared with the back- propagation algorithm for rate neurons by modelling an audio-visual perceptual illusion, the McGurk effect.

APA, Harvard, Vancouver, ISO, and other styles

2

Graves, Alex. "Supervised sequence labelling with recurrent neural networks." kostenfrei, 2008. http://mediatum2.ub.tum.de/doc/673554/673554.pdf.

Full text

APA, Harvard, Vancouver, ISO, and other styles

3

Wang, Yuxuan. "Supervised Speech Separation Using Deep Neural Networks." The Ohio State University, 2015. http://rave.ohiolink.edu/etdc/view?acc_num=osu1426366690.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Hu, Renjie. "Random neural networks for dimensionality reduction and regularized supervised learning." Diss., University of Iowa, 2019. https://ir.uiowa.edu/etd/6960.

Full text

Abstract:

This dissertation explores Random Neural Networks (RNNs) in several aspects and their applications. First, Novel RNNs have been proposed for dimensionality reduction and visualization. Based on Extreme Learning Machines (ELMs) and Self-Organizing Maps (SOMs) a new method is created to identify the important variables and visualize the data. This technique reduces the curse of dimensionality and improves furthermore the interpretability of the visualization and is tested on real nursing survey datasets. ELM-SOM+ is an autoencoder created to preserves the intrinsic quality of SOM and also brings continuity to the projection using two ELMs. This new methodology shows considerable improvement over SOM on real datasets. Second, as a Supervised Learning method, ELMs has been applied to the hierarchical multiscale method to bridge the the molecular dynamics to continua. The method is tested on simulation data and proven to be efficient for passing the information from one scale to another. Lastly, the regularization of ELMs has been studied and a new regularization algorithm for ELMs is created using a modified Lanczos Algorithm. The Lanczos ELM on average divide computational time by 20 and reduce the Normalized MSE by 14% comparing with regular ELMs.

APA, Harvard, Vancouver, ISO, and other styles

5

Aylas, Victor David Sanchez. "Contributions to Supervised Learning of Real-Valued Functions Using Neural Networks." NSUWorks, 1998. http://nsuworks.nova.edu/gscis_etd/395.

Full text

Abstract:

This dissertation presents a new strategy for the automatic design of neural networks. The learning environment addressed is supervised learning from examples. Specifically, Radial Basis Functions (RBF) networks learning real-valued functions of real vectors as in non-linear regression applications are considered. The strategy is based upon the application of strong theoretical relationships between RBF networks and methods from approximation theory, robust statistics, and computational learning theory. The complexity of the network design is examined in detail from the formal definition of the learning problem to the establishment of the corresponding optimization problem. A novel strategy for the systematic and automatic design of RBF networks is developed based upon the coordinated evaluation of memorization and generalization of an incremental architecture. The architecture grows according to the monotonous increase of its generalization. Its corresponding learning method stands out due to its fast convergence and robustness. It represents one of the few learning methods whose computational complexity is precisely stated. It can be used in any non-linear regression tasks which are common in different disciplines of the natural and engineering sciences. Four learning methods are implemented for evaluation. The most complex is the one for the novel self-generating network architecture. Another learning method constitutes a strong contribution to the area of robust learning allowing the automatic detection of data outliers and the removal of their negative influence in the network approximation. It represents the first robust learning method for RBF networks available in the literature and is integrated into the overall strategy introduced in this work. Diverse functions are used to simulate training and test data. Data generated for evaluation is: noise-free, noisy, and with outliers as well as one- and multidimensional. The data with outliers allows the verification of the robustness of the introduced method. In addition, an evaluation example from the area of sensory data processing is chosen. This example consists in localizing a generic object based on range information in the framework of a grasping strategy. The relation to other works and a perspective for further research concludes this work.

APA, Harvard, Vancouver, ISO, and other styles

6

Tatsumi, Keiji. "Studies on supervised learning for neural networks with applications to optimization problems." 京都大学 (Kyoto University), 2006. http://hdl.handle.net/2433/136029.

Full text

APA, Harvard, Vancouver, ISO, and other styles

7

Vančo, Timotej. "Self-supervised učení v aplikacích počítačového vidění." Master's thesis, Vysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií, 2021. http://www.nusl.cz/ntk/nusl-442510.

Full text

Abstract:

The aim of the diploma thesis is to make research of the self-supervised learning in computer vision applications, then to choose a suitable test task with an extensive data set, apply self-supervised methods and evaluate. The theoretical part of the work is focused on the description of methods in computer vision, a detailed description of neural and convolution networks and an extensive explanation and division of self-supervised methods. Conclusion of the theoretical part is devoted to practical applications of the Self-supervised methods in practice. The practical part of the diploma thesis deals with the description of the creation of code for working with datasets and the application of the SSL methods Rotation, SimCLR, MoCo and BYOL in the role of classification and semantic segmentation. Each application of the method is explained in detail and evaluated for various parameters on the large STL10 dataset. Subsequently, the success of the methods is evaluated for different datasets and the limiting conditions in the classification task are named. The practical part concludes with the application of SSL methods for pre-training the encoder in the application of semantic segmentation with the Cityscapes dataset.

APA, Harvard, Vancouver, ISO, and other styles

8

Charles, Eugene Yougarajah Andrew. "Supervised and unsupervised weight and delay adaptation learning in temporal coding spiking neural networks." Thesis, Cardiff University, 2006. http://orca.cf.ac.uk/56168/.

Full text

Abstract:

Artificial neural networks are learning paradigms which mimic the biological neural system. The temporal coding Spiking Neural Network, a relatively new artificial neural network paradigm, is considered to be computationally more powerful than the conventional neural network. Research on the network of spiking neurons is an emerging field and has potential for wider investigation. This research explores alternative learning models with temporal coding spiking neural networks for clustering and classification tasks. Neurons are known to be operating in two modes namely, as integrators and coincidence detectors. Previous temporal coding spiking neural networks, realising spiking neurons as integrators, were utilised for analytical studies. Temporal coding spiking neural networks applied successfully for clustering and classification tasks realised spiking neurons as coincidence detectors and encoded input in formation in the connection delays through a weight adaptation technique. These learning models select suitably delayed connections by enhancing the weights of those connections while weakening the others. This research investigates the learning in temporal coding spiking neural networks with spiking neurons as integrators and coincidence detectors. Focus is given to both supervised and unsupervised learning through weight as well as through delay adaptation. Three novel models for learning in temporal coding spiking neural networks are presented in this research. The first spiking neural network model, Self- Organising Weight Adaptation Spiking Neural Network (SOWA_SNN) realises the spiking neuron as integrator. This model adapts and encodes input information in its connection weights. The second learning model, Self-Organising Delay Adaptation Spiking Neural Network (SODA_SNN) and the third model, Super vised Delay Adaptation Spiking Neural Network (SDA_SNN) realise the spiking neuron as coincidence detector. These two models adapt the connection delays in order to detect temporal patterns through coincidence detection. The first two models were developed for clustering applications and the third for classification tasks. All three models employ Hebbian-based learning rules to update the network connection parameters by utilising the difference between the input and output spike times. The proposed temporal coding spiking neural network models were implemented as discrete models in software and their characteristics and capabilities were analysed through simulations on three bench mark data sets and a high dimensional data set. All three models were able to cluster or classify the analysed data sets efficiently with a high degree of accuracy. The performance of the proposed models, was found to be better than the existing spiking neural network models as well as conventional neural networks. The proposed learning paradigms could be applied to a wide range of applications including manufacturing, business and biomedical domains.

APA, Harvard, Vancouver, ISO, and other styles

9

Tang, Yuxing. "Weakly supervised learning of deformable part models and convolutional neural networks for object detection." Thesis, Lyon, 2016. http://www.theses.fr/2016LYSEC062/document.

Full text

Abstract:

Dans cette thèse, nous nous intéressons au problème de la détection d’objets faiblement supervisée. Le but est de reconnaître et de localiser des objets dans les images, n’ayant à notre disposition durant la phase d’apprentissage que des images partiellement annotées au niveau des objets. Pour cela, nous avons proposé deux méthodes basées sur des modèles différents. Pour la première méthode, nous avons proposé une amélioration de l’approche ”Deformable Part-based Models” (DPM) faiblement supervisée, en insistant sur l’importance de la position et de la taille du filtre racine initial spécifique à la classe. Tout d’abord, un ensemble de candidats est calculé, ceux-ci représentant les positions possibles de l’objet pour le filtre racine initial, en se basant sur une mesure générique d’objectness (par region proposals) pour combiner les régions les plus saillantes et potentiellement de bonne qualité. Ensuite, nous avons proposé l’apprentissage du label des classes latentes de chaque candidat comme un problème de classification binaire, en entrainant des classifieurs spécifiques pour chaque catégorie afin de prédire si les candidats sont potentiellement des objets cible ou non. De plus, nous avons amélioré la détection en incorporant l’information contextuelle à partir des scores de classification de l’image. Enfin, nous avons élaboré une procédure de post-traitement permettant d’élargir et de contracter les régions fournies par le DPM afin de les adapter efficacement à la taille de l’objet, augmentant ainsi la précision finale de la détection. Pour la seconde approche, nous avons étudié dans quelle mesure l’information tirée des objets similaires d’un point de vue visuel et sémantique pouvait être utilisée pour transformer un classifieur d’images en détecteur d’objets d’une manière semi-supervisée sur un large ensemble de données, pour lequel seul un sous-ensemble des catégories d’objets est annoté avec des boîtes englobantes nécessaires pour l’apprentissage des détecteurs. Nous avons proposé de transformer des classifieurs d’images basés sur des réseaux convolutionnels profonds (Deep CNN) en détecteurs d’objets en modélisant les différences entre les deux en considérant des catégories disposant à la fois de l’annotation au niveau de l’image globale et l’annotation au niveau des boîtes englobantes. Cette information de différence est ensuite transférée aux catégories sans annotation au niveau des boîtes englobantes, permettant ainsi la conversion de classifieurs d’images en détecteurs d’objets. Nos approches ont été évaluées sur plusieurs jeux de données tels que PASCAL VOC, ImageNet ILSVRC et Microsoft COCO. Ces expérimentations ont démontré que nos approches permettent d’obtenir des résultats comparables à ceux de l’état de l’art et qu’une amélioration significative a pu être obtenue par rapport à des méthodes récentes de détection d’objets faiblement supervisées
In this dissertation we address the problem of weakly supervised object detection, wherein the goal is to recognize and localize objects in weakly-labeled images where object-level annotations are incomplete during training. To this end, we propose two methods which learn two different models for the objects of interest. In our first method, we propose a model enhancing the weakly supervised Deformable Part-based Models (DPMs) by emphasizing the importance of location and size of the initial class-specific root filter. We first compute a candidate pool that represents the potential locations of the object as this root filter estimate, by exploring the generic objectness measurement (region proposals) to combine the most salient regions and “good” region proposals. We then propose learning of the latent class label of each candidate window as a binary classification problem, by training category-specific classifiers used to coarsely classify a candidate window into either a target object or a non-target class. Furthermore, we improve detection by incorporating the contextual information from image classification scores. Finally, we design a flexible enlarging-and-shrinking post-processing procedure to modify the DPMs outputs, which can effectively match the approximate object aspect ratios and further improve final accuracy. Second, we investigate how knowledge about object similarities from both visual and semantic domains can be transferred to adapt an image classifier to an object detector in a semi-supervised setting on a large-scale database, where a subset of object categories are annotated with bounding boxes. We propose to transform deep Convolutional Neural Networks (CNN)-based image-level classifiers into object detectors by modeling the differences between the two on categories with both image-level and bounding box annotations, and transferring this information to convert classifiers to detectors for categories without bounding box annotations. We have evaluated both our approaches extensively on several challenging detection benchmarks, e.g. , PASCAL VOC, ImageNet ILSVRC and Microsoft COCO. Both our approaches compare favorably to the state-of-the-art and show significant improvement over several other recent weakly supervised detection methods

APA, Harvard, Vancouver, ISO, and other styles

10

Pehrson, Jakob, and Sara Lindstrand. "Support Unit Classification through Supervised Machine Learning." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-281537.

Full text

Abstract:

The purpose of this article is to evaluate the impact a supervised machine learning classification model can have on the process of internal customer support within a large digitized company. Chatbots are becoming a frequently used utility among digital services, though the true general impact is not always clear. The research is separated into the following two questions: (1) Which supervised machine learning algorithm of naïve Bayes, logistic regression, and neural networks can best predict the correct support a user needs and with what accuracy? And (2) What is the effect on the productivity and customer satisfaction of using machine learning to sort customer needs? The data was collected from the internal server database of a large digital company and was then trained on and tested with the three classification algorithms. Furthermore, a survey was collected with questions focused on understanding how the current system affects the involved employees. A first finding indicates that neural networks is the best suited model for the classification task. Though, when the scope and complexity was limited, naïve Bayes and logistic regression performed sufficiently. A second finding of the study is that the classification model potentially improves productivity given that the baseline is met. However, a difficulty exists in drawing conclusions on the exact effects on customer satisfaction since there are many aspects to take into account. Nevertheless, there is a good potential to achieve a positive net effect.
Syftet med artikeln är att utvärdera den påverkan som en klassificeringsmodell kan ha på den interna processen av kundtjänst inom ett stort digitaliserat företag. Chatbotar används allt mer frekvent bland digitala tjänster, även om den generella effekten inte alltid är tydlig. Studien är uppdelad i följande två frågeställningar: (1) Vilken klassificeringsalgoritm bland naive Bayes, logistisk regression, och neurala nätverk kan bäst förutspå den korrekta hjälpen en användare är i behov av och med vilken noggrannhet? Och (2) Vad är effekten på produktivitet och kundnöjdhet för användandet av maskininlärning för sortering av kundbehov? Data samlades från ett stort, digitalt företags interna databas och används sedan i träning och testning med de tre klassificeringsalgoritmerna. Vidare, en enkät skickades ut med fokus på att förstå hur det nuvarande systemet påverkar de berörda arbetarna. Ett första fynd indikerar att neurala nätverk är den mest lämpade modellen för klassificeringen. Däremot, när omfånget och komplexiteten var begränsat presenterade även naive Bayes och logistisk regression tillräckligt. Ett andra fynd av studien är att klassificeringen potentiellt förbättrar produktiviteten givet att baslinjen är mött. Däremot existerar en svårighet i att dra slutsatser om den exakta effekten på kundnöjdhet eftersom det finns många olika aspekter att ta hänsyn till. Likväl finns en god potential i att uppnå en positiv nettoeffekt.

APA, Harvard, Vancouver, ISO, and other styles

11

Bergsten, John, and Konrad Öhman. "Player Analysis in Computer Games Using Artificial Neural Networks." Thesis, Blekinge Tekniska Högskola, Institutionen för kreativa teknologier, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:bth-14812.

Full text

Abstract:

Star Vault AB is a video game development company that has developed the video game Mortal Online. The company has stated that they believe that players new to the game repeatedly find themselves being lost in the game. The objective of this study is to evaluate whether or not an Artificial Neural Network can be used to evaluate when a player is lost in the game Mortal Online. This is done using the free open source library Fast Artifical Neural Network Library. People are invited to a data collection event where they play a tweaked version of the game to facilitate data collection. Players specify whether they are lost or not and the data collected is flagged accordingly. The collected data is then prepared with different parameters to be used when training multiple Artificial Neural Networks. When creating an Artificial Neural Network there exists several parameters which have an impact on its performance. Performance is defined as the balance of high prediction accuracy against low false positive rate. These parameters vary depending on the purpose of the Artificial Neural Network. A quantitative approach is followed where these parameters are varied to investigate which values result in the Artificial Neural Network which best identifies when a player is lost. The parameters are grouped into stages where all combinations of parameter values within each stage are evaluated to reduce the amount of Artificial Neural Networks which have to be trained, with the best performing parameters of each stage being used in subsequent stages. The result is a set of values for the parameters that are considered as ideal as possible. These parameter values are then altered one at a time to verify that they are ideal. The results show that a set of parameters exist which can optimize the Artificial Neural Network model to identify when a player is lost, however not with the high performance that was hoped for. It is theorized that the ambiguity of the word "lost" and the complexity of the game are critical to the low performance.

APA, Harvard, Vancouver, ISO, and other styles

12

Khalid, Fahad. "Measure-based Learning Algorithms : An Analysis of Back-propagated Neural Networks." Thesis, Blekinge Tekniska Högskola, Avdelningen för för interaktion och systemdesign, 2008. http://urn.kb.se/resolve?urn=urn:nbn:se:bth-4795.

Full text

Abstract:

In this thesis we present a theoretical investigation of the feasibility of using a problem specific inductive bias for back-propagated neural networks. We argue that if a learning algorithm is biased towards optimizing a certain performance measure, it is plausible to assume that it will generate a higher performance score when evaluated using that particular measure. We use the term measure function for a multi-criteria evaluation function that can also be used as an inherent function in learning algorithms, in order to customize the bias of a learning algorithm for a specific problem. Hence, the term measure-based learning algorithms. We discuss different characteristics of the most commonly used performance measures and establish similarities among them. The characteristics of individual measures and the established similarities are then correlated to the characteristics of the backpropagation algorithm, in order to explore the applicability of introducing a measure function to backpropagated neural networks. Our study shows that there are certain characteristics of the error back-propagation mechanism and the inherent gradient search method that limit the set of measures that can be used for the measure function. Also, we highlight the significance of taking the representational bias of the neural network into account when developing methods for measure-based learning. The overall analysis of the research shows that measure-based learning is a promising area of research with potential for further exploration. We suggest directions for future research that might help realize measure-based neural networks.
The study is an investigation on the feasibility of using a generic inductive bias for backpropagation artificial neural networks, which could incorporate any one or a combination of problem specific performance metrics to be optimized. We have identified several limitations of both the standard error backpropagation mechanism as well the inherent gradient search approach. These limitations suggest exploration of methods other than backpropagation, as well use of global search methods instead of gradient search. Also, we emphasize the importance of taking the representational bias of the neural network in consideration, since only a combination of both procedural and representational bias can provide highly optimal solutions.

APA, Harvard, Vancouver, ISO, and other styles

13

Oquab, Maxime. "Convolutional neural networks : towards less supervision for visual recognition." Thesis, Paris Sciences et Lettres (ComUE), 2018. http://www.theses.fr/2018PSLEE061.

Full text

Abstract:

Les réseaux de neurones à convolution sont des algorithmes d’apprentissage flexibles qui tirent efficacement parti des importantes masses de données qui leur sont fournies pour l’entraînement. Malgré leur utilisation dans des applications industrielles dès les années 90, ces algorithmes n’ont pas été utilisés pour la reconnaissance d’image à cause de leurs faibles performances avec les images naturelles. C’est finalement grâce a l’apparition d’importantes quantités de données et de puissance de calcul que ces algorithmes ont pu révéler leur réel potentiel lors de la compétition ImageNet, menant à un changement de paradigme en reconnaissance d’image. La première contribution de cette thèse est une méthode de transfert d’apprentissage dans les réseaux à convolution pour la classification d’image. À l’aide d’une procédure de pré-entraînement, nous montrons que les représentations internes d’un réseau à convolution sont assez générales pour être utilisées sur d’autres tâches, et meilleures lorsque le pré-entraînement est réalisé avec plus de données. La deuxième contribution de cette thèse est un système faiblement supervisé pour la classification d’images, pouvant prédire la localisation des objets dans des scènes complexes, en utilisant, lors de l’entraînement, seulement l’indication de la présence ou l’absence des objets dans les images. La troisième contribution de cette thèse est une recherche de pistes de progression en apprentissage non-supervisé. Nous étudions l’algorithme récent des réseaux génératifs adversariaux et proposons l’utilisation d’un test statistique pour l’évaluation de ces modèles. Nous étudions ensuite les liens avec le problème de la causalité, et proposons un test statistique pour la découverte causale. Finalement, grâce a un lien établi récemment avec les problèmes de transport optimal, nous étudions ce que ces réseaux apprennent des données dans le cas non-supervisé
Convolutional Neural Networks are flexible learning algorithms for computer vision that scale particularly well with the amount of data that is provided for training them. Although these methods had successful applications already in the ’90s, they were not used in visual recognition pipelines because of their lesser performance on realistic natural images. It is only after the amount of data and the computational power both reached a critical point that these algorithms revealed their potential during the ImageNet challenge of 2012, leading to a paradigm shift in visual recogntion. The first contribution of this thesis is a transfer learning setup with a Convolutional Neural Network for image classification. Using a pre-training procedure, we show that image representations learned in a network generalize to other recognition tasks, and their performance scales up with the amount of data used in pre-training. The second contribution of this thesis is a weakly supervised setup for image classification that can predict the location of objects in complex cluttered scenes, based on a dataset indicating only with the presence or absence of objects in training images. The third contribution of this thesis aims at finding possible paths for progress in unsupervised learning with neural networks. We study the recent trend of Generative Adversarial Networks and propose two-sample tests for evaluating models. We investigate possible links with concepts related to causality, and propose a two-sample test method for the task of causal discovery. Finally, building on a recent connection with optimal transport, we investigate what these generative algorithms are learning from unlabeled data

APA, Harvard, Vancouver, ISO, and other styles

14

Han, Kun. "Supervised Speech Separation And Processing." The Ohio State University, 2014. http://rave.ohiolink.edu/etdc/view?acc_num=osu1407865723.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Moll, Anton G. [Verfasser], and Rainer [Akademischer Betreuer] Spang. "Semi-supervised Classification of Breast Cancer Expression Profiles Using Neural Networks / Anton G. Moll ; Betreuer: Rainer Spang." Regensburg : Universitätsbibliothek Regensburg, 2018. http://d-nb.info/1153606534/34.

Full text

APA, Harvard, Vancouver, ISO, and other styles

16

Leung, Wing Kai. "The specification, analysis and metrics of supervised feedforward artificial neural networks for applied science and engineering applications." Thesis, Birmingham City University, 2002. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.248495.

Full text

Abstract:

Artificial Neural Networks (ANNs) have been developed for many applications but no detailed study has been made in the measure of their quality such as efficiency and complexity using appropriate metrics. Without an appropriate measurement, it is difficult to tell how an ANN performs on given applications. In addition, it is difficult to provide a measure of the algorithmic complexity of any given application. Further, it is difficult to make use of the results obtained in an application to predict the ANN's quality in a similar application. This research was undertaken to develop metrics, named Neural Metrics, that can be used in the measurement, construction and specification of backpropagation based supervised feedforward ANNs for applied science and engineering applications. A detailed analysis of backpropagation was carried out with a view to studying the mathematical definitions of the proposed metrics. Variants of backpropagation using various optimisation techniques were evaluated with similar computational and metric analysis. The research involved the evaluation of the proposed set of neural metrics using the computer implementation of training algorithms across a number of scientific and engineering benchmark problems including binary and real type training data. The result of the evaluation, for each type of problem, was a specification of values for all neural metrics and network parameters that can be used to successfully solve the same type of problem. With such a specification, neural users can reduce the uncertainty and hence time in choosing the appropriate network details for solving the same type of problem. It is also possible to use the specified neural metric values as reference points to further the experiments with a view to obtaining a better or sub-optimal solution for the problem. In addition, the generalised results obtained in this study provide users not only with a better understanding of the algorithmic complexity of the problem but also with a useful guideline on predicting the values of metrics that are normally determined empirically. It must be emphasised that this study only considers metrics for assessment of construction and off-line training of neural networks. The operational performance (e.g. on-line deployment of the trained networks) is outside the scope. Operational results (e.g. CPU time and run time errors) on training the networks off-line were obtained and discussed for each type of application problem.

APA, Harvard, Vancouver, ISO, and other styles

17

Kilinc, Ismail Ozsel. "Graph-based Latent Embedding, Annotation and Representation Learning in Neural Networks for Semi-supervised and Unsupervised Settings." Scholar Commons, 2017. https://scholarcommons.usf.edu/etd/7415.

Full text

Abstract:

Machine learning has been immensely successful in supervised learning with outstanding examples in major industrial applications such as voice and image recognition. Following these developments, the most recent research has now begun to focus primarily on algorithms which can exploit very large sets of unlabeled examples to reduce the amount of manually labeled data required for existing models to perform well. In this dissertation, we propose graph-based latent embedding/annotation/representation learning techniques in neural networks tailored for semi-supervised and unsupervised learning problems. Specifically, we propose a novel regularization technique called Graph-based Activity Regularization (GAR) and a novel output layer modification called Auto-clustering Output Layer (ACOL) which can be used separately or collaboratively to develop scalable and efficient learning frameworks for semi-supervised and unsupervised settings. First, singularly using the GAR technique, we develop a framework providing an effective and scalable graph-based solution for semi-supervised settings in which there exists a large number of observations but a small subset with ground-truth labels. The proposed approach is natural for the classification framework on neural networks as it requires no additional task calculating the reconstruction error (as in autoencoder based methods) or implementing zero-sum game mechanism (as in adversarial training based methods). We demonstrate that GAR effectively and accurately propagates the available labels to unlabeled examples. Our results show comparable performance with state-of-the-art generative approaches for this setting using an easier-to-train framework. Second, we explore a different type of semi-supervised setting where a coarse level of labeling is available for all the observations but the model has to learn a fine, deeper level of latent annotations for each one. Problems in this setting are likely to be encountered in many domains such as text categorization, protein function prediction, image classification as well as in exploratory scientific studies such as medical and genomics research. We consider this setting as simultaneously performed supervised classification (per the available coarse labels) and unsupervised clustering (within each one of the coarse labels) and propose a novel framework combining GAR with ACOL, which enables the network to perform concurrent classification and clustering. We demonstrate how the coarse label supervision impacts performance and the classification task actually helps propagate useful clustering information between sub-classes. Comparative tests on the most popular image datasets rigorously demonstrate the effectiveness and competitiveness of the proposed approach. The third and final setup builds on the prior framework to unlock fully unsupervised learning where we propose to substitute real, yet unavailable, parent- class information with pseudo class labels. In this novel unsupervised clustering approach the network can exploit hidden information indirectly introduced through a pseudo classification objective. We train an ACOL network through this pseudo supervision together with unsupervised objective based on GAR and ultimately obtain a k-means friendly latent representation. Furthermore, we demonstrate how the chosen transformation type impacts performance and helps propagate the latent information that is useful in revealing unknown clusters. Our results show state-of-the-art performance for unsupervised clustering tasks on MNIST, SVHN and USPS datasets with the highest accuracies reported to date in the literature.

APA, Harvard, Vancouver, ISO, and other styles

18

Hellström, Erik. "Feature learning with deep neural networks for keystroke biometrics : A study of supervised pre-training and autoencoders." Thesis, Luleå tekniska universitet, Datavetenskap, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:ltu:diva-67206.

Full text

Abstract:

Computer security is becoming an increasingly important topic in today’s society, withever increasing connectivity between devices and services. Stolen passwords have thepotential to cause severe damage to companies and individuals alike, leading to therequirement that the security system must be able to detect and prevent fraudulentlogin. Keystroke biometrics is the study of the typing behavior in order to identifythe typist, using features extracted during typing. The features traditionally used inkeystroke biometrics are linear combinations of the timestamps of the keystrokes.This work focuses on feature learning methods and is based on the Carnegie Mellonkeystroke data set. The aim is to investigate if other feature extraction methods canenable improved classification of users. Two methods are employed to extract latentfeatures in the data: Pre-training of an artificial neural network classifier and an autoencoder. Several tests are devised to test the impact of pre-training and compare theresults of a similar network without pre-training. The effect of feature extraction withan autoencoder on a classifier trained on the autoencoder features in combination withthe conventional features is investigated.Using pre-training, I find that the classification accuracy does not improve when using an adaptive learning rate optimizer. However, when a stochastic gradient descentoptimizer is used the accuracy improves by about 8%. Used in conjunction with theconventional features, the features extracted with an autoencoder improve the accuracyof the classifier with about 2%. However, a classifier based on the autoencoder featuresalone is not better than a classifier based on conventional features.

APA, Harvard, Vancouver, ISO, and other styles

19

Ciatto, Giovanni. "Third generation neural networks: formalization as timed automata, validation and learning." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2017. http://amslaurea.unibo.it/12947/.

Full text

Abstract:

Questa tesi mostra una formalizzazione delle reti neurali spiking ottenuta tramite automi temporizzati. Tali reti, a differenza di quelle di seconda generazione, tengono conto anche della dimensione temporale. Sono mostrate due codifiche, sincrona e asincrona, del modello "leaky integrate and fire": i neuroni sono modellati come automi temporizzati che attendono impulsi su dei canali di ingresso e aggiornano il potenziale in base agli input presenti e passati, modulati dai pesi delle rispettive sinapsi e tanto più influenti quanto più recenti. Se il potenziale supera una certa soglia, l'automa emette un segnale sul canale di uscita. Dopo ogni emissione i neuroni rimangono silenti per un periodo refrattario fissato per poi resettarsi. Nel modello asincrono gli input si assumono molto frequenti ma non possono essere contemporanei. In quello sincrono tutti gli impulsi ricevuti nello stesso periodo di accumulazione sono simultanei. Una rete neurale è ottenuta eseguendo in parallelo più automi che condividono canali opportunamente. Anche le sequenze in input sono specificate tramite automi temporizzati, detti generatori, ottenuti tramite un procedimento automatico, da un linguaggio che modella sequenze di spike e pause. Per il modello sincrono si verifica la capacità di riprodurre alcuni comportamenti noti in letteratura. Esso è poi sfruttato per trovare i pesi sinaptici che permettano ad una rete di riprodurre un comportamento dato, espresso tramite logica temporale. Ciò è ottenuto tramite un algoritmo che identifica gli errori commessi dai neuroni di output e applica delle azioni correttive sulle loro sinapsi in ingresso. Le informazioni sulle azioni correttive adeguate vengono poi propagate all'indietro verso gli altri neuroni della rete. Questo processo è ripetuto fino alla riproduzione del comportamento desiderato. Due gli approcci implementativi presentati: uno basato sulla simulazione e uno basato sul model-checking.

APA, Harvard, Vancouver, ISO, and other styles

20

Álvarez, Robles Enrique Josué. "Supervised Learning models with ice hockey data." Thesis, Linköpings universitet, Statistik och maskininlärning, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-167718.

Full text

Abstract:

The technology developments of the last years allow measuring data in almost every field and area nowadays, especially increasing the potential for analytics in branches in which not much analytics have been done due to complicated data access before. The increased number of interest in sports analytics is highly connected to the better technology now available for visual and physical sensors on the one hand and sports as upcoming economic topic holding potentially large revenues and therefore investing interest on the other hand. With the underlying database, precise strategies and individual performance improvements within the field of professional sports are no longer a question of (coach)experience but can be derived from models with statistical accuracy. This thesis aims to evaluate if the available data together with complex and simple supervised machine learning models could generalize from the training data to unseen situations by evaluating performance metrics. Data from games of the ice hockey team of Linköping for the season 2017/2018 is processed with supervised learning algorithms such as binary logistic regression and neural networks. The result of this first step is to determine the strategies of passes by considering both, attempted but failed and successful shots on goals during the game. For that, the original, raw data set was aggregated to game-specific data. After having detected the distinct strategies, they are classified due to their rate of success.

APA, Harvard, Vancouver, ISO, and other styles

21

Chen, Jitong. "On Generalization of Supervised Speech Separation." The Ohio State University, 2017. http://rave.ohiolink.edu/etdc/view?acc_num=osu1492038295603502.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Fischer, Manfred M., and Sucharita Gopal. "Spectral Pattern Recognition and Fuzzy ARTMAP Classification: Design Features, System Dynamics and Real World Simulations." WU Vienna University of Economics and Business, 1996. http://epub.wu.ac.at/4163/1/WSG_DP_5296.pdf.

Full text

Abstract:

Classification of terrain cover from satellite radar imagery represents an area of considerable current interest and research. Most satellite sensors used for land applications are of the imaging type. They record data in a variety of spectral channels and at a variety of ground resolutions. Spectral pattern recognition refers to classification procedures utilizing pixel-by-pixel spectral information as the basis for automated land cover classification. A number of methods have been developed in the past to classify pixels [resolution cells] from multispectral imagery to a priori given land cover categories. Their ability to provide land cover information with high classification accuracies is significant for work where accurate and reliable thematic information is needed. The current trend towards the use of more spectral bands on satellite instruments, such as visible and infrared imaging spectrometers, and finer pixel and grey level resolutions will offer more precise possibilities for accurate identification. But as the complexity of the data grows, so too does the need for more powerful tools to analyse them. It is the major objective of this study to analyse the capabilities and applicability of the neural pattern recognition system, called fuzzy ARTMAP, to generate high quality classifications of urban land cover using remotely sensed images. Fuzzy ARTMAP synthesizes fuzzy logic and Adaptive Resonance Theory (ART) by exploiting the formal similarity between the computations of fuzzy subsethood and the dynamics of category choice, search and learning. The paper describes design features, system dynamics and simulation algorithms of this learning system, which is trained and tested for classification (8 a priori given classes) of a multispectral image of a Landsat-5 Thematic Mapper scene (270 x 360 pixels) from the City of Vienna on a pixel-by-pixel basis. Fuzzy ARTMAP performance is compared with that of an error-based learning system based upon the multi-layer perceptron, and the Gaussian maximum likelihood classifier as conventional statistical benchmark on the same database. Both neural classifiers outperform the conventional classifier in terms of classification accuracy. Fuzzy ARTMAP leads to out-of-sample classification accuracies, very close to maximum performance, while the multi-layer perceptron - like the conventional classifier - shows difficulties to distinguish between some land use categories. (authors' abstract)
Series: Discussion Papers of the Institute for Economic Geography and GIScience

APA, Harvard, Vancouver, ISO, and other styles

23

Bhalala, Smita Ashesh 1966. "Modified Newton's method for supervised training of dynamical neural networks for applications in associative memory and nonlinear identification problems." Thesis, The University of Arizona, 1991. http://hdl.handle.net/10150/277969.

Full text

Abstract:

There have been several innovative approaches towards realizing an intelligent architecture that utilizes artificial neural networks for applications in information processing. The development of supervised training rules for updating the adjustable parameters of neural networks has received extensive attention in the recent past. In this study, specific learning algorithms utilizing modified Newton's method for the optimization of the adjustable parameters of a dynamical neural network are developed. Computer simulation results show that the convergence performance of the proposed learning schemes match very closely that of the LMS learning algorithm for applications in the design of associative memories and nonlinear mapping problems. However, the implementation of the modified Newton's method is complex due to the computation of the slope of the nonlinear sigmoidal function, whereas, the LMS algorithm approximates the slope to be zero.

APA, Harvard, Vancouver, ISO, and other styles

24

Hlynur, Davíð Hlynsson. "Predicting expert moves in the game of Othello using fully convolutional neural networks." Thesis, KTH, Robotik, perception och lärande, RPL, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210914.

Full text

Abstract:

Careful feature engineering is an important factor of artificial intelligence for games. In this thesis I investigate the benefit of delegating the engineering efforts to the model rather than the features, using the board game Othello as a case study. Convolutional neural networks of varying depths are trained to play in a human-like manner by learning to predict actions from tournaments. My main result is that using a raw board state representation, a network can be trained to achieve 57.4% prediction accuracy on a test set, surpassing previous state-of-the-art in this task. The accuracy is increased to 58.3% by adding several common handcrafted features as input to the network but at the cost of more than half again as much the computation time.
Noggrann funktionsteknik är en viktig faktor för artificiell intelligens för spel. I dennaavhandling undersöker jag fördelarna med att delegera teknikarbetet till modellen i ställetför de funktioner, som använder brädspelet Othello som en fallstudie. Konvolutionellaneurala nätverk av varierande djup är utbildade att spela på ett mänskligt sätt genom attlära sig att förutsäga handlingar från turneringar. Mitt främsta resultat är att ett nätverkkan utbildas för att uppnå 57,4% prediktionsnoggrannhet på en testuppsättning, vilketöverträffar tidigare toppmoderna i den här uppgiften. Noggrannheten ökar till 58.3% genomatt lägga till flera vanliga handgjorda funktioner som inmatning till nätverket, tillkostnaden för mer än hälften så mycket beräknatid.

APA, Harvard, Vancouver, ISO, and other styles

25

Berlin, Daniel. "Multi-class Supervised Classification Techniques for High-dimensional Data: Applications to Vehicle Maintenance at Scania." Thesis, KTH, Matematisk statistik, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-209257.

Full text

Abstract:

In vehicle repairs, many times locating the cause of error could turn out more time consuming than the reparation itself. Hence a systematic way to accurately predict a fault causing part would constitute a valuable tool especially for errors difficult to diagnose. This thesis explores the predictive ability of Diagnostic Trouble Codes (DTC’s), produced by the electronic system on Scania vehicles, as indicators for fault causing parts. The statistical analysis is based on about 18800 observations of vehicles where both DTC’s and replaced parts could be identified during the period march 2016 - march 2017. Two different approaches of forming classes is evaluated. Many classes had only few observations and, to give the classifiers a fair chance, it is decided to omit observations of classes based on their frequency in data. After processing, the resulting data could comprise 1547 observations on 4168 features, demonstrating very high dimensionality and making it impossible to apply standard methods of large-sample statistical inference. Two procedures of supervised statistical learning, that are able to cope with high dimensionality and multiple classes, Support Vector Machines and Neural Networks are exploited and evaluated. The analysis showed that on data with 1547 observations of 4168 features (unique DTC’s) and 7 classes SVM yielded an average prediction accuracy of 79.4% compared to 75.4% using NN.The conclusion of the analysis is that DTC’s holds potential to be used as indicators for fault causing parts in a predictive model, but in order to increase prediction accuracy learning data needs improvements. Scope for future research to improve and expand the model, along with practical suggestions for exploiting supervised classifiers at Scania is provided. keywords: Statistical learning, Machine learning, Neural networks, Deep learning, Supervised learning, High dimensionality
Många gånger i samband med fordonsreparationer är felsökningen mer tidskrävande än själva reparationen. Således skulle en systematisk metod för att noggrant prediktera felkällan vara ett värdefullt verktyg för att diagnostisera reparationsåtgärder. I denna uppsats undersöks möjligheten att använda Diagnostic Trouble Codes (DTC:er), som genereras av de elektroniska systemen i Scanias fordon, som indikatorer för att peka ut felorsaken. Till grund för analysen användes ca 18800 observationer av fordon där både DTC:er samt utbytta delar kunnat identifieras under perioden mars 2016 - mars 2017. Två olika strategier för att generera klasser har utvärderats. Till många av klasserna fanns det endast ett fåtal observationer, och för att ge de prediktiva modellerna bra förutsättningar så användes endast klasser med tillräckligt många observationer i träningsdata. Efter bearbetning kunde data innehålla 1547 observationer 4168 attribut, vilket demonstrerar problemets höga dimensionalitet och gör det omöjligt att applicera standard metoder för statistisk analys på stora datamängder. Två metoder för övervakad statistisk inlärning, lämpliga för högdimensionell data med multipla klasser, Södvectormaskiner (SVM) samt Neurala Nätverk (NN) implementeras och deras resultat utvärderas. Analysen visade att på data med 1547 observationer av 4168 attribut (unika DTC:er) och 7 klasser kunde SVM prediktera observationer till klasserna med 79.4% noggrannhet jämfört med 75.4% för NN. De slutsatser som kunde dras av analysen var att DTC:er tycks ha potential att användas för att indikera felorsaker med en prediktiv modell, men att den data som ligger till grund för analysen bör förbättras för att öka noggrannheten i de prediktiva modellerna. Framtida forskningsmöjligheter för att ytterligare förbättra samt utveckla modellen, tillsammans med förslag för hur övervakade klassificerings modeller kan användas på Scnaia har identifierats.

APA, Harvard, Vancouver, ISO, and other styles

26

Veselý, Karel. ""Semi-supervised" trénování hlubokých neuronových sítí pro rozpoznávání řeči." Doctoral thesis, Vysoké učení technické v Brně. Fakulta informačních technologií, 2018. http://www.nusl.cz/ntk/nusl-412596.

Full text

Abstract:

V této dizertační práci nejprve prezentujeme teorii trénování neuronových sítí pro rozpoznávání řeči společně s implementací trénovacího receptu 'nnet1', který je součástí toolkitu s otevřeným kódem Kaldi. Recept se skládá z předtrénování bez učitele pomocí algoritmu RBM, trénování klasifikátoru z řečových rámců s kriteriální funkcí Cross-entropy a ze sekvenčního trénování po větách s kriteriální funkcí sMBR. Následuje hlavní téma práce, kterým je semi-supervised trénování se smíšenými daty s přepisem i bez přepisu. Inspirováni konferenčními články a úvodními experimenty jsme se zaměřili na několik otázek: Nejprve na to, zda je lepší konfidence (t.j. důvěryhodnosti automaticky získaných anotací) počítat po větách, po slovech nebo po řečových rámcích. Dále na to, zda by konfidence měly být použity pro výběr dat nebo váhování dat - oba přístupy jsou kompatibilní s trénováním pomocí metody stochastického nejstrmějšího sestupu, kde jsou gradienty řečových rámců násobeny vahou. Dále jsme se zabývali vylepšováním semi-supervised trénování pomocí kalibrace kofidencí a přístupy, jak model dále vylepšit pomocí dat se správným přepisem. Nakonec jsme navrhli jednoduchý recept, pro který není nutné časově náročné ladění hyper-parametrů trénování, a který je prakticky využitelný pro různé datové sady. Experimenty probíhaly na několika sadách řečových dat: pro rozpoznávač vietnamštiny s 10 přepsaným hodinami (Babel) se chybovost snížila o 2.5%, pro angličtinu se 14 přepsanými hodinami (Switchboard) se chybovost snížila o 3.2%. Zjistili jsme, že je poměrně těžké dále vylepšit přesnost systému pomocí úprav konfidencí, zároveň jsme ale přesvědčení, že naše závěry mají značnou praktickou hodnotu: data bez přepisu je jednoduché nasbírat a naše navrhované řešení přináší dobrá zlepšení úspěšnosti a není těžké je replikovat.

APA, Harvard, Vancouver, ISO, and other styles

27

Gonthier, Nicolas. "Transfer learning of convolutional neural networks for texture synthesis and visual recognition in artistic images." Thesis, université Paris-Saclay, 2021. http://www.theses.fr/2021UPASG024.

Full text

Abstract:

Dans cette thèse, nous étudions le transfert de réseaux de neurones à convolution (abrégés CNN en anglais) pré-entrainés sur des images naturelles, vers des tâches différentes de celles pour lesquelles ils ont été entraînés. Nous avons travaillé sur deux axes de recherche : la synthèse de texture et la reconnaissance visuelle dans les images d'œuvres d'art. Le premier axe consiste à synthétiser une nouvelle image à partir d'une image de référence. La plupart des méthodes récentes sont basées sur l'utilisation des matrices Gram des cartes de caractéristiques issues de CNNs entrainés sur ImageNet. Nous avons développé une stratégie multirésolution pour prendre en compte les structures à grande échelle. Cette stratégie peut être couplée à des contraintes à grande distance, soit par une contrainte basée sur le spectre de Fourier, soit par l'utilisation de l'autocorrélation des cartes de caractéristiques. Elle permet d'obtenir d'excellentes synthèses en haute résolution, tout particulièrement pour les textures régulières. Ces méthodes ont été évaluées de manière quantitatives et perceptuelles. Dans un second temps, nous nous sommes intéressés au transfert d'apprentissage pour la classification des images d'art. Les CNNs peuvent être utilisés comme des extracteurs de caractéristiques ou comme initialisation pour un nouvel entrainement. Nous avons mis en avant la supériorité de cette seconde solution. De plus, nous avons étudié le processus d'apprentissage à l'aide de visualisation de caractéristiques, d'indices de similarité ainsi que des métriques quantitatives. Nous avons aussi étudié le transfert de CNN entrainé pour de la détection d'objets. Nous avons proposé une méthode simple de détection faiblement supervisée (cad uniquement des annotations au niveau de l'image). Elle est basée sur un apprentissage à instances multiples, l'utilisation de caractéristiques figées et de propositions de boîtes issues d'un CNN pré-entrainés. Nous avons expérimentalement montré l'intérêt de nos modèles sur six jeux de données non photoréalistes
In this thesis, we study the transfer of Convolutional Neural Networks (CNN) trained on natural images to related tasks. We follow two axes: texture synthesis and visual recognition in artworks. The first one consists in synthesizing a new image given a reference sample. Most methods are based on enforcing the Gram matrices of ImageNet-trained CNN features. We develop a multi-resolution strategy to take into account large scale structures. This strategy can be coupled with long-range constraints either through a Fourier frequency constraint, or the use of feature maps autocorrelation. This scheme allows excellent high-resolution synthesis especially for regular textures. We compare our methods to alternatives ones with quantitative and perceptual evaluations. In a second axis, we focus on transfer learning of CNN for artistic image classification. CNNs can be used as off-the-shelf feature extractors or fine-tuned. We illustrate the advantage of the last solution. Second, we use feature visualization techniques, CNNs similarity indexes and quantitative metrics to highlight some characteristics of the fine-tuning process. Another possibility is to transfer a CNN trained for object detection. We propose a simple multiple instance method using off-the-shelf deep features and box proposals, for weakly supervised object detection. At training time, only image-level annotations are needed. We experimentally show the interest of our models on six non-photorealistic

APA, Harvard, Vancouver, ISO, and other styles

28

Schennings, Jacob. "Deep Convolutional Neural Networks for Real-Time Single Frame Monocular Depth Estimation." Thesis, Uppsala universitet, Avdelningen för systemteknik, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-336923.

Full text

Abstract:

Vision based active safety systems have become more frequently occurring in modern vehicles to estimate depth of the objects ahead and for autonomous driving (AD) and advanced driver-assistance systems (ADAS). In this thesis a lightweight deep convolutional neural network performing real-time depth estimation on single monocular images is implemented and evaluated. Many of the vision based automatic brake systems in modern vehicles only detect pre-trained object types such as pedestrians and vehicles. These systems fail to detect general objects such as road debris and roadside obstacles. In stereo vision systems the problem is resolved by calculating a disparity image from the stereo image pair to extract depth information. The distance to an object can also be determined using radar and LiDAR systems. By using this depth information the system performs necessary actions to avoid collisions with objects that are determined to be too close. However, these systems are also more expensive than a regular mono camera system and are therefore not very common in the average consumer car. By implementing robust depth estimation in mono vision systems the benefits from active safety systems could be utilized by a larger segment of the vehicle fleet. This could drastically reduce human error related traffic accidents and possibly save many lives. The network architecture evaluated in this thesis is more lightweight than other CNN architectures previously used for monocular depth estimation. The proposed architecture is therefore preferable to use on computationally lightweight systems. The network solves a supervised regression problem during the training procedure in order to produce a pixel-wise depth estimation map. The network was trained using a sparse ground truth image with spatially incoherent and discontinuous data and output a dense spatially coherent and continuous depth map prediction. The spatially incoherent ground truth posed a problem of discontinuity that was addressed by a masked loss function with regularization. The network was able to predict a dense depth estimation on the KITTI dataset with close to state-of-the-art performance.

APA, Harvard, Vancouver, ISO, and other styles

29

Caye, Daudt Rodrigo. "Convolutional neural networks for change analysis in earth observation images with noisy labels and domain shifts." Electronic Thesis or Diss., Institut polytechnique de Paris, 2020. http://www.theses.fr/2020IPPAT033.

Full text

Abstract:

L'analyse de l'imagerie satellitaire et aérienne d'observation de la Terre nous permet d'obtenir des informations précises sur de vastes zones. Une analyse multitemporelle de telles images est nécessaire pour comprendre l'évolution de ces zones. Dans cette thèse, les réseaux de neurones convolutifs sont utilisés pour détecter et comprendre les changements en utilisant des images de télédétection provenant de diverses sources de manière supervisée et faiblement supervisée. Des architectures siamoises sont utilisées pour comparer des paires d'images recalées et identifier les pixels correspondant à des changements. La méthode proposée est ensuite étendue à une architecture de réseau multitâche qui est utilisée pour détecter les changements et effectuer une cartographie automatique simultanément, ce qui permet une compréhension sémantique des changements détectés. Ensuite, un filtrage de classification et un nouvel algorithme de diffusion anisotrope guidée sont utilisés pour réduire l'effet du bruit d'annotation, un défaut récurrent pour les ensembles de données à grande échelle générés automatiquement. Un apprentissage faiblement supervisé est également réalisé pour effectuer une détection de changement au niveau des pixels en utilisant uniquement une supervision au niveau de l'image grâce à l'utilisation de cartes d'activation de classe et d'une nouvelle couche d'attention spatiale. Enfin, une méthode d'adaptation de domaine fondée sur un entraînement adverse est proposée. Cette méthode permet de projeter des images de différents domaines dans un espace latent commun où une tâche donnée peut être effectuée. Cette méthode est testée non seulement pour l'adaptation de domaine pour la détection de changement, mais aussi pour la classification d'images et la segmentation sémantique, ce qui prouve sa polyvalence
The analysis of satellite and aerial Earth observation images allows us to obtain precise information over large areas. A multitemporal analysis of such images is necessary to understand the evolution of such areas. In this thesis, convolutional neural networks are used to detect and understand changes using remote sensing images from various sources in supervised and weakly supervised settings. Siamese architectures are used to compare coregistered image pairs and to identify changed pixels. The proposed method is then extended into a multitask network architecture that is used to detect changes and perform land cover mapping simultaneously, which permits a semantic understanding of the detected changes. Then, classification filtering and a novel guided anisotropic diffusion algorithm are used to reduce the effect of biased label noise, which is a concern for automatically generated large-scale datasets. Weakly supervised learning is also achieved to perform pixel-level change detection using only image-level supervision through the usage of class activation maps and a novel spatial attention layer. Finally, a domain adaptation method based on adversarial training is proposed, which succeeds in projecting images from different domains into a common latent space where a given task can be performed. This method is tested not only for domain adaptation for change detection, but also for image classification and semantic segmentation, which proves its versatility

APA, Harvard, Vancouver, ISO, and other styles

30

Kamat, Sai Shyamsunder. "Analyzing Radial Basis Function Neural Networks for predicting anomalies in Intrusion Detection Systems." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-259187.

Full text

Abstract:

In the 21st century, information is the new currency. With the omnipresence of devices connected to the internet, humanity can instantly avail any information. However, there are certain are cybercrime groups which steal the information. An Intrusion Detection System (IDS) monitors a network for suspicious activities and alerts its owner about an undesired intrusion. These commercial IDS’es react after detecting intrusion attempts. With the cyber attacks becoming increasingly complex, it is expensive to wait for the attacks to happen and respond later. It is crucial for network owners to employ IDS’es that preemptively differentiate a harmless data request from a malicious one. Machine Learning (ML) can solve this problem by recognizing patterns in internet traffic to predict the behaviour of network users. This project studies how effectively Radial Basis Function Neural Network (RBFN) with Deep Learning Architecture can impact intrusion detection. On the basis of the existing framework, it asks how well can an RBFN predict malicious intrusive attempts, especially when compared to contemporary detection practices.Here, an RBFN is a multi-layered neural network model that uses a radial basis function to transform input traffic data. Once transformed, it is possible to separate the various traffic data points using a single straight line in extradimensional space. The outcome of the project indicates that the proposed method is severely affected by limitations. E.g. the model needs to be fine tuned over several trials to achieve a desired accuracy. The results of the implementation show that RBFN is accurate at predicting various cyber attacks such as web attacks, infiltrations, brute force, SSH etc, and normal internet behaviour on an average 80% of the time. Other algorithms in identical testbed are more than 90% accurate. Despite the lower accuracy, RBFN model is more than 94% accurate at recording specific kinds of attacks such as Port Scans and BotNet malware. One possible solution is to restrict this model to predict only malware attacks and use different machine learning algorithm for other attacks.
I det 21: a århundradet är information den nya valutan. Med allnärvaro av enheter anslutna till internet har mänskligheten tillgång till information inom ett ögonblick. Det finns dock vissa grupper som använder metoder för att stjäla information för personlig vinst via internet. Ett intrångsdetekteringssystem (IDS) övervakar ett nätverk för misstänkta aktiviteter och varnar dess ägare om ett oönskat intrång skett. Kommersiella IDS reagerar efter detekteringen av ett intrångsförsök. Angreppen blir alltmer komplexa och det kan vara dyrt att vänta på att attackerna ska ske för att reagera senare. Det är avgörande för nätverksägare att använda IDS:er som på ett förebyggande sätt kan skilja på oskadlig dataanvändning från skadlig. Maskininlärning kan lösa detta problem. Den kan analysera all befintliga data om internettrafik, känna igen mönster och förutse användarnas beteende. Detta projekt syftar till att studera hur effektivt Radial Basis Function Neural Networks (RBFN) med Djupinlärnings arkitektur kan påverka intrångsdetektering. Från detta perspektiv ställs frågan hur väl en RBFN kan förutsäga skadliga intrångsförsök, särskilt i jämförelse med befintliga detektionsmetoder.Här är RBFN definierad som en flera-lagers neuralt nätverksmodell som använder en radiell grundfunktion för att omvandla data till linjärt separerbar. Efter en undersökning av modern litteratur och lokalisering av ett namngivet dataset användes kvantitativ forskningsmetodik med prestanda indikatorer för att utvärdera RBFN: s prestanda. En Random Forest Classifier algorithm användes också för jämförelse. Resultaten erhölls efter en serie finjusteringar av parametrar på modellerna. Resultaten visar att RBFN är korrekt när den förutsäger avvikande internetbeteende i genomsnitt 80% av tiden. Andra algoritmer i litteraturen beskrivs som mer än 90% korrekta. Den föreslagna RBFN-modellen är emellertid mycket exakt när man registrerar specifika typer av attacker som Port Scans och BotNet malware. Resultatet av projektet visar att den föreslagna metoden är allvarligt påverkad av begränsningar. T.ex. så behöver modellen finjusteras över flera försök för att uppnå önskad noggrannhet. En möjlig lösning är att begränsa denna modell till att endast förutsäga malware-attacker och använda andra maskininlärnings-algoritmer för andra attacker.

APA, Harvard, Vancouver, ISO, and other styles

31

Chiaroni, Florent. "Weakly supervised learning for image classification and potentially moving obstacles analysis." Thesis, université Paris-Saclay, 2020. http://www.theses.fr/2020UPASC006.

Full text

Abstract:

Dans le contexte des applications de perception pour le véhicule à conduite déléguée, l’intérêt pour les approches d’apprentissage automatique a continuellement augmenté pendant cette dernière décennie. Cependant, lorsque ces approches doivent être discriminatives, elle nécessitent généralement d’apprendre sur des données manuellement annotées. L’annotation manuelle a un coût non négligeable, tandis que les données non annotées peuvent être facilement obtenues dans le contexte d’un véhicule autonome équipé de capteurs. Il se trouve qu’une catégorie de stratégies d’apprentissage, dite d’apprentissage faiblement supervisé, permet d’exploiter des données partiellement labélisées. Ainsi, nous avons pour objectif dans cette thèse de réduire autant que possible le besoin de labélisation manuelle en proposant des techniques d’apprentissage faiblement supervisées
In the context of autonomous vehicle perception, the interest of the research community for deep learning approaches has continuously grown since the last decade. This can be explained by the fact that deep learning techniques provide nowadays state-of-the-art prediction performances for several computer vision challenges. More specifically, deep learning techniques can provide rich semantic information concerning the complex visual patterns encountered in autonomous driving scenarios. However, such approaches require, as their name implies, to learn on data. In particular, state-of-the-art prediction performances on discriminative tasks often demand hand labeled data of the target application domain. Hand labeling has a significant cost, while, conversely, unlabeled data can be easily obtained in the autonomous driving context. It turns out that a category of learning strategies, referred to as weakly supervised learning, enables to exploit partially labeled data. Therefore, we aim in this thesis at reducing as much as possible the hand labeling requirement by proposing weakly supervised learning techniques.We start by presenting a type of learning methods which are self-supervised. They consist of substituting hand-labels by upstream techniques able to automatically generate exploitable training labels. Self-supervised learning (SSL) techniques have proven their usefulness in the past for offroad obstacles avoidance and path planning through changing environments. However, SSL techniques still leave the door open for detection, segmentation, and classification of static potentially moving obstacles.Consequently, we propose in this thesis three novel weakly supervised learning methods with the final goal to deal with such road users through an SSL framework. The first two proposed contributions of this work aim at dealing with partially labeled image classification datasets, such that the labeling effort can be only focused on our class of interest, the positive class. Then, we propose an approach which deals with training data containing a high fraction of wrong labels, referred to as noisy labels. Next, we demonstrate the potential of such weakly supervised strategies for detection and segmentation of potentially moving obstacles

APA, Harvard, Vancouver, ISO, and other styles

32

Chávez, Escalante Diego Alonso 1988. "Semi-supervised learning with graphs methods using signal processing = Métodos de aprendizado semi-supervisionado com grafos usando processamento de sinais." [s.n.], 2014. http://repositorio.unicamp.br/jspui/handle/REPOSIP/275521.

Full text

Abstract:

Orientador: Siome Klein Goldenstein
Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-25T19:49:49Z (GMT). No. of bitstreams: 1 ChavezEscalante_DiegoAlonso_M.pdf: 1954210 bytes, checksum: c9a77d2f0545d5517700c34dd6cf3324 (MD5) Previous issue date: 2014
Resumo: No aprendizado de máquina, os problemas de classificação de padrões eram tradicionalmente abordados por algoritmos de aprendizado supervisionado que utilizam apenas dados rotulados para treinar-se. Entretanto, os dados rotulados são realmente difíceis de coletar em muitos domínios de problemas, enquanto os dados não rotulados são geralmente mais fáceis de recolher. Também em aprendizado de máquina só o aprendizado não supervisionado é capaz de aprender a topologia e propriedades de um conjunto de dados não rotulados. Portanto, a fim de conseguir uma classificação utilizando o conhecimento a partir de dados rotulados e não rotulados, é necessário o uso de conceitos de aprendizado supervisionado tanto como do não supervisionado. Este tipo de aprendizagem é chamado de aprendizado semi-supervisionado, que declara ter construído melhores classificadores que o tradicional aprendizado supervisionado em algumas condições especificas, porque não só aprende dos dados rotulados, mas também das propriedades naturais dos dados não rotulados como por exemplo a distribuição espacial deles. O aprendizado semi-supervisionado apresenta uma ampla coleção de métodos e técnicas para classificação, e um dos mais interessantes e o aprendizado semi-supervisionado baseado em grafos, o qual modela o problema da classificação semi-supervisionada utilizando a teoria dos grafos. Mas um problema que surge a partir dessa técnica é o custo para treinar conjuntos com grandes quantidades de dados, de modo que o desenvolvimento de algoritmos escaláveis e eficientes de aprendizado semi-supervisionado baseado em grafos e um problema muito interessante e prometedor para lidar com ele. Desta pesquisa foram desenvolvidos dois algoritmos, um para a construção do grafo usando redes neurais não supervisionadas e outro para a regularização do grafo usando processamento de sinais em grafos, especificamente usando filtros de resposta finita sobre o grafo. As duas soluções mostraram resultados comparáveis com os da literatura
Abstract: In machine learning, classification problems were traditionally addressed by supervised learning algorithms, which only use labeled data for training. However, labeled data in many problem domains are really hard to collect, while unlabeled data are usually easy to collect. Also, in machine learning, only unsupervised learning is capable to learn the topology and properties of a set of unlabeled data. In order to do a classification using knowledge from labeled and unlabeled data, it is necessary to use concepts from both supervised and unsupervised learning. This type of learning is called semi-supervised learning, which has claimed to build better classifiers than the traditional supervised learning in some specific conditions, because it does not only learn from the labeled data, but also from the natural properties of unlabeled data as for example spatial distribution. Semi-supervised learning presents a broad collection of methods and techniques for classification. Among them there is graph based semi-supervised learning, which model the problem of semi-supervised classification using graph theory. One problem that arises from this technique is the cost for training large data sets, so the development of scalable and efficient algorithms for graph based semi-supervised learning is a interesting and promising problem to deal with. From this research we developed two algorithms, one for graph construction using unsupervised neural networks; and other for graph regularization using graph signal processing theory, more specifically using FIR filters over a graph. Both solutions showed comparable performance to other literature methods in terms of accuracy
Mestrado
Ciência da Computação
Mestre em Ciência da Computação

APA, Harvard, Vancouver, ISO, and other styles

33

Wikström, Johan. "Evaluating supervised machine learning algorithms to predict recreational fishing success : A multiple species, multiple algorithms approach." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2015. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-172995.

Full text

Abstract:

This report examines three different machine learning algorithms and their effectiveness for predicting recreational fishing success. Recreational fishing is a huge pastime but reliable methods of predicting fishing success have largely been missing. This report compares random forest, linear regression and multilayer perceptron to a reasonable baseline model for predicting fishing success. Fishing success is defined as the expected weight of the fish caught. Previous reports have mainly focused on commercial fishing or limited the research to examining the impact of a single variable. In this exploratory study, multiple attributes and multiple algorithms are examined to determine if supervised machine learning is a viable tool to predict recreational fishing success. Recreational fishing success can potentially be predicted by a large number of attributes, which may be different for different species. In this report, data is fetched from multiple sources and combined into a unified data format. The primary source of data is a database from the fishing app FishBrain, containing data of over 250000 logged catches. Another is the World Weather Online API which supplies weather data. The report focuses on the four most common species in the database, largemouth bass, Micropterus salmoides, northern pike, Esox lucius, rainbow trout, Oncorhynchus mykiss and European perch, Perca fluviatilis with a focus on largemouth bass since it has the most data available. Algorithms are evaluated using the Weka data mining software. Hyperparameters are found using cross-validation and some data is used as a test set to validate the results after cross-validation. Results are measured as the error compared to a baseline algorithm. Random forest is the most effective algorithm in the experiments, reducing error compared to the baseline for all the examined fish species. It is also found that no single variable affects the chosen metric of fishing success much, but rather a combination of most of the examined variables is needed to give optimal predictions. In conclusion, the random forest algorithm can be used to predict fishing success across multiple species. It performs significantly better than linear regression, multilayer perceptron and the baseline on crossvalidation and on the testing set.
I denna rapport evalueras tre olika maskininlärningsalgoritmer och deras effektivitet för att förutsäga framgång inom sportfiske. Sport- fiske är en mycket populär hobby, men pålitliga metoder att förutsäga framgångsrikt sportfiske saknas. Denna rapport jämför random forest, linjär regression och flerlagers neurala nätverk mot en rimlig baselinealgorithm för att förutsäga framgång inom sportfiske. Framgång defineras som fiskens förväntade vikt i kg. Tidigare undersökningar har huvudsakligen fokuserat på kommersiellt fiske eller begränsat undersökningen till påverkan av en enskild variabel. I denna studie undersöks flera attribut och algoritmer för att avgöra om övervakad maskininlärning är ett användbart verktyg för att förutsäga framgång inom sportfiske. Framgång inom sportfiske kan potentiellt påverkas av ett stort antal attribut som kan vara olika för olika arter. I denna studie hämtas data från ett flertal källor som kombineras i ett unifierat dataformat. Den primära datakällan är en databas tillhörande sportfiskeappen FishBrain som innehåller över 250000 loggade fångster. En annan källa är World Weather Online:s API som bidrar med väderdata. Rapporten fokuserar på de fyra vanligaste arterna i databasen, largemouth bass, Micropterus salmoides, gädda, Esox lucius, regnbågsöring, Oncorhynchus mykiss och europeisk abborre, Perca fluviatilis med ett särskilt fokus på largemouth bass eftersom den har mest data tillgängligt. Algoritmerna evalueras med hjälp av data mining-verktyget Weka. Hyperparametrar bestäms med hjälp av korsvalidering och en delmängd av datan separeras och används för att validera resultaten efter korsvalidering. Resultaten mäts relativt en baseline-algoritm. Random forest är den mest effektiva algoritmen i experimenten och reducerar felet jämfört med baseline-algoritmen för alla undersökta fiskarter. Inget enskilt attribut påverkar slutresultatet mycket utan det behövs en kombination av flera attribut för att ge optimala prediktioner. Slutsatsen blir att random forest kan användas för att förutsäga framgång inom sportfiske för flera olika fiskarter. Den presterar signifikant bättre än linjär regression, flerlagers neuralt nätverk och baselinealgoritmen på korsvalidering och på testdelmängden.

APA, Harvard, Vancouver, ISO, and other styles

34

Andersson, Melanie, Arvola Maja, and Sara Hedar. "Sketch Classification with Neural Networks : A Comparative Study of CNN and RNN on the Quick, Draw! data set." Thesis, Uppsala universitet, Institutionen för teknikvetenskaper, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-353504.

Full text

Abstract:

The aim of the study is to apply and compare the performance of two different types of neural networks on the Quick, Draw! dataset and from this determine whether interpreting the sketches as sequences gives a higher accuracy than interpreting them as pixels. The two types of networks constructed were a recurrent neural network (RNN) and a convolutional neural network (CNN). The networks were optimised and the final architectures included five layers. The final evaluation accuracy achieved was 94.2% and 92.3% respectively, leading to the conclusion that the sequential interpretation of the Quick, Draw! dataset is favourable.

APA, Harvard, Vancouver, ISO, and other styles

35

Sarabi, Zahra. "Revealing the Positive Meaning of a Negation." Thesis, University of North Texas, 2019. https://digital.library.unt.edu/ark:/67531/metadc1505158/.

Full text

Abstract:

Negation is a complex phenomenon present in all human languages, allowing for the uniquely human capacities of denial, contradiction, misrepresentation, lying, and irony. It is in the first place a phenomenon of semantical opposition. Sentences containing negation are generally (a) less informative than affirmative ones, (b) morphosyntactically more marked—all languages have negative markers while only a few have affirmative markers, and (c) psychologically more complex and harder to process. Negation often conveys positive meaning. This meaning ranges from implicatures to entailments. In this dissertation, I develop a system to reveal the underlying positive interpretation of negation. I first identify which words are intended to be negated (i.e, the focus of negation) and second, I rewrite those tokens to generate an actual positive interpretation. I identify the focus of negation by scoring probable foci along a continuous scale. One of the obstacles to exploring foci scoring is that no public datasets exist for this task. Thus, to study this problem I create new corpora. The corpora contain verbal, nominal and adjectival negations and their potential positive interpretations along with their scores ranging from 1 to 5. Then, I use supervised learning models for scoring the focus of negation. In order to rewrite the focus of negation with its positive interpretation, I work with negations from Simple Wikipedia, automatically generate potential positive interpretations, and then collect manual annotations that effectively rewrite the negation in positive terms. This procedure yields positive interpretations for approximately 77% of negations, and the final corpus includes over 5,700 negations and over 5,900 positive interpretations. I then use sequence-to-sequence neural models and provide baseline results.

APA, Harvard, Vancouver, ISO, and other styles

36

Othmani-Guibourg, Mehdi. "Supervised learning for distribution of centralised multiagent patrolling strategies." Electronic Thesis or Diss., Sorbonne université, 2019. http://www.theses.fr/2019SORUS534.

Full text

Abstract:

Depuis presque deux décennies, la tâche de la patrouille a fait l'objet d'une attention toute particulière de la part de la communauté multi-agent. La patrouille multi-agent consiste à modéliser comme un système multi-agent une tâche de patrouille à optimiser. Cette optimisation revient à répartir dans l'espace et le temps les agents patrouilleurs sur la zone à surveiller, cela le plus efficacement possible; un tel problème constitue par là même un problème de décision. Un large éventail d'algorithmes basés sur des stratégies d’agent réactives, cognitives, d’apprentissage par renforcement, centralisées et décentralisées, entre autres, ont été développés pour rendre les stratégies de patrouille toujours plus performantes. Cependant, les approches existantes basées sur de l'apprentissage supervisé avaient peu été étudiées jusqu’à présent, bien que quelques travaux aient abordé cette question. L’idée principale et sous-jacente à l'apprentissage supervisé, qui n’est rien de plus qu’un ensemble de méthodes et d'outils permettant d’inférer de nouvelles connaissances, est d’apprendre une fonction associant à tout élément en entrée un élément en sortie, à partir d'un ensemble de données composé de paires d'éléments entrées-sorties; l'apprentissage, dans ce cas, permet au système de faire de la généralisation à de nouvelles données jamais observées auparavant. Jusqu'à présent, la meilleure stratégie de patrouille multi-agent en ligne, à savoir sans calcul préalable, s'est avérée être une stratégie centralisée à coordinateur. Cependant, comme pour tout processus de décision centralisé généralement, une telle stratégie est difficilement échelonnable. L'objectif de ce travail est alors de développer et de mettre en œuvre une nouvelle méthodologie visant à transformer toute stratégie centralisée performante en stratégie distribuée, c'est-à-dire par nature résiliente, plus adaptative aux changements de l'environnement et échelonnable. Ce faisant, le processus de décision centralisé, généralement représenté par un coordinateur dans la patrouille multi-agent, est distribué sur les agents patrouilleurs au moyen de méthodes d’apprentissage supervisé, de sorte que les agents de la stratégie distribuée résultante tendent chacun à capturer ou cristalliser une partie de l’algorithme exécuté par le processus de décision centralisé. Le résultat est alors un nouveau algorithme de prise de décision distribué, qui repose sur de l’apprentissage automatique. Dans cette thèse, une telle procédure de distribution de stratégie centralisée est établie, puis concrètement mise en œuvre en utilisant certaines architectures de réseaux de neurones. Ainsi, après avoir exposé le contexte et les motivations, nous posons la problématique étudiée. Les principales stratégies multi-agent élaborées jusqu'à présent dans le cadre de la patrouille multi-agent sont ensuite décrites, en particulier une stratégie centralisée à haute performance qui est la stratégie centralisée à distribuer ici étudiée, ainsi qu’une stratégie décentralisée assez simple qui est utilisée comme référence pour les stratégies décentralisées. Entre autres, quelques stratégies basées sur de l’apprentissage supervisé sont aussi décrites. Ensuite, le modèle ainsi que certains concept fondamentaux du problème de la patrouille multi-agent sont définis
For nearly two decades, patrolling has received significant attention from the multiagent community. Multiagent patrolling (MAP) consists in modelling a patrol task to optimise as a multiagent system. The problem of optimising a patrol task is to distribute the most efficiently agents over the area to patrol in space and time, which constitutes a decision-making problem. A range of algorithms based on reactive, cognitive, reinforcement learning, centralised and decentralised strategies, amongst others, have been developed to make such a task ever more efficient. However, the existing patrolling-specific approaches based on supervised learning were still at preliminary stages, although a few works addressed this issue. Central to supervised learning, which is a set of methods and tools that allow inferring new knowledge, is the idea of learning a function mapping any input to an output from a sample of data composed of input-output pairs; learning, in this case, enables the system to generalise to new data never observed before. Until now, the best online MAP strategy, namely without precalculation, has turned out to be a centralised strategy with a coordinator. However, as for any centralised decision process in general, such a strategy is hardly scalable. The purpose of this work is then to develop and implement a new methodology aiming at turning any high-performance centralised strategy into a distributed strategy. Indeed, distributed strategies are by design resilient, more adaptive to changes in the environment, and scalable. In doing so, the centralised decision process, generally represented in MAP by a coordinator, is distributed into patrolling agents by means of supervised learning methods, so that each agent of the resultant distributed strategy tends to capture a part of the algorithm executed by the centralised decision process. The outcome is a new distributed decision-making algorithm based on machine learning. In this dissertation therefore, such a procedure of distribution of centralised strategy is established, then concretely implemented using some artificial neural networks architectures. By doing so, after having exposed the context and motivations of this work, we pose the problematic that led our study. The main multiagent strategies devised until now as part of MAP are then described, particularly a high-performance coordinated strategy, which is the centralised strategy studied in this work, as well as a simple decentralised strategy used as reference for decentralised strategies. Among others, some existing strategies based on supervised learning are also described. Thereafter, the model as well as certain of key concepts of MAP are defined. We also define the methodology laid down to address and study this problematic. This methodology comes in the form of a procedure that allows decentralising any centralised strategy by means of supervised learning. Then, the software ecosystem we developed for the needs of this work is also described, particularly PyTrol a discrete-time simulator dedicated to MAP developed with the aim of performing MAP simulation, to assess strategies and generate data, and MAPTrainer, a framework hinging on the PyTorch machine learning library, dedicated to research in machine learning in the context of MAP

APA, Harvard, Vancouver, ISO, and other styles

37

Djikic, Addi. "Segmentation and Depth Estimation of Urban Road Using Monocular Camera and Convolutional Neural Networks." Thesis, KTH, Robotik, perception och lärande, RPL, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-235496.

Full text

Abstract:

Deep learning for safe autonomous transport is rapidly emerging. Fast and robust perception for autonomous vehicles will be crucial for future navigation in urban areas with high traffic and human interplay. Previous work focuses on extracting full image depth maps, or finding specific road features such as lanes. However, in urban environments lanes are not always present, and sensors such as LiDAR with 3D point clouds provide a quite sparse depth perception of road with demanding algorithmic approaches. In this thesis we derive a novel convolutional neural network that we call AutoNet. It is designed as an encoder-decoder network for pixel-wise depth estimation of an urban drivable free-space road, using only a monocular camera, and handled as a supervised regression problem. AutoNet is also constructed as a classification network to solely classify and segment the drivable free-space in real- time with monocular vision, handled as a supervised classification problem, which shows to be a simpler and more robust solution than the regression approach. We also implement the state of the art neural network ENet for comparison, which is designed for fast real-time semantic segmentation and fast inference speed. The evaluation shows that AutoNet outperforms ENet for every performance metrics, but shows to be slower in terms of frame rate. However, optimization techniques are proposed for future work, on how to advance the frame rate of the network while still maintaining the robustness and performance. All the training and evaluation is done on the Cityscapes dataset. New ground truth labels for road depth perception are created for training with a novel approach of fusing pre-computed depth maps with semantic labels. Data collection with a Scania vehicle is conducted, mounted with a monocular camera to test the final derived models. The proposed AutoNet shows promising state of the art performance in regards to road depth estimation as well as road classification.
Deep learning för säkra autonoma transportsystem framträder mer och mer inom forskning och utveckling. Snabb och robust uppfattning om miljön för autonoma fordon kommer att vara avgörande för framtida navigering inom stadsområden med stor trafiksampel. I denna avhandling härleder vi en ny form av ett neuralt nätverk som vi kallar AutoNet. Där nätverket är designat som en autoencoder för pixelvis djupskattning av den fria körbara vägytan för stadsområden, där nätverket endast använder sig av en monokulär kamera och dess bilder. Det föreslagna nätverket för djupskattning hanteras som ett regressions problem. AutoNet är även konstruerad som ett klassificeringsnätverk som endast ska klassificera och segmentera den körbara vägytan i realtid med monokulärt seende. Där detta är hanterat som ett övervakande klassificerings problem, som även visar sig vara en mer simpel och mer robust lösning för att hitta vägyta i stadsområden. Vi implementerar även ett av de främsta neurala nätverken ENet för jämförelse. ENet är utformat för snabb semantisk segmentering i realtid, med hög prediktions- hastighet. Evalueringen av nätverken visar att AutoNet utklassar ENet i varje prestandamätning för noggrannhet, men visar sig vara långsammare med avseende på antal bilder per sekund. Olika optimeringslösningar föreslås för framtida arbete, för hur man ökar nätverk-modelens bildhastighet samtidigt som man behåller robustheten.All träning och utvärdering görs på Cityscapes dataset. Ny data för träning samt evaluering för djupskattningen för väg skapas med ett nytt tillvägagångssätt, genom att kombinera förberäknade djupkartor med semantiska etiketter för väg. Datainsamling med ett Scania-fordon utförs även, monterad med en monoculär kamera för att testa den slutgiltiga härleda modellen. Det föreslagna nätverket AutoNet visar sig vara en lovande topp-presterande modell i fråga om djupuppskattning för väg samt vägklassificering för stadsområden.

APA, Harvard, Vancouver, ISO, and other styles

38

Sommer, Nathan. "A Machine Learning Approach to Controlling Musical Synthesizer Parameters in Real-Time Live Performance." University of Cincinnati / OhioLINK, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1592168963826025.

Full text

APA, Harvard, Vancouver, ISO, and other styles

39

Masko, David. "Calibration in Eye Tracking Using Transfer Learning." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210815.

Full text

Abstract:

This thesis empirically studies transfer learning as a calibration framework for Convolutional Neural Network (CNN) based appearance-based gaze estimation models. A dataset of approximately 1,900,000 eyestripe images distributed over 1682 subjects is used to train and evaluate several gaze estimation models. Each model is initially trained on the training data resulting in generic gaze models. The models are subsequently calibrated for each test subject, using the subject's calibration data, by applying transfer learning through network fine-tuning on the final layers of the network. Transfer learning is observed to reduce the Euclidean distance error of the generic models within the range of 12-21%, which is in line with current state-of-the-art. The best performing calibrated model shows a mean error of 29.53mm and a median error of 22.77mm. However, calibrating heatmap output-based gaze estimation models decreases the performance over the generic models. It is concluded that transfer learning is a viable calibration framework for improving the performance of CNN-based appearance based gaze estimation models.
Detta examensarbete är en empirisk studie på överföringsträning som ramverk för kalibrering av neurala faltningsnätverks (CNN)-baserade bildbaserad blickapproximationsmodeller. En datamängd på omkring 1 900 000 ögonrandsbilder fördelat över 1682 personer används för att träna och bedöma flertalet blickapproximationsmodeller. Varje modell tränas inledningsvis på all träningsdata, vilket resulterar i generiska modeller. Modellerna kalibreras därefter för vardera testperson med testpersonens kalibreringsdata via överföringsträning genom anpassning av de sista lagren av nätverket. Med överföringsträning observeras en minskning av felet mätt som eukilidskt avstånd för de generiska modellerna inom 12-21%, vilket motsvarar de bästa nuvarande modellerna. För den bäst presterande kalibrerade modellen uppmäts medelfelet 29,53mm och medianfelet 22,77mm. Dock leder kalibrering av regionella sannolikhetsbaserade blickapproximationsmodeller till en försämring av prestanda jämfört med de generiska modellerna. Slutsatsen är att överföringsträning är en legitim kalibreringsansats för att förbättra prestanda hos CNN-baserade bildbaserad blickapproximationsmodeller.

APA, Harvard, Vancouver, ISO, and other styles

40

Cherif, Aymen. "Réseaux de neurones, SVM et approches locales pour la prévision de séries temporelles." Thesis, Tours, 2013. http://www.theses.fr/2013TOUR4003/document.

Full text

Abstract:

La prévision des séries temporelles est un problème qui est traité depuis de nombreuses années. On y trouve des applications dans différents domaines tels que : la finance, la médecine, le transport, etc. Dans cette thèse, on s’est intéressé aux méthodes issues de l’apprentissage artificiel : les réseaux de neurones et les SVM. On s’est également intéressé à l’intérêt des méta-méthodes pour améliorer les performances des prédicteurs, notamment l’approche locale. Dans une optique de diviser pour régner, les approches locales effectuent le clustering des données avant d’affecter les prédicteurs aux sous ensembles obtenus. Nous présentons une modification dans l’algorithme d’apprentissage des réseaux de neurones récurrents afin de les adapter à cette approche. Nous proposons également deux nouvelles techniques de clustering, la première basée sur les cartes de Kohonen et la seconde sur les arbres binaires
Time series forecasting is a widely discussed issue for many years. Researchers from various disciplines have addressed it in several application areas : finance, medical, transportation, etc. In this thesis, we focused on machine learning methods : neural networks and SVM. We have also been interested in the meta-methods to push up the predictor performances, and more specifically the local models. In a divide and conquer strategy, the local models perform a clustering over the data sets before different predictors are affected into each obtained subset. We present in this thesis a new algorithm for recurrent neural networks to use them as local predictors. We also propose two novel clustering techniques suitable for local models. The first is based on Kohonen maps, and the second is based on binary trees

APA, Harvard, Vancouver, ISO, and other styles

41

Gardner, Angelica. "Stronger Together? An Ensemble of CNNs for Deepfakes Detection." Thesis, Linnéuniversitetet, Institutionen för datavetenskap och medieteknik (DM), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-97643.

Full text

Abstract:

Deepfakes technology is a face swap technique that enables anyone to replace faces in a video, with highly realistic results. Despite its usefulness, if used maliciously, this technique can have a significant impact on society, for instance, through the spreading of fake news or cyberbullying. This makes the ability of deepfakes detection a problem of utmost importance. In this paper, I tackle the problem of deepfakes detection by identifying deepfakes forgeries in video sequences. Inspired by the state-of-the-art, I study the ensembling of different machine learning solutions built on convolutional neural networks (CNNs) and use these models as objects for comparison between ensemble and single model performances. Existing work in the research field of deepfakes detection suggests that escalated challenges posed by modern deepfake videos make it increasingly difficult for detection methods. I evaluate that claim by testing the detection performance of four single CNN models as well as six stacked ensembles on three modern deepfakes datasets. I compare various ensemble approaches to combine single models and in what way their predictions should be incorporated into the ensemble output. The results I found was that the best approach for deepfakes detection is to create an ensemble, though, the ensemble approach plays a crucial role in the detection performance. The final proposed solution is an ensemble of all available single models which use the concept of soft (weighted) voting to combine its base-learners’ predictions. Results show that this proposed solution significantly improved deepfakes detection performance and substantially outperformed all single models.

APA, Harvard, Vancouver, ISO, and other styles

42

Hassan, K. J. "Application of artificial neural networks for understanding and diagnosing the state of mastitis in dairy cattle." Lincoln University, 2007. http://hdl.handle.net/10182/633.

Full text

Abstract:

Bovine mastitis adversely affects the dairy industry around the world. This disease is caused by a diverse range of bacteria, broadly categorised as minor and major pathogens. In-line tools that help identify these bacterial groupings in the early stages of the disease are advantageous as timely decisions could be made before the cow develops any clinical symptoms. The first objective of this research was to identify the most informative milk parameters for the detection of minor and major bacterial pathogens. The second objective of this research was to evaluate the potential of supervised and unsupervised neural network learning paradigms for the detection of minor infected and major infected quarters in the early stages of the disease. The third objective was to evaluate the effects of different proportions of infected to non-infected cases in the training data set on the correct classification rate of the supervised neural network models as there are proportionately more non-infected cases in a herd than infected cases. A database developed at Lincoln University was used to achieve the research objectives. Starting at calving, quarter milk samples were collected weekly from 112 cows for a period of fourteen weeks, resulting in 4852 samples with complete records for somatic cell count (SCC), electrical resistance, protein percentage, fat percentage, and bacteriological status. To account for the effects of the stage of lactation on milk parameters with respect to days in milking, data was divided into three days in milk ranges. In addition, cow variation was accounted for by the sire family from which the cow originated and the lactation number of each cow. Data was pre-processed before the application of advanced analytical techniques. Somatic cell score (SCS) and electrical resistance index were derived from somatic cell count and electrical resistance, respectively. After pre-processing, the data was divided into training and validation sets for the unsupervised neural network modelling experiment and, for the supervised neural network modelling experiments, the data was divided into training, calibration and validation sets. Prior to any modelling experiments, the data was analysed using statistical and multivariate visualisation techniques. Correlations (p<0.05) were found between the infection status of a quarter and its somatic cell score (SCS, 0.86), electrical resistance index (ERI, -0.59) and protein percentage (PP, 0.33). The multivariate parallel visualisation analysis validated the correlation analysis. Due to significant multicolinearity [Correlations: SCS and ERI (-0.65: p<0.05); SCS and PP (0.32: p<0.05); ERI and PP (-0.35: p<0.05)], the original variables were decorrelated using principle component analysis. SCS and ERI were found to be the most informative variables for discriminating between non-infected, minor infected and major infected cases. Unsupervised neural network (USNN) model was trained using the training data set which was extracted from the database, containing approximately equal number of randomly selected records for each bacteriological status [not infected (NI), infected with a major pathogen (MJI) and infected with a minor pathogen (MNI)]. The USNN model was validated with the remaining data using the four principle components, days in milk (DIM), lactation number (LN), sire number, and bacteriological status (BS). The specificity of the USNN model in correctly identifying non infected cases was 97%. Sensitivities for correctly detecting minor and major infections were 89% and 80%, respectively. The supervised neural network (SNN) models were trained, calibrated and validated with several sets of training, calibration and validation data, which were randomly extracted from the database in such a way that each set has a different proportion of infected to non-infected cases ranging from 1:1 to 1:10. The overall accuracy of these models based on validation data sets gradually increased with increase in the number of non-infected cases in the data sets (80% for the 1:1, 84% for 1:2, 86% for 1:4 and 93% for 1:10). Specificities of the best models for correctly recognising non-infected cases for the four data sets were 82% for 1:1, 91% for 1:2, 94% for 1:4 and 98% for 1:10. Sensitivities for correctly recognising minor infected cases for the four data sets were 86% for 1:1, 76% for 1:2, 71% for 1:4 and 44% for 1:10. Sensitivities for correctly recognising major infected cases for the four data sets were 20% for 1:1, 20% for 1:2, 30% for 1:4 and 40% for 1:10. Overall, sensitivity for the minor infected cases decreased while that of major infected cases increased with increase in the number non-infected cases in the training data set. Due to the very low prevalence of MJI category in this particular herd, results for this category may be inconclusive. This research suggests that somatic cell score and electrical resistance index of milk were the most effective variables for detecting the infection status of a quarter followed by milk protein and fat percentage. The neural network models were able to differentiate milk containing minor and major bacterial pathogens based on milk parameters associated with mastitis. It is concluded that the neural network models can be developed and incorporated into milking machines to provide an efficient and effective method for the diagnosis of mastitis.

APA, Harvard, Vancouver, ISO, and other styles

43

Sonnert, Adrian. "Predicting inter-frequency measurements in an LTE network using supervised machine learning : a comparative study of learning algorithms and data processing techniques." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-148553.

Full text

Abstract:

With increasing demands on network reliability and speed, network suppliers need to effectivize their communications algorithms. Frequency measurements are a core part of mobile network communications, increasing their effectiveness would increase the effectiveness of many network processes such as handovers, load balancing, and carrier aggregation. This study examines the possibility of using supervised learning to predict the signal of inter-frequency measurements by investigating various learning algorithms and pre-processing techniques. We found that random forests have the highest predictive performance on this data set, at 90.7\% accuracy. In addition, we have shown that undersampling and varying the discriminator are effective techniques for increasing the performance on the positive class on frequencies where the negative class is prevalent. Finally, we present hybrid algorithms in which the learning algorithm for each model depends on attributes of the training data set. These algorithms perform at a much higher efficiency in terms of memory and run-time without heavily sacrificing predictive performance.

APA, Harvard, Vancouver, ISO, and other styles

44

Buttar, Sarpreet Singh. "Applying Artificial Neural Networks to Reduce the Adaptation Space in Self-Adaptive Systems : an exploratory work." Thesis, Linnéuniversitetet, Institutionen för datavetenskap och medieteknik (DM), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-87117.

Full text

Abstract:

Self-adaptive systems have limited time to adjust their configurations whenever their adaptation goals, i.e., quality requirements, are violated due to some runtime uncertainties. Within the available time, they need to analyze their adaptation space, i.e., a set of configurations, to find the best adaptation option, i.e., configuration, that can achieve their adaptation goals. Existing formal analysis approaches find the best adaptation option by analyzing the entire adaptation space. However, exhaustive analysis requires time and resources and is therefore only efficient when the adaptation space is small. The size of the adaptation space is often in hundreds or thousands, which makes formal analysis approaches inefficient in large-scale self-adaptive systems. In this thesis, we tackle this problem by presenting an online learning approach that enables formal analysis approaches to analyze large adaptation spaces efficiently. The approach integrates with the standard feedback loop and reduces the adaptation space to a subset of adaptation options that are relevant to the current runtime uncertainties. The subset is then analyzed by the formal analysis approaches, which allows them to complete the analysis faster and efficiently within the available time. We evaluate our approach on two different instances of an Internet of Things application. The evaluation shows that our approach dramatically reduces the adaptation space and analysis time without compromising the adaptation goals.

APA, Harvard, Vancouver, ISO, and other styles

45

Holm, Rasmus. "Prediction of Inter-Frequency Measurements in a LTE Network with Deep Learning." Thesis, Linköpings universitet, Statistik och maskininlärning, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-151879.

Full text

Abstract:

The telecommunications industry faces difficult challenges as more and more devices communicate over the internet. A telecommunications network is a complex system with many parts and some are candidates for further automation. We have focused on interfrequency measurements that are used during inter-frequency handovers, among other procedures. A handover is the procedure when for instance a phone changes the base station it communicates with and the inter-frequency measurements are rather expensive to perform. More specifically, we have investigated the possibility of using deep learning—an ever expanding field in machine learning—for predicting inter-frequency measurements in a Long Term Evolution (LTE) network. We have focused on the multi-layer perceptron and extended it with (variational) autoencoders or modified it through dropout such that it approximate the predictive distribution of a Gaussian process. The telecommunications network consist of many cells and each cell gather its own data. One of the strengths of deep learning models is that they usually increase their performance as more and more data is used. We have investigated whether we do see an increase in performance if we combine data from multiple cells and the results show that this is not necessarily the case. The performances are comparable between models trained on combined data from multiple cells and models trained on data from individual cells. We can expect the multi-layer perceptron to perform better than a linear regression model. The best performing multi-layer perceptron architectures have been rather shallow, 1-2 hidden layers, and the extensions/modifications we have used/done have not shown any significant improvements to warrant their presence. For the particular LTE network we have worked with we would recommend to use shallow multi-layer perceptron architectures as far as deep learning models are concerned.

APA, Harvard, Vancouver, ISO, and other styles

46

Chahla, Charbel. "Non-linear feature extraction for object re-identification in cameras networks." Thesis, Troyes, 2017. http://www.theses.fr/2017TROY0023.

Full text

Abstract:

La réplication du système visuel utilisé par le cerveau pour traiter l'information est un domaine de grand intérêt. Cette thèse se situe dans le cadre d'un système automatisé capable d'analyser les traits du visage lorsqu'une personne est proche des caméras et suivre son identité lorsque ces traits ne sont plus traçables. La première partie est consacrée aux procédures d'estimation de pose de visage pour les utiliser dans les scénarios de reconnaissance faciale. Nous avons proposé une nouvelle méthode basée sur une représentation sparse et on l'a appelé Sparse Label sensible Local Preserving Projections. Dans un environnement incontrôlé, la ré-identification de personne reposant sur des données biométriques n'est pas réalisable. Par contre, les caractéristiques basées sur l'apparence des personnes peuvent être exploitées plus efficacement. Dans ce contexte, nous proposons une nouvelle approche pour la ré-identification dans un réseau de caméras non chevauchantes. Pour fournir une mesure de similarité, chaque image est décrite par un vecteur de similarité avec une collection de prototypes. La robustesse de l'algorithme est améliorée en proposant la procédure Color Categorisation. Dans la dernière partie de cette thèse, nous proposons une architecture Siamese de deux réseaux neuronaux convolutionnels (CNN), chaque CNN étant réduit à seulement onze couches. Cette architecture permet à une machine d'être alimentée directement avec des données brutes pour faire la classification
Replicating the visual system that the brain uses to process the information is an area of substantial interest. This thesis is situated in the context of a fully automated system capable of analyzing facial features when the target is near the cameras, and tracking his identity when his facial features are no more traceable. The first part of this thesis is devoted to face pose estimation procedures to be used in face recognition scenarios. We proposed a new label-sensitive embedding based on a sparse representation called Sparse Label sensitive Locality Preserving Projections. In an uncontrolled environment observed by cameras from an unknown distance, person re-identification relying upon conventional biometrics such as face recognition is not feasible. Instead, visual features based on the appearance of people can be exploited more reliably. In this context, we propose a new embedding scheme for single-shot person re-identification under non overlapping target cameras. Each person is described as a vector of kernel similarities to a collection of prototype person images. The robustness of the algorithm is improved by proposing the Color Categorization procedure. In the last part of this thesis, we propose a Siamese architecture of two Convolutional Neural Networks (CNN), with each CNN reduced to only eleven layers. This architecture allows a machine to be fed directly with raw data and to automatically discover the representations needed for classification

APA, Harvard, Vancouver, ISO, and other styles

47

Santos, Rosiane Correia. "LearnInPlanner: uma abordagem de aprendizado supervisionado com redes neurais para solução de problemas de planejamento clássico." Universidade de São Paulo, 2013. http://www.teses.usp.br/teses/disponiveis/100/100131/tde-25012014-115621/.

Full text

Abstract:

A busca progressiva no espaço de estados é uma das abordagens mais populares de Planejamento Automatizado. O desempenho dos algoritmos de busca progressiva é influenciado pela heurística independente de domínio utilizada para guiá-lo. Nesse contexto, o foco do presente trabalho consiste em investigar técnicas de aprendizado de máquina supervisionadas que possibilitaram agregar à heurística do plano relaxado, comumente utilizada em abordagens atuais de planejamento, informações sobre o domínio em questão que viessem a ser úteis ao algoritmo de busca. Essas informações foram representadas por meio de um espaço de características do problema de planejamento e uma rede neural MLP foi aplicada para estimar uma nova função heurística para guiar a busca por meio de um processo de regressão não linear. Uma vez que o conjunto de características disponíveis para a construção da nova função heurística é grande, foi necessário a definição de um processo de seleção de características capaz de determinar qual conjunto de características de entrada da rede resultaria em melhor desempenho para o modelo de regressão. Portanto, para a seleção de características, aplicou-se uma abordagem de algoritmos genéticos. Como principal resultado, tem-se uma análise comparativa do desempenho entre a utilização da heurística proposta neste trabalho e a utilização da heurística do plano relaxado para guiar o algoritmo de busca na tarefa de planejamento. Para a análise empírica foram utilizados domínios de diferentes complexidades disponibilizados pela Competições Internacionais de Planejamento. Além dos resultados empíricos e análises comparativas, as contribuições deste trabalho envolvem o desenvolvimento de um novo planejador independente de domínio, denominado LearnInPlanner. Esse planejador utiliza a nova função heurística estimada por meio do processo de aprendizado e o algoritmo de Busca Gulosa para solucionar os problemas de planejamento.
The forward state-space search is one of the most popular Automated Planning approaches. The performance of forward search algorithms is affected by the domain-independent heuristic being used. In this context, the focus of this work consisted on investigating techniques of supervised machine learning that make possible to agregate to the relaxed plan heuristic, commonly used in current planning approaches, information about the domain which could be useful to the search algorithm. This information has been represented through a feature space of planning problem and a MLP neural network has been applied to estimate a new heuristic function for guiding the search through a non-linear regression process. Once the set of features available for the construction of the new heuristic function is large, it was necessary to define a feature selection process capable of determining which set of neural network input features would result in the best performance for the regression model. Therefore, for selecting features, an approach of genetic algorithms has been applied. As the main result, one has obtained a comparative performance analysis between the use of heuristic proposed in this work and the use of the relaxed plan heuristic to guide the search algorithm in the planning task. For the empirical analysis were used domains with different complexities provided by the International Planning Competitions. In addition to the empirical results and comparative analysis, the contributions of this work involves the development of a new domain-independent planner, named LearnInPlanner. This planner uses the new heuristic function estimated by the learning process and the Greedy Best-First search algorithm to solve planning problems.

APA, Harvard, Vancouver, ISO, and other styles

48

Tran, Khanh-Hung. "Semi-supervised dictionary learning and Semi-supervised deep neural network." Thesis, université Paris-Saclay, 2021. http://www.theses.fr/2021UPASP014.

Full text

Abstract:

Depuis les années 2010, l’apprentissage automatique (ML) est l’un des sujets qui retient beaucoup l'attention des chercheurs scientifiques. De nombreux modèles de ML ont démontré leur capacité produire d’excellent résultats dans des divers domaines comme Vision par ordinateur, Traitement automatique des langues, Robotique… Toutefois, la plupart de ces modèles emploient l’apprentissage supervisé, qui requiert d’un massive annotation. Par conséquent, l’objectif de cette thèse est d’étudier et de proposer des approches semi-supervisées qui ont plusieurs avantages par rapport à l’apprentissage supervisé. Au lieu d’appliquer directement un classificateur semi-supervisé sur la représentation originale des données, nous utilisons plutôt des types de modèle qui intègrent une phase de l’apprentissage de représentation avant de la phase de classification, pour mieux s'adapter à la non linéarité des données. Dans le premier temps, nous revisitons des outils qui permettent de construire notre modèles semi-supervisés. Tout d’abord, nous présentons deux types de modèle qui possèdent l’apprentissage de représentation dans leur architecture : l’apprentissage de dictionnaire et le réseau de neurones, ainsi que les méthodes d’optimisation pour chaque type de model, en plus, dans le cas de réseau de neurones, nous précisons le problème avec les exemples contradictoires. Ensuite, nous présentons les techniques qui accompagnent souvent avec l’apprentissage semi-supervisé comme l’apprentissage de variétés et le pseudo-étiquetage. Dans le deuxième temps, nous travaillons sur l’apprentissage de dictionnaire. Nous synthétisons en général trois étapes pour construire un modèle semi-supervisée à partir d’un modèle supervisé. Ensuite, nous proposons notre modèle semi-supervisée pour traiter le problème de classification typiquement dans le cas d’un faible nombre d’échantillons d’entrainement (y compris tous labellisés et non labellisés échantillons). D'une part, nous appliquons la préservation de la structure de données de l’espace original à l’espace de code parcimonieux (l’apprentissage de variétés), ce qui est considéré comme la régularisation pour les codes parcimonieux. D'autre part, nous intégrons un classificateur semi-supervisé dans l’espace de code parcimonieux. En outre, nous effectuons le codage parcimonieux pour les échantillons de test en prenant en compte aussi la préservation de la structure de données. Cette méthode apporte une amélioration sur le taux de précision par rapport à des méthodes existantes. Dans le troisième temps, nous travaillons sur le réseau de neurones. Nous proposons une approche qui s’appelle "manifold attack" qui permets de renforcer l’apprentissage de variétés. Cette approche est inspirée par l’apprentissage antagoniste : trouver des points virtuels qui perturbent la fonction de coût sur l’apprentissage de variétés (en la maximisant) en fixant les paramètres du modèle; ensuite, les paramètres du modèle sont mis à jour, en minimisant cette fonction de coût et en fixant les points virtuels. Nous fournissons aussi des critères pour limiter l’espace auquel les points virtuels appartiennent et la méthode pour les initialiser. Cette approche apporte non seulement une amélioration sur le taux de précision mais aussi une grande robustesse contre les exemples contradictoires. Enfin, nous analysons des similarités et des différences, ainsi que des avantages et inconvénients entre l’apprentissage de dictionnaire et le réseau de neurones. Nous proposons quelques perspectives sur ces deux types de modèle. Dans le cas de l’apprentissage de dictionnaire semi-supervisé, nous proposons quelques techniques en inspirant par le réseau de neurones. Quant au réseau de neurones, nous proposons d’intégrer "manifold attack" sur les modèles génératifs
Since the 2010's, machine learning (ML) has been one of the topics that attract a lot of attention from scientific researchers. Many ML models have been demonstrated their ability to produce excellent results in various fields such as Computer Vision, Natural Language Processing, Robotics... However, most of these models use supervised learning, which requires a massive annotation. Therefore, the objective of this thesis is to study and to propose semi-supervised learning approaches that have many advantages over supervised learning. Instead of directly applying a semi-supervised classifier on the original representation of data, we rather use models that integrate a representation learning stage before the classification stage, to better adapt to the non-linearity of the data. In the first step, we revisit tools that allow us to build our semi-supervised models. First, we present two types of model that possess representation learning in their architecture: dictionary learning and neural network, as well as the optimization methods for each type of model. Moreover, in the case of neural network, we specify the problem with adversarial examples. Then, we present the techniques that often accompany with semi-supervised learning such as variety learning and pseudo-labeling. In the second part, we work on dictionary learning. We synthesize generally three steps to build a semi-supervised model from a supervised model. Then, we propose our semi-supervised model to deal with the classification problem typically in the case of a low number of training samples (including both labelled and non-labelled samples). On the one hand, we apply the preservation of the data structure from the original space to the sparse code space (manifold learning), which is considered as regularization for sparse codes. On the other hand, we integrate a semi-supervised classifier in the sparse code space. In addition, we perform sparse coding for test samples by taking into account also the preservation of the data structure. This method provides an improvement on the accuracy rate compared to other existing methods. In the third step, we work on neural network models. We propose an approach called "manifold attack" which allows reinforcing manifold learning. This approach is inspired from adversarial learning : finding virtual points that disrupt the cost function on manifold learning (by maximizing it) while fixing the model parameters; then the model parameters are updated by minimizing this cost function while fixing these virtual points. We also provide criteria for limiting the space to which the virtual points belong and the method for initializing them. This approach provides not only an improvement on the accuracy rate but also a significant robustness to adversarial examples. Finally, we analyze the similarities and differences, as well as the advantages and disadvantages between dictionary learning and neural network models. We propose some perspectives on both two types of models. In the case of semi-supervised dictionary learning, we propose some techniques inspired by the neural network models. As for the neural network, we propose to integrate manifold attack on generative models

APA, Harvard, Vancouver, ISO, and other styles

49

Gomes, Leonaldo da Silva. "Redes Neurais Aplicadas Ã InferÃncia dos Sinais de Controle de Dosagem de Coagulantes em uma ETA por FiltraÃÃo RÃpida." Universidade Federal do CearÃ, 2012. http://www.teses.ufc.br/tde_busca/arquivo.php?codArquivo=8105.

Full text

Abstract:

Considerando a importÃncia do controle da coagulaÃÃo quÃmica para o processo de tratamento de Ãgua por filtraÃÃo rÃpida, esta dissertaÃÃo propÃe a aplicaÃÃo de redes neurais artificiais para inferÃncia dos sinais de controle de dosagem de coagulantes principal e auxiliar, no processo de coagulaÃÃo quÃmica em uma estaÃÃo de tratamento de Ãgua por filtraÃÃo rÃpida. Para tanto, foi feito uma anÃlise comparativa da aplicaÃÃo de modelos baseados em redes neurais do tipo: alimentada adiante focada atrasada no tempo (FTLFN); alimentada adiante atrasada no tempo distribuÃda (DTLFN); recorrente de Elman (ERN) e auto-regressiva nÃo-linear com entradas exÃgenas (NARX). Da anÃlise comparativa, o modelo baseado em redes NARX apresentou melhores resultados, evidenciando o potencial do modelo para uso em casos reais, o que contribuirÃ para a viabilizaÃÃo de projetos desta natureza em estaÃÃes de tratamento de Ãgua de pequeno porte.
Considering the importance of the chemical coagulation control for the water treatment by direct filtration, this work proposes the application of artificial neural networks for inference of dosage control signals of principal and auxiliary coagulant, in the chemical coagulation process in a water treatment plant by direct filtration. To that end, was made a comparative analysis of the application of models based on neural networks, such as: Focused Time Lagged Feedforward Network (FTLFN); Distributed Time Lagged Feedforward Network (DTLFN); Elman Recurrent Network (ERN) and Non-linear Autoregressive with exogenous inputs (NARX). From the comparative analysis, the model based on NARX networks showed better results, demonstrating the potential of the model for use in real cases, which will contribute to the viability of projects of this nature in small size water treatment plants.

APA, Harvard, Vancouver, ISO, and other styles

50

Choi, Jin-Woo. "Action Recognition with Knowledge Transfer." Diss., Virginia Tech, 2021. http://hdl.handle.net/10919/101780.

Full text

Abstract:

Recent progress on deep neural networks has shown remarkable action recognition performance from videos. The remarkable performance is often achieved by transfer learning: training a model on a large-scale labeled dataset (source) and then fine-tuning the model on the small-scale labeled datasets (targets). However, existing action recognition models do not always generalize well on new tasks or datasets because of the following two reasons. i) Current action recognition datasets have a spurious correlation between action types and background scene types. The models trained on these datasets are biased towards the scene instead of focusing on the actual action. This scene bias leads to poor generalization performance. ii) Directly testing the model trained on the source data on the target data leads to poor performance as the source, and target distributions are different. Fine-tuning the model on the target data can mitigate this issue. However, manual labeling small- scale target videos is labor-intensive. In this dissertation, I propose solutions to these two problems. For the first problem, I propose to learn scene-invariant action representations to mitigate the scene bias in action recognition models. Specifically, I augment the standard cross-entropy loss for action classification with 1) an adversarial loss for the scene types and 2) a human mask confusion loss for videos where the human actors are invisible. These two losses encourage learning representations unsuitable for predicting 1) the correct scene types and 2) the correct action types when there is no evidence. I validate the efficacy of the proposed method by transfer learning experiments. I trans- fer the pre-trained model to three different tasks, including action classification, temporal action localization, and spatio-temporal action detection. The results show consistent improvement over the baselines for every task and dataset. I formulate human action recognition as an unsupervised domain adaptation (UDA) problem to handle the second problem. In the UDA setting, we have many labeled videos as source data and unlabeled videos as target data. We can use already exist- ing labeled video datasets as source data in this setting. The task is to align the source and target feature distributions so that the learned model can generalize well on the target data. I propose 1) aligning the more important temporal part of each video and 2) encouraging the model to focus on action, not the background scene, to learn domain-invariant action representations. The proposed method is simple and intuitive while achieving state-of-the-art performance without training on a lot of labeled target videos. I relax the unsupervised target data setting to a sparsely labeled target data setting. Then I explore the semi-supervised video action recognition, where we have a lot of labeled videos as source data and sparsely labeled videos as target data. The semi-supervised setting is practical as sometimes we can afford a little bit of cost for labeling target data. I propose multiple video data augmentation methods to inject photometric, geometric, temporal, and scene invariances to the action recognition model in this setting. The resulting method shows favorable performance on the public benchmarks.
Doctor of Philosophy
Recent progress on deep learning has shown remarkable action recognition performance. The remarkable performance is often achieved by transferring the knowledge learned from existing large-scale data to the small-scale data specific to applications. However, existing action recog- nition models do not always work well on new tasks and datasets because of the following two problems. i) Current action recognition datasets have a spurious correlation between action types and background scene types. The models trained on these datasets are biased towards the scene instead of focusing on the actual action. This scene bias leads to poor performance on the new datasets and tasks. ii) Directly testing the model trained on the source data on the target data leads to poor performance as the source, and target distributions are different. Fine-tuning the model on the target data can mitigate this issue. However, manual labeling small-scale target videos is labor-intensive. In this dissertation, I propose solutions to these two problems. To tackle the first problem, I propose to learn scene-invariant action representations to mitigate background scene- biased human action recognition models for the first problem. Specifically, the proposed method learns representations that cannot predict the scene types and the correct actions when there is no evidence. I validate the proposed method's effectiveness by transferring the pre-trained model to multiple action understanding tasks. The results show consistent improvement over the baselines for every task and dataset. To handle the second problem, I formulate human action recognition as an unsupervised learning problem on the target data. In this setting, we have many labeled videos as source data and unlabeled videos as target data. We can use already existing labeled video datasets as source data in this setting. The task is to align the source and target feature distributions so that the learned model can generalize well on the target data. I propose 1) aligning the more important temporal part of each video and 2) encouraging the model to focus on action, not the background scene. The proposed method is simple and intuitive while achieving state-of-the-art performance without training on a lot of labeled target videos. I relax the unsupervised target data setting to a sparsely labeled target data setting. Here, we have many labeled videos as source data and sparsely labeled videos as target data. The setting is practical as sometimes we can afford a little bit of cost for labeling target data. I propose multiple video data augmentation methods to inject color, spatial, temporal, and scene invariances to the action recognition model in this setting. The resulting method shows favorable performance on the public benchmarks.

APA, Harvard, Vancouver, ISO, and other styles

Dissertations / Theses on the topic 'Supervised neural networks'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles