Dissertationen zum Thema „Self-supervised learninig“

The aim of the diploma thesis is to make research of the self-supervised learning in computer vision applications, then to choose a suitable test task with an extensive data set, apply self-supervised methods and evaluate. The theoretical part of the work is focused on the description of methods in computer vision, a detailed description of neural and convolution networks and an extensive explanation and division of self-supervised methods. Conclusion of the theoretical part is devoted to practical applications of the Self-supervised methods in practice. The practical part of the diploma thesis deals with the description of the creation of code for working with datasets and the application of the SSL methods Rotation, SimCLR, MoCo and BYOL in the role of classification and semantic segmentation. Each application of the method is explained in detail and evaluated for various parameters on the large STL10 dataset. Subsequently, the success of the methods is evaluated for different datasets and the limiting conditions in the classification task are named. The practical part concludes with the application of SSL methods for pre-training the encoder in the application of semantic segmentation with the Cityscapes dataset.

In speaker recognition, i-vectors have been the state-of-the-art unsupervised technique over the last few years, whereas x-vectors is becoming the state-of-the-art supervised technique, these days. Recent advances in Deep Learning (DL) approaches to speaker recognition have improved the performance but are constrained to the need of labels for the background data. In practice, labeled background data is not easily accessible, especially when large training data is required. In i-vector based speaker recognition, cosine and Probabilistic Linear Discriminant Analysis (PLDA) are the two basic scoring techniques. Cosine scoring is unsupervised whereas PLDA parameters are typically trained using speaker-labeled background data. This makes a big performance gap between these two scoring techniques. The question is: how to fill this performance gap without using speaker labels for the background data? In this thesis, the above mentioned problem has been addressed using DL approaches without using and/or limiting the use of labeled background data. Three DL based proposals have been made. In the first proposal, a Restricted Boltzmann Machine (RBM) vector representation of speech is proposed for the tasks of speaker clustering and tracking in TV broadcast shows. This representation is referred to as RBM vector. The experiments on AGORA database show that in speaker clustering the RBM vectors gain a relative improvement of 12% in terms of Equal Impurity (EI). For speaker tracking task RBM vectors are used only in the speaker identification part, where the relative improvement in terms of Equal Error Rate (EER) is 11% and 7% using cosine and PLDA scoring, respectively. In the second proposal, DL approaches are proposed in order to increase the discriminative power of i-vectors in speaker verification. We have proposed the use of autoencoder in several ways. Firstly, an autoencoder will be used as a pre-training for a Deep Neural Network (DNN) using a large amount of unlabeled background data. Then, a DNN classifier will be trained using relatively small labeled data. Secondly, an autoencoder will be trained to transform i-vectors into a new representation to increase their discriminative power. The training will be carried out based on the nearest neighbor i-vectors which will be chosen in an unsupervised manner. The evaluation was performed on VoxCeleb-1 database. The results show that using the first system, we gain a relative improvement of 21% in terms of EER, over i-vector/PLDA. Whereas, using the second system, a relative improvement of 42% is gained. If we use the background data in the testing part, a relative improvement of 53% is gained. In the third proposal, we will train a self-supervised end-to-end speaker verification system. The idea is to utilize impostor samples along with the nearest neighbor samples to make client/impostor pairs in an unsupervised manner. The architecture will be based on a Convolutional Neural Network (CNN) encoder, trained as a siamese network with two branch networks. Another network with three branches will also be trained using triplet loss, in order to extract unsupervised speaker embeddings. The experimental results show that both the end-to-end system and the speaker embeddings, despite being unsupervised, show a comparable performance to the supervised baseline. Moreover, their score combination can further improve the performance. The proposed approaches for speaker verification have respective pros and cons. The best result was obtained using the nearest neighbor autoencoder with a disadvantage of relying on background i-vectors in the testing. On the contrary, the autoencoder pre-training for DNN is not bound by this factor but is a semi-supervised approach. The third proposal is free from both these constraints and performs pretty reasonably. It is a self-supervised approach and it does not require the background i-vectors in the testing phase.
Los avances recientes en Deep Learning (DL) para el reconocimiento del hablante están mejorado el rendimiento de los sistemas tradicionales basados en i-vectors. En el reconocimiento de locutor basado en i-vectors, la distancia coseno y el análisis discriminante lineal probabilístico (PLDA) son las dos técnicas más usadas de puntuación. La primera no es supervisada, pero la segunda necesita datos etiquetados por el hablante, que no son siempre fácilmente accesibles en la práctica. Esto crea una gran brecha de rendimiento entre estas dos técnicas de puntuación. La pregunta es: ¿cómo llenar esta brecha de rendimiento sin usar etiquetas del hablante en los datos de background? En esta tesis, el problema anterior se ha abordado utilizando técnicas de DL sin utilizar y/o limitar el uso de datos etiquetados. Se han realizado tres propuestas basadas en DL. En la primera, se propone una representación vectorial de voz basada en la máquina de Boltzmann restringida (RBM) para las tareas de agrupación de hablantes y seguimiento de hablantes en programas de televisión. Los experimentos en la base de datos AGORA, muestran que en agrupación de hablantes los vectores RBM suponen una mejora relativa del 12%. Y, por otro lado, en seguimiento del hablante, los vectores RBM,utilizados solo en la etapa de identificación del hablante, muestran una mejora relativa del 11% (coseno) y 7% (PLDA). En la segunda, se utiliza DL para aumentar el poder discriminativo de los i-vectors en la verificación del hablante. Se ha propuesto el uso del autocodificador de varias formas. En primer lugar, se utiliza un autocodificador como preentrenamiento de una red neuronal profunda (DNN) utilizando una gran cantidad de datos de background sin etiquetar, para posteriormente entrenar un clasificador DNN utilizando un conjunto reducido de datos etiquetados. En segundo lugar, se entrena un autocodificador para transformar i-vectors en una nueva representación para aumentar el poder discriminativo de los i-vectors. El entrenamiento se lleva a cabo en base a los i-vectors vecinos más cercanos, que se eligen de forma no supervisada. La evaluación se ha realizado con la base de datos VoxCeleb-1. Los resultados muestran que usando el primer sistema obtenemos una mejora relativa del 21% sobre i-vectors, mientras que usando el segundo sistema, se obtiene una mejora relativa del 42%. Además, si utilizamos los datos de background en la etapa de prueba, se obtiene una mejora relativa del 53%. En la tercera, entrenamos un sistema auto-supervisado de verificación de locutor de principio a fin. Utilizamos impostores junto con los vecinos más cercanos para formar pares cliente/impostor sin supervisión. La arquitectura se basa en un codificador de red neuronal convolucional (CNN) que se entrena como una red siamesa con dos ramas. Además, se entrena otra red con tres ramas utilizando la función de pérdida triplete para extraer embeddings de locutores. Los resultados muestran que tanto el sistema de principio a fin como los embeddings de locutores, a pesar de no estar supervisados, tienen un rendimiento comparable a una referencia supervisada. Cada uno de los enfoques propuestos tienen sus pros y sus contras. El mejor resultado se obtuvo utilizando el autocodificador con el vecino más cercano, con la desventaja de que necesita los i-vectors de background en el test. El uso del preentrenamiento del autocodificador para DNN no tiene este problema, pero es un enfoque semi-supervisado, es decir, requiere etiquetas de hablantes solo de una parte pequeña de los datos de background. La tercera propuesta no tienes estas dos limitaciones y funciona de manera razonable. Es un en

Semi-supervised self-learning algorithms have been shown to improve classifier accuracy under a variety of conditions. In this thesis, semi-supervised self-learning using ensembles of random forests and fuzzy c-means clustering similarity was applied to three data sets to show where improvement is possible over random forests alone. Two of the data sets are emulations of large simulations in which the data may be distributed. Additionally, the ratio of majority to minority class examples in the training set was altered to examine the effect of training set bias on performance when applying the semi-supervised algorithm.

Automotive technologies and fully autonomous driving have seen a tremendous growth in recent times and have benefitted from extensive deep learning research. State-of-the-art deep learning methods are largely supervised and require labelled data for training. However, the annotation process for image data is time-consuming and costly in terms of human efforts. It is of interest to find informative samples for labelling by Content Based Image Retrieval (CBIR). Generally, a CBIR method takes a query image as input and returns a set of images that are semantically similar to the query image. The image retrieval is achieved by transforming images to feature representations in a latent space, where it is possible to reason about image similarity in terms of image content. In this thesis, a self-supervised method is developed to learn feature representations of road scenes images. The self-supervised method learns feature representations for images by adapting intermediate convolutional features from an existing deep Convolutional Neural Network (CNN). A contrastive approach based on Noise Contrastive Estimation (NCE) is used to train the feature learning model. For complex images like road scenes where mutiple image aspects can occur simultaneously, it is important to embed all the salient image aspects in the feature representation. To achieve this, the output feature representation is obtained as an ensemble of feature embeddings which are learned by focusing on different image aspects. An attention mechanism is incorporated to encourage each ensemble member to focus on different image aspects. For comparison, a self-supervised model without attention is considered and a simple dimensionality reduction approach using SVD is treated as the baseline. The methods are evaluated on nine different evaluation datasets using CBIR performance metrics. The datasets correspond to different image aspects and concern the images at different spatial levels - global, semi-global and local. The feature representations learned by self-supervised methods are shown to perform better than the SVD approach. Taking into account that no labelled data is required for training, learning representations for road scenes images using self-supervised methods appear to be a promising direction. Usage of multiple query images to emphasize a query intention is investigated and a clear improvement in CBIR performance is observed. It is inconclusive whether the addition of an attentive mechanism impacts CBIR performance. The attention method shows some positive signs based on qualitative analysis and also performs better than other methods for one of the evaluation datasets containing a local aspect. This method for learning feature representations is promising but requires further research involving more diverse and complex image aspects.

Visual odometry is one of the prevalent techniques for the positioning of autonomous agents equipped with cameras. Several recent works in this field have in various ways attempted to exploit the capabilities of deep neural networks to improve the performance of visual odometry solutions. One of such approaches is using an end-to-end learning-based solution to infer the egomotion of the camera from a sequence of input images. The state of the art end-to-end solutions employ a common self-supervised training strategy that minimises a notion of photometric error formed by the view synthesis of the input images. As this error is a function of the predicted egomotion, its minimisation corresponds to the learning of egomotion estimation by the network. However, this also requires the depth information of the images, for which an additional depth estimation network is introduced in training. This implies that for end-to-end learning of camera egomotion, a set of parameters are required to be learned, which are not used in inference. In this work, we propose a novel learning strategy using epipolar geometry, which does not rely on depth estimations. Empirical evaluation of our method demonstrates its comparable performance to the baseline work that relies on explicit depth estimations for training.
Visuell odometri är en av de vanligast förekommande teknikerna för positionering av autonoma agenter utrustade med kameror. Flera senare arbeten inom detta område har på olika sätt försökt utnyttja kapaciteten hos djupa neurala nätverk för att förbättra prestandan hos lösningar baserade på visuell odometri. Ett av dessa tillvägagångssätt består i att använda en inlärningsbaserad lösning för att härleda kamerans rörelse utifrån en sekvens av bilder. Gemensamt för de flesta senare lösningar är en självövervakande träningsstrategi som minimerar det uppfattade fotometriska fel som uppskattas genom att syntetisera synvinkeln utifrån givna bildsekvenser. Eftersom detta fel är en funktion av den estimerade kamerarörelsen motsvarar minimering av felet att nätverket lär sig uppskatta kamerarörelsen. Denna inlärning kräver dock även information om djupet i bilderna, vilket fås genom att introducera ett nätverk specifikt för estimering av djup. Detta innebär att för uppskattning av kamerans rörelse krävs inlärning av ytterligare en uppsättning parametrar vilka inte används i den slutgiltiga uppskattningen. I detta arbete föreslår vi en ny inlärningsstrategi baserad på epipolär geometri, vilket inte beror på djupskattningar. Empirisk utvärdering av vår metod visar att dess resultat är jämförbara med tidigare metoder som använder explicita djupskattningar för träning.

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Includes bibliographical references (leaves 178-192).
This thesis presents a self-supervised framework for perceptual and motor learning based upon correlations in different sensory modalities. The brain and cognitive sciences have gathered an enormous body of neurological and phenomenological evidence in the past half century demonstrating the extraordinary degree of interaction between sensory modalities during the course of ordinary perception. We develop a framework for creating artificial perceptual systems that draws on these findings, where the primary architectural motif is the cross-modal transmission of perceptual information to enhance each sensory channel individually. We present self-supervised algorithms for learning perceptual grounding, intersensory influence, and sensorymotor coordination, which derive training signals from internal cross-modal correlations rather than from external supervision. Our goal is to create systems that develop by interacting with the world around them, inspired by development in animals. We demonstrate this framework with: (1) a system that learns the number and structure of vowels in American English by simultaneously watching and listening to someone speak. The system then cross-modally clusters the correlated auditory and visual data.
(cont.) It has no advance linguistic knowledge and receives no information outside of its sensory channels. This work is the first unsupervised acquisition of phonetic structure of which we are aware, outside of that done by human infants. (2) a system that learns to sing like a zebra finch, following the developmental stages of a juvenile zebra finch. It first learns the song of an adult male and then listens to its own initially nascent attempts at mimicry through an articulatory synthesizer. In acquiring the birdsong to which it was initially exposed, this system demonstrates self-supervised sensorimotor learning. It also demonstrates afferent and efferent equivalence - the system learns motor maps with the same computational framework used for learning sensory maps.
by Michael Harlan Coen.
Ph.D.

With the ever growing media and music catalogs, tools that search and navigate this data are important. For more complex search queries, meta-data is needed, but to manually label the vast amounts of new content is impossible. In this thesis, automatic labeling of musical instrument activities in song mixes is investigated, with a focus on ways to alleviate the lack of annotated data for instrument activity detection models. Two methods for alleviating the problem of small amounts of data are proposed and evaluated. Firstly, a self-supervised approach based on automatic labeling and mixing of randomized instrument stems is investigated. Secondly, a domain-adaptation approach that trains models on sampled MIDI files for instrument activity detection on recorded music is explored. The self-supervised approach yields better results compared to the baseline and points to the fact that deep learning models can learn instrument activity detection without an intrinsic musical structure in the audio mix. The domain-adaptation models trained solely on sampled MIDI files performed worse than the baseline, however using MIDI data in conjunction with recorded music boosted the performance. A hybrid model combining both self-supervised learning and domain adaptation by using both sampled MIDI data and recorded music produced the best results overall.
I och med de ständigt växande media- och musikkatalogerna krävs verktyg för att söka och navigera i dessa. För mer komplexa sökförfrågningar så behövs det metadata, men att manuellt annotera de enorma mängderna av ny data är omöjligt. I denna uppsats undersöks automatisk annotering utav instrumentsaktivitet inom musik, med ett fokus på bristen av annoterad data för modellerna för instrumentaktivitetsigenkänning. Två metoder för att komma runt bristen på data föreslås och undersöks. Den första metoden bygger på självövervakad inlärning baserad på automatisk annotering och slumpartad mixning av olika instrumentspår. Den andra metoden använder domänadaption genom att träna modeller på samplade MIDI-filer för detektering av instrument i inspelad musik. Metoden med självövervakning gav bättre resultat än baseline och pekar på att djupinlärningsmodeller kan lära sig instrumentigenkänning trots att ljudmixarna saknar musikalisk struktur. Domänadaptionsmodellerna som endast var tränade på samplad MIDI-data presterade sämre än baseline, men att använda MIDI-data tillsammans med data från inspelad musik gav förbättrade resultat. En hybridmodell som kombinerade både självövervakad inlärning och domänadaption genom att använda både samplad MIDI-data och inspelad musik gav de bästa resultaten totalt.

Depth detection is a very common computer vision problem. It shows up primarily in robotics, automation, or 3D visualization domains, as it is essential for converting images to point clouds. One of the poster child applications is self driving cars. Currently, the best methods for depth detection are either very expensive, like LIDAR, or require precise calibration, like stereo cameras. These costs have given rise to attempts to detect depth from a monocular camera (a single camera). While this is possible, it is harder than LIDAR or stereo methods since depth can't be measured from monocular images, it has to be inferred. A good example is covering one eye: you still have some idea how far away things are, but it's not exact. Neural networks are a natural fit for this. Here, we build on previous neural network methods by applying a recent state of the art model to panoramic images in addition to pinhole ones and performing a comparative evaluation. First, we create a simulated depth detection dataset that lends itself to panoramic comparisons and contains pre-made cylindrical and spherical panoramas. We then modify monodepth2 to support cylindrical and cubemap panoramas, incorporating current best practices for depth detection on those panorama types, and evaluate its performance for each type of image using our dataset. We also consider the resources used in training and other qualitative factors.

Dissertação para obtenção do Grau de Mestre em Engenharia Eletrotécnica e de Computadores
This dissertation presents a system for haptic interaction and self-supervised learning mechanisms to ascertain navigation affordances from depth cues. A simple pan-tilt telescopic arm and a structured light sensor, both fitted to the robot’s body frame, provide the required haptic and depth sensory feedback. The system aims at incrementally develop the ability to assess the cost of navigating in natural environments. For this purpose the robot learns a mapping between the appearance of objects, given sensory data provided by the sensor, and their bendability, perceived by the pan-tilt telescopic arm. The object descriptor, representing the object in memory and used for comparisons with other objects, is rich for a robust comparison and simple enough to allow for fast computations. The output of the memory learning mechanism allied with the haptic interaction point evaluation prioritize interaction points to increase the confidence on the interaction and correctly identifying obstacles, reducing the risk of the robot getting stuck or damaged. If the system concludes that the object is traversable, the environment change detection system allows the robot to overcome it. A set of field trials show the ability of the robot to progressively learn which elements of environment are traversable.

Due to the long duration required to perform manual knowledge entry by human knowledge engineers it is desirable to find methods to automatically acquire knowledge about the world by accessing online information. In this work I examine using the Cyc ontology to guide the creation of Naïve Bayes classifiers to provide knowledge about items described in Wikipedia articles. Given an initial set of Wikipedia articles the system uses the ontology to create positive and negative training sets for the classifiers in each category. The order in which classifiers are generated and used to test articles is also guided by the ontology. The research conducted shows that a system can be created that utilizes statistical text classification methods to extract information from an ad-hoc generated information source like Wikipedia for use in a formal semantic ontology like Cyc. Benefits and limitations of the system are discussed along with future work.

Molecular property prediction has a vast range of applications in the chemical industry. A powerful molecular property prediction model can promote experiments and production processes. The idea behind this degree program lies in the use of transfer learning to predict molecular properties. The project is divided into two parts. The first part is to build and pre-train the model. The model, which is constructed with pure attention-based Transformer Layer, is pre-trained through a Masked Edge Recovery task with large-scale unlabeled data. Then, the performance of this pre- trained model is tested with different molecular property prediction tasks and finally verifies the effectiveness of transfer learning.The results show that after self-supervised pre-training, this model shows its excellent generalization capability. It is possible to be fine-tuned with a short period and performs well in downstream tasks. And the effectiveness of transfer learning is reflected in the experiment as well. The pre-trained model not only shortens the task- specific training time but also obtains better performance and avoids overfitting due to too little training data for molecular property prediction.
Prediktion av molekylers egenskaper har en stor mängd tillämpningar inom kemiindustrin. Kraftfulla metoder för att predicera molekylära egenskaper kan främja vetenskapliga experiment och produktionsprocesser. Ansatsen i detta arbete är att använda överförd inlärning (eng. transfer learning) för att predicera egenskaper hos molekyler. Projektet är indelat i två delar. Den första delen fokuserar på att utveckla och förträna en modell. Modellen består av Transformer-lager med attention- mekanismer och förtränas genom att återställa maskerade kanter i molekylgrafer från storskaliga mängder icke-annoterad data. Efteråt utvärderas prestandan hos den förtränade modellen i en mängd olika uppgifter baserade på prediktion av molekylegenskaper vilket bekräftar fördelen med överförd inlärning.Resultaten visar att modellen efter självövervakad förträning besitter utmärkt förmåga till att generalisera. Den kan finjusteras med liten tidskostnad och presterar väl i specialiserade uppgifter. Effektiviteten hos överförd inlärning visas också i experimenten. Den förtränade modellen förkortar inte bara tiden för uppgifts-specifik inlärning utan uppnår även bättre prestanda och undviker att övertränas på grund otillräckliga mängder data i uppgifter för prediktion av molekylegenskaper.

Hyperspectral imaging is an expanding topic within the field of computer vision, that uses images of high spectral granularity. Contrastive learning is a discrim- inative approach to self-supervised learning, a form of unsupervised learning where the network is trained using self-created pseudo-labels. This work com- bines these two research areas and investigates how a pretrained network based on contrastive learning can be used for hyperspectral images. The hyperspectral images used in this work are generated from simulated RGB images and spec- tra from a spectral library. The network is trained with a pretext task based on data augmentations, and is evaluated through transfer learning and fine-tuning for a downstream task. The goal is to determine the impact of the pretext task on the downstream task and to determine the required amount of labelled data. The results show that the downstream task (a classifier) based on the pretrained network barely performs better than a classifier without a pretrained network. In the end, more research needs to be done to confirm or reject the benefit of a pretrained network based on contrastive learning for hyperspectral images. Also, the pretrained network should be tested on real-world hyperspectral data and trained with a pretext task designed for hyperspectral images.

For both humans and machines, understanding the visual world requires relating new percepts with past experience. We argue that a good visual representation for an image should encode what makes it similar to other images, enabling the recall of associated experiences. Current machine implementations of visual representations can capture some aspects of similarity, but fall far short of human ability overall. Even if one explicitly labels objects in millions of images to tell the computer what should be considered similar—a very expensive procedure—the labels still do not capture everything that might be relevant. This thesis shows that one can often train a representation which captures similarity beyond what is labeled in a given dataset. That means we can begin with a dataset that has uninteresting labels, or no labels at all, and still build a useful representation. To do this, we propose to using pretext tasks: tasks that are not useful in and of themselves, but serve as an excuse to learn a more general-purpose representation. The labels for a pretext task can be inexpensive or even free. Furthermore, since this approach assumes training labels differ from the desired outputs, it can handle output spaces where the correct answer is ambiguous, and therefore impossible to annotate by hand. The thesis explores two broad classes of supervision. The first isweak image-level supervision, which is exploited to train mid-level discriminative patch classifiers. For example, given a dataset of street-level imagery labeled only with GPS coordinates, patch classifiers are trained to differentiate one specific geographical region (e.g. the city of Paris) from others. The resulting classifiers each automatically collect and associate a set of patches which all depict the same distinctive architectural element. In this way, we can learn to detect elements like balconies, signs, and lamps without annotations. The second type of supervision requires no information about images other than the pixels themselves. Instead, the algorithm is trained to predict the context around image patches. The context serves as a sort of weak label: to predict well, the algorithm must associate similar-looking patches which also have similar contexts. After training, the feature representation learned using this within-image context indeed captures visual similarity across images, which ultimately makes it useful for real tasks like object detection and geometry estimation.

Semi-supervised learning (SSL) is the most practical approach for classification among machine learning algorithms. It is similar to the humans way of learning and thus has great applications in text/image classification, bioinformatics, artificial intelligence, robotics etc. Labeled data is hard to obtain in real life experiments and may need human experts with experimental equipments to mark the labels, which can be slow and expensive. But unlabeled data is easily available in terms of web pages, data logs, images, audio, video les and DNA/RNA sequences. SSL uses large unlabeled and few labeled data to build better classifying functions which acquires higher accuracy and needs lesser human efforts. Thus it is of great empirical and theoretical interest. We contribute two SSL algorithms (i) adaptive anomaly detection (AAD) (ii) hybrid anomaly detection (HAD), which are self evolving and very efficient to detect anomalies in a large scale and complex data distributions. Our algorithms are capable of modifying an existing classier by both retiring old data and adding new data. This characteristic enables the proposed algorithms to handle massive and streaming datasets where other existing algorithms fail and run out of memory. As an application to semi-supervised anomaly detection and for experimental illustration, we have implemented a prototype of the AAD and HAD systems and conducted experiments in an on-campus cloud computing environment. Experimental results show that the detection accuracy of both algorithms improves as they evolves and can achieve 92.1% detection sensitivity and 83.8% detection specificity, which makes it well suitable for anomaly detection in large and streaming datasets. We compared our algorithms with two popular SSL methods (i) subspace regularization (ii) ensemble of Bayesian sub-models and decision tree classifiers. Our contributed algorithms are easy to implement, significantly better in terms of space, time complexity and accuracy than these two methods for semi-supervised anomaly detection mechanism.

Many automotive safety applications in modern cars make use of cameras and object detection to analyze the surrounding environment. Pedestrians, animals and other vehicles can be detected and safety actions can be taken before dangerous situations arise. To detect occurrences of the different objects, these systems are traditionally trained to learn a classification model using a set of images that carry labels corresponding to their content. To obtain high performance with a variety of object appearances, the required amount of data is very large. Acquiring unlabeled images is easy, while the manual work of labeling is both time-consuming and costly. Semi-supervised learning refers to methods that utilize both labeled and unlabeled data, a situation that is highly desirable if it can lead to improved accuracy and at the same time alleviate the demand of labeled data. This has been an active area of research in the last few decades, but few studies have investigated the performance of these algorithms in larger systems. In this thesis, we investigate if and how semi-supervised learning can be used in a large-scale pedestrian detection system. With the area of application being automotive safety, where real-time performance is of high importance, the work is focused around boosting classifiers. Results are presented on a few publicly available UCI data sets and on a large data set for pedestrian detection captured in real-life traffic situations. By evaluating the algorithms on the pedestrian data set, we add the complexity of data set size, a large variety of object appearances and high input dimension. It is possible to find situations in low dimensions where an additional set of unlabeled data can be used successfully to improve a classification model, but the results show that it is hard to efficiently utilize semi-supervised learning in large-scale object detection systems. The results are hard to scale to large data sets of higher dimensions as pair-wise computations are of high complexity and proper similarity measures are hard to find.

This thesis presents an investigation of statistical analytical methods applied to the analysis of the shape of the arterial pressure waveform. The arterial pulse is analysed by a selection of both supervised and unsupervised methods of learning. Supervised learning methods are generally better known as regression. Unsupervised learning methods seek patterns in data without the specification of a target variable. The theoretical relationship between arterial pressure and wave shape is first investigated by study of a transmission line model of the arterial tree. A meta-database of pulse waveforms obtained by the SphygmoCor"??" device is then analysed by the unsupervised learning technique of Self Organising Maps (SOM). The map patterns indicate that the observed arterial pressures affect the wave shape in a similar way as predicted by the theoretical model. A database of continuous arterial pressure obtained by catheter line during sleep is used to derive supervised models that enable estimation of arterial pressures, based on the measured wave shapes. Independent component analysis (ICA) is also used in a supervised learning methodology to show the theoretical plausibility of separating the pressure signals from unwanted noise components. The accuracy and repeatability of the SphygmoCor?? device is measured and discussed. Alternative regression models are introduced that improve on the existing models in the estimation of central cardiovascular parameters from peripheral arterial wave shapes. Results of this investigation show that from the information in the wave shape, it is possible, in theory, to estimate the continuous underlying pressures within the artery to a degree of accuracy acceptable to the Association for the Advancement of Medical Instrumentation. This could facilitate a new role for non-invasive sphygmographic devices, to be used not only for feature estimation but as alternatives to invasive arterial pressure sensors in the measurement of continuous blood pressure.

Les xarxes neuronals convolucionals profundes (CNNs) han assolit resultats molt positius en diverses aplicacions de reconeixement visual, tals com classificació, detecció o segmentació d’imatges. En aquesta tesis, abordem dues limitacions de les CNNs. La primera, entrenar CNNs profundes requereix grans quantitats de dades etiquetades, les quals són molt costoses i àrdues d’aconseguir. La segona és que entrenar CNNs en sistemes d’aprenentatge continuu és un problema obert per a la recerca. L’oblit catastròfic en xarxes és molt comú quan s’adapta un model entrenat a nous entorns o noves tasques. Per tant, en aquesta tesis, tenim com a objectiu millorar les CNNs per a les aplicacions amb dades limitades i adaptar-les de forma contínua en noves tasques. L’aprenentatge auto-supervisat compensa la falta de dades etiquetades amb la introducció de tasques auxiliars en les quals les dades estan fàcilment disponibles. En la primera part de la tesis, mostrem com els rànquings es poden utilitzar de forma semblant a una tasca auto-supervisada per a problemes de regressió. Després, proposem una tècnica de propagació cap endarrera eficient per a xarxes siameses que prevenen el còmput redundant introduït per les arquitectures de xarxa multi-branca. A més a més, demostrem que mesurar la incertesa de les xarxes en les tasques semblants a les auto-supervisades és una bona mesura de la quantitat d’informació que contenen les dades no etiquetades. Aquesta mesura pot ser, aleshores, utilitzada per a l’execució de algoritmes d’aprenentatge actiu. Aquests marcs que proposem els apliquem doncs a dos problemes de regressió: Avaluació de la Qualitat d’Imatge (IQA) i el comptador de persones. En els dos casos, mostrem com generar de forma automàtica grups d’imatges ranquejades per a les dades no etiquetades. Els nostres resultats mostren que les xarxes entrenades per a la regressió de les anotacions de les dades etiquetades a la vegada que per aprendre a ordenar els rànquings de les dades no etiquetades, obtenen significativament millors resultats que superen l’estat de l’art. També demostrem que l’aprenentatge actiu utilitzant rànquings pot reduir la quantitat d’etiquetatge en un 50% per ambdues tasques de IQA i comptador de persones. A la segona part de la tesis, proposem dosmètodes per a evitar l’oblit catastròfic en escenaris d’aprenentatge seqüencial de tasques. El primer mètode deriva del de Consolidació Elàstica de Pesos, el qual utilitza la diagonal de la Matriu d’Informació de Fisher (FIM) per a mesurar la importància dels paràmetres de la xarxa. No obstant, l’aproximació assumida no és realista. Per tant, diagonalitzem aproximadament la FIMutilitzant un grup de paràmetres de rotació factoritzada proporcionant una millora significativa del rendiment de tasques seqüencials en el cas de l’aprenentatge continu. Per al segon mètode, demostrem que l’oblit es manifesta de forma diferent en cada capa de la xarxa i proposem un mètode híbrid on la destil·lació s’utilitza per a l’extractor de característiques i la rememoració en el classificador mitjançant generació de característiques. El nostremètode soluciona la limitació de la rememoració mitjançant la generació d’imatges i la destil·lació de probabilitats (com l’utilitzat en el mètode Aprenentatge Sense Oblit), i pot afegir de forma natural noves tasques en un únic classificador ben calibrat. Els experiments confirmen que el mètode proposat sobrepassa les mètriques de referència i part de l’estat de l’art.
Las redes neuronales convolucionales profundas (CNNS) han alcanzado resultados muy positivos en diferentes aplicaciones de reconocimiento visual, tales como clasificación, detección o segmentación de imágenes. En esta tesis, abordamos dos limitaciones de las CNNs. La primera, entrenar CNNs profundas requiere grandes cantidades de datos etiquetados, los cuales sonmuy costosos y arduos de conseguir. La segunda es que entrenar en sistemas de aprendizaje continuo es un problema abierto para la investigación. El olvido catastrófico en redes es muy común cuando se adapta un modelo entrenado a nuevos entornos o nuevas tareas. Por lo tanto, en esta tesis, tenemos como objetivo mejorar las CNNs para aplicaciones con datos limitados y adaptarlas de forma continua a nuevas tareas. El aprendizaje auto-supervisado compensa la falta de datos etiquetados con la introducción de tareas auxiliares en las cuales los datos están fácilmente disponibles. En la primera parte de la tesis, mostramos cómo los ránquings se pueden utilizar de forma parecida a una tarea auto-supervisada para los problemas de regresión. Después, proponemos una técnica de propagación hacia atrás eficiente para redes siamesas que previene el computo redundante introducido por las arquitecturas de red multi-rama. Además, demostramos quemedir la incertidumbre de las redes en las tareas parecidas a las auto-supervisadas, es una buena medida de la cantidad de información que contienen los datos no etiquetados. Dicha medida puede ser entonces usada para la ejecución de algoritmos de aprendizaje activo. Estosmarcos que proponemos los aplicamos entonces a dos problemas de regresión: Evaluación de Calidad de Imagen (IQA) y el contador de personas. En los dos casos, mostramos cómo generar de forma automática grupos de imágenes ranqueadas para los datos no etiquetados. Nuestros resultados muestran que las redes entrenadas para la regresión de las anotaciones de los datos etiquetados, a la vez que para aprender a ordenar los ránquings de los datos no etiquetados, obtienen resultados significativamente mejores al estado del arte. También demostramos que el aprendizaje activo utilizando ránquings puede reducir la cantidad de etiquetado en un 50% para ambas tareas de IQA y contador de personas. En la segunda parte de la tesis, proponemos dos métodos para evitar el olvido catastrófico en escenarios de aprendizaje secuencial de tareas. El primer método deriva del de Consolidación Elástica de Pesos, el cuál utiliza la diagonal de laMatriz de Información de Fisher (FIM) para medir la importancia de los pesos de la red. No obstante, la aproximación asumida no es realista. Por lo tanto, diagonalizamos la aproximación de la FIM utilizando un grupo de parámetros de rotación factorizada proporcionando una mejora significativa en el rendimiento de tareas secuenciales para el caso del aprendizaje continuo. Para el segundo método, demostramos que el olvido se manifiesta de forma diferente en cada capa de la red y proponemos un método híbrido donde la destilación se utiliza para el extractor de características y la rememoración en el clasificador mediante generación de características. Nuestro método soluciona la limitación de la rememoración mediante generación de imágenes y la destilación de probabilidades (como la utilizada en elmétodo Aprendizaje Sin Olvido), y puede añadir de forma natural nuevas tareas en un único clasificador bien calibrado. Los experimentos confirman que el método propuesto sobrepasa las métricas de referencia y parte del estado del arte.
Deep convolutional neural networks (CNNs) have achieved superior performance in many visual recognition application, such as image classification, detection and segmentation. In this thesis we address two limitations of CNNs. Training deep CNNs requires huge amounts of labeled data, which is expensive and labor intensive to collect. Another limitation is that training CNNs in a continual learning setting is still an open research question. Catastrophic forgetting is very likely when adapting trainedmodels to new environments or new tasks. Therefore, in this thesis, we aim to improve CNNs for applications with limited data and to adapt CNNs continually to new tasks. Self-supervised learning leverages unlabelled data by introducing an auxiliary task for which data is abundantly available. In the first part of the thesis, we show how rankings can be used as a proxy self-supervised task for regression problems. Then we propose an efficient backpropagation technique for Siamese networks which prevents the redundant computation introduced by the multi-branch network architecture. In addition, we show that measuring network uncertainty on the self-supervised proxy task is a good measure of informativeness of unlabeled data. This can be used to drive an algorithm for active learning. We then apply our framework on two regression problems: Image Quality Assessment (IQA) and Crowd Counting. For both, we show how to automatically generate ranked image sets from unlabeled data. Our results show that networks trained to regress to the ground truth targets for labeled data and to simultaneously learn to rank unlabeled data obtain significantly better, state-of-the-art results. We further show that active learning using rankings can reduce labeling effort by up to 50% for both IQA and crowd counting. In the second part of the thesis, we propose two approaches to avoiding catastrophic forgetting in sequential task learning scenarios. The first approach is derived from ElasticWeight Consolidation, which uses a diagonal Fisher InformationMatrix (FIM) tomeasure the importance of the parameters of the network. However the diagonal assumption is unrealistic. Therefore, we approximately diagonalize the FIM using a set of factorized rotation parameters. This leads to significantly better performance on continual learning of sequential tasks. For the second approach, we show that forgetting manifests differently at different layers in the network and propose a hybrid approach where distillation is used in the feature extractor and replay in the classifier via feature generation. Our method addresses the limitations of generative image replay and probability distillation (i.e. learning without forgetting) and can naturally aggregate new tasks in a single, well-calibrated classifier. Experiments confirmthat our proposed approach outperforms the baselines and some start-of-the-art methods.

Self-adaptive systems are capable of autonomously adjusting their behavior at runtime to accomplish particular adaptation goals. The most common way to realize self-adaption is using a feedback loop(s) which contains four actions: collect runtime data from the system and its environment, analyze the collected data, decide if an adaptation plan is required, and act according to the adaptation plan for achieving the adaptation goals. Existing approaches achieve the adaptation goals by using formal methods, and exhaustively verify all the available adaptation options, i.e., adaptation space. However, verifying the entire adaptation space is often not feasible since it requires time and resources. In this thesis, we present an approach which uses machine learning to reduce the adaptation space in self-adaptive systems. The approach integrates with the feedback loop and selects a subset of the adaptation options that are valid in the current situation. The approach is applied on the simulator of a self-adaptive Internet of Things application which is deployed in KU Leuven, Belgium. We compare our results with a formal model based self-adaptation approach called ActivFORMS. The results show that on average the adaptation space is reduced by 81.2% and the adaptation time by 85% compared to ActivFORMS while achieving the same quality guarantees.

Algoritmos de aprendizado semissupervisionado aprendem a partir de uma combinação de dados rotulados e não rotulados. Assim, eles podem ser aplicados em domínios em que poucos exemplos rotulados e uma vasta quantidade de exemplos não rotulados estão disponíveis. Além disso, os algoritmos semissupervisionados podem atingir um desempenho superior aos algoritmos supervisionados treinados nos mesmos poucos exemplos rotulados. Uma poderosa abordagem ao aprendizado semissupervisionado, denominada aprendizado multidescrição, pode ser usada sempre que os exemplos de treinamento são descritos por dois ou mais conjuntos de atributos disjuntos. A classificação de textos é um domínio de aplicação no qual algoritmos semissupervisionados vêm obtendo sucesso. No entanto, o aprendizado semissupervisionado multidescrição ainda não foi bem explorado nesse domínio dadas as diversas maneiras possíveis de se descrever bases de textos. O objetivo neste trabalho é analisar o desempenho de algoritmos semissupervisionados multidescrição na classificação de textos, usando unigramas e bigramas para compor duas descrições distintas de documentos textuais. Assim, é considerado inicialmente o difundido algoritmo multidescrição CO-TRAINING, para o qual são propostas modificações a fim de se tratar o problema dos pontos de contenção. É também proposto o algoritmo COAL, o qual pode melhorar ainda mais o algoritmo CO-TRAINING pela incorporação de aprendizado ativo como uma maneira de tratar pontos de contenção. Uma ampla avaliação experimental desses algoritmos foi conduzida em bases de textos reais. Os resultados mostram que o algoritmo COAL, usando unigramas como uma descrição das bases textuais e bigramas como uma outra descrição, atinge um desempenho significativamente melhor que um algoritmo semissupervisionado monodescrição. Levando em consideração os bons resultados obtidos por COAL, conclui-se que o uso de unigramas e bigramas como duas descrições distintas de bases de textos pode ser bastante compensador
Semi-supervised learning algorithms learn from a combination of both labeled and unlabeled data. Thus, they can be applied in domains where few labeled examples and a vast amount of unlabeled examples are available. Furthermore, semi-supervised learning algorithms may achieve a better performance than supervised learning algorithms trained on the same few labeled examples. A powerful approach to semi-supervised learning, called multi-view learning, can be used whenever the training examples are described by two or more disjoint sets of attributes. Text classification is a domain in which semi-supervised learning algorithms have shown some success. However, multi-view semi-supervised learning has not yet been well explored in this domain despite the possibility of describing textual documents in a myriad of ways. The aim of this work is to analyze the effectiveness of multi-view semi-supervised learning in text classification using unigrams and bigrams as two distinct descriptions of text documents. To this end, we initially consider the widely adopted CO-TRAINING multi-view algorithm and propose some modifications to it in order to deal with the problem of contention points. We also propose the COAL algorithm, which further improves CO-TRAINING by incorporating active learning as a way of dealing with contention points. A thorough experimental evaluation of these algorithms was conducted on real text data sets. The results show that the COAL algorithm, using unigrams as one description of text documents and bigrams as another description, achieves significantly better performance than a single-view semi-supervised algorithm. Taking into account the good results obtained by COAL, we conclude that the use of unigrams and bigrams as two distinct descriptions of text documents can be very effective

Submitted by Lara Oliveira (lara@ufersa.edu.br) on 2018-03-14T20:25:58Z No. of bitstreams: 1 HiuryNA_DISSERT.pdf: 3188162 bytes, checksum: d40d42a78787557868ebc6d3cd5af945 (MD5)
Approved for entry into archive by Vanessa Christiane (referencia@ufersa.edu.br) on 2018-06-18T16:58:58Z (GMT) No. of bitstreams: 1 HiuryNA_DISSERT.pdf: 3188162 bytes, checksum: d40d42a78787557868ebc6d3cd5af945 (MD5)
Approved for entry into archive by Vanessa Christiane (referencia@ufersa.edu.br) on 2018-06-18T16:59:18Z (GMT) No. of bitstreams: 1 HiuryNA_DISSERT.pdf: 3188162 bytes, checksum: d40d42a78787557868ebc6d3cd5af945 (MD5)
Made available in DSpace on 2018-06-18T16:59:31Z (GMT). No. of bitstreams: 1 HiuryNA_DISSERT.pdf: 3188162 bytes, checksum: d40d42a78787557868ebc6d3cd5af945 (MD5) Previous issue date: 2017-11-17
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Data classification is a task applied in various areas of knowledge, therefore, the focus of ongoing research. Data classification can be divided according to the available data, which are labeled or not labeled. One approach has proven very effective when working with data sets containing labeled and unlabeled data, this called semi-supervised learning, your objective is to label the unlabeled data by using the amount of labeled data in the data set, improving their success rate. Such data can be classified with more than one label, known as multi-label classification. Furthermore, these data can be organized hierarchically, thus containing a relation therebetween, this called hierarchical classification. This work proposes the use of multi-view semi-supervised learning, which is one of the semissupervisionado learning aspects, in problems of hierarchical multi-label classification, with the objective of investigating whether semi-supervised learning is an appropriate approach to solve the problem of low dimensionality of data. An experimental analysis of the methods found that supervised learning had a better performance than semi-supervised approaches, however, semi-supervised learning may be a widely used approach, because, there is plenty to be contributed in this area
classificação de dados é uma tarefa aplicada em diversas áreas do conhecimento, sendo assim, foco de constantes pesquisas. A classificação de dados pode ser dividida de acordo com a disposição dos dados, sendo estes rotulados ou não rotulados. Uma abordagem vem se mostrando bastante eficiente ao se trabalhar com conjuntos de dados contendo dados rotulados e não rotulados, esta chamada de aprendizado semissupervisionado, seu objetivo é classificar os dados não rotulados através da quantidade de dados rotulados contidos no conjunto, melhorando sua taxa de acerto. Tais dados podem ser classificados com mais de um rótulo, conhecida como classificação multirrótulo. Além disso, estes dados podem estar organizados de forma hierárquica, contendo assim, uma relação entre os mesmos, esta, por sua vez, denominada classificação hierárquica. Neste trabalho é proposto a utilização do aprendizado semissupervisionado multidescrição, que é uma das vertentes do aprendizado semissupervisionado, em problemas de classificação hierárquica multirrótulo, com o objetivo de investigar se o aprendizado semissupervisionado é uma abordagem apropriada para resolver o problema de baixa dimensionalidade de dados. Uma análise experimental dos métodos verificou que o aprendizado supervisionado obteve melhor desempenho contra as abordagens semissupervisionadas, contudo, o aprendizado semissupervisionado pode vir a ser uma abordagem amplamente utilizada, pois, há bastante o que ser contribuído nesta área
2018-03-14

Modelos para aprendizado não supervisionado podem fornecer restrições complementares úteis para melhorar a capacidade de generalização de classificadores. Baseando-se nessa premissa, um algoritmo existente, denominado de C3E (Consensus between Classification and Clustering Ensembles), recebe como entradas estimativas de distribuições de probabilidades de classes para objetos de um conjunto alvo, bem como uma matriz de similaridades entre esses objetos. Tal matriz é tipicamente construída por agregadores de agrupadores de dados, enquanto que as distribuições de probabilidades de classes são obtidas por um agregador de classificadores induzidos por um conjunto de treinamento. Como resultado, o C3E fornece estimativas refinadas das distribuições de probabilidades de classes como uma forma de consenso entre classificadores e agrupadores. A ideia subjacente é de que objetos similares são mais propensos a compartilharem o mesmo rótulo de classe. Nesta tese, uma versão mais simples do algoritmo C3E, baseada em uma função de perda quadrática (C3E-SL), foi investigada em uma abordagem que permitiu a estimação automática (a partir dos dados) de seus parâmetros críticos. Tal abordagem faz uso de um nova estratégia evolutiva concebida especialmente para tornar o C3E-SL mais prático e flexível, abrindo caminho para que variantes do algoritmo pudessem ser desenvolvidas. Em particular, para lidar com a escassez de dados rotulados, um novo algoritmo que realiza aprendizado semissupervisionado foi proposto. Seu mecanismo explora estruturas intrínsecas dos dados a partir do C3E-SL em um procedimento de autotreinamento (self-training). Esta noção também inspirou a concepção de um outro algoritmo baseado em aprendizado ativo (active learning), o qual é capaz de se autoadaptar para aprender novas classes que possam surgir durante a predição de novos dados. Uma extensa análise experimental, focada em problemas do mundo real, mostrou que os algoritmos propostos são bastante úteis e promissores. A combinação de classificadores e agrupadores resultou em modelos de classificação com grande potencial prático e que são menos dependentes do usuário ou do especialista de domínio. Os resultados alcançados foram tipicamente melhores em comparação com os obtidos por classificadores tradicionalmente usados.
Unsupervised learning models can provide a variety of supplementary constraints to improve the generalization capability of classifiers. Based on this assumption, an existing algorithm, named C3E (from Consensus between Classification and Clustering Ensembles), receives as inputs class probability distribution estimates for objects in a target set as well as a similarity matrix. Such a similarity matrix is typically built from clusterers induced on the target set, whereas the class probability distributions are obtained by an ensemble of classifiers induced from a training set. As a result, C3E provides refined estimates of the class probability distributions, from the consensus between classifiers and clusterers. The underlying idea is that similar new objects in the target set are more likely to share the same class label. In this thesis, a simpler version of the C3E algorithm, based on a Squared Loss function (C3E-SL), was investigated from an approach that enables the automatic estimation (from data) of its critical parameters. This approach uses a new evolutionary strategy designed to make C3E-SL more practical and flexible, making room for the development of variants of the algorithm. To address the scarcity of labeled data, a new algorithm that performs semi-supervised learning was proposed. Its mechanism exploits the intrinsic structure of the data by using the C3E-SL algorithm in a self-training procedure. Such a notion inspired the development of another algorithm based on active learning, which is able to self-adapt to learn new classes that may emerge when classifying new data. An extensive experimental analysis, focused on real-world problems, showed that the proposed algorithms are quite useful and promising. The combination of supervised and unsupervised learning yielded classifiers of great practical value and that are less dependent on user-defined parameters. The achieved results were typically better than those obtained by traditional classifiers.

O presente relatório foi produzido no âmbito da unidade curricular Prática de Ensino Supervisionada, que faz parte do Mestrado em Ensino do Português no 3º Ciclo do Ensino Básico e Ensino Secundário e de Espanhol nos Ensinos Básico e Secundário, sob a orientação da Professora Doutora Ângela Maria Franco Martins Coelho de Paiva Balça. Identifica-se, na sua essência basilar, como um trabalho reflexivo-descritivo sobre a prática aplicada e efetuada no ano letivo 2015/2016, no lecionamento das disciplinas de Português em duas turmas de 10º ano, e de Espanhol – Língua Estrangeira I numa de 7º ano, na Escola Secundária/3 Rainha Santa Isabel, de Estremoz. Além do mais, também constitui o expoente de todo o processo levado a cabo durante os dois anos do Mestrado, o qual permitiu e conduziu à revisão, modificação, inovação e progressão em matéria de conceitos, ideias, noções, ações e teorias, quer fossem mais antigas ou recentes. Este é o produto final e contributo para o desenvolvimento e melhoria a nível pessoal e profissional. Através do conhecimento da literatura teórica e da sua aplicação na ação, a reflexão compromete-nos a cumprir uma prática fundamentada e apoiada em toda a documentação mundial, europeia e portuguesa normativa e de referência para o exercício da profissão docente o mais completo e eficaz possível. Mais do que um relatório, é uma avaliação orientativa da dimensão transformadora no desempenho docente que, na sua parte mais cogitativa, expõe estruturalmente: a observação e o seu registo; a observação em contexto; a planificação; a orientação; a componente letiva – aulas lecionadas (análise, aprendizagem e melhorias) e a pesquisa reflexiva na abordagem dos inquéritos passados nas turmas de Português e de Espanhol; e, por fim, a abordagem reflexiva sobre a avaliação formativa das aprendizagens realizada às turmas de 10º ano, na disciplina de Português; ABSTRACT: This report was produced in the scope of Supervised Teaching Practice’s curricular unit, which is part of the Master’s Degree in Teaching Portuguese for the 3rd stage of Primary Education and Secondary Education, and Spanish Foreign Language Teaching for Primary and Secondary Education, under the supervision of Dr. Ângela Maria Franco Martins Coelho de Paiva Balça. In its basic essence, this is a reflective and descriptive paper about practices applied and performed for the 2015-2016 school year to teach Portuguese, in two tenth grade classes, and Spanish as a Foreign Language, in one seventh grade class at Rainha Santa Isabel School of Estremoz. Furthermore, it outlines the entire process carried out during the two years of the Master’s Degree, which provided and led to review, change, breakthrough, and advancement regarding concepts, ideas, assumptions, and theories, whether they were pre-existing or more recent. This is the final product and the contribution towards development and improvement in personal and professional terms. Through knowledge of theoretical literature and applying it to practice, the reflection leads us to compile substantiated and supported practice in all worldwide, European, and Portuguese standards and reference documentation for the most effective pursuit of the profession. More than a report, this is an evaluation of transformation in teaching performance that structurally examines the following: observation and its registration; observation in the field; lesson design; guidance and monitoring; a teaching component (analysis, apprenticeships, and improvements) with a reflective element based on the results of the Portuguese and Spanish class surveys; and, finally, a reflexive approach about formative assessment of student learning that took place within the Portuguese course.

La compréhension automatique de la parole est au confluent des deux grands domaines que sont la reconnaissance automatique de la parole et l'apprentissage automatique. Un des problèmes majeurs dans ce domaine est l'obtention d'un corpus de données conséquent afin d'obtenir des modèles statistiques performants. Les corpus de parole pour entraîner des modèles de compréhension nécessitent une intervention humaine importante, notamment dans les tâches de transcription et d'annotation sémantique. Leur coût de production est élevé et c'est la raison pour laquelle ils sont disponibles en quantité limitée.Cette thèse vise principalement à réduire ce besoin d'intervention humaine de deux façons : d'une part en réduisant la quantité de corpus annoté nécessaire à l'obtention d'un modèle grâce à des techniques d'apprentissage semi-supervisé (Self-Training, Co-Training et Active-Learning) ; et d'autre part en tirant parti des réponses de l'utilisateur du système pour améliorer le modèle de compréhension.Ce dernier point touche à un second problème rencontré par les systèmes de compréhension automatique de la parole et adressé par cette thèse : le besoin d'adapter régulièrement leurs modèles aux variations de comportement des utilisateurs ou aux modifications de l'offre de services du système

Made available in DSpace on 2016-06-02T19:05:51Z (GMT). No. of bitstreams: 1 3777.pdf: 3225691 bytes, checksum: 38e3ba8f3c842f4e05d42710339e897a (MD5) Previous issue date: 2011-02-17
Machine Learning (ML) can be seen as research area within the Artificial Intelligence (AI) that aims to develop computer programs that can evolve with new experiences. The main ML purpose is the search for methods and techniques that enable the computer system improve its performance autonomously using information learned through its use. This feature can be considered the fundamental mechanisms of the processes of automatic learning. The main goal in this research project was to investigate, propose and implement methods and algorithms to allow the construction of a continuous learning system capable of extracting knowledge from the Web in Portuguese, throughout the creation of a knowledge base which can be constantly updated as new knowledge is extracted.
O Aprendizado de Máquina (AM) pode ser visto como uma área de pesquisa dentro da Inteligência Artificial (IA) que busca o desenvolvimento de programas de computador que possam evoluir à medida que vão sendo expostos a novas experiências. O principal objetivo de AM é a busca por métodos e técnicas que permitem a concepção de sistemas computacionais capazes de melhorar seu desempenho, de maneira autônoma, usando informações obtidas ao longo de seu uso; tal característica pode, de certa forma, ser considerada como um dos mecanismos fundamentais que regem os processos de aprendizado automático. O principal objetivo da pesquisa descrita neste documento foi investigar, propor e implementar métodos e algoritmos que permitissem a construção de um sistema computacional de aprendizado contínuo capaz de realizar a extração de conhecimento a partir da Web em português, por meio da criação de uma base de conhecimento atualizada constantemente à medida que novos conhecimentos vão sendo extraídos.

Hlavným zámerom tejto diplomovej práce bola automatická segmentácia lézií sklerózy multiplex na snímkoch MRI. V rámci práce boli otestované najnovšie metódy segmentácie s využitím hlbokých neurónových sietí a porovnané prístupy inicializácie váh sietí pomocou preneseného učenia (transfer learning) a samoriadeného učenia (self-supervised learning). Samotný problém automatickej segmentácie lézií sklerózy multiplex je veľmi náročný, a to primárne kvôli vysokej nevyváženosti datasetu (skeny mozgov zvyčajne obsahujú len malé množstvo poškodeného tkaniva). Ďalšou výzvou je manuálna anotácia týchto lézií, nakoľko dvaja rozdielni doktori môžu označiť iné časti mozgu ako poškodené a hodnota Dice Coefficient týchto anotácií je približne 0,86. Možnosť zjednodušenia procesu anotovania lézií automatizáciou by mohlo zlepšiť výpočet množstva lézií, čo by mohlo viesť k zlepšeniu diagnostiky individuálnych pacientov. Našim cieľom bolo navrhnutie dvoch techník využívajúcich transfer learning na predtrénovanie váh, ktoré by neskôr mohli zlepšiť výsledky terajších segmentačných modelov. Teoretická časť opisuje rozdelenie umelej inteligencie, strojového učenia a hlbokých neurónových sietí a ich využitie pri segmentácii obrazu. Následne je popísaná skleróza multiplex, jej typy, symptómy, diagnostika a liečba. Praktická časť začína predspracovaním dát. Najprv boli skeny mozgu upravené na rovnaké rozlíšenie s rovnakou veľkosťou voxelu. Dôvodom tejto úpravy bolo využitie troch odlišných datasetov, v ktorých boli skeny vytvárané rozličnými prístrojmi od rôznych výrobcov. Jeden dataset taktiež obsahoval lebku, a tak bolo nutné jej odstránenie pomocou nástroju FSL pre ponechanie samotného mozgu pacienta. Využívali sme 3D skeny (FLAIR, T1 a T2 modality), ktoré boli postupne rozdelené na individuálne 2D rezy a použité na vstup neurónovej siete s enkodér-dekodér architektúrou. Dataset na trénovanie obsahoval 6720 rezov s rozlíšením 192 x 192 pixelov (po odstránení rezov, ktorých maska neobsahovala žiadnu hodnotu). Využitá loss funkcia bola Combo loss (kombinácia Dice Loss s upravenou Cross-Entropy). Prvá metóda sa zameriavala na využitie predtrénovaných váh z ImageNet datasetu na enkodér U-Net architektúry so zamknutými váhami enkodéra, resp. bez zamknutia a následného porovnania s náhodnou inicializáciou váh. V tomto prípade sme použili len FLAIR modalitu. Transfer learning dokázalo zvýšiť sledovanú metriku z hodnoty približne 0,4 na 0,6. Rozdiel medzi zamknutými a nezamknutými váhami enkodéru sa pohyboval okolo 0,02. Druhá navrhnutá technika používala self-supervised kontext enkodér s Generative Adversarial Networks (GAN) na predtrénovanie váh. Táto sieť využívala všetky tri spomenuté modality aj s prázdnymi rezmi masiek (spolu 23040 obrázkov). Úlohou GAN siete bolo dotvoriť sken mozgu, ktorý bol prekrytý čiernou maskou v tvare šachovnice. Takto naučené váhy boli následne načítané do enkodéru na aplikáciu na náš segmentačný problém. Tento experiment nevykazoval lepšie výsledky, s hodnotou DSC 0,29 a 0,09 (nezamknuté a zamknuté váhy enkodéru). Prudké zníženie metriky mohlo byť spôsobené použitím predtrénovaných váh na vzdialených problémoch (segmentácia a self-supervised kontext enkodér), ako aj zložitosť úlohy kvôli nevyváženému datasetu.

Self-adaptive systems have limited time to adjust their configurations whenever their adaptation goals, i.e., quality requirements, are violated due to some runtime uncertainties. Within the available time, they need to analyze their adaptation space, i.e., a set of configurations, to find the best adaptation option, i.e., configuration, that can achieve their adaptation goals. Existing formal analysis approaches find the best adaptation option by analyzing the entire adaptation space. However, exhaustive analysis requires time and resources and is therefore only efficient when the adaptation space is small. The size of the adaptation space is often in hundreds or thousands, which makes formal analysis approaches inefficient in large-scale self-adaptive systems. In this thesis, we tackle this problem by presenting an online learning approach that enables formal analysis approaches to analyze large adaptation spaces efficiently. The approach integrates with the standard feedback loop and reduces the adaptation space to a subset of adaptation options that are relevant to the current runtime uncertainties. The subset is then analyzed by the formal analysis approaches, which allows them to complete the analysis faster and efficiently within the available time. We evaluate our approach on two different instances of an Internet of Things application. The evaluation shows that our approach dramatically reduces the adaptation space and analysis time without compromising the adaptation goals.

O presente relatório da Prática de Ensino Supervisionada (PES) em Educação Pré-Escolar e Ensino do 1.º Ciclo do Ensino Básico tem como finalidade basilar dar a conhecer o meu percurso de intervenção e investigação nesses contextos de ensino, com enfoque para a temática: “Avaliar para aprender”. São os principais objetivos deste relatório: abordar as funções da avaliação na construção da práxis educativa; compreender como é realizada a avaliação das aprendizagens das crianças/alunos inerentes às valências de ensino supracitadas; apresentar possíveis estratégias para que a avaliação assuma um cariz formativo com recurso, por exemplo, ao feedback; assumir a avaliação como elemento regulador do ensino e da aprendizagem e mostrar que tentei assegurar uma avaliação holística e integrada. Para alcançar tais objetivos foram promovidas algumas estratégias que realçassem a avaliação como um elemento integrante do processo de ensinoaprendizagem, assumindo um cariz formativo e garantindo a participação dos diversos elementos da comunidade educativa; Supervised Teaching Practice in a Preschool and Primary School Educational Level: Assessment to learn ABSTRACT: This report of Supervised Teaching Practice (STP) in a Pre-school and Primary School Educational Level is to present the research and personal involvement in these educational contexts, with focus on the theme: "Assessment to Learn." The main objectives of this report are: reviewing the functions of an assessment in the construction of an educational praxis; understanding how the learning assessment of children/students involved in the above teaching skills is perfomed; presenting possible strategies so that the assessment takes a formative nature using, for example, feedback; using the evaluation as regulatory teaching and learning element and ensuring an holistic and integrated review. To achieve these objectives some strategies were promoted that presented an assessment as an integral part of the process of teaching and learning, electing a formative nature and ensuring participation of the various members of the educational community.

O aprendizado de máquina consiste de conceitos e técnicas que permitem aos computadores melhorar seu desempenho com a experiência, ou, em outras palavras, aprender com dados. Um dos principais tópicos do aprendizado de máquina é o agrupamento de dados que, como o nome sugere, procura agrupar os dados de acordo com sua similaridade. Apesar de sua definição relativamente simples, o agrupamento é uma tarefa computacionalmente complexa, tornando proibitivo o emprego de algoritmos exaustivos, na busca pela solução ótima do problema. A importância do agrupamento de dados, aliada aos seus desafios, faz desse campo um ambiente de intensa pesquisa. Também a classe de fenômenos naturais conhecida como comportamento coletivo tem despertado muito interesse. Isso decorre da observação de um estado organizado e global que surge espontaneamente das interações locais presentes em grandes grupos de indivíduos, caracterizando, pois, o que se chama auto-organização ou emergência, para ser mais preciso. Os desafios intrínsecos e a relevância do tema vêm motivando sua pesquisa em diversos ramos da ciência e da engenharia. Ao mesmo tempo, técnicas baseadas em comportamento coletivo vêm sendo empregadas em tarefas de aprendizado de máquina, mostrando-se promissoras e ganhando bastante atenção. No presente trabalho, objetivou-se o desenvolvimento de técnicas de agrupamento baseadas em comportamento coletivo. Faz-se cada item do conjunto de dados corresponder a um indivíduo, definem-se as leis de interação local, e então os indivíduos são colocados a interagir entre si, de modo que os padrões que surgem reflitam os padrões originalmente presentes no conjunto de dados. Abordagens baseadas em dinâmica de troca de energia foram propostas. Os dados permanecem fixos em seu espaço de atributos, mas carregam certa informação a energia , a qual é progressivamente trocada entre eles. Os grupos são estabelecidos entre dados que tomam estados de energia semelhantes. Este trabalho abordou também o aprendizado semissupervisionado, cuja tarefa é rotular dados em bases parcialmente rotuladas. Nesse caso, foi adotada uma abordagem baseada na movimentação dos próprios dados pelo espaço de atributos. Procurou-se, durante todo este trabalho, não apenas propor novas técnicas de aprendizado, mas principalmente, por meio de muitas simulações e ilustrações, mostrar como elas se comportam em diferentes cenários, num esforço em mostrar onde reside a vantagem de se utilizar a dinâmica coletiva na concepção dessas técnicas
Machine learning consists of concepts and techniques that enable computers to improve their performance with experience, i.e., enable computers to learn from data. Data clustering (or just clustering) is one of its main topics, which aims to group data according to their similarities. Regardless of its simple definition, clustering is a complex computational task. Its relevance and challenges make this field an environment of intense research. The class of natural phenomena known as collective behavior has also attracted much interest. This is due to the observation that global patterns may spontaneously arise from local interactions among large groups of individuals, what is know as self-organization (or emergence). The challenges and relevance of the subject are encouraging its research in many branches of science and engineering. At the same time, techniques based on collective behavior are being employed in machine learning tasks, showing to be promising. The objective of the present work was to develop clustering techniques based on collective behavior. Each dataset item corresponds to an individual. Once the local interactions are defined, the individuals begin to interact with each other. It is expected that the patterns arising from these interactions match the patterns originally present in the dataset. Approaches based on dynamics of energy exchange have been proposed. The data are kept fixed in their feature space, but they carry some sort of information (the energy), which is progressively exchanged among them. The groups are established among data that take similar energy states. This work has also addressed the semi-supervised learning task, which aims to label data in partially labeled datasets. In this case, it has been proposed an approach based on the motion of the data themselves around the feature space. More than just providing new machine learning techniques, this research has tried to show how the techniques behave in different scenarios, in an effort to show where lies the advantage of using collective dynamics in the design of such techniques

Key physicochemical properties for a wide spectrum of chemical pollutants are unknown. This thesis analyses the prospect of assessing the environmental distribution of chemicals directly from supervised learning algorithms using molecular descriptors, rather than from multimedia environmental models (MEMs) using several physicochemical properties estimated from QSARs. Dimensionless compartmental mass ratios of 468 validation chemicals were compared, in logarithmic units, between: a) SimpleBox 3, a Level III MEM, propagating random property values within statistical distributions of widely recommended QSARs; and, b) Support Vector Regressions (SVRs), acting as Quantitative Structure-Fate Relationships (QSFRs), linking mass ratios to molecular weight and constituent counts (atoms, bonds, functional groups and rings) for training chemicals. Best predictions were obtained for test and validation chemicals optimally found to be within the domain of applicability of the QSFRs, evidenced by low MAE and high q2 values (in air, MAE≤0.54 and q2≥0.92; in water, MAE≤0.27 and q2≥0.92).
Las propiedades fisicoquímicas de un gran espectro de contaminantes químicos son desconocidas. Esta tesis analiza la posibilidad de evaluar la distribución ambiental de compuestos utilizando algoritmos de aprendizaje supervisados alimentados con descriptores moleculares, en vez de modelos ambientales multimedia alimentados con propiedades estimadas por QSARs. Se han comparado fracciones másicas adimensionales, en unidades logarítmicas, de 468 compuestos entre: a) SimpleBox 3, un modelo de nivel III, propagando valores aleatorios de propiedades dentro de distribuciones estadísticas de QSARs recomendados; y, b) regresiones de vectores soporte (SVRs) actuando como relaciones cuantitativas de estructura y destino (QSFRs), relacionando fracciones másicas con pesos moleculares y cuentas de constituyentes (átomos, enlaces, grupos funcionales y anillos) para compuestos de entrenamiento. Las mejores predicciones resultaron para compuestos de test y validación correctamente localizados dentro del dominio de aplicabilidad de los QSFRs, evidenciado por valores bajos de MAE y valores altos de q2 (en aire, MAE≤0.54 y q2≥0.92; en agua, MAE≤0.27 y q2≥0.92).

Cette thèse traite de la détection de fraude par carte de crédit. Selon la Banque Centrale Européenne, la valeur des fraudes utilisant des cartes en 2016 s'élevait à 1,8 milliard d'euros. Le défis pour les institutions est de réduire ces fraudes. En règle générale, les systèmes de détection de la fraude sont consistués d'un système automatique construit à base de règles "si-alors" qui contrôlent toutes les transactions en entrée et déclenchent une alerte si la transaction est considérée suspecte. Un groupe expert vérifie l'alerte et décide si cette dernière est vrai ou pas. Les critères utilisés dans la sélection des règles maintenues opérationnelles sont principalement basés sur la performance individuelle des règles. Cette approche ignore en effet la non-additivité des règles. Nous proposons une nouvelle approche utilisant des indices de puissance. Cette approche attribue aux règles un score normalisé qui quantifie l'influence de la règle sur les performances globales du groupe de règles. Les indice utilisés sont le "Shapley Value" et le "Banzhaf Value". Leurs applications sont: 1) Aide à la décision de conserver ou supprimer une règle; 2) Sélection du nombre k de règles les mieux classées, afin de travailler avec un ensemble plus compact. En utilisant des données réelles de fraude par carte de crédit, nous montrons que: 1) Cette approche permet de mieux évaluer les performances du groupe plutot que de les évaluer isolément. 2) La performance de l'ensemble des règles peut être atteinte en conservant le dixième des règles. Nous observons que cette application peut être comsidérée comme une tâche de sélection de caractéristiques:ainsi nous montrons que notre approche est comparable aux algorithmes courants de sélection des caractéristiques. Il présente un avantage dans la gestion des règles, car attribue un score normalisé à chaque règle. Ce qui n'est pas le cas pour la plupart des algorithmes, qui se concentrent uniquement sur une solution d'ensemble. Nous proposons une nouvelle version du Banzhaf Value, à savoir le k-Banzhaf; qui surclasse la précedente en terme de temps de calcul et possède des performances comparables. Enfin, nous mettons en œuvre un processus d’auto-apprentissage afin de renforcer l’apprentissage dans un algorithme. Nous comparons ces derniers avec nos trois indices de puissance pour effectuer une classification sur les données de fraude par carte de crédit. En conclusion, nous observons que la sélection de caractéristiques basée sur les indices de puissance a des résultats comparables avec les autres algorithmes dans le processus d'auto-apprentissage
This thesis deals with the detection of credit card fraud. According to the European Central Bank, the value of frauds using cards in 2016 amounted to 1.8 billion euros. The challenge for institutions is to reduce these frauds. In general, fraud detection systems consist of an automatic system built with "if-then" rules that control all incoming transactions and trigger an alert if the transaction is considered suspicious. An expert group checks the alert and decides whether it is true or not. The criteria used in the selection of the rules that are kept operational are mainly based on the individual performance of the rules. This approach ignores the non-additivity of the rules. We propose a new approach using power indices. This approach assigns to the rules a normalized score that quantifies the influence of the rule on the overall performance of the group. The indexes we use are the Shapley Value and Banzhaf Value. Their applications are 1) Decision support to keep or delete a rule; 2) Selection of the number k of best-ranked rules, in order to work with a more compact set. Using real credit card fraud data, we show that: 1) This approach performs better than the one that evaluates the rules in isolation. 2) The performance of the set of rules can be achieved by keeping one-tenth of the rules. We observe that this application can be considered as a task of selection of characteristics: We show that our approach is comparable to the current algorithms of the selection of characteristics. It has an advantage in rule management because it assigns a standard score to each rule. This is not the case for most algorithms, which focus only on an overall solution. We propose a new version of Banzhaf Value, namely k-Banzhaf; which outperforms the previous in terms of computing time and has comparable performance. Finally, we implement a self-learning process to reinforce the learning in an automatic learning algorithm. We compare these with our power indices to rank credit card fraud data. In conclusion, we observe that the selection of characteristics based on the power indices has comparable results with the other algorithms in the self-learning process

No
We present a novel self-supervised framework for monocular image depth learning and confidence estimation. Our framework reduces the amount of ground truth annotation data required for training Convolutional Neural Networks (CNNs), which is often a challenging problem for the fast deployment of CNNs in many computer vision tasks. Our DepthNet adopts a novel fully differential patch-based cost function through the Zero-Mean Normalized Cross Correlation (ZNCC) to take multi-scale patches as matching and learning strategies. This approach greatly increases the accuracy and robustness of the depth learning. Whilst the proposed patch-based cost function naturally provides a 0-to-1 confidence, it is then used to self-supervise the training of a parallel network for confidence map learning and estimation by exploiting the fact that ZNCC is a normalized measure of similarity which can be approximated as the confidence of the depth estimation. Therefore, the proposed corresponding confidence map learning and estimation operate in a self-supervised manner and is a parallel network to the DepthNet. Evaluation on the KITTI depth prediction evaluation dataset and Make3D dataset show that our method outperforms the state-of-the-art results.

abstract: In recent years, Convolutional Neural Networks (CNNs) have been widely used in not only the computer vision community but also within the medical imaging community. Specifically, the use of pre-trained CNNs on large-scale datasets (e.g., ImageNet) via transfer learning for a variety of medical imaging applications, has become the de facto standard within both communities. However, to fit the current paradigm, 3D imaging tasks have to be reformulated and solved in 2D, losing rich 3D contextual information. Moreover, pre-trained models on natural images never see any biomedical images and do not have knowledge about anatomical structures present in medical images. To overcome the above limitations, this thesis proposes an image out-painting self-supervised proxy task to develop pre-trained models directly from medical images without utilizing systematic annotations. The idea is to randomly mask an image and train the model to predict the missing region. It is demonstrated that by predicting missing anatomical structures when seeing only parts of the image, the model will learn generic representation yielding better performance on various medical imaging applications via transfer learning. The extensive experiments demonstrate that the proposed proxy task outperforms training from scratch in six out of seven medical imaging applications covering 2D and 3D classification and segmentation. Moreover, image out-painting proxy task offers competitive performance to state-of-the-art models pre-trained on ImageNet and other self-supervised baselines such as in-painting. Owing to its outstanding performance, out-painting is utilized as one of the self-supervised proxy tasks to provide generic 3D pre-trained models for medical image analysis.
Dissertation/Thesis
Masters Thesis Computer Science 2020

碩士
國立臺灣大學
電子工程學研究所
107
Autonomous mobile robots and vision-base intelligent assistive technolo- gies are going to be indispensable in everybody’s lives. Among these appli- cations, traversability estimation (TE) is one of the most significant com- ponent to guarantee people’s safety. Besides accuracy, transferability across different environments and platforms is another important design consider- ation for TE. In this thesis, a two-stage convolutional neural network is proposed for TE. Input color images are first translated to relative depth maps by domain-specific depth estimation networks. These networks are used to align different domains and are trained with an extended version of self- supervised learning method dedicated to fisheye images. After that, an universal classification network trained in the source domain is used to es- timate traversability with relative depth maps from all domains. Experiments on the real-world data show that the proposed method has competitive accuracy with higher transferability among different environ- ments. In addition, it has a reasonable model complexity under domain- specific premise and requires only a single fisheye camera, which is also suitable for resource-constrained platforms.

abstract: To ensure system integrity, robots need to proactively avoid any unwanted physical perturbation that may cause damage to the underlying hardware. In this thesis work, we investigate a machine learning approach that allows robots to anticipate impending physical perturbations from perceptual cues. In contrast to other approaches that require knowledge about sources of perturbation to be encoded before deployment, our method is based on experiential learning. Robots learn to associate visual cues with subsequent physical perturbations and contacts. In turn, these extracted visual cues are then used to predict potential future perturbations acting on the robot. To this end, we introduce a novel deep network architecture which combines multiple sub- networks for dealing with robot dynamics and perceptual input from the environment. We present a self-supervised approach for training the system that does not require any labeling of training data. Extensive experiments in a human-robot interaction task show that a robot can learn to predict physical contact by a human interaction partner without any prior information or labeling. Furthermore, the network is able to successfully predict physical contact from either depth stream input or traditional video input or using both modalities as input.
Dissertation/Thesis
Masters Thesis Computer Science 2017

A significant barrier to applying the techniques of machine learning to the domain of object boundary detection is the need to obtain a large database of correctly labeled examples. Inspired by developmental psychology, this paper proposes that boundary detection can be learned from the output of a motion tracking algorithm that separates moving objects from their static surroundings. Motion segmentation solves the database problem by providing cheap, unlimited, labeled training data. A probabilistic model of the textural and shape properties of object boundaries can be trained from this data and then used to efficiently detect boundaries in novel images via loopy belief propagation.
Singapore-MIT Alliance (SMA)

L'efficacité des données reste un défi majeur dans l'apprentissage par renforcement profond. Bien que les techniques modernes soient capables d'atteindre des performances élevées dans des tâches extrêmement complexes, y compris les jeux de stratégie comme le StarCraft, les échecs, le shogi et le go, ainsi que dans des domaines visuels exigeants comme les jeux Atari, cela nécessite généralement d'énormes quantités de données interactives, limitant ainsi l'application pratique de l'apprentissage par renforcement. Dans ce mémoire, nous proposons la SPR, une méthode inspirée des récentes avancées en apprentissage auto-supervisé de représentations, conçue pour améliorer l'efficacité des données des agents d'apprentissage par renforcement profond. Nous évaluons cette méthode sur l'environement d'apprentissage Atari, et nous montrons qu'elle améliore considérablement les performances des agents avec un surcroît de calcul modéré. Lorsqu'on lui accorde à peu près le même temps d'apprentissage qu'aux testeurs humains, un agent d'apprentissage par renforcement augmenté de SPR atteint des performances surhumaines dans 7 des 26 jeux, une augmentation de 350% par rapport à l'état de l'art précédent, tout en améliorant fortement les performances moyennes et médianes. Nous évaluons également cette méthode sur un ensemble de tâches de contrôle continu, montrant des améliorations substantielles par rapport aux méthodes précédentes. Le chapitre 1 présente les concepts nécessaires à la compréhension du travail présenté, y compris des aperçus de l'apprentissage par renforcement profond et de l'apprentissage auto-supervisé de représentations. Le chapitre 2 contient une description détaillée de nos contributions à l'exploitation de l'apprentissage de représentation auto-supervisé pour améliorer l'efficacité des données dans l'apprentissage par renforcement. Le chapitre 3 présente quelques conclusions tirées de ces travaux, y compris des propositions pour les travaux futurs.
Data efficiency remains a key challenge in deep reinforcement learning. Although modern techniques have been shown to be capable of attaining high performance in extremely complex tasks, including strategy games such as StarCraft, Chess, Shogi, and Go as well as in challenging visual domains such as Atari games, doing so generally requires enormous amounts of interactional data, limiting how broadly reinforcement learning can be applied. In this thesis, we propose SPR, a method drawing from recent advances in self-supervised representation learning designed to enhance the data efficiency of deep reinforcement learning agents. We evaluate this method on the Atari Learning Environment, and show that it dramatically improves performance with limited computational overhead. When given roughly the same amount of learning time as human testers, a reinforcement learning agent augmented with SPR achieves super-human performance on 7 out of 26 games, an increase of 350% over the previous state of the art, while also strongly improving mean and median performance. We also evaluate this method on a set of continuous control tasks, showing substantial improvements over previous methods. Chapter 1 introduces concepts necessary to understand the work presented, including overviews of Deep Reinforcement Learning and Self-Supervised Representation learning. Chapter 2 contains a detailed description of our contributions towards leveraging self-supervised representation learning to improve data-efficiency in reinforcement learning. Chapter 3 provides some conclusions drawn from this work, including a number of proposals for future work.

Extraire une représentation de tous les facteurs de haut niveau de l'état d'un agent à partir d'informations sensorielles de bas niveau est une tâche importante, mais difficile, dans l'apprentissage automatique. Dans ce memoire, nous explorerons plusieurs approches non supervisées pour apprendre ces représentations. Nous appliquons et analysons des méthodes d'apprentissage de représentations non supervisées existantes dans des environnements d'apprentissage par renforcement, et nous apportons notre propre suite d'évaluations et notre propre méthode novatrice d'apprentissage de représentations d'état. Dans le premier chapitre de ce travail, nous passerons en revue et motiverons l'apprentissage non supervisé de représentations pour l'apprentissage automatique en général et pour l'apprentissage par renforcement. Nous introduirons ensuite un sous-domaine relativement nouveau de l'apprentissage de représentations : l'apprentissage auto-supervisé. Nous aborderons ensuite deux approches fondamentales de l'apprentissage de représentations, les méthodes génératives et les méthodes discriminatives. Plus précisément, nous nous concentrerons sur une collection de méthodes discriminantes d'apprentissage de représentations, appelées méthodes contrastives d'apprentissage de représentations non supervisées (CURL). Nous terminerons le premier chapitre en détaillant diverses approches pour évaluer l'utilité des représentations. Dans le deuxième chapitre, nous présenterons un article de workshop dans lequel nous évaluons un ensemble de méthodes d'auto-supervision standards pour les problèmes d'apprentissage par renforcement. Nous découvrons que la performance de ces représentations dépend fortement de la dynamique et de la structure de l'environnement. À ce titre, nous déterminons qu'une étude plus systématique des environnements et des méthodes est nécessaire. Notre troisième chapitre couvre notre deuxième article, Unsupervised State Representation Learning in Atari, où nous essayons d'effectuer une étude plus approfondie des méthodes d'apprentissage de représentations en apprentissage par renforcement, comme expliqué dans le deuxième chapitre. Pour faciliter une évaluation plus approfondie des représentations en apprentissage par renforcement, nous introduisons une suite de 22 jeux Atari entièrement labellisés. De plus, nous choisissons de comparer les méthodes d'apprentissage de représentations de façon plus systématique, en nous concentrant sur une comparaison entre méthodes génératives et méthodes contrastives, plutôt que les méthodes générales du deuxième chapitre choisies de façon moins systématique. Enfin, nous introduisons une nouvelle méthode contrastive, ST-DIM, qui excelle sur ces 22 jeux Atari.
Extracting a representation of all the high-level factors of an agent’s state from level-level sensory information is an important, but challenging task in machine learning. In this thesis, we will explore several unsupervised approaches for learning these state representations. We apply and analyze existing unsupervised representation learning methods in reinforcement learning environments, as well as contribute our own evaluation benchmark and our own novel state representation learning method. In the first chapter, we will overview and motivate unsupervised representation learning for machine learning in general and for reinforcement learning. We will then introduce a relatively new subfield of representation learning: self-supervised learning. We will then cover two core representation learning approaches, generative methods and discriminative methods. Specifically, we will focus on a collection of discriminative representation learning methods called contrastive unsupervised representation learning (CURL) methods. We will close the first chapter by detailing various approaches for evaluating the usefulness of representations. In the second chapter, we will present a workshop paper, where we evaluate a handful of off-the-shelf self-supervised methods in reinforcement learning problems. We discover that the performance of these representations depends heavily on the dynamics and visual structure of the environment. As such, we determine that a more systematic study of environments and methods is required. Our third chapter covers our second article, Unsupervised State Representation Learning in Atari, where we try to execute a more thorough study of representation learning methods in RL as motivated by the second chapter. To facilitate a more thorough evaluation of representations in RL we introduce a benchmark of 22 fully labelled Atari games. In addition, we choose the representation learning methods for comparison in a more systematic way by focusing on comparing generative methods with contrastive methods, instead of the less systematically chosen off-the-shelf methods from the second chapter. Finally, we introduce a new contrastive method, ST-DIM, which excels at the 22 Atari games.

With the growing presence of robots in human populated environments, it becomes necessary to render the relation between robots and humans natural, rather than invasive. For that purpose, robots need to make sure the acoustic noise induced by their motion does not disturb people that are in the same environment. This dissertation proposes a method that allows a robot to learn how to control the amount of noise it produces, taking into account the environmental context and the robot’s mechanical characteristics. The robot performs its task while adapting its speed, so it produces less acoustic noise than the environment’s, and thus, not disturbing nearby people. Before the robot can execute a task in a given environment, it needs to learn how much noise it induces while moving at different speeds on that environment. For that, a microphone was installed on a wheeled robot to gather information about it’s acoustic noise. To validate the proposed solution, tests in different environments with different background sounds were performed. After that, a PIR sensor was installed on the robot to verify the ability of the robot to use the motion controller when someone enters the sensor’s field of vision. This second test demonstrated the possibility of using the proposed solution in another systems.
Com a crescente presença dos robôs no espaço ocupado pelos seres humanos, é necessário que a relação entre robôs e humans seja natural e não invasiva. Para alcançar este objectivo, é preciso que os robôs tenham formas de assegurar que o ruído causado pelos seus movimentos não incomodam as pessoas inseridas no meio envolvente. Esta dissertação propôe uma solução que permite que um robô aprenda a controlar a quantidade de ruído que produz, tendo em conta as suas caracteristicas e o meio ambiente onde é colocado. O robô concretiza as suas tarefas adaptando a sua velocidade de forma a produzir menos ruído que o presente no meio ambiente e, assim, não incomodar as pessoas. Para que o robô possa executar alguma tarefa num determinado ambiente, é necessário aprender a quantidade de ruído que faz ao movimentar-se a diferentes velocidades nesse ambiente. Para tal, um microfone foi instalado num robô com rodas para obter informação sobre a sua acústica. Para validar a solução proposta, foram realizados testes, com sucesso, em diferentes ambientes com diferentes ruídos de fundo. Posteriormente, foi instalado no robô um sensor PIR para analizar a capacidade do robô executar o controlador de velocidade quando alguém entra no campo de visão do sensor. Este segundo teste demonstrou a possibilidade de incluir a solução proposta em outros sistemas.