Dissertations / Theses on the topic 'Deep learning with uncertainty'

Diese Arbeit konzentriert sich auf die Lösung des Problems der Unsicherheitsmessung und ihrer Auswirkungen auf Geschäftsentscheidungen, wobei zwei Ziele verfolgt werden: Erstens die Entwicklung und Validierung robuster Modelle zur Quantifizierung der Unsicherheit, wobei insbesondere sowohl die etablierten statistischen Modelle als auch neu entwickelte maschinelle Lernwerkzeuge zum Einsatz kommen. Das zweite Ziel dreht sich um die industrielle Anwendung der vorgeschlagenen Modelle. Die Anwendung auf reale Fälle bei der Messung der Volatilität oder bei einer riskanten Entscheidung ist mit einem direkten und erheblichen Gewinn oder Verlust verbunden. Diese These begann mit der Untersuchung der impliziten Volatilität (IV) als Proxy für die Wahrnehmung der Unsicherheit von Anlegern für eine neue Klasse von Vermögenswerten - Kryptowährungen. Das zweite Papier konzentriert sich auf Methoden zur Identifizierung risikofreudiger Händler und nutzt die DNN-Infrastruktur, um das Risikoverhalten von Marktakteuren, das auf Unsicherheit beruht und diese aufrechterhält, weiter zu untersuchen. Das dritte Papier befasste sich mit dem herausfordernden Bestreben der Betrugserkennung 3 und bot das Entscheidungshilfe-modell, das eine genauere und interpretierbarere Bewertung der zur Prüfung eingereichten Finanzberichte ermöglichte. Angesichts der Bedeutung der Risikobewertung und der Erwartungen der Agenten für die wirtschaftliche Entwicklung und des Aufbaus der bestehenden Arbeiten von Baker (2016) bot das vierte Papier eine neuartige DL-NLP-basierte Methode zur Quantifizierung der wirtschaftspolitischen Unsicherheit. Die neuen Deep-Learning-basierten Lösungen bieten eine überlegene Leistung gegenüber bestehenden Ansätzen zur Quantifizierung und Erklärung wirtschaftlicher Unsicherheiten und ermöglichen genauere Prognosen, verbesserte Planungskapazitäten und geringere Risiken. Die angebotenen Anwendungsfälle bilden eine Plattform für die weitere Forschung.
This thesis focuses on solving the problem of uncertainty measurement and its impact on business decisions while pursuing two goals: first, develop and validate accurate and robust models for uncertainty quantification, employing both the well established statistical models and newly developed machine learning tools, with particular focus on deep learning. The second goal revolves around the industrial application of proposed models, applying them to real-world cases when measuring volatility or making a risky decision entails a direct and substantial gain or loss. This thesis started with the exploration of implied volatility (IV) as a proxy for investors' perception of uncertainty for a new class of assets - crypto-currencies. The second paper focused on methods to identify risk-loving traders and employed the DNN infrastructure for it to investigate further the risk-taking behavior of market actors that both stems from and perpetuates uncertainty. The third paper addressed the challenging endeavor of fraud detection and offered the decision support model that allowed a more accurate and interpretable evaluation of financial reports submitted for audit. Following the importance of risk assessment and agents' expectations in economic development and building on the existing works of Baker (2016) and their economic policy uncertainty (EPU) index, it offered a novel DL-NLP-based method for the quantification of economic policy uncertainty. In summary, this thesis offers insights that are highly relevant to both researchers and practitioners. The new deep learning-based solutions exhibit superior performance to existing approaches to quantify and explain economic uncertainty, allowing for more accurate forecasting, enhanced planning capacities, and mitigated risks. The offered use-cases provide a road-map for further development of the DL tools in practice and constitute a platform for further research.

Deep learning and convolutional neural networks have become the dominant tool for computer vision. These techniques excel at learning complicated representations from data using supervised learning. In particular, image recognition models now out-perform human baselines under constrained settings. However, the science of computer vision aims to build machines which can see. This requires models which can extract richer information than recognition, from images and video. In general, applying these deep learning models from recognition to other problems in computer vision is significantly more challenging. This thesis presents end-to-end deep learning architectures for a number of core computer vision problems; scene understanding, camera pose estimation, stereo vision and video semantic segmentation. Our models outperform traditional approaches and advance state-of-the-art on a number of challenging computer vision benchmarks. However, these end-to-end models are often not interpretable and require enormous quantities of training data. To address this, we make two observations: (i) we do not need to learn everything from scratch, we know a lot about the physical world, and (ii) we cannot know everything from data, our models should be aware of what they do not know. This thesis explores these ideas using concepts from geometry and uncertainty. Specifically, we show how to improve end-to-end deep learning models by leveraging the underlying geometry of the problem. We explicitly model concepts such as epipolar geometry to learn with unsupervised learning, which improves performance. Secondly, we introduce ideas from probabilistic modelling and Bayesian deep learning to understand uncertainty in computer vision models. We show how to quantify different types of uncertainty, improving safety for real world applications.

Several computer vision approaches have been proposed for tackling food analysis problems, due to the challenging problem it poses, the ease collection of food images, and its numerous applications to health and leisure. However, high food ambiguity, interclass variability and intra-class similarity define a real challenge for the Deep learning and Computer Vision algorithms. With the advent of Convolutional Neural Networks, the complex problem of visual food analysis has experienced significant improvement. Despite this, for real applications, where thousands of foods must be analyzed and recognized it is necessary to better understand what the model learns and, from this, guide its learning on more discriminatives features to improve its accurate and robustness. In this thesis we address the problem of analyzing food images through methods based on deep learning algorithms. There are two distinguishable parts. In the first part, we focus on the food recognition task and delve into uncertainty modeling. First, we propose a new multi-task model that is able to simultaneously predict different food-related tasks. Here, we extend the homoscedastic uncertainty modeling to allow single-label and multilabel classification and propose a regularization term, which jointly weighs the tasks as well as their correlations. Second, we propose a novel prediction scheme based on a class hierarchy that considers local classifiers, in addition to a flat classifier. For this, we define criteria based on the Epistemic Uncertainty estimated from the ’children’ classifiers and the prediction from the ’parent’ classifier to decide the approach to use. And third, we propose three new data augmentation strategies that analysis class-level or sample-level epistemic uncertainty to guide the model training. In the second part we contribute to the design of new methods for food detection (food/nonfood classification), for ensemble of food classifiers and for semantic food detection. First, we proposes an overview of the last advances on food/non-food classification and an optimal model based on the GoogLeNet architecture, Principal Component Analysis, and a Support Vector Machine. Second, we propose a combination of multiple classifiers for food recognition based on two different Convolutional models that complement each other and thus, achieve an improvement in performance. And third, we address the problem of automatic food tray analysis in canteens and restaurants environment through a new approach that integrates in the same framework food localization, recognition and segmentation for semantic food detection. All the methods designed in this thesis are validated and contrasted over relevant public food datasets and the results obtained are reported in detail.
El desafiante problema que plantea el análisis de alimentos, la facilidad para recopilar imágenes de alimentos y sus numerosas aplicaciones para la salud y el ocio son algunos de los factores principales que han incentivado la generación de varios enfoques de visión por computadora para abordar este problema. Sin embargo, la ambigüedad alimentaria, variabilidad entre clases y similitud dentro de la clase definen un desafío real para los algoritmos de aprendizaje profundo y visión por computadora. Con la llegada de las redes neuronales convolucionales, el complejo problema del análisis visual de los alimentos ha experimentado una mejora significativa. A pesar de ello, para aplicaciones reales, donde se deben analizar y reconocer miles de alimentos, es necesario comprender mejor lo que aprende el modelo y, a partir de ello, orientar su aprendizaje en aspectos más discriminatorios para mejorar su precisión y robustez. En esta tesis abordamos el problema del análisis de imágenes de alimentos mediante métodos basados en algoritmos de aprendizaje profundo. Hay dos partes distinguibles. En la primera parte, nos centramos en la tarea de reconocimiento de alimentos y profundizamos en el modelado de incertidumbre. Primero, proponemos un nuevo modelo multi-tarea que es capaz de predecir simultáneamente diferentes tareas relacionadas con los alimentos. Aquí, ampliamos el modelo de incertidumbre homocedástica para permitir la clasificación tanto de etiqueta única como de etiquetas múltiples, y proponemos un término de regularización, que pondera conjuntamente las tareas y sus correlaciones. En segundo lugar, proponemos un novedoso esquema de predicción basado en una jerarquía de clases que considera clasificadores locales y un clasificador plano. Para decidir el enfoque a utilizar (plano o local), definimos criterios basados en la incertidumbre epistémica estimada a partir de los clasificadores de 'hijos' y la predicción del clasificador de 'padres'. Y tercero, proponemos tres nuevas estrategias de aumento de datos que analizan la incertidumbre epistémica a nivel de clase o de muestra para guiar el entrenamiento del modelo. En la segunda parte contribuimos al diseño de nuevos métodos para la detección de alimentos (clasificación food/non-food), para generar predicciones a partir de un conjunto de clasificadores de alimentos y para la detección semántica de alimentos. Primero, establecemos en estado del arte en cuanto a últimos avances en clasificación de food/non-food y proponemos un modelo óptimo basado en la arquitectura GoogLeNet, Análisis de Componentes Principales (PCA) y una Máquina de Vector de Soporte (SVM). En segundo lugar, proponemos medidas difusas para combinar múltiples clasificadores para el reconocimiento de alimentos basados en dos arquitecturas convolucionales diferentes que se complementan y de este modo, logran una mejora en el rendimiento. Y tercero, abordamos el problema del análisis automático de bandejas de alimentos en el entorno de comedores y restaurantes a través de un nuevo enfoque que integra en un mismo marco la localización, el reconocimiento y la segmentación de alimentos para la detección semántica de alimentos. Todos los métodos diseñados en esta tesis están validados y contrastados sobre conjuntos de datos de alimentos públicos relevantes y los resultados obtenidos se informan en detalle.

Deep Neural Networks (DNNs) have shown promising results in classifying known Low-probability-of-intercept (LPI) radar signals in noisy environments. However, regular DNNs produce low-quality confidence and uncertainty estimates, making them unreliable, which inhibit deployment in real-world settings. Hence, the need for robust uncertainty estimation methods has grown, and two categories emerged, Bayesian approximation and ensemble learning. As autonomous LPI radar classification is deployed in safety-critical environments, this study compares Bayesian Neural Networks (BNNs) and Deep Ensembles (DEs) as uncertainty estimation methods. We synthetically generate a training and test data set, as well as a shifted data set where subtle changes are made to the signal parameters. The methods are evaluated on predictive performance, relevant confidence and uncertainty estimation metrics, and method-related metrics such as model size, training, and inference time. Our results show that our DE achieves slightly higher predictive performance than the BNN on both in-distribution and shifted data with an accuracy of 74% and 32%, respectively. Further, we show that both methods exhibit more cautiousness in their predictions compared to a regular DNN for in-distribution data, while the confidence quality significantly degrades on shifted data. Uncertainty in predictions is evaluated as predictive entropy, and we show that both methods exhibit higher uncertainty on shifted data. We also show that the signal-to-noise ratio affects uncertainty compared to a regular DNN. However, none of the methods exhibit uncertainty when making predictions on unseen signal modulation patterns, which is not a desirable behavior. Further, we conclude that the amount of available resources could influence the choice of the method since DEs are resource-heavy, requiring more memory than a regular DNN or BNN. On the other hand, the BNN requires a far longer training time.
Tidigare studier har visat att djupa neurala nätverk (DNN) kan klassificera signalmönster för en speciell typ av radar (LPI) som är skapad för att vara svår att identifiera och avlyssna. Traditionella neurala nätverk saknar dock ett naturligt sätt att skatta osäkerhet, vilket skadar deras pålitlighet och förhindrar att de används i säkerhetskritiska miljöer. Osäkerhetsskattning för djupinlärning har därför vuxit och på senare tid blivit ett stort område med två tydliga kategorier, Bayesiansk approximering och ensemblemetoder. LPI radarklassificering är av stort intresse för försvarsindustrin, och tekniken kommer med största sannolikhet att appliceras i säkerhetskritiska miljöer. I denna studie jämför vi Bayesianska neurala nätverk och djupa ensembler för LPI radarklassificering. Resultaten från studien pekar på att en djup ensemble uppnår högre träffsäkerhet än ett Bayesianskt neuralt nätverk och att båda metoderna uppvisar återhållsamhet i sina förutsägelser jämfört med ett traditionellt djupt neuralt nätverk. Vi skattar osäkerhet som entropi och visar att osäkerheten i metodernas slutledningar ökar både på höga brusnivåer och på data som är något förskjuten från den kända datadistributionen. Resultaten visar dock att metodernas osäkerhet inte ökar jämfört med ett vanligt nätverk när de får se tidigare osedda signal mönster. Vi visar också att val av metod kan influeras av tillgängliga resurser, eftersom djupa ensembler kräver mycket minne jämfört med ett traditionellt eller Bayesianskt neuralt nätverk.

Magíster en Ciencias de la Ingeniería, Mención Mecánica
El desarrollo en tecnología de mediciones ha permitido el monitoreo continuo de sistemas complejos a través de múltiples sensores, generando así grandes bases de datos. Estos datos normalmente son almacenados para ser posteriormente analizados con técnicas tradicionales de Prognostics and Health Management (PHM). Sin embargo, muchas veces, gran parte de esta información es desperdiciada, ya que los métodos tradicionales de PHM requieren de conocimiento experto sobre el sistema para su implementación. Es por esto que, para estimar parámetros relacionados a confiabilidad, los enfoques basados en análisis de datos pueden utilizarse para complementar los métodos de PHM. El objetivo de esta tesis consiste en desarrollar e implementar un marco de trabajo basado en técnicas de Aprendizaje Profundo para la estimación del estado de salud de sistemas y componentes, utilizando datos multisensoriales de monitoreo. Para esto, se definen los siguientes objetivos específicos: Desarrollar una arquitectura capaz de extraer características temporales y espaciales de los datos. Proponer un marco de trabajo para la estimación del estado de salud, y validarlo utilizando dos conjuntos de datos: C-MAPSS turbofan engine, y baterías ion-litio CS2. Finalmente, entregar una estimación de la propagación de la incertidumbre en los pronósticos del estado de salud. Se propone una estructura que integre las ventajas de relación espacial de las Convolutional Neural Networks, junto con el análisis secuencial de las Long-Short Term Memory Recurrent Neural Networks. Utilizando Dropout tanto para la regularización, como también para una aproximación bayesiana para la estimación de incertidumbre de los modelos. De acuerdo con lo anterior, la arquitectura propuesta recibe el nombre CNNBiLSTM. Para los datos de C-MAPSS se entrenan cuatro modelos diferentes, uno para cada subconjunto de datos, con el objetivo de estimar la vida remanente útil. Los modelos arrojan resultados superiores al estado del arte en la raíz del error medio cuadrado (RMSE), mostrando robustez en el proceso de entrenamiento, y baja incertidumbre en sus predicciones. Resultados similares se obtienen para el conjunto de datos CS2, donde el modelo entrenado con todas las celdas de batería logra estimar el estado de carga y el estado de salud con un bajo RMSE y una pequeña incertidumbre sobre su estimación de valores. Los resultados obtenidos por los modelos entrenados muestran que la arquitectura propuesta es adaptable a diferentes sistemas y puede obtener relaciones temporales abstractas de los datos sensoriales para la evaluación de confiabilidad. Además, los modelos muestran robustez durante el proceso de entrenamiento, así como una estimación precisa con baja incertidumbre.

Dans ce manuscrit de thèse, nous développons de nouveaux algorithmes et modèles pour résoudre les problèmes d'apprentissage profond sur de la donnée séquentielle, en partant des problématiques posées par l'apprentissage des modèles de langage basés sur des réseaux de neurones. Un premier axe de recherche développe de nouveaux modèles génératifs profonds basés sur des méthodes de Monte Carlo Séquentielles (SMC), qui permettent de mieux modéliser la diversité du langage, ou de mieux quantifier l'incertitude pour des problèmes de régression séquentiels. Un deuxième axe de recherche vise à faciliter l'utilisation de techniques de SMC dans le cadre de l'apprentissage profond, en développant un nouvel algorithme de lissage à coût computationnel largement réduit, et qui s'applique à un scope plus large de modèles à espace d'états, notamment aux modèles génératifs profonds. Finalement, un troisième axe de recherche propose le premier algorithme d'apprentissage par renforcement permettant d'apprendre des modèles de langage conditionnels "ex-nihilo" (i.e sans jeu de données supervisé), basé sur un mécanisme de troncation de l'espace d'actions par un modèle de langage pré-entrainé
In this thesis, we develop new models and algorithms to solve deep learning tasks on sequential data problems, with the perspective of tackling the pitfalls of current approaches for learning language models based on neural networks. A first research work develops a new deep generative model for sequential data based on Sequential Monte Carlo Methods, that enables to better model diversity in language modelling tasks, and better quantify uncertainty in sequential regression problems. A second research work aims to facilitate the use of SMC techniques within deep learning architectures, by developing a new online smoothing algorithm with reduced computational cost, and applicable on a wider scope of state-space models, including deep generative models. Finally, a third research work proposes the first reinforcement learning that enables to learn conditional language models from scratch (i.e without supervised datasets), based on a truncation mechanism of the natural language action space with a pretrained language model

A machine learning approach to damage detection is presented for a bridge structural health monitoring system, validated on the renowned Z-24 bridge benchmark dataset where a sensor instrumented, threespan bridge was realistically damaged in stages. A Bayesian autoencoder neural network is trained to reconstruct raw sensor data sequences, with uncertainty bounds in prediction. The reconstruction error is then compared with a healthy-state error distribution and the sequence determined to come from a healthy state or not. Several realistic damage stages were successfully detected, making this a viable approach in a data-based monitoring system of an operational bridge. This is a fully operational, machine learning based bridge damage detection system, that is learned directly from raw sensor data.
En maskininlärningsmetod för strukturell skadedetektering av broar presenteras. Metoden valideras på det kända referensdataset Z-24, där en sensor-instrumenterad trespannsbro stegvist skadats. Ett Bayesianskt neuralt nätverk med autoenkoders tränas till att rekonstruera råa sensordatasekvenser, med osäkerhetsgränser i förutsägningen. Rekonstrueringsavvikelsen jämförs med avvikelsesfördelningen i oskadat tillstånd och sekvensen bedöms att komma från ett skadad eller icke skadat tillstånd. Flera realistiska stegvisa skadetillstånd upptäcktes, vilket gör metoden användbar i ett databaserat skadedetektionssystem för en bro i full storlek. Detta är ett lovande steg mot ett helt operativt databaserat skadedetektionssystem.

Sensor technology is rapidly developing and, consequently, the generation of point cloud data is constantly increasing. Since the recent release of PointNet, it is possible to process this unordered 3-dimensional data directly in a neural network. The company TLT Screen AB, which develops cutting-edge tracking technology, seeks to optimize the localization of the fingertips of a hand in a point cloud. To do so, the identification of relevant 3D neural network models for modeling hands and detection of fingertips in various hand orientations is essential. The Hand PointNet processes point clouds of hands directly and generate estimations of fixed points (joints), including fingertips, of the hands. Therefore, this model was selected to optimize the localization of fingertips for TLT Screen AB and forms the subject of this research. The model has advantages over conventional convolutional neural networks (CNN). First of all, in contrast to the 2D CNN, the Hand PointNet can use the full 3-dimensional spatial information. Compared to the 3D CNN, moreover, it avoids unnecessarily voluminous data and enables more efficient learning. The model was trained and evaluated on the public dataset MRSA Hand. In contrast to previously published work, the main object of this investigation is the estimation of only 5 joints, for the fingertips. The behavior of the model with a reduction from the usual 21 to 11 and only 5 joints are examined. It is found that the reduction of joints contributed to an increase in the mean error of the estimated joints. Furthermore, the examination of the distribution of the residuals of the estimate for fingertips is found to be less dense. MC dropout to study the prediction uncertainty for the fingertips has shown that the uncertainty increases when the joints are decreased. Finally, the results show that the uncertainty is greatest for the prediction of the thumb tip. Starting from the tip of the thumb, it is observed that the uncertainty of the estimates decreases with each additional fingertip.

This thesis addresses the problem of unreliable perception from vision models, particularly when encountering new, unexpected inputs. The research contributions presented in this thesis enable vision models to jointly predict visual information alongside an uncertainty for this prediction, allowing the model to indicate when it does not know. These methods are designed for the application of robotics, where complex vision models must operate accurately, robustly, and subject to computational constraints.

Ce travail se concentre sur la quantification de l'incertitude pour les réseaux de neurones profonds, qui est vitale pour la fiabilité et la précision de l'apprentissage profond. Cependant, la conception complexe du réseau et les données d'entrée limitées rendent difficile l'estimation des incertitudes. Parallèlement, la quantification de l'incertitude pour les tâches de régression a reçu moins d'attention que pour celles de classification en raison de la sortie standardisée plus simple de ces dernières et de leur grande importance. Cependant, des problèmes de régression sont rencontrés dans un large éventail d'applications en vision par ordinateur. Notre principal axe de recherche porte sur les méthodes post-hoc, et notamment les réseaux auxiliaires, qui constituent l'un des moyens les plus efficaces pour estimer l'incertitude des prédictions des tâches principales sans modifier le modèle de la tâche principale. Dans le même temps, le scénario d'application se concentre principalement sur les tâches de régression visuelle. En outre, nous fournissons également une méthode de quantification de l'incertitude basée sur le modèle modifié de tâche principale et un ensemble de données permettant d'évaluer la qualité et la robustesse des estimations de l'incertitude.Nous proposons d'abord Side Learning Uncertainty for Regression Problems (SLURP), une approche générique pour l'estimation de l'incertitude de régression via un réseau auxiliaire qui exploite la sortie et les représentations intermédiaires générées par le modèle pour la tâche principale. Le réseau auxiliaire apprend l'erreur de prédiction du modèle pour la tâche principale et peut fournir des estimations d'incertitude comparables à celles des approches des ensembles pour différentes tâches de régression par pixel.Pour être considéré comme robuste, un estimateur d'incertitude auxiliaire doit être capable de maintenir ses performances et de déclencher des incertitudes plus élevées tout en rencontrant des entrées des examples Out-Of-Distribution (OOD), c'est-à-dire de fournir une incertitude aléatoire et épistémique robuste. Nous considérons que SLURP est principalement adapté aux estimations de l'incertitude aléatoires. De plus, la robustesse des estimateurs auxiliaires d'incertitude n'a pas été explorée. Notre deuxième travail propose un schéma d'estimateur d'incertitude auxiliaire généralisé, introduisant la distribution de Laplace pour l'estimation aléatoire robuste de l'incertitude et le Discretization-Induced Dirichlet pOsterior (DIDO) pour l'incertitude épistémique. Des expériences approfondies confirment la robustesse dans diverses tâches.De plus, pour présenter DIDO, nous présentons un article d'évaluation des solutions qui appliquent des stratégies de discrétisation aux tâches de régression, développant une solution de quantification d'incertitude post-hoc, baptisée Expectation of Distance (E-Dist), qui surpasse les autres solutions post-hoc dans les mêmes conditions.De plus, nous étudions les méthodes de quantification de l'incertitude en un seul passage basées sur le modèle de tâche principale ajusté. Nous proposons Latent Discreminant deterministic Uncertainty (LDU), qui fait progresser l'estimation déterministe de l'incertitude évolutive et rivalise avec les Deep Ensembles sur les tâches d'estimation de profondeur monoculaire.En termes d'évaluation de la quantification de l'incertitude, nous proposons un ensemble de données Multiple Uncertainty Autonomous Driving (MUAD), prenant en charge diverses tâches de vision par ordinateur dans différents scénarios urbains avec des différents exemples OOD difficiles.En résumé, nous proposons de nouvelles solutions et références pour la quantification de l'incertitude de l'apprentissage profond, notamment SLURP, E-Dist, DIDO et LDU. De plus, nous proposons l'ensemble de données MUAD pour fournir une évaluation plus complète des scénarios de conduite autonome avec différentes sources d'incertitude
This work focuses on uncertainty quantification for deep neural networks, which is vital for reliability and accuracy in deep learning. However, complex network design and limited training data make estimating uncertainties challenging. Meanwhile, uncertainty quantification for regression tasks has received less attention than for classification ones due to the more straightforward standardized output of the latter and their high importance. However, regression problems are encountered in a wide range of applications in computer vision. Our main research direction is on post-hoc methods, and especially auxiliary networks, which are one of the most effective means of estimating the uncertainty of main task predictions without modifying the main task model. At the same time, the application scenario mainly focuses on visual regression tasks. In addition, we also provide an uncertainty quantification method based on the modified main task model and a dataset for evaluating the quality and robustness of uncertainty estimates.We first propose Side Learning Uncertainty for Regression Problems (SLURP), a generic approach for regression uncertainty estimation via an auxiliary network that exploits the output and the intermediate representations generated by the main task model. This auxiliary network effectively captures prediction errors and competes with ensemble methods in pixel-wise regression tasks.To be considered robust, an auxiliary uncertainty estimator must be capable of maintaining its performance and triggering higher uncertainties while encountering Out-of-Distribution (OOD) inputs, i.e., to provide robust aleatoric and epistemic uncertainty. We consider that SLURP is mainly adapted for aleatoric uncertainty estimates. Moreover, the robustness of the auxiliary uncertainty estimators has not been explored. Our second work presents a generalized auxiliary uncertainty estimator scheme, introducing the Laplace distribution for robust aleatoric uncertainty estimation and Discretization-Induced Dirichlet pOsterior (DIDO) for epistemic uncertainty. Extensive experiments confirm robustness in various tasks.Furthermore, to introduce DIDO, we provide a survey paper on regression with discretization strategies, developing a post-hoc uncertainty quantification solution, dubbed Expectation of Distance (E-Dist), which outperforms the other post-hoc solutions under the same settings. Additionally, we investigate single-pass uncertainty quantification methods, introducing Discriminant deterministic Uncertainty (LDU), which advances scalable deterministic uncertainty estimation and competes with Deep Ensembles on monocular depth estimation tasks.In terms of uncertainty quantification evaluation, we offer the Multiple Uncertainty Autonomous Driving dataset (MUAD), supporting diverse computer vision tasks in varying urban scenarios with challenging out-of-distribution examples.In summary, we contribute new solutions and benchmarks for deep learning uncertainty quantification, including SLURP, E-Dist, DIDO, and LDU. In addition, we propose the MUAD dataset to provide a more comprehensive evaluation of autonomous driving scenarios with different uncertainty sources

This thesis addresses the issues of data sparsity in the sonar domain. A data pipeline is set up to generate and enhance sonar data. The possibilities and limitations of using cycleGAN as a tool to enhance simulated sonar images for the purpose of training neural networks for detection and classification is studied. A neural network is trained on the enhanced simulated sonar images and tested on real sonar images to evaluate the quality of these images.The novelty of this work lies in extending previous methods to a more general framework and showing that GAN enhanced simulations work for complex tasks on field data.Using real sonar images to enhance the simulated images, resulted in improved classification compared to a classifier trained on solely simulated images.
Denna rapport ämnar undersöka problemet med gles data för djupinlärning i sonardomänen. Ett dataflöde för att generera och höja kvalitén hos simulerad sonardata sätts upp i syfte att skapa en stor uppsättning data för att träna ett neuralt nätverk. Möjligheterna och begränsningarna med att använda cycleGAN för att höja kvalitén hos simulerad sonardata studeras och diskuteras. Ett neuralt nätverk för att upptäcka och klassificera objekt i sonarbilder tränas i syfte att evaluera den förbättrade simulerade sonardatan.Denna rapport bygger vidare på tidigare metoder genom att generalisera dessa och visa att metoden har potential även för komplexa uppgifter baserad på icke trivial data.Genom att träna ett nätverk för klassificering och detektion på simulerade sonarbilder som använder cycleGAN för att höja kvalitén, ökade klassificeringsresultaten markant jämfört med att träna på enbart simulerade bilder.

The last decade has seen great advances within the field of artificial intelligence. One of the most noteworthy areas is that of deep learning, which is nowadays used in everything from self driving cars to automated cancer screening. During the same time, the amount of spatial data encompassing not only two but three dimensions has also grown and whole cities and countries are being scanned. Combining these two technological advances enables the creation of detailed maps with a multitude of applications, civilian as well as military.This thesis aims at combining two data sources covering most of Sweden; laser data from LiDAR scans and surface model from aerial images, with deep learning to create maps of the terrain. The target is to learn a simplified version of orienteering maps as these are created with high precision by experienced map makers, and are a representation of how easy or hard it would be to traverse a given area on foot. The performance on different types of terrain are measured and it is found that open land and larger bodies of water is identified at a high rate, while trails are hard to recognize.It is further researched how the different densities found in the source data affect the performance of the models, and found that some terrain types, trails for instance, benefit from higher density data, Other features of the terrain, like roads and buildings are predicted with higher accuracy by lower density data.Finally, the certainty of the predictions is discussed and visualised by measuring the average entropy of predictions in an area. These visualisations highlight that although the predictions are far from perfect, the models are more certain about their predictions when they are correct than when they are not.

Methods for uncertainty quantification (UQ) and mitigation in the electrical power system are very basic, Monte Carlo (MC) method and its meta methods are generally deployed in most applications, due to its simplicity and easy to be generalised. They are adequate for a traditional power system when the load is predictable, and generation is controllable. However, the large penetration of low carbon technologies, such as solar panels, electric vehicles, and energy storage, has necessitated the needs for more comprehensive approaches to uncertainty as these technologies introduce new sources of uncertainties with larger volume and diverse characteristics, understanding source and consequences of uncertainty becomes highly complex issues. Traditional methods assume that for a given system it has a unique uncertainty characteristic, hence deal with the uncertainty of the system as a single component in applications. However, this view is no longer applicable in the new context as it neglects the important underlying information associated with individual uncertainty components. Therefore, this thesis aims at: i) systematically developing UQ methodologies to identify, discriminate, and quantify different uncertainty components (forward UQ), and critically to model and trace the associated sources independently (inverse UQ) to deliver new uncertainty information, such as, how uncertainty components generated from its sources, how uncertainty components correlate with each other and how uncertainty components propagate through system aggregation; ii) applying the new uncertainty information to further improve a range of fundamental power system applications from Load Forecasting (LF) to Energy Management System (EMS).In the EMS application, the proposed forward UQ methods enable the development of a decentralised system that is able to tap into the new uncertainty information concerning the correlations between load pattern across individual households, the characteristics of uncertainty components and their propagation through aggregation. The decentralised EMS was able to achieve peak and uncertainty reduction by 18% and 45% accordingly at the grid level. In the LF application, this thesis developed inverse UQ through a deep learning model to directly build the connection between uncertainty components and its corresponding sources. For Load Forecasting on expectation (point LF) and probability (probabilistic LF) and witnessed 20%/12% performance improvement compared to the state-of-the-art, such as Support Vector Regression (SVR), Autoregressive Integrated Moving Average (ARIMA), and Multiple Linear Quantile Regression (MLQR).

Opinion classification is used to classify and analyze the opinions in text-based product and service reviews. However, due to the uncertainty of opinion data, it is difficult to gain satisfactory classification accuracy using existing machine learning algorithms. Therefore, how to deal with uncertainty in opinions to improve the performance of machine learning is a challenge. This thesis develops a three-way decision-making framework to support two-stage decision making. It first divides opinions into positive, negative, and boundary regions using fuzzy concepts, and then classifies the boundaries again using semantic features and deep learning. It provides a promising method for opinion classification.

Deep convolutional neural networks (CNN’s) have achieved state-of-the-art accuraciesfor multi-class segmentation in biomedical image science. In this thesis, A 3D U-Net isused to segment 4D flow Magnetic Resonance Images that include the heart and its largevessels. The 4 dimensional flow MRI dataset has been segmented and validated using amulti-atlas based registration technique. This multi-atlas based technique resulted in highquality segmentations, with the disadvantage of long computation times typically requiredby three-dimensional registration techniques. The 3D U-Net framework learns to classifyvoxels by transforming the information about the segmentation into a latent feature spacein a contracting path and upsampling them to semantic segmentation in an expandingpath. A CNN trained using a sufficiently diverse set of volumes at different time intervalsof the diastole and systole should be able to handle more extreme morphological differencesbetween subjects. Evaluation of the results is based on metric for segmentation evaluationsuch as Dice coefficient. Uncertainty is estimated using a bayesian implementationof the 3D U-Net of similar architecture.

The presentation was online over zoom due to covid19 restrictions.

Chromosome based genetic diagnostics, the detection of specific chromosomes, plays an increasingly important role in medicine as the molecular basis of hu- man disease is defined. The current diagnostic process is performed mainly by karyotyping specialists. They first put chromosomes in pairs and generate an image listing all the chromosome pairs in order. This process is called kary- otyping, and the generated image is called karyogram. Then they analyze the images based on the shapes, size, and relationships of different image segments and then make diagnostic decisions. Manual inspection is time-consuming, labor-intensive, and error-prone.This thesis investigates supervised methods for genetic diagnostics on karyo- grams. Mainly, the theory targets abnormality detection and gives the confi- dence of the result in the chromosome domain. This thesis aims to divide chromosome pictures into normal and abnormal categories and give the con- fidence level. The main contributions of this thesis are (1) an empirical study of chromosome and karyotyping; (2) appropriate data preprocessing; (3) neu- ral networks building by using transfer learning; (4) experiments on different systems and conditions and comparison of them; (5) a right choice for our requirement and a way to improve the model; (6) a method to calculate the confidence level of the result by uncertainty estimation.Empirical research shows that the karyogram is ordered as a whole, so preprocessing such as rotation and folding is not appropriate. It is more rea- sonable to choose noise or blur. In the experiment, two neural networks based on VGG16 and InceptionV3 were established using transfer learning and com- pared their effects under different conditions. We hope to minimize the error of assuming normal cases because we cannot accept that abnormal chromo- somes are predicted as normal cases. This thesis describes how to use Monte Carlo Dropout to do uncertainty estimation like a non-Bayesian model[1].
Kromosombaserad genetisk diagnostik, detektering av specifika kromosomer, kommer att spela en allt viktigare roll inom medicin eftersom den molekylära grunden för mänsklig sjukdom definieras. Den nuvarande diagnostiska pro- cessen utförs huvudsakligen av specialister på karyotypning. De sätter först kromosomer i par och genererar en bild som listar alla kromosompar i ord- ning. Denna process kallas karyotypning, och den genererade bilden kallas karyogram. Därefter analyserar de bilderna baserat på former, storlek och för- hållanden för olika bildsegment och fattar sedan diagnostiska beslut.Denna avhandling undersöker övervakade metoder för genetisk diagnostik på karyogram. Huvudsakligen riktar teorin sig mot onormal detektion och ger förtroendet för resultatet i kromosomdomänen. Manuell inspektion är tidskrä- vande, arbetskrävande och felbenägen. Denna uppsats syftar till att dela in kro- mosombilder i normala och onormala kategorier och ge konfidensnivån. Dess huvudsakliga bidrag är (1) en empirisk studie av kromosom och karyotyp- ning; (2) lämplig förbehandling av data; (3) Neurala nätverk byggs med hjälp av transfer learning; (4) experiment på olika system och förhållanden och jäm- förelse av dem; (5) ett rätt val för vårt krav och ett sätt att förbättra modellen;    en metod för att beräkna resultatets konfidensnivå genom osäkerhetsupp- skattning.    Empirisk forskning visar att karyogrammet är ordnat som en helhet, så förbehandling som rotation och vikning är inte lämpligt. Det är rimligare att välja brus, oskärpa etc. I experimentet upprättades två neurala nätverk base- rade på VGG16 och InceptionV3 med hjälp av transfer learning och jämförde deras effekter under olika förhållanden. När vi väljer utvärderingsindikatorer, eftersom vi inte kan acceptera att onormala kromosomer bedöms förväntas, hoppas vi att minimera felet att anta som vanligt. Denna avhandling beskriver hur man använder Monte Carlo Dropout för att göra osäkerhetsberäkningar som en icke-Bayesisk modell [1].

In recent years, development of Convolutional Neural Networks has enabled high performing semantic segmentation models. Generally, these deep learning based segmentation methods require a large amount of annotated data. Acquiring such annotated data for semantic segmentation is a tedious and expensive task. Within machine learning, active learning involves in the selection of new data in order to limit the usage of annotated data. In active learning, the model is trained for several iterations and additional samples are selected that the model is uncertain of. The model is then retrained on additional samples and the process is repeated again. In this thesis, an active learning framework has been applied to road segmentation which is semantic segmentation of objects related to road scenes. The uncertainty in the samples is estimated with Monte Carlo dropout. In Monte Carlo dropout, several dropout masks are applied to the model and the variance is captured, working as an estimate of the model’s uncertainty. Other metrics to rank the uncertainty evaluated in this work are: a baseline method that selects samples randomly, the entropy in the default predictions and three additional variations/extensions of Monte Carlo dropout. Both the active learning framework and uncertainty estimation are implemented in the thesis. Monte Carlo dropout performs slightly better than the baseline in 3 out of 4 metrics. Entropy outperforms all other implemented methods in all metrics. The three additional methods do not perform better than Monte Carlo dropout. An analysis of what kind of uncertainty Monte Carlo dropout capture is performed together with a comparison of the samples selected by baseline and Monte Carlo dropout. Future development and possible improvements are also discussed.

Táto práca sa zaoberá určením neistoty v predikciách modelov hlbokého učenia. Aj keď sa týmto modelom darí dosahovať vynikajúce výsledky v mnohých oblastiach počítačového videnia, ich výstupy sú väčšinou deterministické a neposkytujú mnoho informácií o tom, ako si je model istý svojou predpoveďou. To je obzvlášť dôležité pri analýze lekárskych obrazových dát, kde môžu mať omyly vysokú cenu a schopnosť detekovať neisté predikcie by umožnila dohliadajúcemu lekárovi spracovať relevantné prípady manuálne. V tejto práci aplikujem niekoľko rôznych metrík vyvinutých v nedávnom výskume pre určenie neistoty na modely hlbokého učenia natrénované pre lokalizáciu cefalometrických landmarkov. Následne ich vyhodnotím a porovnávam v sade experimentov, ktorých úlohou je určiť, nakoľko jednotlivé metriky poskytujú užitočnú informáciu o tom, ako si je model istý svojou predpoveďou.

Les réseaux de neurones profonds (DNN) ont obtenu un succès remarquable sur de nombreuses applications du monde réel (par exemple, la reconnaissance de formes et la segmentation sémantique), mais sont toujours confrontés au problème de la gestion de l'incertitude. La théorie de Dempster-Shafer (DST) fournit un cadre bien fondé et élégant pour représenter et raisonner avec des informations incertaines. Dans cette thèse, nous avons proposé un nouveau framework utilisant DST et DNNs pour résoudre les problèmes d'incertitude. Dans le cadre proposé, nous hybridons d'abord DST et DNN en branchant une couche de réseau neuronal basée sur DST suivie d'une couche utilitaire à la sortie d'un réseau neuronal convolutif pour la classification à valeur définie. Nous étendons également l'idée à la segmentation sémantique en combinant des réseaux entièrement convolutifs et DST. L'approche proposée améliore les performances des modèles DNN en attribuant des modèles ambigus avec une incertitude élevée, ainsi que des valeurs aberrantes, à des ensembles multi-classes. La stratégie d'apprentissage utilisant des étiquettes souples améliore encore les performances des DNN en convertissant des données d'étiquettes imprécises et non fiables en fonctions de croyance. Nous avons également proposé une stratégie de fusion modulaire utilisant ce cadre proposé, dans lequel un module de fusion agrège les sorties de la fonction de croyance des DNN évidents selon la règle de Dempster. Nous utilisons cette stratégie pour combiner des DNN formés à partir d'ensembles de données hétérogènes avec différents ensembles de classes tout en conservant des performances au moins aussi bonnes que celles des réseaux individuels sur leurs ensembles de données respectifs. De plus, nous appliquons la stratégie pour combiner plusieurs réseaux superficiels et obtenir une performance similaire d'un DNN avancé pour une tâche compliquée
Deep neural networks (DNNs) have achieved remarkable success on many realworld applications (e.g., pattern recognition and semantic segmentation) but still face the problem of managing uncertainty. Dempster-Shafer theory (DST) provides a wellfounded and elegant framework to represent and reason with uncertain information. In this thesis, we have proposed a new framework using DST and DNNs to solve the problems of uncertainty. In the proposed framework, we first hybridize DST and DNNs by plugging a DSTbased neural-network layer followed by a utility layer at the output of a convolutional neural network for set-valued classification. We also extend the idea to semantic segmentation by combining fully convolutional networks and DST. The proposed approach enhances the performance of DNN models by assigning ambiguous patterns with high uncertainty, as well as outliers, to multi-class sets. The learning strategy using soft labels further improves the performance of the DNNs by converting imprecise and unreliable label data into belief functions. We have also proposed a modular fusion strategy using this proposed framework, in which a fusion module aggregates the belief-function outputs of evidential DNNs by Dempster’s rule. We use this strategy to combine DNNs trained from heterogeneous datasets with different sets of classes while keeping at least as good performance as those of the individual networks on their respective datasets. Further, we apply the strategy to combine several shallow networks and achieve a similar performance of an advanced DNN for a complicated task

Nowadays, lung cancer assessment is a complex and tedious task mainly per- formed by radiological visual inspection of suspicious pulmonary nodules, using computed tomography (CT) scan images taken to patients over time. Several computational tools relying on conventional artificial intelligence and computer vision algorithms have been proposed for supporting lung cancer de- tection and classification. These solutions mostly rely on the analysis of indi- vidual lung CT images of patients and on the use of hand-crafted image de- scriptors. Unfortunately, this makes them unable to cope with the complexity and variability of the problem. Recently, the advent of deep learning has led to a major breakthrough in the medical image domain, outperforming conven- tional approaches. Despite recent promising achievements in nodule detection, segmentation, and lung cancer classification, radiologists are still reluctant to use these solutions in their day-to-day clinical practice. One of the main rea- sons is that current solutions do not provide support to automatic analysis of the temporal evolution of lung tumours. The difficulty to collect and annotate longitudinal lung CT cases to train models may partially explain the lack of deep learning studies that address this issue. In this dissertation, we investigate how to automatically provide lung can- cer assessment through deep learning algorithms and computer vision pipelines, especially taking into consideration the temporal evolution of the pulmonary nodules. To this end, our first goal consisted on obtaining accurate methods for lung cancer assessment (diagnostic ground truth) based on individual lung CT images. Since these types of labels are expensive and difficult to collect (e.g. usually after biopsy), we proposed to train different deep learning models, based on 3D convolutional neural networks (CNN), to predict nodule malig- nancy based on radiologist visual inspection annotations (which are reasonable to obtain). These classifiers were built based on ground truth consisting of the nodule malignancy, the position and the size of the nodules to classify. Next, we evaluated different ways of synthesizing the knowledge embedded by the nodule malignancy neural network, into an end-to-end pipeline aimed to detect pul- monary nodules and predict lung cancer at the patient level, given a lung CT image. The positive results confirmed the convenience of using CNNs for mod- elling nodule malignancy, according to radiologists, for the automatic prediction of lung cancer. Next, we focused on the analysis of lung CT image series. Thus, we first faced the problem of automatically re-identifying pulmonary nodules from dif- ferent lung CT scans of the same patient. To do this, we present a novel method based on a Siamese neural network (SNN) to rank similarity between nodules, overpassing the need for image registration. This change of paradigm avoided introducing potentially erroneous image deformations and provided computa- tionally faster results. Different configurations of the SNN were examined, in- cluding the application of transfer learning, using different loss functions, and the combination of several feature maps of different network levels. This method obtained state-of-the-art performances for nodule matching both in an isolated manner and embedded in an end-to-end nodule growth detection pipeline. Afterwards, we moved to the core problem of supporting radiologists in the longitudinal management of lung cancer. For this purpose, we created a novel end-to-end deep learning pipeline, composed of four stages that completely au- tomatize from the detection of nodules to the classification of cancer, through the detection of growth in the nodules. In addition, the pipeline integrated a novel approach for nodule growth detection, which relies on a recent hierarchi- cal probabilistic segmentation network adapted to report uncertainty estimates. Also, a second novel method was introduced for lung cancer nodule classification, integrating into a two stream 3D-CNN the estimated nodule malignancy prob- abilities derived from a pre-trained nodule malignancy network. The pipeline was evaluated in a longitudinal cohort and the reported outcomes (i.e. nodule detection, re-identification, growth quantification, and malignancy prediction) were comparable with state-of-the-art work, focused on solving one or a few of the functionalities of our pipeline. Thereafter, we also investigated how to help clinicians to prescribe more accurate tumour treatments and surgical planning. Thus, we created a novel method to forecast nodule growth given a single image of the nodule. Partic- ularly, the method relied on a hierarchical, probabilistic and generative deep neural network able to produce multiple consistent future segmentations of the nodule at a given time. To do this, the network learned to model the mul- timodal posterior distribution of future lung tumour segmentations by using variational inference and injecting the posterior latent features. Eventually, by applying Monte-Carlo sampling on the outputs of the trained network, we esti- mated the expected tumour growth mean and the uncertainty associated with the prediction. Although further evaluation in a larger cohort would be highly recommended, the proposed methods reported accurate results to adequately support the ra- diological workflow of pulmonary nodule follow-up. Beyond this specific appli- cation, the outlined innovations, such as the methods for integrating CNNs into computer vision pipelines, the re-identification of suspicious regions over time based on SNNs, without the need to warp the inherent image structure, or the proposed deep generative and probabilistic network to model tumour growth considering ambiguous images and label uncertainty, they could be easily appli- cable to other types of cancer (e.g. pancreas), clinical diseases (e.g. Covid-19) or medical applications (e.g. therapy follow-up).
Avui en dia, l’avaluació del càncer de pulmó ´es una tasca complexa i tediosa, principalment realitzada per inspecció visual radiològica de nòduls pulmonars sospitosos, mitjançant imatges de tomografia computada (TC) preses als pacients al llarg del temps. Actualment, existeixen diverses eines computacionals basades en intel·ligència artificial i algorismes de visió per computador per donar suport a la detecció i classificació del càncer de pulmó. Aquestes solucions es basen majoritàriament en l’anàlisi d’imatges individuals de TC pulmonar dels pacients i en l’ús de descriptors d’imatges fets a mà. Malauradament, això les fa incapaces d’afrontar completament la complexitat i la variabilitat del problema. Recentment, l’aparició de l’aprenentatge profund ha permès un gran avenc¸ en el camp de la imatge mèdica. Malgrat els prometedors assoliments en detecció de nòduls, segmentació i classificació del càncer de pulmó, els radiòlegs encara són reticents a utilitzar aquestes solucions en el seu dia a dia. Un dels principals motius ´es que les solucions actuals no proporcionen suport automàtic per analitzar l’evolució temporal dels tumors pulmonars. La dificultat de recopilar i anotar cohorts longitudinals de TC pulmonar poden explicar la manca de treballs d’aprenentatge profund que aborden aquest problema. En aquesta tesi investiguem com abordar el suport automàtic a l’avaluació del càncer de pulmó, construint algoritmes d’aprenentatge profund i pipelines de visió per ordinador que, especialment, tenen en compte l’evolució temporal dels nòduls pulmonars. Així doncs, el nostre primer objectiu va consistir a obtenir mètodes precisos per a l’avaluació del càncer de pulmó basats en imatges de CT pulmonar individuals. Atès que aquests tipus d’etiquetes són costoses i difícils d’obtenir (per exemple, després d’una biòpsia), vam dissenyar diferents xarxes neuronals profundes, basades en xarxes de convolució 3D (CNN), per predir la malignitat dels nòduls basada en la inspecció visual dels radiòlegs (més senzilles de recol.lectar). A continuació, vàrem avaluar diferents maneres de sintetitzar aquest coneixement representat en la xarxa neuronal de malignitat, en una pipeline destinada a proporcionar predicció del càncer de pulmó a nivell de pacient, donada una imatge de TC pulmonar. Els resultats positius van confirmar la conveniència d’utilitzar CNN per modelar la malignitat dels nòduls, segons els radiòlegs, per a la predicció automàtica del càncer de pulmó. Seguidament, vam dirigir la nostra investigació cap a l’anàlisi de sèries d’imatges de TC pulmonar. Per tant, ens vam enfrontar primer a la reidentificació automàtica de nòduls pulmonars de diferents tomografies pulmonars. Per fer-ho, vam proposar utilitzar xarxes neuronals siameses (SNN) per classificar la similitud entre nòduls, superant la necessitat de registre d’imatges. Aquest canvi de paradigma va evitar possibles pertorbacions de la imatge i va proporcionar resultats computacionalment més ràpids. Es van examinar diferents configuracions del SNN convencional, que van des de l’aplicació de l’aprenentatge de transferència, utilitzant diferents funcions de pèrdua, fins a la combinació de diversos mapes de característiques de diferents nivells de xarxa. Aquest mètode va obtenir resultats d’estat de la tècnica per reidentificar nòduls de manera aïllada, i de forma integrada en una pipeline per a la quantificació de creixement de nòduls. A més, vam abordar el problema de donar suport als radiòlegs en la gestió longitudinal del càncer de pulmó. Amb aquesta finalitat, vam proposar una nova pipeline d’aprenentatge profund, composta de quatre etapes que s’automatitzen completament i que van des de la detecció de nòduls fins a la classificació del càncer, passant per la detecció del creixement dels nòduls. A més, la pipeline va integrar un nou enfocament per a la detecció del creixement dels nòduls, que es basava en una recent xarxa de segmentació probabilística jeràrquica adaptada per informar estimacions d’incertesa. A més, es va introduir un segon mètode per a la classificació dels nòduls del càncer de pulmó, que integrava en una xarxa 3D-CNN de dos fluxos les probabilitats estimades de malignitat dels nòduls derivades de la xarxa pre-entrenada de malignitat dels nòduls. La pipeline es va avaluar en una cohort longitudinal i va informar rendiments comparables a l’estat de la tècnica utilitzats individualment o en pipelines però amb menys components que la proposada. Finalment, també vam investigar com ajudar els metges a prescriure de forma més acurada tractaments tumorals i planificacions quirúrgiques més precises. Amb aquesta finalitat, hem realitzat un nou mètode per predir el creixement dels nòduls donada una única imatge del nòdul. Particularment, el mètode es basa en una xarxa neuronal profunda jeràrquica, probabilística i generativa capaç de produir múltiples segmentacions de nòduls futurs consistents del nòdul en un moment determinat. Per fer-ho, la xarxa aprèn a modelar la distribució posterior multimodal de futures segmentacions de tumors pulmonars mitjançant la utilització d’inferència variacional i la injecció de les característiques latents posteriors. Finalment, aplicant el mostreig de Monte-Carlo a les sortides de la xarxa, podem estimar la mitjana de creixement del tumor i la incertesa associada a la predicció. Tot i que es recomanable una avaluació posterior en una cohort més gran, els mètodes proposats en aquest treball han informat resultats prou precisos per donar suport adequadament al flux de treball radiològic del seguiment dels nòduls pulmonars. Més enllà d’aquesta aplicació especifica, les innovacions presentades com, per exemple, els mètodes per integrar les xarxes CNN a pipelines de visió per ordinador, la reidentificació de regions sospitoses al llarg del temps basades en SNN, sense la necessitat de deformar l’estructura de la imatge inherent o la xarxa probabilística per modelar el creixement del tumor tenint en compte imatges ambigües i la incertesa en les prediccions, podrien ser fàcilment aplicables a altres tipus de càncer (per exemple, pàncrees), malalties clíniques (per exemple, Covid-19) o aplicacions mèdiques (per exemple, seguiment de la teràpia).

Vehicle Path Prediction can be used to support Advanced Driver Assistance Systems (ADAS) that covers different technologies like Autonomous Braking System, Adaptive Cruise Control, etc. In this thesis, the vehicle’s future path, parameterized as 5 coordinates along the path, is predicted by using only visual data collected by a front vision sensor. This approach provides cheaper application opportunities without using different sensors. The predictions are done by deep convolutional neural networks (CNN) and the goal of the project is to use recurrent neural networks (RNN) and to investigate the benefits of using reccurence to the task. Two different approaches are used for the models. The first approach is a single-frame approach that makes predictions by using only one image frame as input and predicts the future location points of the car. The single-frame approach is the baseline model. The second approach is a sequential approach that enables the network the usage of historical information of previous image frames in order to predict the vehicle’s future path for the current frame. With this approach, the effect of using recurrence is investigated. Moreover, uncertainty is important for the model reliability. Having a small uncertainty in most of the predictions or having a high uncertainty in unfamiliar situations for the model will increase success of the model. In this project, the uncertainty estimation approach is based on capturing the uncertainty by following a method that allows to work on deep learning models. The uncertainty approach uses the same models that are defined by the first two approaches. Finally, the evaluation of the approaches are done by the mean absolute error and defining two different reasonable tolerance levels for the distance between the prediction path and the ground truth path. The difference between two tolerance levels is that the first one is a strict tolerance level and the the second one is a more relaxed tolerance level. When using strict tolerance level based on distances on test data, 36% of the predictions are accepted for single-frame model, 48% for the sequential model, 27% and 13% are accepted for single-frame and sequential models of uncertainty models. When using relaxed tolerance level on test data, 60% of the predictions are accepted by single-frame model, 67% for the sequential model, 65% and 53% are accepted for single-frame and sequential models of uncertainty models. Furthermore, by using stored information for each sequence, the methods are evaluated for different conditions such as day/night, road type and road cover. As a result, the sequential model outperforms in the majority of the evaluation results.

L'utilisation croissante de solutions technologiques fondées sur des algorithmes d'apprentissage profond a connu une explosion ces dernières années en raison de leurs performances sur des tâches de détection d'objets, de segmentation d'images et de vidéos ou encore de classification, et ce dans de nombreux domaines tels que la médecine, la finance, la conduite autonome ... Dans ce contexte, la recherche en apprentissage profond se concentre de plus en plus sur l'amélioration des performances et une meilleure compréhension des algorithmes utilisés en essayant de quantifier l'incertitude associée à leurs prédictions. Fournir cette incertitude est clé pour une dissémination massive de ces nouveaux outils dans l'industrie et lever les freins actuels pour des systèmes critiques notamment. En effet, fournir l'information de l'incertitude peut revêtir une importance réglementaire dans certains secteurs d'activité.Ce manuscrit expose nos travaux menés sur la quantification de l'incertitude au sein des réseaux de neurones. Pour commencer, nous proposons un état des lieux approfondi en explicitant les concepts clés impliqués dans un cadre métrologique. Ensuite, nous avons fait le choix de nous concentrer sur la propagation de l'incertitude des entrées à travers un réseau de neurones d'ores-et-déjà entraîné afin de répondre à un besoin industriel pressant. La méthode de propagation de l'incertitude des entrées proposée, nommée WGMprop, modélise les sorties du réseau comme des mixtures de gaussiennes dont la propagation de l'incertitude est assurée par un algorithme Split&Merge muni d'une mesure de divergence choisie comme la distance de Wasserstein. Nous nous sommes ensuite focalisés sur la quantification de l'incertitude inhérente aux paramètres du réseau. Dans ce cadre, une étude comparative des méthodes à l'état de l'art a été réalisée. Cette étude nous a notamment conduit à proposer une méthode de caractérisation locale des ensembles profonds, méthode faisant office de référence à l'heure actuelle. Notre méthodologie, nommée WEUQ, permet une exploration des bassins d'attraction du paysage des paramètres des réseaux de neurones en prenant en compte la diversité des prédicteurs. Enfin, nous présentons notre cas d'application, consistant en la mesure automatisée de la distribution des tailles de nanoparticules de dioxyde de titane à partir d'images acquises par microscopie électronique à balayage (MEB). Nous décrivons à cette occasion le développement de la brique technologique utilisée ainsi que les choix méthodologiques de quantification d'incertitudes découlant de nos recherches
The growing use of technological solutions based on deep learning algorithms has exploded in recent years, due to their performance on tasks such as object detection, image and video segmentation and classification, in many fields such as medicine, finance, autonomous driving... In this context, deep learning research is increasingly focusing on improving the performance and understanding of the algorithms used, by attempting to quantify the uncertainty associated with their predictions. Providing this uncertainty is key to the mass dissemination of these new tools in industry, and to overcoming the current obstacles to their use, particularly in critical systems. Indeed, providing information on uncertainty may be of regulatory importance in certain sectors of activity.This manuscript presents our work on uncertainty quantification in neural networks. To begin with, we provide an in-depth overview, explaining the key concepts involved in a metrological framework. Next, we have chosen to focus on the propagation of input uncertainty through an already-trained neural network, in response to a pressing industrial need. The proposed input uncertainty propagation method, named WGMprop, models the network outputs as mixtures of Gaussians, whose uncertainty propagation is ensured by a Split&Merge algorithm equipped with a divergence measure chosen as the Wasserstein distance. We then focused on quantifying the uncertainty inherent in the network parameters. In this context, a comparative study of state-of-the-art methods was carried out. In particular, this study led us to propose a method for local characterization of deep ensembles, which is currently the standard. Our methodology, named WEUQ, enables an exploration of the basins of attraction of the neural network parameter landscape, taking into account the diversity of predictors. Finally, we present our case study, involving the automated measurement of the size distribution of titanium dioxide nanoparticles from images acquired by scanning electron microscopy (SEM). We take this opportunity to describe the development of the technology used, and the methodological choices for quantifying the uncertainties arising from our research

Dans ce manuscrit, nous développons des architectures d'apprentissage profond pour modéliser la consommation énergétique et la qualité de l'air de bâtiments.Nous présentons d'abord une méthodologie de bout-en-bout permettant d'optimiser la demande énergétique tout en améliorant le confort, en substituant au traditionnel simulateur physique un modèle num'eriquement plus efficace.A partir de données historiques, nous vérifions que les simulations de ce métamodèle correspondent aux conditions réelles du bâtiment.Cependant, les performances des prédictions sont dégradées dans certaines situations à cause de différents facteurs alétoires.Nous proposons alors de quantifier l'incertitude des prédictions en combinant des modèles à espaces d'état à des modèles d'apprentissage profond pour les séries temporelles.Dans une première approche, nous montrons comment les poids d'un modèle peuvent être affinés par des méthodes de Monte Carlo séquentielles, afin de prendre en compte l'incertitude sur la dernière couche.Nous proposons un second modèle génératif à états latents discrets, permettant une procédure d'apprentissage moins coûteuse par Inférence Variationnelle ayant des performances équivalentes sur une tâche de prévision de l'humidité relative.Enfin, notre dernière contribution étend l'utilisation de ces modèles discrets, en proposant une nouvelle loi a priori basée sur des ponts de diffusion.En apprenant à corrompre puis à reconstruire des échantillons de l'espace latent, notre modèle est capable d'apprendre la distribution a priori, quelle que soit la nature des données
In this thesis, we develop deep learning architectures for modelling building energy consumption and air quality.We first present an end-to-end methodology for optimizing energy demand while improving indoor comfort, by substituting the traditionally used physical simulators with a much faster surrogate model.Using historic data, we can ensure that simulations from this metamodel match the real conditions of the buildings.Yet some differences remain, due to unavailable and random factors.We propose to quantify this uncertainty by combining state space models with time series deep learning models.In a first approach, we show how the weights of a model can be finetuned through Sequential Monte Carlo methods, in order to take into account uncertainty on the last layer.We propose a second generative model with discrete latent states, allowing for a simpler training procedure through Variational Inference and equivalent performances on a relative humidity forecasting task.Finally, our last work extends on these quantized models, by proposing a new prior based on diffusion bridges.By learning to corrupt and reconstruct samples from the latent space, our model is able to learn the complex prior distribution, regardless of the nature of the data

La dosimétrie biologique est la branche de la physique de la santé qui se préoccupe de l’estimation de doses de rayonnement ionisants à partir de biomarqueurs. Dans le procédé standard défini par l’AIEA, la dose est calculée en estimant la fréquence d’apparition de chromosomes dicentriques lors de la métaphase des lymphocytes périphériques. La variabilité morphologique des chromosomes, ainsi que celle des conditions d’acquisition des images rend ce problème de détection d’objets complexe. De plus, l’estimation fiable de cette fréquence nécessite le traitement d’un grand nombre d’image. Étant donné les limites du comptage humain (faible nombre de personnes qualifiées, charge cognitive), l’automatisation est une nécessité dans le contexte d’exposition de masse. Dans ce contexte, l’objectif de cette thèse est de tirer parti des progrès récents en vision par ordinateur (et plus spécifiquement en détection d’objets) apportés par l’apprentissage profond. La contribution principale de ce travail est une preuve de concept pour un modèle de détection de chromosomes dicentriques. Ce système repose sur l’agrégation de modèles pour parvenir à un haut niveau de performance, ainsi qu’a une bonne quantification de son incertitude, une exigence essentielle dans un contexte médical
Biological dosimetry is the branch of health physics dealing with the estimation of ionizing radiation doses from biomarkers. The current gold standard (defined by the IAEA) relies on estimating how frequently dicentric chromosomes appear in peripheral blood lymphocytes. Variations in acquisition conditions and chromosome morphology makes this a challenging object detection problem. Furthermore, the need for an accurate estimation of the average number of dicentric per cell means that a large number of image has to be processed. Human counting is intrinsically limited, as cognitive load is high and the number of specialist insufficient in the context of a large-scale exposition. The main goal of this PhD is to use recent developments in computer vision brought by deep learning, especially for object detection. The main contribution of this thesis is a proof of concept for a dicentric chromosome detection model. This model agregates several Unet models to reach a high level of performance and quantify its prediction uncertainty, which is a stringent requirement in a medical setting

Most of the decisions in real-life problems need to be made in the absence of complete knowledge about the consequences of the decision. Furthermore, in some of these problems, the probability and/or the number of different outcomes are also unknown (named deep uncertainty). Therefore, all the probability-based approaches (such as stochastic programming) are unable to address these problems. On the other hand, involving various stakeholders with different (possibly conflicting) criteria in the problems brings additional complexity. The main aim and primary motivation for writing this thesis have been to deal with deep uncertainty in Multi-Criteria Decision-Making (MCDM) problems, especially with long-term decision-making processes such as strategic planning problems. To achieve these aims, we first introduced a two-stage scenario-based structure for dealing with deep uncertainty in Multi-Objective Optimisation (MOO)/MCDM problems. The proposed method extends the concept of two-stage stochastic programming with recourse to address the capability of dealing with deep uncertainty through the use of scenario planning rather than statistical expectation. In this research, scenarios are used as a dimension of preference (a component of what we term the meta-criteria) to avoid problems relating to the assessment and use of probabilities under deep uncertainty. Such scenario-based thinking involved a multi-objective representation of performance under different future conditions as an alternative to expectation, which fitted naturally into the broader multi-objective problem context. To aggregate these objectives of the problem, the Generalised Goal Programming (GGP) approach is used. Due to the capability of this approach to handle large numbers of objective functions/criteria, the GGP is significantly useful in the proposed framework. Identifying the goals for each criterion is the only action that the Decision Maker (DM) needs to take without needing to investigate the trade-offs between different criteria. Moreover, the proposed two-stage framework has been expanded to a three-stage structure and a moving horizon concept to handle the existing deep uncertainty in more complex problems, such as strategic planning. As strategic planning problems will deal with more than two stages and real processes are continuous, it follows that more scenarios will continuously be unfolded that may or may not be periodic. "Stages", in this study, are artificial constructs to structure thinking of an indefinite future. A suitable length of the planning window and stages in the proposed methodology are also investigated. Philosophically, the proposed two-stage structure always plans and looks one step ahead while the three-stage structure considers the conditions and consequences of two upcoming steps in advance, which fits well with our primary objective. Ignoring long-term consequences of decisions as well as likely conditions could not be a robust strategic approach. Therefore, generally, by utilising the three-stage structure, we may expect a more robust decision than with a two-stage representation. Modelling time preferences in multi-stage problems have also been introduced to solve the fundamental problem of comparability of the two proposed methodologies because of the different time horizon, as the two-stage model is ignorant of the third stage. This concept has been applied by a differential weighting in models. Importance weights, then, are primarily used to make the two- and three-stage models more directly comparable, and only secondarily as a measure of risk preference. Differential weighting can help us apply further preferences in the model and lead it to generate more preferred solutions. Expanding the proposed structure to the problems with more than three stages which usually have too many meta-scenarios may lead us to a computationally expensive model that cannot easily be solved, if it all. Moreover, extension to a planning horizon that too long will not result in an exact plan, as nothing in nature is predictable to this level of detail, and we are always surprised by new events. Therefore, beyond the expensive computation in a multi-stage structure for more than three stages, defining plausible scenarios for far stages is not logical and even impossible. Therefore, the moving horizon models in a T-stage planning window has been introduced. To be able to run and evaluate the proposed two- and three-stage moving horizon frameworks in longer planning horizons, we need to identify all plausible meta-scenarios. However, with the assumption of deep uncertainty, this identification is almost impossible. On the other hand, even with a finite set of plausible meta-scenarios, comparing and computing the results in all plausible meta-scenarios are hardly possible, because the size of the model grows exponentially by raising the length of the planning horizon. Furthermore, analysis of the solutions requires hundreds or thousands of multi-objective comparisons that are not easily conceivable, if it all. These issues motivated us to perform a Simulation-Optimisation study to simulate the reasonable number of meta-scenarios and enable evaluation, comparison and analysis of the proposed methods for the problems with a T-stage planning horizon. In this Simulation-Optimisation study, we started by setting the current scenario, the scenario that we were facing it at the beginning of the period. Then, the optimisation model was run to get the first-stage decisions which can implement immediately. Thereafter, the next scenario was randomly generated by using Monte Carlo simulation methods. In deep uncertainty, we do not have enough knowledge about the likelihood of plausible scenarios nor the probability space; therefore, to simulate the deep uncertainty we shall not use anything of scenario likelihoods in the decision models. The two- and three-stage Simulation-Optimisation algorithms were also proposed. A comparison of these algorithms showed that the solutions to the two-stage moving horizon model are feasible to the other pattern (three-stage). Also, the optimal solution to the three-stage moving horizon model is not dominated by any solutions of the other model. So, with no doubt, it must find better, or at least the same, goal achievement compared to the two-stage moving horizon model. Accordingly, the three-stage moving horizon model evaluates and compares the optimal solution of the corresponding two-stage moving horizon model to the other feasible solutions, then, if it selects anything else it must either be better in goal achievement or be robust in some future scenarios or a combination of both. However, the cost of these supremacies must be considered (as it may lead us to a computationally expensive problem), and the efficiency of applying this structure needs to be approved. Obviously, using the three-stage structure in comparison with the two-stage approach brings more complexity and calculations to the models. It is also shown that the solutions to the three-stage model would be preferred to the solutions provided by the two-stage model under most circumstances. However, by the "efficiency" of the three-stage framework in our context, we want to know that whether utilising this approach and its solutions is worth the expense of the additional complexity and computation. The experiments in this study showed that the three-stage model has advantages under most circumstances(meta-scenarios), but that the gains are quite modest. This issue is frequently observed when comparing these methods in problems with a short-term (say less than five stages) planning window. Nevertheless, analysis of the length of the planning horizon and its effects on the solutions to the proposed frameworks indicate that utilising the three-stage models is more efficient for longer periods because the differences between the solutions of the two proposed structures increase by any iteration of the algorithms in moving horizon models. Moreover, during the long-term calculations, we noticed that the two-stage algorithm failed to find the optimal solutions for some iterations while the three-stage algorithm found the optimal value in all cases. Thus, it seems that for the planning horizons with more than ten stages, the efficiency of the three-stage model be may worth the expenses of the complexity and computation. Nevertheless, if the DM prefers to not use the three-stage structure because of the complexity and/or calculations, the two-stage moving horizon model can provide us with some reasonable solutions, although they might not be as good as the solutions generated by a three-stage framework. Finally, to examine the power of the proposed methodology in real cases, the proposed two-stage structure was applied in the sugarcane industry to analyse the whole infrastructure of the sugar and bioethanol Supply Chain (SC) in such a way that all economics (Max profit), environmental (Min CO₂), and social benefits (Max job-creations) were optimised under six key uncertainties, namely sugarcane yield, ethanol and refined sugar demands and prices, and the exchange rate. Moreover, one of the critical design questions - that is, to design the optimal number and technologies as well as the best place(s) for setting up the ethanol plant(s) - was also addressed in this study. The general model for the strategic planning of sugar- bioethanol supply chains (SC) under deep uncertainty was formulated and also examined in a case study based on the South African Sugar Industry. This problem is formulated as a Scenario-Based Mixed-Integer Two-Stage Multi-Objective Optimisation problem and solved by utilising the Generalised Goal Programming Approach. To sum up, the proposed methodology is, to the best of our knowledge, a novel approach that can successfully handle the deep uncertainty in MCDM/MOO problems with both short- and long-term planning horizons. It is generic enough to use in all MCDM problems under deep uncertainty. However, in this thesis, the proposed structure only applied in Linear Problems (LP). Non-linear problems would be an important direction for future research. Different solution methods may also need to be examined to solve the non-linear problems. Moreover, many other real-world optimisation and decision-making applications can be considered to examine the proposed method in the future.

Selon le rapport annuel de la Fédération Mondiale du Cœur de 2023, les maladies cardiovasculaires (MCV) représentaient près d'un tiers de tous les décès mondiaux en 2021. Comparativement aux pays à revenu élevé, plus de 80% des décès par MCV surviennent dans les pays à revenu faible et intermédiaire. La répartition inéquitable des ressources de diagnostic et de traitement des MCV demeure toujours non résolue. Face à ce défi, les dispositifs abordables d'échographie de point de soins (POCUS) ont un potentiel significatif pour améliorer le diagnostic des MCV. Avec l'aide de l'intelligence artificielle (IA), le POCUS permet aux non-experts de contribuer, améliorant ainsi largement l'accès aux soins, en particulier dans les régions moins desservies.L'objectif de cette thèse est de développer des algorithmes robustes et automatiques pour analyser la fonction cardiaque à l'aide de dispositifs POCUS, en mettant l'accent sur l'échocardiographie et l'électrocardiogramme. Notre premier objectif est d'obtenir des caractéristiques cardiaques explicables à partir de chaque modalité individuelle. Notre deuxième objectif est d'explorer une approche multimodale en combinant les données d'échocardiographie et d'électrocardiogramme.Nous commençons par présenter deux nouvelles structures d'apprentissage profond (DL) pour la segmentation de l'échocardiographie et l'estimation du mouvement. En incorporant des connaissance a priori de forme et de mouvement dans les modèles DL, nous démontrons, grâce à des expériences approfondies, que de tels a priori contribuent à améliorer la précision et la généralisation sur différentes séries de données non vues. De plus, nous sommes en mesure d'extraire la fraction d'éjection du ventricule gauche (FEVG), la déformation longitudinale globale (GLS) et d'autres indices utiles pour la détection de l'infarctus du myocarde (IM).Ensuite, nous proposons un modèle DL explicatif pour la décomposition non supervisée de l'électrocardiogramme. Ce modèle peut extraire des informations explicables liées aux différentes sous-ondes de l'ECG sans annotation manuelle. Nous appliquons ensuite ces paramètres à un classificateur linéaire pour la détection de l'infarctus du myocarde, qui montre une bonne généralisation sur différentes séries de données.Enfin, nous combinons les données des deux modalités pour une classification multimodale fiable. Notre approche utilise une fusion au niveau de la décision intégrant de l'incertitude, permettant l'entraînement avec des données multimodales non appariées. Nous évaluons ensuite le modèle entraîné à l'aide de données multimodales appariées, mettant en évidence le potentiel de la détection multimodale de l'IM surpassant celle d'une seule modalité.Dans l'ensemble, nos algorithmes proposés robustes et généralisables pour l'analyse de l'échocardiographie et de l'ECG démontrent un potentiel significatif pour l'analyse de la fonction cardiaque portable. Nous anticipons que notre cadre pourrait être davantage validé à l'aide de dispositifs portables du monde réel
According to the 2023 annual report of the World Heart Federation, cardiovascular diseases (CVD) accounted for nearly one third of all global deaths in 2021. Compared to high-income countries, more than 80% of CVD deaths occurred in low and middle-income countries. The inequitable distribution of CVD diagnosis and treatment resources still remains unresolved. In the face of this challenge, affordable point-of-care ultrasound (POCUS) devices demonstrate significant potential to improve the diagnosis of CVDs. Furthermore, by taking advantage of artificial intelligence (AI)-based tools, POCUS enables non-experts to help, thus largely improving the access to care, especially in less-served regions.The objective of this thesis is to develop robust and automatic algorithms to analyse cardiac function for POCUS devices, with a focus on echocardiography (ECHO) and electrocardiogram (ECG). Our first goal is to obtain explainable cardiac features from each single modality respectively. Our second goal is to explore a multi-modal approach by combining ECHO and ECG data.We start by presenting two novel deep learning (DL) frameworks for echocardiography segmentation and motion estimation tasks, respectively. By incorporating shape prior and motion prior into DL models, we demonstrate through extensive experiments that such prior can help improve the accuracy and generalises well on different unseen datasets. Furthermore, we are able to extract left ventricle ejection fraction (LVEF), global longitudinal strain (GLS) and other useful indices for myocardial infarction (MI) detection.Next, we propose an explainable DL model for unsupervised electrocardiogram decomposition. This model can extract interpretable information related to different ECG subwaves without manual annotation. We further apply those parameters to a linear classifier for myocardial infarction detection, which showed good generalisation across different datasets.Finally, we combine data from both modalities together for trustworthy multi-modal classification. Our approach employs decision-level fusion with uncertainty, allowing training with unpaired multi-modal data. We further evaluate the trained model using paired multi-modal data, showcasing the potential of multi-modal MI detection to surpass that from a single modality.Overall, our proposed robust and generalisable algorithms for ECHO and ECG analysis demonstrate significant potential for portable cardiac function analysis. We anticipate that our novel framework could be further validated using real-world portable devices. We envision that such advanced integrative tools may significantly contribute towards better identification of CVD patients

La vision par ordinateur est une branche de l'intelligence artificielle dont le but est de permettre à une machine d'analyser, de traiter et de comprendre le contenu d'images numériques. La compréhension de scène en particulier est un enjeu majeur en vision par ordinateur. Elle passe par une caractérisation à la fois sémantique et structurelle de l'image, permettant d'une part d'en décrire le contenu et, d'autre part, d'en comprendre la géométrie. Cependant tandis que l'espace réel est de nature tridimensionnelle, l'image qui le représente, elle, est bidimensionnelle. Une partie de l'information 3D est donc perdue lors du processus de formation de l'image et il est d'autant plus complexe de décrire la géométrie d'une scène à partir d'images 2D de celle-ci.Il existe plusieurs manières de retrouver l'information de profondeur perdue lors de la formation de l'image. Dans cette thèse nous nous intéressons à l’estimation d'une carte de profondeur étant donné une seule image de la scène. Dans ce cas, l'information de profondeur correspond, pour chaque pixel, à la distance entre la caméra et l'objet représenté en ce pixel. L'estimation automatique d'une carte de distances de la scène à partir d'une image est en effet une brique algorithmique critique dans de très nombreux domaines, en particulier celui des véhicules autonomes (détection d’obstacles, aide à la navigation).Bien que le problème de l'estimation de profondeur à partir d'une seule image soit un problème difficile et intrinsèquement mal posé, nous savons que l'Homme peut apprécier les distances avec un seul œil. Cette capacité n'est pas innée mais acquise et elle est possible en grande partie grâce à l'identification d'indices reflétant la connaissance a priori des objets qui nous entourent. Par ailleurs, nous savons que des algorithmes d'apprentissage peuvent extraire ces indices directement depuis des images. Nous nous intéressons en particulier aux méthodes d’apprentissage statistique basées sur des réseaux de neurones profond qui ont récemment permis des percées majeures dans de nombreux domaines et nous étudions le cas de l'estimation de profondeur monoculaire
Computer vision is a branch of artificial intelligence whose purpose is to enable a machine to analyze, process and understand the content of digital images. Scene understanding in particular is a major issue in computer vision. It goes through a semantic and structural characterization of the image, on one hand to describe its content and, on the other hand, to understand its geometry. However, while the real space is three-dimensional, the image representing it is two-dimensional. Part of the 3D information is thus lost during the process of image formation and it is therefore non trivial to describe the geometry of a scene from 2D images of it.There are several ways to retrieve the depth information lost in the image. In this thesis we are interested in estimating a depth map given a single image of the scene. In this case, the depth information corresponds, for each pixel, to the distance between the camera and the object represented in this pixel. The automatic estimation of a distance map of the scene from an image is indeed a critical algorithmic brick in a very large number of domains, in particular that of autonomous vehicles (obstacle detection, navigation aids).Although the problem of estimating depth from a single image is a difficult and inherently ill-posed problem, we know that humans can appreciate distances with one eye. This capacity is not innate but acquired and made possible mostly thanks to the identification of indices reflecting the prior knowledge of the surrounding objects. Moreover, we know that learning algorithms can extract these clues directly from images. We are particularly interested in statistical learning methods based on deep neural networks that have recently led to major breakthroughs in many fields and we are studying the case of the monocular depth estimation

Cette thèse explore l'intersection cruciale entre l'apprentissage automatique (IA) et la sécurité, visant à résoudre les défis liés au déploiement de systèmes intelligents dans des scénarios réels. Malgré des progrès significatifs en IA, des préoccupations liées à la confidentialité, à l'équité et à la fiabilité ont émergé, incitant à renforcer la fiabilité des systèmes d'IA. L'objectif central de la thèse est de permettre aux algorithmes d'IA d'identifier les écarts par rapport au comportement normal, contribuant ainsi à la sécurité globale des systèmes intelligents.La thèse commence par établir les concepts fondamentaux de la détection des données hors distribution (OOD) et de la détection des erreurs de classification dans le chapitre 1, fournissant une littérature essentielle et expliquant les principes clés. L'introduction souligne l'importance de traiter les problèmes liés au comportement non intentionnel et nuisible en IA, en particulier lorsque les systèmes d'IA produisent des résultats inattendus en raison de divers facteurs tels que des divergences dans les distributions de données.Dans le chapitre 2, la thèse introduit une nouvelle méthode de détection de données hors distribution basée sur la distance géodésique Fisher-Rao entre les distributions de probabilité. Cette approche unifie la formulation des scores de détection pour les logits du réseau et les espaces latents, contribuant à une robustesse et une fiabilité accrues dans l'identification des échantillons en dehors de la distribution d'entraînement.Le chapitre 3 présente une méthode de détection des données hors distribution non supervisée qui analyse les trajectoires neuronales sans nécessiter de supervision ou d'ajustement d'hyperparamètres. Cette méthode vise à identifier les trajectoires d'échantillons atypiques à travers diverses couches, améliorant l'adaptabilité des modèles d'IA à des scénarios divers.Le chapitre 4 se concentre sur la consolidation et l'amélioration de la détection hors distribution en combinant efficacement plusieurs détecteurs. La thèse propose une méthode universelle pour combiner des détecteurs existants, transformant le problème en un test d'hypothèse multivarié et tirant parti d'outils de méta-analyse. Cette approche améliore la détection des changements de données, en en faisant un outil précieux pour la surveillance en temps réel des performances des modèles dans des environnements dynamiques et évolutifs.Dans le chapitre 5, la thèse aborde la détection des erreurs de classification et l'estimation de l'incertitude par une approche axée sur les données, introduisant une solution pratique en forme fermée. La méthode quantifie l'incertitude par rapport à un observateur, distinguant entre prédictions confiantes et incertaines même face à des données difficiles. Cela contribue à une compréhension plus nuancée de la confiance du modèle et aide à signaler les prédictions nécessitant une intervention humaine.La thèse se termine en discutant des perspectives futures et des orientations pour améliorer la sécurité en IA et en apprentissage automatique, soulignant l'évolution continue des systèmes d'IA vers une plus grande transparence, robustesse et fiabilité. Le travail collectif présenté dans la thèse représente une avancée significative dans le renforcement de la sécurité en IA, contribuant au développement de modèles d'apprentissage automatique plus fiables et dignes de confiance, capables de fonctionner efficacement dans des scénarios réels divers et dynamiques
This thesis explores the intersection of machine learning (ML) and safety, aiming to address challenges associated with the deployment of intelligent systems in real-world scenarios. Despite significant progress in ML, concerns related to privacy, fairness, and trustworthiness have emerged, prompting the need for enhancing the reliability of AI systems. The central focus of the thesis is to enable ML algorithms to detect deviations from normal behavior, thereby contributing to the overall safety of intelligent systems.The thesis begins by establishing the foundational concepts of out-of-distribution (OOD) detection and misclassification detection in Chapter 1, providing essential background literature and explaining key principles. The introduction emphasizes the importance of addressing issues related to unintended and harmful behavior in ML, particularly when AI systems produce unexpected outcomes due to various factors such as mismatches in data distributions.In Chapter 2, the thesis introduces a novel OOD detection method based on the Fisher-Rao geodesic distance between probability distributions. This approach unifies the formulation of detection scores for both network logits and feature spaces, contributing to improved robustness and reliability in identifying samples outside the training distribution.Chapter 3 presents an unsupervised OOD detection method that analyzes neural trajectories without requiring supervision or hyperparameter tuning. This method aims to identify atypical sample trajectories through various layers, enhancing the adaptability of ML models to diverse scenarios.Chapter 4 focuses on consolidating and enhancing OOD detection by combining multiple detectors effectively. It presents a universal method for ensembling existing detectors, transforming the problem into a multi-variate hypothesis test and leveraging meta-analysis tools. This approach improves data shift detection, making it a valuable tool for real-time model performance monitoring in dynamic and evolving environments.In Chapter 5, the thesis addresses misclassification detection and uncertainty estimation through a data-driven approach, introducing a practical closed-form solution. The method quantifies uncertainty relative to an observer, distinguishing between confident and uncertain predictions even in the face of challenging or unfamiliar data. This contributes to a more nuanced understanding of the model's confidence and helps flag predictions requiring human intervention.The thesis concludes by discussing future perspectives and directions for improving safety in ML and AI, emphasizing the ongoing evolution of AI systems towards greater transparency, robustness, and trustworthiness. The collective work presented in the thesis represents a significant step forward in advancing AI safety, contributing to the development of more reliable and trustworthy machine learning models that can operate effectively in diverse and dynamic real-world scenarios

The primary objective of this thesis is to investigate whether evolutionary concepts can improve the performance, speed and convenience of algorithms in various active areas of machine learning research. Deep neural networks are exhibiting an explosion in the number of parameters that need to be trained, as well as the number of permutations of possible network architectures and hyper-parameters. There is little guidance on how to choose these and brute-force experimentation is prohibitively time consuming. We show that evolutionary algorithms can help tame this explosion of freedom, by developing an algorithm that robustly evolves near optimal deep neural network architectures and hyper-parameters across a wide range of image and sentiment classification problems. We further develop an algorithm that automatically determines whether a given data science problem is of classification or regression type, successfully choosing the correct problem type with more than 95% accuracy. Together these algorithms show that a great deal of the current "art" in the design of deep learning networks - and in the job of the data scientist - can be automated. Having discussed the general problem of optimising deep learning networks the thesis moves on to a specific application: the automated extraction of human sentiment from text and images of human faces. Our results reveal that our approach is able to outperform several public and/or commercial text sentiment analysis algorithms using an evolutionary algorithm that learned to encode and extend sentiment lexicons. A second analysis looked at using evolutionary algorithms to estimate text sentiment while simultaneously compressing text data. An extensive analysis of twelve sentiment datasets reveal that accurate compression is possible with 3.3% loss in classification accuracy even with 75% compression of text size, which is useful in environments where data volumes are a problem. Finally, the thesis presents improvements to automated sentiment analysis of human faces to identify emotion, an area where there has been a tremendous amount of progress using convolutional neural networks. We provide a comprehensive critique of past work, highlight recommendations and list some open, unanswered questions in facial expression recognition using convolutional neural networks. One serious challenge when implementing such networks for facial expression recognition is the large number of trainable parameters which results in long training times. We propose a novel method based on evolutionary algorithms, to reduce the number of trainable parameters whilst simultaneously retaining classification performance, and in some cases achieving superior performance. We are robustly able to reduce the number of parameters on average by 95% with no loss in classification accuracy. Overall our analyses show that evolutionary algorithms are a valuable addition to machine learning in the deep learning era: automating, compressing and/or improving results significantly, depending on the desired goal.

Deep learning has long been criticised as a black-box model for lacking sound theoretical explanation. During the PhD course, I explore and establish theoretical foundations for deep learning. In this thesis, I present my contributions positioned upon existing literature: (1) analysing the generalizability of the neural networks with residual connections via complexity and capacity-based hypothesis complexity measures; (2) modeling stochastic gradient descent (SGD) by stochastic differential equations (SDEs) and their dynamics, and further characterizing the generalizability of deep learning; (3) understanding the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems, which sheds light in reconciling the over-representation and excellent generalizability of deep learning; and (4) discovering the interplay between generalization, privacy preservation, and adversarial robustness, which have seen rising concerns in deep learning deployment.

Il grande successo che il Deep Learning ha ottenuto in ambiti strategici per la nostra società quali l'industria, la difesa, la medicina etc., ha portanto sempre più realtà a investire ed esplorare l'utilizzo di questa tecnologia. Ormai si possono trovare algoritmi di Machine Learning e Deep Learning quasi in ogni ambito della nostra vita. Dai telefoni, agli elettrodomestici intelligenti fino ai veicoli che guidiamo. Quindi si può dire che questa tecnologia pervarsiva è ormai a contatto con le nostre vite e quindi dobbiamo confrontarci con essa. Da questo nasce l’eXplainable Artificial Intelligence o XAI, uno degli ambiti di ricerca che vanno per la maggiore al giorno d'oggi in ambito di Deep Learning e di Intelligenza Artificiale. Il concetto alla base di questo filone di ricerca è quello di rendere e/o progettare i nuovi algoritmi di Deep Learning in modo che siano affidabili, interpretabili e comprensibili all'uomo. Questa necessità è dovuta proprio al fatto che le reti neurali, modello matematico che sta alla base del Deep Learning, agiscono come una scatola nera, rendendo incomprensibile all'uomo il ragionamento interno che compiono per giungere ad una decisione. Dato che stiamo delegando a questi modelli matematici decisioni sempre più importanti, integrandole nei processi più delicati della nostra società quali, ad esempio, la diagnosi medica, la guida autonoma o i processi di legge, è molto importante riuscire a comprendere le motivazioni che portano questi modelli a produrre determinati risultati. Il lavoro presentato in questa tesi consiste proprio nello studio e nella sperimentazione di algoritmi di Deep Learning integrati con tecniche di Intelligenza Artificiale simbolica. Questa integrazione ha un duplice scopo: rendere i modelli più potenti, consentendogli di compiere ragionamenti o vincolandone il comportamento in situazioni complesse, e renderli interpretabili. La tesi affronta due macro argomenti: le spiegazioni ottenute grazie all'integrazione neuro-simbolica e lo sfruttamento delle spiegazione per rendere gli algoritmi di Deep Learning più capaci o intelligenti. Il primo macro argomento si concentra maggiormente sui lavori svolti nello sperimentare l'integrazione di algoritmi simbolici con le reti neurali. Un approccio è stato quelli di creare un sistema per guidare gli addestramenti delle reti stesse in modo da trovare la migliore combinazione di iper-parametri per automatizzare la progettazione stessa di queste reti. Questo è fatto tramite l'integrazione di reti neurali con la Programmazione Logica Probabilistica (PLP) che consente di sfruttare delle regole probabilistiche indotte dal comportamento delle reti durante la fase di addestramento o ereditate dall'esperienza maturata dagli esperti del settore. Queste regole si innescano allo scatenarsi di un problema che il sistema rileva durate l'addestramento della rete. Questo ci consente di ottenere una spiegazione di cosa è stato fatto per migliorare l'addestramento una volta identificato un determinato problema. Un secondo approccio è stato quello di far cooperare sistemi logico-probabilistici con reti neurali per la diagnosi medica da fonti di dati eterogenee. La seconda tematica affrontata in questa tesi tratta lo sfruttamento delle spiegazioni che possiamo ottenere dalle rete neurali. In particolare, queste spiegazioni sono usate per creare moduli di attenzione che aiutano a vincolare o a guidare le reti neurali portandone ad avere prestazioni migliorate. Tutti i lavori sviluppati durante il dottorato e descritti in questa tesi hanno portato alle pubblicazioni elencate nel Capitolo 14.2.
The great success that Machine and Deep Learning has achieved in areas that are strategic for our society such as industry, defence, medicine, etc., has led more and more realities to invest and explore the use of this technology. Machine Learning and Deep Learning algorithms and learned models can now be found in almost every area of our lives. From phones to smart home appliances, to the cars we drive. So it can be said that this pervasive technology is now in touch with our lives, and therefore we have to deal with it. This is why eXplainable Artificial Intelligence or XAI was born, one of the research trends that are currently in vogue in the field of Deep Learning and Artificial Intelligence. The idea behind this line of research is to make and/or design the new Deep Learning algorithms so that they are interpretable and comprehensible to humans. This necessity is due precisely to the fact that neural networks, the mathematical model underlying Deep Learning, act like a black box, making the internal reasoning they carry out to reach a decision incomprehensible and untrustable to humans. As we are delegating more and more important decisions to these mathematical models, it is very important to be able to understand the motivations that lead these models to make certain decisions. This is because we have integrated them into the most delicate processes of our society, such as medical diagnosis, autonomous driving or legal processes. The work presented in this thesis consists in studying and testing Deep Learning algorithms integrated with symbolic Artificial Intelligence techniques. This integration has a twofold purpose: to make the models more powerful, enabling them to carry out reasoning or constraining their behaviour in complex situations, and to make them interpretable. The thesis focuses on two macro topics: the explanations obtained through neuro-symbolic integration and the exploitation of explanations to make the Deep Learning algorithms more capable or intelligent. The neuro-symbolic integration was addressed twice, by experimenting with the integration of symbolic algorithms with neural networks. A first approach was to create a system to guide the training of the networks themselves in order to find the best combination of hyper-parameters to automate the design of these networks. This is done by integrating neural networks with Probabilistic Logic Programming (PLP). This integration makes it possible to exploit probabilistic rules tuned by the behaviour of the networks during the training phase or inherited from the experience of experts in the field. These rules are triggered when a problem occurs during network training. This generates an explanation of what was done to improve the training once a particular issue was identified. A second approach was to make probabilistic logic systems cooperate with neural networks for medical diagnosis on heterogeneous data sources. The second topic addressed in this thesis concerns the exploitation of explanations. In particular, the explanations one can obtain from neural networks are used in order to create attention modules that help in constraining and improving the performance of neural networks. All works developed during the PhD and described in this thesis have led to the publications listed in Chapter 14.2.

Uncertainty propagation across components of complex probabilistic models is vital for improving regularisation. Unfortunately, for many interesting models based on non-linear Gaussian processes (GPs), straightforward propagation of uncertainty is computationally and mathematically intractable. This thesis is concerned with solving this problem through developing novel variational inference approaches. From a modelling perspective, a key contribution of the thesis is the development of deep Gaussian processes (deep GPs). Deep GPs generalise several interesting GP-based models and, hence, motivate the development of uncertainty propagation techniques. In a deep GP, each layer is modelled as the output of a multivariate GP, whose inputs are governed by another GP. The resulting model is no longer a GP but, instead, can learn much more complex interactions between data. In contrast to other deep models, all the uncertainty in parameters and latent variables is marginalised out and both supervised and unsupervised learning is handled. Two important special cases of a deep GP can equivalently be seen as its building components and, historically, were developed as such. Firstly, the variational GP-LVM is concerned with propagating uncertainty in Gaussian process latent variable models. Any observed inputs (e.g. temporal) can also be used to correlate the latent space posteriors. Secondly, this thesis develops manifold relevance determination (MRD) which considers a common latent space for multiple views. An adapted variational framework allows for strong model regularisation, resulting in rich latent space representations to be learned. The developed models are also equipped with algorithms that maximise the information communicated between their different stages using uncertainty propagation, to achieve improved learning when partially observed values are present. The developed methods are demonstrated in experiments with simulated and real data. The results show that the developed variational methodologies improve practical applicability by enabling automatic capacity control in the models, even when data are scarce.

Deep learning, in particular neural networks, achieved remarkable success in the recent years. However, most of it is based on supervised learning, and relies on ever larger datasets, and immense computing power. One step towards general artificial intelligence is to build a model of the world, with enough knowledge to acquire a kind of ``common sense''. Representations learned by such a model could be reused in a number of other tasks. It would reduce the requirement for labelled samples and possibly acquire a deeper understanding of the problem. The vast quantities of knowledge required to build common sense precludes the use of supervised learning, and suggests to rely on unsupervised learning instead.

The concept of uncertainty is central to unsupervised learning. The task is usually to learn a complex, multimodal distribution. Density estimation and generative models aim at representing the whole distribution of the data, while predictive learning consists of predicting the state of the world given the context and, more often than not, the prediction is not unique. That may be because the model lacks the capacity or the computing power to make a certain prediction, or because the future depends on parameters that are not part of the observation. Finally, the world can be chaotic of truly stochastic. Representing complex, multimodal continuous distributions with deep neural networks is still an open problem.

In this thesis, we first assess the difficulties of representing probabilities in high dimensional spaces, and review the related work in this domain. We then introduce two methods to address the problem of video prediction, first using a novel form of linearizing auto-encoders and latent variables, and secondly using Generative Adversarial Networks (GANs). We show how GANs can be seen as trainable loss functions to represent uncertainty, then how they can be used to disentangle factors of variation. Finally, we explore a new non-probabilistic framework for GANs.

Deep Brain Stimulation (DBS) is a surgical intervention that is known to reduce or eliminate the symptoms of common movement disorders, such as Parkinson.s disease, dystonia, or tremor. During the intervention the surgeon places electrodes inside of the patient.s brain to stimulate speci.c regions. Since these regions span only a couple of millimeters, and electrode misplacement has severe consequences, reliable and accurate navigation is of great importance. Usually the surgeon relies on fused CT and MRI data sets, as well as direct feedback from the patient. More recently Microelectrode Recordings (MER), which support navigation by measuring the electric .eld of the patient.s brain, are also used. We propose a visualization system that fuses the different modalities: imaging data, MER and patient checks, as well as the related uncertainties, in an intuitive way to present placement-related information in a consistent view with the goal of supporting the surgeon in the .nal placement of the stimulating electrode. We will describe the design considerations for our system, the technical realization, present the outcome of the proposed system, and provide an evaluation.

Die Topologieoptimierung ist die Suche einer optimalen Bauteilgeometrie in Abhängigkeit des Einsatzfalls. Für komplexe Probleme kann die Topologieoptimierung aufgrund eines hohen Detailgrades viel Zeit- und Rechenkapazität erfordern. Diese Nachteile der Topologieoptimierung sollen mittels Deep Learning reduziert werden, so dass eine Topologieoptimierung dem Konstrukteur als sekundenschnelle Hilfe dient. Das Deep Learning ist die Erweiterung künstlicher neuronaler Netzwerke, mit denen Muster oder Verhaltensregeln erlernt werden können. So soll die bislang numerisch berechnete Topologieoptimierung mit dem Deep Learning Ansatz gelöst werden. Hierzu werden Ansätze, Berechnungsschema und erste Schlussfolgerungen vorgestellt und diskutiert.

Les avancées récentes en apprentissage profond et en traitement d'image présentent l'opportunité d'unifier ces deux champs de recherche complémentaires pour une meilleure résolution du problème de classification d'images dans des catégories sémantiques. L'apprentissage profond apporte au traitement d'image le pouvoir de représentation nécessaire à l'amélioration des performances des méthodes de classification d'images. Cette thèse propose de nouvelles méthodes d'apprentissage de représentations visuelles profondes pour la résolution de cette tache. L'apprentissage profond a été abordé sous deux angles. D'abord nous nous sommes intéressés à l'apprentissage non supervisé de représentations latentes ayant certaines propriétés à partir de données en entrée. Il s'agit ici d'intégrer une connaissance à priori, à travers un terme de régularisation, dans l'apprentissage d'une machine de Boltzmann restreinte (RBM). Nous proposons plusieurs formes de régularisation qui induisent différentes propriétés telles que la parcimonie, la sélectivité et l'organisation en structure topographique. Le second aspect consiste au passage graduel de l'apprentissage non supervisé à l'apprentissage supervisé de réseaux profonds. Ce but est réalisé par l'introduction sous forme de supervision, d'une information relative à la catégorie sémantique. Deux nouvelles méthodes sont proposées. Le premier est basé sur une régularisation top-down de réseaux de croyance profonds à base de RBMs. Le second optimise un cout intégrant un critre de reconstruction et un critre de supervision pour l'entrainement d'autoencodeurs profonds. Les méthodes proposées ont été appliquées au problme de classification d'images. Nous avons adopté le modèle sac-de-mots comme modèle de base parce qu'il offre d'importantes possibilités grâce à l'utilisation de descripteurs locaux robustes et de pooling par pyramides spatiales qui prennent en compte l'information spatiale de l'image. L'apprentissage profonds avec agrÉgation spatiale est utilisé pour apprendre un dictionnaire hiÉrarchique pour l'encodage de reprÉsentations visuelles de niveau intermÉdiaire. Cette mÉthode donne des rÉsultats trs compétitifs en classification de scènes et d'images. Les dictionnaires visuels appris contiennent diverses informations non-redondantes ayant une structure spatiale cohérente. L'inférence est aussi très rapide. Nous avons par la suite optimisé l'étape de pooling sur la base du codage produit par le dictionnaire hiérarchique précédemment appris en introduisant introduit une nouvelle paramétrisation dérivable de l'opération de pooling qui permet un apprentissage par descente de gradient utilisant l'algorithme de rétro-propagation. Ceci est la premire tentative d'unification de l'apprentissage profond et du modèle de sac de mots. Bien que cette fusion puisse sembler évidente, l'union de plusieurs aspects de l'apprentissage profond de représentations visuelles demeure une tache complexe à bien des égards et requiert encore un effort de recherche important
Recent advancements in the areas of deep learning and visual information processing have presented an opportunity to unite both fields. These complementary fields combine to tackle the problem of classifying images into their semantic categories. Deep learning brings learning and representational capabilities to a visual processing model that is adapted for image classification. This thesis addresses problems that lead to the proposal of learning deep visual representations for image classification. The problem of deep learning is tackled on two fronts. The first aspect is the problem of unsupervised learning of latent representations from input data. The main focus is the integration of prior knowledge into the learning of restricted Boltzmann machines (RBM) through regularization. Regularizers are proposed to induce sparsity, selectivity and topographic organization in the coding to improve discrimination and invariance. The second direction introduces the notion of gradually transiting from unsupervised layer-wise learning to supervised deep learning. This is done through the integration of bottom-up information with top-down signals. Two novel implementations supporting this notion are explored. The first method uses top-down regularization to train a deep network of RBMs. The second method combines predictive and reconstructive loss functions to optimize a stack of encoder-decoder networks. The proposed deep learning techniques are applied to tackle the image classification problem. The bag-of-words model is adopted due to its strengths in image modeling through the use of local image descriptors and spatial pooling schemes. Deep learning with spatial aggregation is used to learn a hierarchical visual dictionary for encoding the image descriptors into mid-level representations. This method achieves leading image classification performances for object and scene images. The learned dictionaries are diverse and non-redundant. The speed of inference is also high. From this, a further optimization is performed for the subsequent pooling step. This is done by introducing a differentiable pooling parameterization and applying the error backpropagation algorithm. This thesis represents one of the first attempts to synthesize deep learning and the bag-of-words model. This union results in many challenging research problems, leaving much room for further study in this area

Monte Carlo methods in derivative pricing are computationally expensive, in particular for evaluating models partial derivatives with regard to inputs. This research proposes the use of deep learning to approximate such valuation models for highly exotic derivatives, using automatic differentiation to evaluate input sensitivities. Deep learning models are trained to approximate Phoenix Autocall valuation using a proprietary model used by Svenska Handelsbanken AB. Models are trained on large datasets of low-accuracy (10^4 simulations) Monte Carlo data, successfully learning the true model with an average error of 0.1% on validation data generated by 10^8 simulations. A specific model parametrisation is proposed for 2-day valuation only, to be recalibrated interday using transfer learning. Automatic differentiation approximates sensitivity to (normalised) underlying asset prices with a mean relative error generally below 1.6%. Overall error when predicting sensitivity to implied volatililty is found to lie within 10%-40%. Near identical results are found by finite difference as automatic differentiation in both cases. Automatic differentiation is not successful at capturing sensitivity to interday contract change in value, though errors of 8%-25% are achieved by finite difference. Model recalibration by transfer learning proves to converge over 15 times faster and with up to 14% lower relative error than training using random initialisation. The results show that deep learning models can efficiently learn Monte Carlo valuation, and that these can be quickly recalibrated by transfer learning. The deep learning model gradient computed by automatic differentiation proves a good approximation of the true model sensitivities. Future research proposals include studying optimised recalibration schedules, using training data generated by single Monte Carlo price paths, and studying additional parameters and contracts.

Bridging the abstraction gap between artifacts and concepts is the essence of software engineering (SE) research problems. SE researchers regularly use machine learning to bridge this gap, but there are three fundamental issues with traditional applications of machine learning in SE research. Traditional applications are too reliant on labeled data. They are too reliant on human intuition, and they are not capable of learning expressive yet efficient internal representations. Ultimately, SE research needs approaches that can automatically learn representations of massive, heterogeneous, datasets in situ, apply the learned features to a particular task and possibly transfer knowledge from task to task. Improvements in both computational power and the amount of memory in modern computer architectures have enabled new approaches to canonical machine learning tasks. Specifically, these architectural advances have enabled machines that are capable of learning deep, compositional representations of massive data depots. The rise of deep learning has ushered in tremendous advances in several fields. Given the complexity of software repositories, we presume deep learning has the potential to usher in new analytical frameworks and methodologies for SE research and the practical applications it reaches. This dissertation examines and enables deep learning algorithms in different SE contexts. We demonstrate that deep learners significantly outperform state-of-the-practice software language models at code suggestion on a Java corpus. Further, these deep learners for code suggestion automatically learn how to represent lexical elements. We use these representations to transmute source code into structures for detecting similar code fragments at different levels of granularity—without declaring features for how the source code is to be represented. Then we use our learning-based framework for encoding fragments to intelligently select and adapt statements in a codebase for automated program repair. In our work on code suggestion, code clone detection, and automated program repair, everything for representing lexical elements and code fragments is mined from the source code repository. Indeed, our work aims to move SE research from the art of feature engineering to the science of automated discovery.

L'objectif de cette thèse est de rendre l'apprentissage profond plus efficace en termes de ressources en appliquant le principe de modularité. La thèse comporte plusieurs contributions principales : une étude de la littérature sur la modularité dans l'apprentissage profond; la conception d'OmniPrint et de Meta-Album, des outils qui facilitent l'étude de la modularité des données; des études de cas examinant les effets de l'apprentissage épisodique, un exemple de modularité des données; un mécanisme d'évaluation modulaire appelé LTU pour évaluer les risques en matière de protection de la vie privée; et la méthode RRR pour réutiliser des modèles modulaires pré-entraînés afin d'en construire des versions plus compactes. La modularité, qui implique la décomposition d'une entité en sous-entités, est un concept répandu dans diverses disciplines. Cette thèse examine la modularité sur trois axes de l'apprentissage profond : les données, la tâche et le modèle. OmniPrint et Meta-Album facilitent de comparer les modèles modulaires et d'explorer les impacts de la modularité des données. LTU garantit la fiabilité de l'évaluation de la protection de la vie privée. RRR améliore l'efficacité de l'utilisation des modèles modulaires pré-entraînés. Collectivement, cette thèse fait le lien entre le principe de modularité et l'apprentissage profond et souligne ses avantages dans certains domaines de l'apprentissage profond, contribuant ainsi à une intelligence artificielle plus efficace en termes de ressources
This Ph.D. thesis is dedicated to enhancing the efficiency of Deep Learning by leveraging the principle of modularity. It contains several main contributions: a literature survey on modularity in Deep Learning; the introduction of OmniPrint and Meta-Album, tools that facilitate the investigation of data modularity; case studies examining the effects of episodic few-shot learning, an instance of data modularity; a modular evaluation mechanism named LTU for assessing privacy risks; and the method RRR for reusing pre-trained modular models to create more compact versions. Modularity, which involves decomposing an entity into sub-entities, is a prevalent concept across various disciplines. This thesis examines modularity across three axes of Deep Learning: data, task, and model. OmniPrint and Meta-Album assist in benchmarking modular models and exploring data modularity's impacts. LTU ensures the reliability of the privacy assessment. RRR significantly enhances the utilization efficiency of pre-trained modular models. Collectively, this thesis bridges the modularity principle with Deep Learning and underscores its advantages in selected fields of Deep Learning, contributing to more resource-efficient Artificial Intelligence

Environmental models are important tools; however uncertainty is pervasive in the modeling process.   Current research has shown that under­standing and representing these uncertainties is critical when decisions are expected to be made from the modeling results.  One critical question has become: how focused should uncertainty intervals be with consideration of characteristics of uncertain input data, model equation representations, and output observations?   This thesis delves into this issue with applied research in four independent studies.  These studies developed a diverse array of simply-structured process models (catchment hydrology, soil carbon dynamics, wetland P cycling, stream rating); employed field data observations with wide ranging characteristics (e.g., spatial variability, suspected systematic error); and explored several variations of probabilistic and non-probabilistic uncertainty schemes for model calibrations.  A key focus has been on how the design of various schemes impacted the resulting uncertainty intervals, and more importantly the ability to justify conclusions.  In general, some uncertainty in uncertainty (u2) resulted in all studies, in various degrees.  Subjectivity was intrinsic in the non-probabilistic results.  One study illustrated that such subjectivity could be partly mitigated using a “limits of acceptability” scheme with posterior validation of errors.  u2 was also a factor from probabilistic calibration algorithms, as residual errors were not wholly stochastic.  Overall however, u2 was not a deterrent to drawing conclusions from each study. One insight on the value of data for modeling was that there can be substantial redundant information in some hydrological time series.  Several process insights resulted: there can be substantial fractions of relatively inert soil carbon in agricultural systems; the lowest achievable outflow phosphorus concentration in an engineered wetland seemed partly controlled by rapid turnover and decomposition of the specific vegetation in that system.  Additionally, consideration of uncertainties in a stage-discharge rating model enabled more confident detection of change in long-term river flow patterns.

QC 20121105

In this thesis we study the problem of learning under uncertainty using the statistical learning paradigm. We rst propose a linear maximum margin classi er that deals with uncertainty in data input. More speci cally, we reformulate the standard Support Vector Machine (SVM) framework such that each training example can be modeled by a multi-dimensional Gaussian distribution described by its mean vector and its covariance matrix { the latter modeling the uncertainty. We address the classi cation problem and de ne a cost function that is the expected value of the classical SVM cost when data samples are drawn from the multi-dimensional Gaussian distributions that form the set of the training examples. Our formulation approximates the classical SVM formulation when the training examples are isotropic Gaussians with variance tending to zero. We arrive at a convex optimization problem, which we solve e - ciently in the primal form using a stochastic gradient descent approach. The resulting classi er, which we name SVM with Gaussian Sample Uncertainty (SVM-GSU), is tested on synthetic data and ve publicly available and popular datasets; namely, the MNIST, WDBC, DEAP, TV News Channel Commercial Detection, and TRECVID MED datasets. Experimental results verify the e ectiveness of the proposed method. Next, we extended the aforementioned linear classi er so as to lead to non-linear decision boundaries, using the RBF kernel. This extension, where we use isotropic input uncertainty and we name Kernel SVM with Isotropic Gaussian Sample Uncertainty (KSVM-iGSU), is used in the problems of video event detection and video aesthetic quality assessment. The experimental results show that exploiting input uncertainty, especially in problems where only a limited number of positive training examples are provided, can lead to better classi cation, detection, or retrieval performance. Finally, we present a preliminary study on how the above ideas can be used under the deep convolutional neural networks learning paradigm so as to exploit inherent sources of uncertainty, such as spatial pooling operations, that are usually used in deep networks.

Since 2006, deep learning algorithms which rely on deep architectures with several layers of increasingly complex representations have been able to outperform state-of-the-art methods in several settings. Deep architectures can be very efficient in terms of the number of parameters required to represent complex operations which makes them very appealing to achieve good generalization with small amounts of data. Although training deep architectures has traditionally been considered a difficult problem, a successful approach has been to employ an unsupervised layer-wise pre-training step to initialize deep supervised models. First, unsupervised learning has many benefits w.r.t. generalization because it only relies on unlabeled data which is easily found. Second, the possibility to learn representations layer by layer instead of all layers at once improves generalization further and reduces computational time. However, deep learning is a very recent approach and still poses a lot of theoretical and practical questions concerning the consistency of layer-wise learning with many layers and difficulties such as evaluating performance, performing model selection and optimizing layers. In this thesis we first discuss the limitations of the current variational justification for layer-wise learning which does not generalize well to many layers. We ask if a layer-wise method can ever be truly consistent, i.e. capable of finding an optimal deep model by training one layer at a time without knowledge of the upper layers. We find that layer-wise learning can in fact be consistent and can lead to optimal deep generative models. To do this, we introduce the Best Latent Marginal (BLM) upper bound, a new criterion which represents the maximum log-likelihood of a deep generative model where the upper layers are unspecified. We prove that maximizing this criterion for each layer leads to an optimal deep architecture, provided the rest of the training goes well. Although this criterion cannot be computed exactly, we show that it can be maximized effectively by auto-encoders when the encoder part of the model is allowed to be as rich as possible. This gives a new justification for stacking models trained to reproduce their input and yields better results than the state-of-the-art variational approach. Additionally, we give a tractable approximation of the BLM upper-bound and show that it can accurately estimate the final log-likelihood of models. Taking advantage of these theoretical advances, we propose a new method for performing layer-wise model selection in deep architectures, and a new criterion to assess whether adding more layers is warranted. As for the difficulty of training layers, we also study the impact of metrics and parametrization on the commonly used gradient descent procedure for log-likelihood maximization. We show that gradient descent is implicitly linked with the metric of the underlying space and that the Euclidean metric may often be an unsuitable choice as it introduces a dependence on parametrization and can lead to a breach of symmetry. To mitigate this problem, we study the benefits of the natural gradient and show that it can restore symmetry, regrettably at a high computational cost. We thus propose that a centered parametrization may alleviate the problem with almost no computational overhead.

Imitation learning refers to an agent's ability to mimic a desired behaviour by learning from observations. The field is rapidly gaining attention due to recent advances in computational and communication capabilities as well as rising demand for intelligent applications. The goal of imitation learning is to describe the desired behaviour by providing demonstrations rather than instructions. This enables agents to learn complex behaviours with general learning methods that require minimal task specific information. However, imitation learning faces many challenges. The objective of this thesis is to advance the state of the art in imitation learning by adopting deep learning methods to address two major challenges of learning from demonstrations. Firstly, representing the demonstrations in a manner that is adequate for learning. We propose novel Convolutional Neural Networks (CNN) based methods to automatically extract feature representations from raw visual demonstrations and learn to replicate the demonstrated behaviour. This alleviates the need for task specific feature extraction and provides a general learning process that is adequate for multiple problems. The second challenge is generalizing a policy over unseen situations in the training demonstrations. This is a common problem because demonstrations typically show the best way to perform a task and don't offer any information about recovering from suboptimal actions. Several methods are investigated to improve the agent's generalization ability based on its initial performance. Our contributions in this area are three fold. Firstly, we propose an active data aggregation method that queries the demonstrator in situations of low confidence. Secondly, we investigate combining learning from demonstrations and reinforcement learning. A deep reward shaping method is proposed that learns a potential reward function from demonstrations. Finally, memory architectures in deep neural networks are investigated to provide context to the agent when taking actions. Using recurrent neural networks addresses the dependency between the state-action sequences taken by the agent. The experiments are conducted in simulated environments on 2D and 3D navigation tasks that are learned from raw visual data, as well as a 2D soccer simulator. The proposed methods are compared to state of the art deep reinforcement learning methods. The results show that deep learning architectures can learn suitable representations from raw visual data and effectively map them to atomic actions. The proposed methods for addressing generalization show improvements over using supervised learning and reinforcement learning alone. The results are thoroughly analysed to identify the benefits of each approach and situations in which it is most suitable.