Dissertationen zum Thema „Autonomus driving“

According to the report from the EU,every year more than 40 000 lives are lost in a road accident. More than 90% of accidents are caused by human error. Advance Driver-Assistance Systems (ADAS) can help drivers to avoid these accidents or at least to reduce the consequence. Such systems use a multitude of sensors to perceive the environment around the vehicle,and act consequently. In the future, cars will be a part of an intelligent transport system where all players are connected. With these abilities, an electronic horizon is created pushing forward the sensor range. Information regarding traffic or hazard can be obtained from other cars or from the cloud. With the use of ADAS, the driver can be assisted to avoid dangerous situations. In a near future, all this information will be elaborated and used from the vehicle itself, reducing the possibility of human error. The thesis is a study on how and if it is possible to create an enhanced Adaptive Cruise Control (ACC) function relying on e-Horizon data. Standard ACC use a radar to keep distance from the car in front, by accelerating up to a preset speed or braking. The goal is to research if curvature can be taken into consideration to create a more comfortable and safe speed profile. To do so, the onboard function obtains curvature information from different data sources. The first one, is the navigation data, provided in real time to the car through Vehicle to Cloud connection. The second one, is an onboard camera used for ADAS functions, which outputs the curvature. The recording from a GPS tracker is used to create an ideal path description, to be used as benchmark. Geographic coordinates from map service provider are elaborated, with the use of Simulink models and MATLAB to obtain a curvature profile. This will be used in the test environment to create and actuate the speed profile. The parameters, used to evaluate the performance of the speed profile, are lateral and longitudinal accelerations and jerk.

Målet med examensarbetet har varit att samla information åt ett tekniskt konsultföretag för att öka deras kunskap om autonoma system och fordonskommunikation. Statusen på arbetet kring dessa aktiva säkerhetssystem hos olika aktörer och hur systemen implementeras i dagens och framtidens fordon har undersökts genom omfattande litteraturstudier, intervjuer och marknadsanalyser. De autonoma systemen kan samla information från omgivningen genom sensorer och bidra till ett jämnare trafikflöde, ökad säkerhet, lättare bilar och bättre miljö. Genom fordonskommunikationen kan fordon kommunicera med varandra samt infrastrukturen och garantera en säker bilfärd. År 2030 utgörs innerstaden av autonom, elektrifierad kollektivtrafik för att transportera människor på begäran, samtidigt som personbilar till viss del förbjuds. Potentiella behov för människan i en fullt autonom bil har identifierats och diverse produktutvecklingsmetoder har tillämpats för att utforma två konceptuella lösningar för en framtida bilinteriör. Lösningarna visar interaktionen mellan människa och system eftersom underhållning och bekvämlighet blir viktigt i en fullt autonom bil. Respektive lösning är statsägd och rymmer fyra passagerare. I lösningarna är sittplatserna placerade på ett sätt som underlättar kommunikation mellan passagerarna. Passagerarna kan underhållas eller informeras individuellt eller gemensamt via text, ljud och bild.
The goal with this thesis project has been to collect information for a technical consulting company in order to increase their knowledge about autonomous systems and vehicular communication. The status of how various operators work with active safety systems and how the systems are implemented in current and future vehicles has been investigated through extensive literature studies, interviews and market research. The autonomous systems can collect information from the surrounding through sensors and contribute to better traffic efficiency, increased safety, lighter cars and a better environment. Through vehicle communication, the vehicle can communicate with each other in order to guarantee a safe ride. In 2030 the inner city constitutes of autonomous, electrified public transport to transport people on demand, meanwhile private cars are prohibited. Potential needs for the human in a fully, autonomous car has been identified and various product development methods has been applied in order to develop two conceptual solutions for a future car interior. The solutions show the interaction between human and system since entertainment and comfort becomes important in a fully, autonomous car. Each solution is state-owned and holds four passengers. In the solutions, the seats are placed in regard to facilitate communication between the passengers. The passengers can be entertained or informed individually or collectively by text, sound and images.

L'objectif premier de cette thèse est de tester et d'étendre des modèles de navigation initialement développés pour de la navigation en intérieur et de les appliquer à la délégation de conduite des véhicules autonomes. La particularité de ces architectures de contrôle pour robot mobile est qu'elles sont issus d'une approche bio-inspirée et sont dite neurorobotiques : elles reproduisent les mécanismes neuronaux impliqués dans les comportements de navigation observés chez les animaux.Ces travaux concourent à introduire auprès de la communauté du véhicule autonome les approches bio-inspirées et leur potentiel au travers des modèles présentés.Cette thèse commence par aborder les technologies utilisées (GPS, lidar, ...) et les différentes stratégies de navigation (réactive, avec carte). Cette présentation se termine en circonscrivant le problème de la navigation des véhicules autonomes à la reconnaissance de lieux par la vision.Le manuscrit présente ensuite les approches bio-inspirée et neurorobotique avant de donner les notions d'anatomie facilitant la compréhension des modèles bio-inspirés décrits par la suite.Un modèle de reconnaissance de lieux par la vision est présenté en détail. Des expériences de simulations avec des données réelles permettent de tester ce modèle, initialement conçu pour la navigation intérieure, dans des environnements propres aux véhicules. Les résultats de ces tests montrent que le modèle construit une représentation de l'environnement capable de discriminer et de reconnaître des lieux.Une architecture neuronale de navigation réactive reposant sur le précédent système, le modèle PerAc est décrite ensuite. Ce modèle utilise les lieux construits par le modèle de reconnaissance et associe chaque lieu à une direction à suivre. Par un ensemble d'associations, il est possible d'apprendre à suivre de manière robuste des trajectoires ou une position à rejoindre. Ce modèle, conçu pour de la navigation en intérieur, est testé sur un véhicule dans des expériences servant à évaluer sa capacité à fonctionner sur véhicule. Les résultats prouvent que le modèle est capable d'apprendre des trajectoires avec une voiture en environnement extérieur.Ma dernière contribution porte sur un modèle de contextes visuels permettant d'accroître les performances computationnelles de la reconnaissance de lieux. Une chaîne de traitements visuelle inspirée du cortex visuel primaire effectue un découpage de l'environnement qui vient moduler la reconnaissance de lieux du modèle précédent.Ce manuscrit se termine en proposant un ensemble de pistes à même d'améliorer les différents aspects des modèles présentés, par exemple, en modélisant plus finement certaines zones du cerveau ou en utilisant un codage épars afin de réduire le coût computationnel
The main aim of this thesis is to extend navigation models built for indoor environments to autonomous vehicles.These models follow a neurorobotics approach: they reproduce neural substrates that sustain the mechanism responsible for spatial cognition in the mammalian brain. This thesis introduces and shows the potential of bio-inspired robotics and neurorobotics to the community of autonomous driving.This manuscrit begins by giving a state of the art of robotic navigation. It presents the subsidiary question of visual recognition.The notions of bio-inspired robotics and neurorobotics are then introduced. An overview of anatomical observations necessary to the comprehension of bio-inspired models is also given.The document describes a model for visual place recognition inspired by hippocampal place cells. This model is tested in outdoor on-road environments. The results show that the model is able to build a representation of the environment that can be used in navigation tasks.A reactive navigation model (PerAc) is then presented. This model uses the representation built by the visual place recognition model to learn sensorimotor units linking places and actions. A set of these associations allows the model to learn robust trajectories or goal positions. This model was initially built for robots evolving in indoor environments. It is embbedded and tested on an autonomous vehicle. The experiments show that the model is able to learn trajectory with a car in outdoor environments.My last contribution presented is about a visual contexts model. A second visual processing chain inspired by the primary visual cortex is proposed to build a coarse representation of the environment that modulates the activities of the visual place cells.The essay ends by giving a set of leads to improve different aspects of the proposed models, by modeling in more depth other brain structures or change the encoding of data for instance

Autonomous Driving and Advance Driver Assistance Systems (ADAS) are revolutionizing the way we drive and the future of mobility. Among ADAS, Traffic Sign Classification is an important technique which assists the driver to easily interpret traffic signs on the road. In this thesis, we used the powerful combination of Image Processing and Deep Learning to pre-process and classify the traffic signs. Recent studies in Deep Learning show us how good a Convolutional Neural Network (CNN) is for image classification and there are several state-of-the-art models with classification accuracies over 99 % existing out there. This shaped our thesis to focus more on tackling the current challenges and some open-research cases. We focussed more on performance tuning by modifying the existing architectures with a trade-off between computations and accuracies. Our research areas include enhancement in low light/noisy conditions by adding Recurrent Neural Network (RNN) connections, and contribution to a universal-regional dataset with Generative Adversarial Networks (GANs). The results obtained on the test data are comparable to the state-of-the-art models and we reached accuracies above 98% after performance evaluation in different frameworks

Mestrado em Engenharia Mecânica
Foram simulados numericamente jogos de recursos públicos em redes usando algoritmo de Monte Carlo. Foram usadas redes regulares unidimensionais em anel, redes regulares bidimensionais (rede quadrada) e redes scale-free. São apresentados os métodos seguidos, a teoria e os algoritmos usados. Estes jogos apresentam uma transição de fase entre uma fase dominada por oportunistas de uma fase dominada por cooperadores em função de um parâmetro de rendimento das contribuições. Foi encontrado um intervalo, dependente do número médio de vizinhos, para o qual a fracção de configurações sobreviventes tende para 1 quando o tamanho da rede aumenta. Foi também encontrada uma dependência no valor de parâmetro crítico de transição no número médio de vizinhos para as configurações sobreviventes. Esses efeitos foram observados em todos os tipos de rede estudados neste trabalho. ABSTRACT: Public goods games were numerically simulated in networks using Monte Carlo Algorithm. Regular one-dimensional ring networks, regular two-dimensional lattice networks and scale-free networks had been used. The methods followed, the theory and the algorithms used are presented. This games have a phase transition between one phase dominated by defectors from one dominated by cooperators in function of the value of efficiency from the contributions. It was found an interval, dependent on the average number of neighbors, where the fraction of surviving configurations tens to 1 when the size of the network increases. It was found dependence in the critical value of transition value with the average number of neighbors. Both effects were observed in all types of networks studied in this work.

L'automatisation de la conduite est un processus en cours qui est en train de changer radicalement la façon dont les gens voyagent et passent du temps dans leur voiture pendant leurs déplacements. Les véhicules conditionnellement automatisés libèrent les conducteurs humains de la surveillance et de la supervision du système et de l'environnement de conduite, leur permettant d'effectuer des activités secondaires pendant la conduite, mais requièrent qu’ils puissent reprendre la tâche de conduite si nécessaire. Pour les conducteurs, il est essentiel de comprendre les capacités et les limites du système, d’en reconnaître les notifications et d'interagir de manière adéquate avec le véhicule pour assurer leur propre sécurité et celle des autres usagers de la route. À cause de la diversité des situations de conduite que le conducteur peut rencontrer, les programmes traditionnels de formation peuvent ne pas être suffisants pour assurer une compréhension efficace de l'interaction entre le conducteur humain et le véhicule pendant les transitions de contrôle. Il est donc nécessaire de permettre aux conducteurs de vivre ces situations avant leur première utilisation du véhicule. Dans ce contexte, la Réalité Mixte constitue un outil d'apprentissage et d'évaluation des compétences potentiellement efficace qui permettrait aux conducteurs de se familiariser avec le véhicule automatisé et d'interagir avec le nouvel équipement dans un environnement sans risque. Si jusqu'à il y a quelques années, les plates-formes de Réalité Mixte étaient destinées à un public de niche, la démocratisation et la diffusion à grande échelle des dispositifs immersifs ont rendu leur adoption plus accessible en termes de coût, de facilité de mise en œuvre et de configuration. L'objectif de cette thèse est d'étudier le rôle de la réalité mixte dans l'acquisition de compétences pour l'interaction d'un conducteur avec un véhicule conditionnellement automatisé. En particulier, nous avons exploré le rôle de l'immersion dans le continuum de la réalité mixte en étudiant différentes combinaisons d'espaces de visualisation et de manipulation et la correspondance entre le monde virtuel et le monde réel. Du fait des contraintes industrielles, nous avons limité les candidats possibles à des systèmes légers portables, peu chers et facilement accessibles; et avons analysé l’impact des incohérences sensorimotrices que ces systèmes peuvent provoquer sur la réalisation des activités dans l’environnement virtuel. À partir de ces analyses, nous avons conçu un programme de formation visant l'acquisition des compétences, des règles et des connaissances nécessaires à l'utilisation d'un véhicule conditionnellement automatisé. Nous avons proposé des scénarios routiers simulés de plus en plus complexes pour permettre aux apprenants d’interagir avec ce type de véhicules dans différentes situations de conduite. Des études expérimentales ont été menées afin de déterminer l'impact de l'immersion sur l'apprentissage, la pertinence du programme de formation conçu et, à plus grande échelle, de valider l'efficacité de l'ensemble des plateformes de formation par des mesures subjectives et objectives. Le transfert de compétences de l'environnement de formation à la situation réelle a été évalué par des essais sur simulateurs de conduite haut de gamme et sur des véhicules réels sur la voie publique
Driving automation is an ongoing process that is radically changing how people travel and spend time in their cars during journeys. Conditionally automated vehicles free human drivers from the monitoring and supervision of the system and driving environment, allowing them to perform secondary activities during automated driving, but requiring them to resume the driving task if necessary. For the drivers, understanding the system’s capabilities and limits, recognizing the system’s notifications, and interacting with the vehicle in the appropriate way is crucial to ensuring their own safety and that of other road users. Because of the variety of unfamiliar driving situations that the driver may encounter, traditional handover and training programs may not be sufficient to ensure an effective understanding of the interaction between the human driver and the vehicle during transitions of control. Thus, there is the need to let drivers experience these situations before their first ride. In this context, Mixed Reality provides potentially valuable learning and skill assessment tools which would allow drivers to familiarize themselves with the automated vehicle and interact with the novel equipment involved in a risk-free environment. If until a few years ago these platforms were destined to a niche audience, the democratization and the large-scale spread of immersive devices since then has made their adoption more accessible in terms of cost, ease of implementation, and setup. The objective of this thesis is to investigate the role of Mixed Reality in the acquisition of competences needed for a driver’s interaction with a conditionally automated vehicle. In particular, we explored the role of immersion along the Mixed Reality continuum by investigating different combinations of visualization and manipulation spaces and the correspondence between the virtual and the real world. For industrial constraints, we restricted the possible candidates to light systems that are portable, cost-effective and accessible; we thus analyzed the impact of the sensorimotor incoherences that these systems may cause on the execution of tasks in the virtual environment. Starting from these analyses, we designed a training program aimed at the acquisition of skills, rules and knowledge necessary to operate a conditionally automated vehicle. In addition, we proposed simulated road scenarios with increasing complexity to suggest what it feels like to be a driver at this level of automation in different driving situations. Experimental user studies were conducted in order to determine the impact of immersion on learning and the pertinence of the designed training program and, on a larger scale, to validate the effectiveness of the entire training platform with self-reported and objective measures. Furthermore, the transfer of skills from the training environment to the real situation was assessed with test drives using both high-end driving simulators and actual vehicles on public roads

This degree project investigates the suitability of using Behavior Trees (BT) as an architecture for the behavioral layer in autonomous driving. BTs originate from video game development but have received attention in robotics research the past couple of years. This project also includes implementation of a simulated traffic environment using the Unity3D engine, where the use of BTs is evaluated and compared to an implementation using finite-state machines (FSM). After the initial implementation, the simulation along with the control architectures were extended with additional behaviors in four steps. The different versions were evaluated using software maintainability metrics (Cyclomatic complexity and Maintainability index) in order to extrapolate and reason about more complex implementations as would be required in a real autonomous vehicle. It is concluded that as the AI requirements scale and grow more complex, the BTs likely become substantially more maintainable than FSMs and hence may prove a viable alternative for autonomous driving.

L’objectiu d’aquesta tesi és estudiar algoritmes de reconstrucció 3D adequats per a la conducció autònoma. Per fer-ho, necessitem implementacions i representacions ràpides de l’entorn 3D que tinguin en compte la informació geomètrica i semàntica. L’ús de paral·lelització CUDA i GPU permet aprofitar maquinari flexible i programable d’alt rendiment per complir els requisits de temps exigents. La tesi presenta tres contribucions principals. En primer lloc, descrivim la paral·lelització del conegut algorisme d’estèreo basat en el Semi-Global Matching (SGM), que estima la profunditat a partir de dues imatges estèreo. Desplegem un disseny de paral·lelització eficient que funciona a les GPU de baix consum energètic i aconsegueix un rendiment en temps real. Com a segona contribució, presentem una millora del model de representació 3D anomenat Stixel World, que representa les superfícies inclinades. L’extensió del model ajuda a representar escenes reals que fallen sota els supòsits anteriors i, mitjançant una regularització eficient del model, manté la mateixa precisió que el model anterior. També proposem una estratègia algorítmica per accelerar el procés, que redueix la quantitat de combinacions Stixel provades. Finalment, expliquem les nostres estratègies de paral·lelització per a l’algorisme de segmentació de Stixel. Proposem una estratègia de paral·lelització que s’adapti a l’arquitectura massivament paral·lela de les GPU. També estudiem les diferents tècniques d’acceleració disponibles per a Stixels i com es poden implementar de manera eficient per a aquesta arquitectura. A més, el nostre enfocament redueix la complexitat computacional de l’algorisme mitjançant la reformulació del model.
El objetivo de esta tesis es estudiar algoritmos de reconstrucción 3D aptos para la conducción autónoma. Para ello, necesitamos implementaciones y representaciones rápidas del entorno 3D que tengan en cuenta la información geométrica y semántica. El uso de la paralelización de CUDA y GPU permite aprovechar el hardware de alto rendimiento flexible y programable para cumplir con los estrictos requisitos de tiempo. La tesis presenta tres contribuciones principales. Primero, describimos la paralelización del conocido algoritmo de estéreo basado en Semi-Global Matching (SGM), que estima la profundidad de dos imágenes estéreo. Implementamos un diseño de paralelización eficiente que se ejecuta sobre GPU de bajo consumo de energía y logra un rendimiento en tiempo real. Como segunda contribución, presentamos una mejora del modelo de representación 3D llamado Stixel World que da cuenta de las superficies inclinadas. La extensión del modelo ayuda a representar escenas reales que fallan bajo los supuestos anteriores y, mediante una regularización eficiente del modelo, mantiene la misma precisión del modelo anterior. También proponemos una estrategia algorítmica para acelerar el proceso, lo que reduce la cantidad de combinaciones de Stixel probadas. Finalmente, explicamos nuestras estrategias de paralelización para el algoritmo de segmentación de Stixel. Proponemos una estrategia de paralelización que se adapta a la arquitectura masivamente paralela de las GPU. También estudiamos las diferentes técnicas de aceleración disponibles para Stixels y cómo se pueden implementar de manera eficiente para esta arquitectura. Además, nuestro enfoque reduce la complejidad computacional del algoritmo al reformular el modelo.
The objective of this thesis is to study 3D reconstruction algorithms suitable for autonomous driving. In order to do so, we need fast implementations and representations of the 3D environment that take into account geometric and semantic information. The use of CUDA and GPU parallelization allows to leverage flexible and programmable high performance hardware to fulfill the strong time requirements. The thesis presents three main contributions. First, we describe the parallelization of the well-known stereo matching algorithm based on Semi-Global Matching (SGM), which estimates depth from two stereo images. We deploy an efficient parallelization design that runs on top of low-energy consumption GPUs and achieves real-time performance. As our second contribution, we present an improvement of the 3D representation model called the Stixel World that accounts for slanted surfaces. The extension of the model helps representing real scenes that fail under the previous assumptions, and, by an efficient model regularization, keeps the same accuracy of the previous model. We also propose an algorithmic strategy to speed up the process, which reduces the amount of Stixel combinations tested by the dynamic programming approach. Finally, we explain our parallelization strategies for the Stixel segmentation algorithm. We propose a parallelization strategy that fits the massively parallel architecture of GPUs. We also study the different speed up techniques available for Stixels and how they can be implemented efficiently for this architecture. Additionally, our approach reduces the computational complexity of the algorithm by reformulating the measurement depth model, relying on the confidence of the depth estimation and the identification of invalid values to handle outliers.

Autonomous driving, or self-driving, is the ability of a vehicle to drive itself without human input. To achieve this, the vehicle uses mechanical and electronic parts, sensors, actuators and an on-board computer. The on-board computer runs sophisticated software which allows the vehicle to perceive and understand its environment based on sensor input, localise itself in that environment and plan the optimal route from point A to point B. Autonomous driving is no longer a thing of the future, and to develop autonomous driving solutions is a highly valuable skill in today’s software engineering field. Robot Operating System (ROS) is a meta-operating system that simplifies the process of robotics programming. This master’s thesis aims to demonstrate how ROS could be used to develop autonomous driving software by analysing autonomous driving problems, examining existing solutions and developing a prototype vehicle using ROS. This thesis provides an overview of autonomous driving and usage of ROS in the development of autonomous driving, then elaborates on the benefits and challenges of using ROS for autonomous car development. The research methods used in this master’s thesis are design science research (DSR) and a literature review. An artefact is developed and evaluated—a remote-controlled (RC) car equipped with Raspberry Pi 3 board as the on-board computer, an Arduino Uno board, Teensy LC board, a set of sensors and ROS-based software. The thesis is supported by the author’s employer, automotive software company called Elektrobit. By following the steps described in this thesis, it is possible to develop an autonomous driving RC car which runs on ROS. Additionally, this thesis shows why ROS provides good solutions for the autonomous driving issues. It points to the benefits of ROS: open sourced, peer-to-peer, network-based meta-operating system with ready-made components for autonomous driving, and highlights some of the challenges of ROS: security issues and single point of failure.

Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 77-83).
The safe, successful deployment of autonomous systems under real-world conditions, in part, hinges upon providing rigorous performance and safety guarantees. This thesis considers the problem of establishing and verifying the safety of autonomous systems. To this end, we present a novel framework for the synthesis of safety constraints for autonomous systems, so-called safety contracts, that can be applied to and used by a wide set of real-world systems by acting as a design requirement for the controller implementation of the system. The contracts consider a large variety of road models, guarantee that the controlled system will remain safe with respect to probabilistic models of traffic behavior, and ensure that it will follow the rules of the road. We generate contracts using reachability analysis in a reach-avoid problem under consideration of dynamic obstacles, i.e., other traffic participants. Contracts are then derived directly from the reachable sets. By decomposing large road networks into local road geometries and defining assume-guarantee contracts between local geometries, we enable computational tractability over large spatial domains. To efficiently account for the behavior of other traffic participants, we iteratively alternate between falsification to generate new traffic scenarios that violate the safety contract and reachable set computation to update the safety contract. These counterexamples to collision-free behavior are found by solving a gradient-based trajectory optimization problem. We demonstrate the practical effectiveness of the proposed methods in a set of experiments involving the Manhattan road network as well as interacting multi-car traffic scenarios.
by Lucas Liebenwein.
S.M.

The thesis develops a trajectory planner which operates in a formula racing scenario. The proposed trajectory planner gives time-optimal off-road trajectory planning solutions and generates sequences of control signals for the vehicle to follow the trajectory. Outputs of the trajectory planner are time-optimal trajectory, steering angle, resultant force of brake and throttle. The trajectory planner is designed to have two modes, the Exploring mode which is based on Rapidly-exploring Random Tree (RRT), and the optimization mode which is built upon optimal Rapidly-exploring random tree (RRT*). The exploring mode can generate a valid and safe trajectory in real  time, but the solution is not optimal; optimization mode gives optimal trajectories but it can only do offline planning. The system structure makes it possible that the exploring mode keeps running while the optimization mode runs in the background; once the optimization process is complete, the vehicle could then follow the optimal trajectory. A local trajectory planning method which considers the constraint of off-road vehicle dynamics is designed and integrated in the planner. System performance is evaluated by simulations on a racing track. Many aspects including time optimality, vehicle stability have been taken into account. A new, fast, local steering method has been proposed; the method generates trajectory based on random input. By a more efficient implementation of the planner in the future, for example by using parallel computing, the optimization mode is promising for real-time trajectory generation.
Detta examensarbete utvecklar en trajektoriegenerering som baseras på ett Formula racing scenario. Den föreslagna trajektoriegenerering ger en tidsoptimal off-road-lösning och medför sekventiella kontrollsignaler till fordonen att följa trajektorien. Utsignaler från trajektoriegenerering är den tidsoptimal trajektorien, styrvinkel och motsvarande kraft for broms och gaspådrag. Trajektoriegenerering är designad med två funktioner, utforskningsläget baserat på Rapidly-exploring Random Tree (RRT) och optimeringsläget som baseras på det optimala Rapidly-exploring Random Tree (RRT). Utforskningsläget genererar en tillåten och säker trajektorie i realtid, men den är inte optimal; Optimeringsläget ger den optimala trajektorien men kan inte bli genererad i realtid. Systemstrukturen gör det möjligt att ha utforskningsläget körandes, medans optimeringsläget körs i bakgrunden; när optimeringsprocessen är färdig kan fordonet följa den optimala trajektorien. En lokal trajektoriegenererings-metod tar med dynamiska off-road krav för fordonet när planeringen görs. Prestandan av systemet är avgjort på en simulerat racerbana.Många aspekter såsom tidoptima och fordonsstabilitet har tagits med i beräkningen. En ny styrmetod har föreslagits; där metoden snabbt genererar en trajektorie baserad på slumpmässiga insignaler. Med mer effektiv framtida implementering av planeringen, exempelvis parallellberäkning, blir optimeringsläget en lovande trajektoriegenerering i realtid.

Dans cette thèse, nous abordons les défis de la rareté des annotations et la fusion de données hétérogènes tels que les nuages de points 3D et images 2D. D’abord, nous adoptons une stratégie de conduite de bout en bout où un réseau de neurones est entraîné pour directement traduire l'entrée capteur (image caméra) en contrôles-commandes, ce qui rend cette approche indépendante des annotations dans le domaine visuel. Nous utilisons l’apprentissage par renforcement profond où l'algorithme apprend de la récompense, obtenue par interaction avec un simulateur réaliste. Nous proposons de nouvelles stratégies d'entraînement et fonctions de récompense pour une meilleure conduite et une convergence plus rapide. Cependant, le temps d’apprentissage reste élevé. C'est pourquoi nous nous concentrons sur la perception dans le reste de cette thèse pour étudier la fusion de nuage de points et d'images. Nous proposons deux méthodes différentes pour la fusion 2D-3D. Premièrement, nous projetons des nuages de points LiDAR 3D dans l’espace image 2D, résultant en des cartes de profondeur éparses. Nous proposons une nouvelle architecture encodeur-décodeur qui fusionne les informations de l’image et la profondeur pour la tâche de complétion de carte de profondeur, améliorant ainsi la résolution du nuage de points projeté dans l'espace image. Deuxièmement, nous fusionnons directement dans l'espace 3D pour éviter la perte d'informations dû à la projection. Pour cela, nous calculons les caractéristiques d’image issues de plusieurs vues avec un CNN 2D, puis nous les projetons dans un nuage de points 3D global pour les fusionner avec l’information 3D. Par la suite, ce nuage de point enrichi sert d'entrée à un réseau "point-based" dont la tâche est l'inférence de la sémantique 3D par point. Sur la base de ce travail, nous introduisons la nouvelle tâche d'adaptation de domaine non supervisée inter-modalités où on a accès à des données multi-capteurs dans une base de données source annotée et une base cible non annotée. Nous proposons une méthode d’apprentissage inter-modalités 2D-3D via une imitation mutuelle entre les réseaux d'images et de nuages de points pour résoudre l’écart de domaine source-cible. Nous montrons en outre que notre méthode est complémentaire à la technique unimodale existante dite de pseudo-labeling
In this thesis, we address the challenges of label scarcity and fusion of heterogeneous 3D point clouds and 2D images. We adopt the strategy of end-to-end race driving where a neural network is trained to directly map sensor input (camera image) to control output, which makes this strategy independent from annotations in the visual domain. We employ deep reinforcement learning where the algorithm learns from reward by interaction with a realistic simulator. We propose new training strategies and reward functions for better driving and faster convergence. However, training time is still very long which is why we focus on perception to study point cloud and image fusion in the remainder of this thesis. We propose two different methods for 2D-3D fusion. First, we project 3D LiDAR point clouds into 2D image space, resulting in sparse depth maps. We propose a novel encoder-decoder architecture to fuse dense RGB and sparse depth for the task of depth completion that enhances point cloud resolution to image level. Second, we fuse directly in 3D space to prevent information loss through projection. Therefore, we compute image features with a 2D CNN of multiple views and then lift them all to a global 3D point cloud for fusion, followed by a point-based network to predict 3D semantic labels. Building on this work, we introduce the more difficult novel task of cross-modal unsupervised domain adaptation, where one is provided with multi-modal data in a labeled source and an unlabeled target dataset. We propose to perform 2D-3D cross-modal learning via mutual mimicking between image and point cloud networks to address the source-target domain shift. We further showcase that our method is complementary to the existing uni-modal technique of pseudo-labeling

Mestrado em Engenharia de Computadores e Telemática
Condução autónoma constitui a deslocação de um agente, robô ou veículo, de um qualquer ponto no espaço para um outro, sem qualquer intervenção humana, por forma a atingir objectivos pré-estabelecidos. Para conduzir de forma autónoma, usando planeamento de trajectória, é crucial que o agente consiga representar abstractamente tanto o conhecimento a priori acerca do mundo, como a informação que este vai adquirindo à medida que avança. Para alcançar este propósito, desenvolveu-se um sistema para ser usado na pista da Competição de Condução Autónoma do Festival Nacional de Robótica. Este sistema caracteriza-se por ser flexível e modular. Tais características permitem não são a adição componentes na pista acima referida, mas também a fácil expansão do suporte a outros tipos de pistas ou circuitos. Concluiu-se, pois, que o modelo de representação mais adequado para o sistema que se pretendia desenvolver seria um modelo híbrido, na medida em que, ao nível global tal representação seria topológica e ao nível local métrica. Ou seja, dividindo a pista em secções, estas são a base para a representação topológica, sendo depois cada secção mapeada internamente de forma métrica. Ao integrar o trabalho desta dissertação com o sistema global lograva-se alcançar um sistema de Condução Autónoma susceptível de planear a curto e médio prazo, com vista a melhorar o desempenho dos robôs usados no projecto, relativamente à solução anteriormente usada, que era baseada num sistema reactivo com alguma memória e noção de estado, mas sem planeamento de trajectória. ABSTRACT: Autonomous driving is the movement of an agent, robot or vehicle, from some point in space to another one, without any human intervention, in order to achieve predetermined goals. To drive autonomously using trajectory planning, it is vital to have an abstraction of the knowledge about the world, be it a priori or information that the agent acquires during the driving. For this, we developed a system capable of abstractly represent, not only the track for the Autonomous Driving Competition of the Portuguese Robotics Open, but also, tracks with similar characteristics. The system was developed in a exible and modular manner, in order to allow the addition of new elements to the stated track and the easy expansion to support other types of tracks and circuits. The conclusion was that the most appropriate representation model for the system we were trying to develop was an hybrid model, in that, at a global level the representation would be topological and at a local level it would be metrical. In other words, dividing the track into sections, these are the basis for the topological representation, being each of the sections then mapped internally using a metrical representation. Integrating the work of this dissertation in the global system, one hoped to achieve a Autonomous Driving system capable of short and medium term planning, with the goal of improve the performance of the ROTA project robots, comparatively with the previous solution, which was based in a reactive system with some memory and to some degree stateful.

Mestrado em Engenharia de Computadores e Telemática
Visão por computador e fusão sensorial são temas relativamente recentes, no entanto largamente adoptados no desenvolvimento de robôs autónomos que exigem adaptabilidade ao seu ambiente envolvente. Esta dissertação foca-se numa abordagem a estes dois temas para alcançar percepção no contexto de condução autónoma. O uso de câmaras para atingir este fim é um processo bastante complexo. Ao contrário dos meios sensoriais clássicos que fornecem sempre o mesmo tipo de informação precisa e atingida de forma determinística, as sucessivas imagens adquiridas por uma câmara estão repletas da mais variada informação e toda esta ambígua e extremamente difícil de extrair. A utilização de câmaras como meio sensorial em robótica é o mais próximo que chegamos na semelhança com aquele que é o de maior importância no processo de percepção humana, o sistema de visão. Visão por computador é uma disciplina científica que engloba àreas como: processamento de sinal, inteligência artificial, matemática, teoria de controlo, neurobiologia e física. A plataforma de suporte ao estudo desenvolvido no âmbito desta dissertação é o ROTA (RObô Triciclo Autónomo) e todos os elementos que consistem o seu ambiente. No contexto deste, são descritas abordagens que foram introduzidas com fim de desenvolver soluções para todos os desafios que o robô enfrenta no seu ambiente: detecção de linhas de estrada e consequente percepção desta, detecção de obstáculos, semáforos, zona da passadeira e zona de obras. É também descrito um sistema de calibração e aplicação da remoção da perspectiva da imagem, desenvolvido de modo a mapear os elementos percepcionados em distâncias reais. Em consequência do sistema de percepção, é ainda abordado o desenvolvimento de auto-localização integrado numa arquitectura distribuída incluindo navegação com planeamento inteligente. Todo o trabalho desenvolvido no decurso da dissertação é essencialmente centrado no desenvolvimento de percepção robótica no contexto de condução autónoma.
Computer vision and sensor fusion are subjects that are quite recent, however widely adopted in the development of autonomous robots that require adaptability to their surrounding environment. This thesis gives an approach on both in order to achieve perception in the scope of autonomous driving. The use of cameras to achieve this goal is a rather complex subject. Unlike the classic sensorial devices that provide the same type of information with precision and achieve this in a deterministic way, the successive images acquired by a camera are replete with the most varied information, that this ambiguous and extremely dificult to extract. The use of cameras for robotic sensing is the closest we got within the similarities with what is of most importance in the process of human perception, the vision system. Computer vision is a scientific discipline that encompasses areas such as signal processing, artificial intelligence, mathematics, control theory, neurobiology and physics. The support platform in which the study within this thesis was developed, includes ROTA (RObô Triciclo Autónomo) and all elements comprising its environment. In its context, are described approaches that introduced in the platform in order to develop solutions for all the challenges facing the robot in its environment: detection of lane markings and its consequent perception, obstacle detection, trafic lights, crosswalk and road maintenance area. It is also described a calibration system and implementation for the removal of the image perspective, developed in order to map the elements perceived in actual real world distances. As a result of the perception system development, it is also addressed self-localization integrated in a distributed architecture that allows navigation with long term planning. All the work developed in the course of this work is essentially focused on robotic perception in the context of autonomous driving.

In recent years, autonomous driving has become an increasingly practical technology. With state-of-the-art computer and sensor engineering, autonomous vehicles may be produced and widely used for travel and logistics in the near future. They have great potential to reduce traffic accidents, improve transportation efficiency, and release people from driving tasks while commuting. Researchers have built autonomous vehicles that can drive on public roads and handle normal surrounding traffic and obstacles. However, in situations like lane changing and merging, the autonomous vehicle faces the challenge of performing smooth interaction with human-driven vehicles. To do this, autonomous vehicle intelligence still needs to be improved so that it can better understand and react to other human drivers on the road. In this thesis, we argue for the importance of implementing ”socially cooperative driving”, which is an integral part of everyday human driving, in autonomous vehicles. An intention-integrated Prediction- and Cost function-Based algorithm (iPCB) framework is proposed to enable an autonomous vehicles to perform cooperative social behaviors. We also propose a behavioral planning framework to enable the socially cooperative behaviors with the iPCB algorithm. The new architecture is implemented in an autonomous vehicle and can coordinate the existing Adaptive Cruise Control (ACC) and Lane Centering interface to perform socially cooperative behaviors. The algorithm has been tested in over 500 entrance ramp and lane change scenarios on public roads in multiple cities in the US and over 10; 000 in simulated case and statistical testing. Results show that the proposed algorithm and framework for autonomous vehicle improves the performance of autonomous lane change and entrance ramp handling. Compared with rule-based algorithms that were previously developed on an autonomous vehicle for these scenarios, over 95% of potentially unsafe situations are avoided.

Autonomous driving is becoming popular nowadays. In order for autonomouscars to be fully accepted, high demands are placed on the safety side. Onesafety critical issue is the robustness to disturbances. In this work, a robustmodel predictive controller is designed for an autonomous vehicle. More specifically,robust output feedback model predictive control (ROFMPC) is used, androbustness is guaranteed through the use of robust invariant sets. The vehicleis modeled using a discretized, and linearized, version of a simple kinematicbicycle model, expressed in road-aligned coordinates. It is investigated for howlarge uncertainties robustness, and stability, can be guaranteed. Both externaldisturbances and measurement noise are considered. A steady-state Kalmanfilter is used to estimate the state of the vehicle. Two cases have been studiedin simulation; straight line and curved line trajectory following. Results fromsimulations show that robustness can be ensured if the uncertainties in the systemare sufficiently small. Finally, the ROFMPC algorithm is implemented onan F1/10 RC car.
Självkörande bilar blir alltmer populärt. För att självkörande bilar ska bliallmänt accepterade, har höga krav ställts på säkerheten. En viktig sak ursäkerhetssynpunkt är huruvida sytemet kan hantera störningar. I det här arbetetdesignas en robust modelprediktiv regulator för ett självkörande fordon.Mer specifikt används ”Robust Output Feedback Model Predictive Control”(ROFMPC) och robusthet, gentemot både externa störningar och mätbrus,garanteras genom användingen av robust invarianta mängder. Fordonet modellerasmed hjälp av en diskretiserad och linjäriserad enkel cykelliknande modell,uttryckt i naturliga koordinater. Två olika fall studeras genom simulering; delsdå fordonet ska följa en rak bana och dels då det ska följa en krökt bana. Ettstationärt Kalman filter används till att uppskatta fordonets tillstånd. Resultatenfrån simuleringarna visar på att robusthet kan garanteras om störningarnaär tillräckligt små. Slutligen är den robusta MPC-regulatorn implementerad påen F1/10 RC bil.

The urban population is growing at an exponential rate throughout the world leading to the problems related to swift and speedy mobility or issues caused by convectional mobility options. This study illustrates and explores the new ways to transport people specially taking into account the self-driving cars concept and discusses the concept of mobility 4.0 (smart / intelligent mobility) and briefly highlights the technological aspects of autonomous vehicles, adaptation advantages and progress in laws and legislations of autonomous vehicle. The study is primarily qualitative and relies on the work of previous researcher, technical reports and blogs but the part of this study is quantitative where empirical data was collected from the experts in a conference held by BBG Austria. The result of the studies shows adaptation readiness of Austrian professional market and business prospects associated with autonomous vehicles Moreover, different business models are suggested, which could be adopted to incorporate the driverless vehicles in day-to-day life of an individual living in urban environment. The models basically suggest that the adaptation of the technology would help curbing transport externalities especially external cost associated to transportation of each individual; which includes congestion, accident, infrastructure costs and environmental costs which are incurred by least efficient conventional cars and would also help shrinking the diseases like premature mortality, aggravation of respiratory as well as cardiovascular disease and sleep disturbance which are the result of city level congestion and pollution. (author's abstract)
Series: Schriftenreihe des Instituts für Transportwirtschaft und Logistik - Verkehr

Nowadays, autonomous driving revolution is getting closer to reality. To achieve the Autonomous driving the first step is to develop the Advanced Driver Assistance System (ADAS). Driver-assistance systems are one of the fastest-growing segments in automotive electronics since already there are many forms of ADAS available. To investigate state of art of development of ADAS towards Autonomous Driving, we develop Systematic Mapping Study (SMS). SMS methodology is used to collect, classify, and analyze the relevant publications. A classification is introduced based on the developments carried out in ADAS towards Autonomous driving. According to SMS methodology, we identified 894 relevant publications about ADAS and its developmental journey toward Autonomous Driving completed from 2012 to 2016. We classify the area of our research under three classifications: technical classifications, research types and research contributions. The related publications are classified under thirty-three technical classifications. This thesis sheds light on a better understanding of the achievements and shortcomings in this area. By evaluating collected results, we answer our seven research questions. The result specifies that most of the publications belong to the Models and Solution Proposal from the research type and contribution. The least number of the publications belong to the Automated…Autonomous driving from the technical classification which indicated the lack of publications in this area.

Outside of the city environment, there are many unstructured and rough environments that are challenging in vehicle navigation tasks. In these environments, vehicle vibrations caused by rough terrain can be harmful for humans. In addition, a human operator can not work around the clock. A promising solution is to use artificial intelligence to replace human operators. I test this by using the artificial intelligence technique know as reinforcement learning, with the algorithm Proximal Policy Optimization, to perform some basic locomotion tasks in a simulated environment with a simple terrain vehicle. The terrain vehicle consists of two chassis, each having two wheels attached, connected to each other with an articulation joint that can rotate to turn the vehicle. I show that a trained model can learn to operate the terrain vehicle and complete basic tasks, such as finding and following a path while avoiding obstacles. I tested robustness by evaluating performance on sloped terrains with a model trained to operate on flat ground. The results from the tests with different slopes show that, for most environments, the trained model could handle slopes up to around 7.5-10 degrees without much issue, even though it had no way of detecting the slope. This tells us that the models can perform their tasks quite well even when disturbances are introduced, as long as these disturbances doesn't require them to significantly change their behaviors.

Questa tesi rappresenta il risultato di un tirocinio svolto presso il reparto di test di Siemens Industry Software NV Leuven, in Belgio. Il primo obiettivo è stato quello di avere una visione generale sul settore della guida autonoma e delle relative tecnologie: si presenta quindi un'analisi della letteratura. Il campo è stato quindi ristretto ai riconoscitori di oggetti 2D che utilizzano sensori automotive come dispositivi di input. Dopo uno studio dello stato dell'arte di architetture di reti neurali e dei corrispondenti dataset usati per l'allenamento in questo settore, la domanda di ricerca è stata come validare la robustezza del sistema in condizioni reali, in particolare in degradate condizioni del manto stradale. Non essendo stato possibile portare a termine una campagna di test su strada per via dell'emergenza sanitaria, la decisione è stata di proseguire lo studio mediante la sperimentazione e testing virtuale. Questo inoltre ha permesso di estendere gli studi sulla scarsità di dati che affligge l'apprendimento automatico, consentendo di verificare se un ambiente virtuale può essere utile per la generazione di dati per l'allenamento e se può aiutare ad affrontare il problema del'adattamento del dominio. Quest'ultimo è stato affontato sotto due sfumature diverse. Da un lato l'intrinsica differenza tra dati reali e dati sintetici, dall'altro l'abilità di un sistema di adattarsi ad un nuovo dominio reale, che quindi presenta, ad esempio, delle condizioni ambientali differenti.

Articulated vehicles are widely used in the economically vital cargo industry as they provide a greater maneuverability than their rigid counterparts. Hence, autonomous driving systems for articulated vehicles have become the subject of intense research in the robotic community. This thesis analyzes the reverse motion of an articulated vehicle, namely a tractor-trailer with one on-axle hitched semitrailer, and develops a full autonomous driving system that enables reverse parking in the presence of static obstacles. The motion controller used in the autonomous driving system is based on a two-level feedback control system, with a path stabilization controller in the first level and a hitch angle controller in the second level. The path planner used is a modified RRT planner where the Dubins path has been incorporated in order to enable the planning towards a goal pose rather than merely a goal region. The modifications made have resulted in several improvements, such as more accurate planning and higher computational efficiency. Using a 1:32 scale remote controlled tractor-trailer, and a Qualisys motion capture system for pose estimation, the autonomous driving system was successfully implemented and validated.

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 75-79).
We introduce CarSpeak, a communication system for autonomous driving. CarSpeak enables a car to query and access sensory information captured by other cars in a manner similar to how it accesses information from its local sensors. CarSpeak adopts a content-centric approach where information objects - i.e., regions along the road - are first class citizens. It names and accesses road regions using a multi-resolution system, which allows it to scale the amount of transmitted data with the available bandwidth. CarSpeak also changes the MAC protocol so that, instead of having nodes contend for the medium, contention is between road regions, and the medium share assigned to any region depends on the number of cars interested in that region. CarSpeak is implemented in a state-of-the-art autonomous driving system and tested on indoor and outdoor hardware testbeds including an autonomous golf car and 10 iRobot Create robots. In comparison with a baseline that directly uses 802.11, CarSpeak reduces the time for navigating around obstacles by 2.4x, and reduces the probability of a collision due to limited visibility by 14 x.
by Swarun Suresh Kumar.
S.M.

Autonomous vehicles have been the subject of intense research, resulting in many of the latest cars being at least partly self driving. Cooperative driving extends this to a group of vehicles called a platoon, relying on com-munication between the vehicles in order to increase safety and improve the ˛ow of tra°c. This thesis is partly done in context of Grand Cooperative Driving Challenge (GCDC) 2016 where KTH has participated with a Scania truck and the Research Concept Vehicle (RCV), an electric prototype car.Trajectory planning is investigated for the longitudinal control of both the truck and the RCV. This planner is to ensure that the vehicles reached a position in a given time and a desired velocity. This is done using Pon-tryagin's minimum principle and interpolation.A more advanced planner based on Model Predictive Control (MPC) is used to avoid collisions in two di˙erent scenarios. One considers obstacle avoidance in the form of an overtake and the other a lane change scenario were the vehicle needs to decide how to position itself relative to the other vehicles.Simulations of the longitudinal control and planning of the truck did show that it could time the position and speed with a position error of less than 2m and speed error less than 0.2 m/s, assuming a distance of 120-200 m, a time interval of 40s and goal speed of 7m/s. The same simulation for the RCV had a distance error of less than 0.3m and a speed error below 0.2m.Simulations of the RCV using MPC planners showed that overtaking and lane changes could be performed. When performing the lane change the RCV managed to maintain a longitudinal distance of at least 1m, even if the other vehicles are slowing down or increasing their speed. The overtaking could also be successfully performed although with small margins, having a lateral distance of 0.5 m to the vehicle being overtaken.
Autonoma fordon har länge varit ett intensivt forskningsområde vilket resulterat i att många av de senaste bilana är åtminstone delvis självkörande. Cooperative driving utvidgar detta till en grupp fordon som kommunicerar med varandra för att öka säkerheten och få trafiken att flyta bättre. Den här uppsatsen baseras på Grand Cooperative Divers Challange (GCDC) 2016 där KTH deltog med en Scania lastbil och en elbil kallad Research Concept Vehicle (RCV). Rörelseplannering har undersökts för longitudinell kontrol av lastbilen och RCV. Den här planeraren ska se till att fordonet når en given position inom en viss tid och med en önskad hastighet. För detta ändamål används vad som kallas "Pontryagin's minimum principle" och interpolation. En mer avancerad planerare baserad på Model Predictive Control (MPC) används för att undvika kollisioner i två olika situationer. Den ena simulerar en omkörning och den andra ett filbyte med flera andra fordon i den intilliggande filen. Simuleringar av den longitudinella kontrollen av lastbilen visade att den kunde nå en position och hastighet med ett fel mindre än 2m respektive 0,2m/s då sträckor mellan 120-200m, ett tidsintervall på 40s och önskad hastighet på 7m/s används. Samma simuleringar med RCV hade ett positionsfel mindre än 0,3m och hastighetsfel under 0,2m/s. Simuleringar med RCV då MPC används visade att omkörningar och filbyten kunde genomföras. Filbyten kunde genomföras med ett longitudinellt avstånd på minst 1m, även då övriga fordon saktar ner eller ökar farten. Omkörningar kunde också genomföras om än med små marginaler. Det laterala avståndet var 0,5m till det omkörda fordonet.

The field of automated vehicle demands outstanding reliability figures to be matched by the artificially driving agents. The software architectures commonly used originate from decades of automation engineering, when robots operated only in confined environments on predefined tasks. On the other hand, autonomous driving represents an “into the wild” application for robotics. The architectures embraced until now may not be sufficiently robust to comply with such an ambitious goal. This research activity proposes a bio-inspired sensorimotor architecture for cognitive robots that addresses the lack of autonomy inherent to the rules-based paradigm. The new architecture finds its realization in an agent for autonomous driving named “Co-driver”. The Agent synthesis was extensively inspired by biological principles that contribute to give the Co-driver some cognitive abilities. Worth to be mentioned are the “simulation hypothesis of cognition” and the “affordance competition hypothesis”. The former is mainly concerned with how the Agent builds its driving skills, whereas the latter yields an interpretable agent notwithstanding the complex behaviors produced. Throughout the essay, the Agent is explained in detail, together with the bottom-up learning framework adopted. Overall, the research effort bore an effectively performing autonomous driving agent whose underlying architecture provides considerable adaptation capability. The thesis also discusses the aspects related to the implementation of the proposed ideas into a versatile software that supports both simulation environments and real vehicle platforms. The step-by-step explanation of the Co-driver is made up of theoretical considerations supported by working simulation examples, some of which are also released open-source to the research community as a driving benchmark. Eventually, guidelines are given for future research activities that may originate from the Agent and the hierarchical training framework devised. First and foremost, the exploitation of the hierarchical training framework to discover optimized longer-term driving policies.

Machine Learning, and in particular Deep Learning, are extremely capable tools for solving problems which are difficult, or intractable to tackle analytically. Application areas include pattern recognition, computer vision, speech and natural language processing. With the automotive industry aiming for increasing amount of automation in driving, the problems to solve become increasingly complex, which appeals to the use of supervised learning methods from Machine Learning and Deep Learning. With this approach, solutions to the problems are learned implicitly from training data, and inspecting their correctness is not possible directly. This presents concerns when the resulting systems are used to support safety-critical functions, as is the case with autonomous driving of automotive vehicles. This thesis studies the safety concerns related to learning systems within autonomous driving and applies a safety monitoring approach to a collision avoidance scenario. Experiments are performed using a simulated environment, with a deep learning system supporting perception for vehicle control, and a safety monitor for collision avoidance. The related operational situations and safety constraints are studied for an autonomous driving function, with potential faults in the learning system introduced and examined. Also, an example is considered for a measure that indicates trustworthiness of the learning system during operation.
Maskininlärning, och i synnerhet deep learning, är extremt kapabla verktyg för att lösa problem  som är svåra, eller omöjliga att hantera analytiskt. Applikationsområden inkluderar  mönsterigenkänning, datorseende, tal‐ och språkförståelse. När utvecklingen inom bilindustrin  går mot en ökad grad av automatisering, blir problemen som måste lösas alltmer komplexa,  vilket har lett till ett ökat användande av metoder från maskininlärning och deep learning. Med  detta tillvägagångssätt lär sig systemet lösningen till ett problem implicit från träningsdata och  man kan inte direkt utvärdera lösningens korrekthet. Detta innebär problem när systemet i  fråga är del av en säkerhetskritisk funktion, vilket är fallet för självkörande fordon. Detta  examensarbete behandlar säkerhetsaspekter relaterade till maskininlärningssystem i autonoma  fordon och applicerar en safety monitoring‐metodik på en kollisionsundvikningsfunktion.  Simuleringar utförs, med ett deep learning‐system som del av systemet för perception, som ger  underlag för styrningen av fordonet, samt en safety monitor för kollisionsundvikning. De  relaterade operationella situationerna och säkerhetsvillkoren studeras för en autonom  körnings‐funktion, där potentiella fel i det lärande systemet introduceras och utvärderas.  Vidare introduceras ett förslag på ett mått på trovärdighet hos det lärande systemet under  drift.

Alloy is a lightweight formal modeling tool that generates instances of a software specification to check properties of the design. This work demonstrates the use of Alloy for the rapid development of autonomous vehicle driving protocols. We contribute two driving protocols: a Normal protocol that represents the unpredictable yet safe driving behavior of typical human drivers, and a Connected protocol that employs connected technology for cooperative autonomous driving. Using five properties that define safe and productive driving actions, we analyze the performance of our protocols in mixed traffic. Lightweight formal modeling is a valuable way to reason about driving protocols early in the development process because it can automate the checking of safety and productivity properties and prevent costly design flaws.

We present an autonomous driving agent on a simulated highway driving scenario with vehicles such as cars and trucks moving with stochastically variable velocity profiles. The focus of the simulated environment is to test tactical decision making in highway driving scenarios. When an agent (vehicle) maintains an optimal range of velocity it is beneficial both in terms of energy efficiency and greener environment. In order to maintain an optimal range of velocity, in this thesis work I proposed two novel reward structures: (a) gaussian reward structure and (b) exponential rise and fall reward structure. I trained respectively two deep reinforcement learning agents to study their differences and evaluate their performance based on a set of parameters that are most relevant in highway driving scenarios. The algorithm implemented in this thesis work is double-dueling deep-Q-network with prioritized experience replay buffer. Experiments were performed by adding noise to the inputs, simulating Partially Observable Markov Decision Process in order to obtain reliability comparison between different reward structures. Velocity occupancy grid was found to be better than binary occupancy grid as input for the algorithm. Furthermore, methodology for generating fuel efficient policies has been discussed and demonstrated with an example.
Vi presenterar ett autonomt körföretag på ett simulerat motorvägsscenario med fordon som bilar och lastbilar som rör sig med stokastiskt variabla hastighetsprofiler. Fokus för den simulerade miljön är att testa taktiskt beslutsfattande i motorvägsscenarier. När en agent (fordon) upprätthåller ett optimalt hastighetsområde är det fördelaktigt både när det gäller energieffektivitet och grönare miljö. För att upprätthålla ett optimalt hastighetsområde föreslog jag i detta avhandlingsarbete två nya belöningsstrukturer: (a) gaussisk belöningsstruktur och (b) exponentiell uppgång och nedgång belöningsstruktur. Jag utbildade respektive två djupförstärkande inlärningsagenter för att studera deras skillnader och utvärdera deras prestanda baserat på en uppsättning parametrar som är mest relevanta i motorvägsscenarier. Algoritmen som implementeras i detta avhandlingsarbete är dubbel-duell djupt Q- nätverk med prioriterad återuppspelningsbuffert. Experiment utfördes genom att lägga till brus i ingångarna, simulera delvis observerbar Markov-beslutsprocess för att erhålla tillförlitlighetsjämförelse mellan olika belöningsstrukturer. Hastighetsbeläggningsgaller visade sig vara bättre än binärt beläggningsgaller som inmatning för algoritmen. Dessutom har metodik för att generera bränsleeffektiv politik diskuterats och demonstrerats med ett exempel.

The field of automated vehicle demands outstanding reliability figures to be matched by the artificially driving agents. The software architectures commonly used originate from decades of automation engineering, when robots operated only in confined environments on predefined tasks. On the other hand, autonomous driving represents an “into the wild” application for robotics. The architectures embraced until now may not be sufficiently robust to comply with such an ambitious goal. This research activity proposes a bio-inspired sensorimotor architecture for cognitive robots that addresses the lack of autonomy inherent to the rules-based paradigm. The new architecture finds its realization in an agent for autonomous driving named “Co-driver”. The Agent synthesis was extensively inspired by biological principles that contribute to give the Co-driver some cognitive abilities. Worth to be mentioned are the “simulation hypothesis of cognition” and the “affordance competition hypothesis”. The former is mainly concerned with how the Agent builds its driving skills, whereas the latter yields an interpretable agent notwithstanding the complex behaviors produced. Throughout the essay, the Agent is explained in detail, together with the bottom-up learning framework adopted. Overall, the research effort bore an effectively performing autonomous driving agent whose underlying architecture provides considerable adaptation capability. The thesis also discusses the aspects related to the implementation of the proposed ideas into a versatile software that supports both simulation environments and real vehicle platforms. The step-by-step explanation of the Co-driver is made up of theoretical considerations supported by working simulation examples, some of which are also released open-source to the research community as a driving benchmark. Eventually, guidelines are given for future research activities that may originate from the Agent and the hierarchical training framework devised. First and foremost, the exploitation of the hierarchical training framework to discover optimized longer-term driving policies.

Autonomous vehicles will help society if they can easily support a broad range of driving environments, conditions, and vehicles. Achieving this requires reducing the complexity of the algorithmic system, easing the collection of training data, and verifying operation using real-world experiments. Our work addresses these issues by utilizing a reflexive neural network that translates images into steering and throttle commands. This network is trained using simulation data from Grand Theft Auto V~\cite{gtav}, which we augment to reduce the number of simulation hours driven. We then validate our work using a RC car system through numerous tests. Our system successfully drive 98 of 100 laps of a track with multiple road types and difficult turns; it also successfully avoids collisions with another vehicle in 90\% of the trials.

Developments in distributed system technology facilitate the sharing of resources, even at a global level. This thesis explores sharing computational resources using mobile computations, agents, and autonomic techniques. We propose autonomous mobile programs (AMPs) which are aware of their resource needs and sensitive to the environment in which they execute. AMPs periodically use a cost model to decide where to execute in a network. Unusually this form of autonomous mobility affects only where the program executes and not what it does. We present a generic AMP cost modei, together with a validated instantiation and comparative performance results for four AMPs. We demonstrate that AMPs are able to dynamically relocate themselves to minimise execution time in the presence of varying network resources. Collections of AMPs effectively perform decentralised dynamic load balancing. Experiments on small LANs show that collections of AMPs quickly obtain and maintain optimal or near-optimal ball1nce. The advantages of our decentralised approach are that it has the potential to scale to very large and dynamic networks, and to achieve improved balance, and offers guarantees to limit overheads under reasonable assumptions. In an autonomous mobile program, the program must contain explicit control of self-aware mobile coordination. To encapsulate this for common patterns of computation over collections, autonomous mobility skeletons (AMSs) are proposed. These are akin to algorithmic skeletons in being polymorphic higher order functions, but where algorithmic skeletons abstract over parallel coordination, AMSs abstract over autonomous mobile coordination. AMS cost models have been built over collection iterations. The automap, autofold and Autolterator AMSs are presented, together with performance measurements for Jocaml, Java Voyager, and JavaGo implementations on LANs. An AMS considers only the cost of the current collective computation, but it is more useful to know the cost of the entire program. We have extended our AMS cost models to be parameterised on the cost of the remainder of the program. A cost calculus to estimate the costs for the remainder of a computation at arbitrary points has been built. An automatic Jocaml cost analyser based oil the calculus produces cost equations parameterised on program variables in context, and may be used to find both cost in higher-order functions and the cost for the remainder of the program. Costed autonomous mobility skeletons (CAMSs) have been built, which not only encapsulate common patterns of autonomous mobility but take additional cost parameters to provide costs for the remainder of the program. Potential performance improvements are assessed by comparing CAMS to AMS programs. The results show that CAMS programs perform more effectively than AMS programs, because they have more accurate cost information. Hence a CAMS program may move to a faster location when the corresponding AMS program does not.

For safe mobility, an autonomous vehicle must perceive the surroundings accurately. There are many perception tasks associated with understanding the local environment such as object detection, localization, and lane analysis. Object detection, in particular, plays a vital role in determining an object’s location and classifying it correctly and is one of the challenging tasks in the self-driving research area. Before employing an object detection module in autonomous vehicle testing, an organization needs to have a precise analysis of the module. Hence, it becomes crucial for a company to have an evaluation framework to evaluate an object detection algorithm’s performance. This thesis develops a comprehensive framework for evaluating and analyzing object detection algorithms, both 2D (camera images based) and 3D (LiDAR point cloud-based). The pipeline developed in this thesis provides the ability to evaluate multiple models with ease, signified by the key performance metrics, Average Precision, F-score, and Mean Average Precision. 40-point interpolation method is used to calculate the Average Precision.
För säker rörlighet måste ett autonomt fordon uppfatta omgivningen exakt. Det finns många uppfattningsuppgifter associerade med att förstå den lokala miljön, såsom objektdetektering, lokalisering och filanalys. I synnerhet objektdetektering spelar en viktig roll för att bestämma ett objekts plats och klassificera det korrekt och är en av de utmanande uppgifterna inom det självdrivande forskningsområdet. Innan en anställd detekteringsmodul används i autonoma fordonsprovningar måste en organisation ha en exakt analys av modulen. Därför blir det avgörande för ett företag att ha en utvärderingsram för att utvärdera en objektdetekteringsalgoritms prestanda. Denna avhandling utvecklar ett omfattande ramverk för utvärdering och analys av objektdetekteringsalgoritmer, både 2 D (kamerabilder baserade) och 3 D (LiDAR-punktmolnbaserade). Rörledningen som utvecklats i denna avhandling ger möjlighet att enkelt utvärdera flera modeller, betecknad med nyckelprestandamätvärdena, Genomsnittlig precision, F-poäng och genomsnittlig genomsnittlig precision. 40-punkts interpoleringsmetod används för att beräkna medelprecisionen.

Syftet med denna studie var att bygga, träna och testa two olika självkörande agenter med hjälp av maskininlärningstekniker, specifikt neurala nätverk. För att träna agenterna användes en teknik kallad Imitationsinlärning. Imitationsinlärning är en teknik för att lära agenter sekventiell beslutsfattning genom demostrering från en expert (vanligtvis en människa). Två något olika nätverksarkitekturer jämfördes. Skillnaden mellan dessa var att den kontrollmodul för en specifik intention (vilket här betecknar t.ex. åt vilket håll man ska köra i en korsning) var placerad antingen tidigt eller sent i det neurala nätverket. Testningen av de tränade modellerna observerade körningsförmåga såsom antalet lyckade intentioner och hastighet. Den tidiga modellen hade en signifikant högre hastighet än den sena modellen och verkade också köra generellt “bättre” än den sena modellen dock utan signifikans från den statistiska utvärderingen. Detta kan vara en effekt av en för liten stickprovstorlek, vilket kunde åtgärdats med användning av andra verktyg vid träning och testning. Det upptäcktes också att den tidiga modellens gas-värden som ackumulerades under testningen, var närmare expertens gas-distribution.
This study aimed to build, train, and test two different autonomous vehicle (AV) agents by using machine learning techniques, specifically neural network architectures. To be able to train the agents a technique called Imitation Learning was used. Imitation Learning is an approach for learning sequential decision-making from demonstrations provided by an expert. Two slightly different neural network architectures were compared. The difference was that the intentional command module (which denotes what direction to take in an intersection for example) was located either in the beginning or the end of the respective networks. The testing of the trained models was looking at their driving capabilities such as intentions completed and speed. The early model was significantly faster than the late model and seemed to be “better” at driving in general but with no significant difference from the late network. This could be an effect of the sample size being too small, which could have been rectified with different tools used in the training and testing. Additionally, it was found that the early model’s gas values, acquired at testing time, were closer to the expert gas distribution.

Autonomous driving technology is now evolving at an unprecedented speed. HD maps, which are embedded with highly precise and detailed road spatial and object information, play an important role in supporting autonomous vehicles. This thesis presents the development of a semi-automated HD map creation and updating method that is capable of extracting basic road feature information to HD maps by employing raw MLS point cloud data. The proposed HD map creation method consists of four steps: Road edge extraction, road surface extraction, road marking extraction and driving line generation. First, an existing curb-based road edge detection method is applied to extract road edge candidate points according to the elevation difference and slope between points. This thesis develops an edge vectorization algorithm based on the point's distance-to-trajectory. Then, the road surface is extracted by filtering the points inside fitted edges on the XY plane within a range of the ground elevation. In the next step, instead of using intensity to detect road markings used by most studies, this thesis fuses point clouds and images to assign each point with an RGB value to segment marking points. Marking objects are extracted by conditional Euclidean clustering and classified according to a manually defined decision tree. Finally, driving lines are generated based on the vectorized road edge and lane markings. The HD map update method varies depending on which data source is updated for the road segments, including updating images only, updating point clouds only and updating both images and point clouds. The method is evaluated by six point clouds and image datasets collected from different types of roads. The extracted road edges are assessed by both length- and buffer-based assessment methods. The results indicate that the road edge extraction algorithm performs well in all three dimensions. The road surface extraction results confirm the high accuracy of extracted edges. In addition, the quantitative evaluations of road markings demonstrate that the proposed road marking extraction method achieves an average recall, precision, and F1-score of 94.50%, 81.65% and 87.09%.

Mestrado em Engenharia de Computadores e Telemática
Os robôs autónomos e os veículos não tripulados são vertentes da robótica de forte investigação durante os últimos anos, especialmente para o desenvolvimento de veículos autónomos destinados à exploração de lugares inóspitos. Um robô autónomo é uma máquina que consegue ser independente e regida pelas suas próprias leis, um sistema que consegue sobreviver num ambiente natural sem intervenção humana. Hoje em dia são muitos os sistemas disponíveis (como por exemplo GPS e visão computorizada) que ajudam os robôs a sobreviver, sendo o grande desafio integrá-los de forma a produzir um ser mais inteligente, sólido e confiável. O Departamento de Electrónica, Telecomunicações e Informática da Universidade de Aveiro tem realizado, nos últimos anos, trabalho na área da condução autónoma. Um dos objectivos desse trabalho tem sido o desenvolvimento de veículos autónomos para participação na prova de condução autónoma do Festival Nacional de Robótica, na qual participa desde 2001. O software de controlo de alto nível do veículo actual assentava numa estrutura sequencial o que torna complexa a tarefa de manutenção e integração de novas funcionalidades. Actualmente existem módulos para a descrição do mundo bem como para o planeamento de trajectórias sobre esses modelos e um módulo para percepção a partir de imagem. O objectivo deste trabalho foi o de reestruturar todo o software de alto nível, tornando-o modular e concorrente permitindo dessa forma uma melhor manutenção, actualização e evolução. Fica assim mais fácil a substituição de módulos, a incorporação de novas funcionalidades e o trabalho de equipa. A nova estrutura assenta na existência de uma memória central e partilhada, tipo blackboard, onde os vários módulos recolhem os dados de que necessitam e depositam os dados produzidos. Este novo modelo arquitectural foi implementado no veículo e testado durante a edição de 2010 do Festival Nacional de Robótica, tendo alcançado o terceiro lugar. A arquitectura apresentada incorporou vários sistemas já existentes, tendo como principais vantagens a modularidade e extensibilidade dentro de um ambiente concorrente e com informação distribuída.
The autonomous robots and unmanned vehicles have been an intensive fi eld of research in the last years for driverless cars and harsh environments exploration. An autonomous robot is a machine that can work in an independent way and subject to its own laws only, a system that can survive in the real-world environment without human intervention. Today many systems are available (e.g. GPS, Computer Vision) that aid the robots to survive, being the biggest challenge put all together and create an agent more intelligent, robust and reliable. The Department of Electronics, Telecommunications and Informatics of University of Aveiro in the last years has worked in the autonomous driving area. One point of this work has been the developing of autonomous vehicles for participation in the autonomous driving inside competition of the Portuguese Robotics Open, where it has participated since 2001. The high level software that controls the actual vehicle is based on a sequential structure that turns maintenance and integration of new modules in a complex task. Currently, there are modules to carry out several basic tasks, namely, image perception and integration, \world" representation, and creation of trajectory plans The aim of this work is the reorganization of the existent high-level software, following a modular and concurrent paradigm. The new software organization makes it easier to replace software modules and to add new functionalities, enhancing team work development and maintenance. The new structure is based on the existence of a central shared memory, like a blackboard, where the modules collect data that they need as well as publish produced data. This new architectural approach has been implemented in the ROTA robot and it was tested in the 2010 Portuguese Robotics Open (where it ranked 3rd). The proposed architecture links several existing systems and has as strongest points modularity and extensibility in concurrent environment with distributed information.

Les véhicules autonomes, qualifiés aussi de véhicules sans conducteur, deviennent dans certains contextes une réalité tangible et partageront très bientôt nos routes avec d’autres véhicules classiques. Pour qu’un véhicule autonome se déplace de manière sécurisée, il doit savoir où il se trouve et ce qui l’entoure dans l’environnement. Pour la première tâche, pour déterminer sa position dans l’environnement, il doit se localiser selon six degrés de liberté (position et angles de rotation). Alors que pour la deuxième tâche, une bonne connaissance de cet environnement « proche » est nécessaire, ce qui donne lieu à une solution sous forme de cartographie. Par conséquent, pour atteindre le niveau de sécurité souhaité des véhicules autonomes, une localisation précise est primordiale. Cette localisation précise permet au véhicule non seulement de se positionner avec précision, mais également de trouver sa trajectoire optimale et d’éviter efficacement les collisions avec des objets statiques et dynamiques sur son trajet. Actuellement, la solution la plus répandue est le système de positionnement (GPS). Ce système ne permet qu’une précision limitée (de l’ordre de plusieurs mètres) et bien que les systèmes RTK (RealTime Kinematic) et DGPS (Differential GPS) aient atteint une précision bien plus satisfaisante, ces systèmes restent sensibles au masquage des signaux, et aux réflexions multiples, en particulier dans les zones urbaines denses. Toutes ces déficiences rendent ces systèmes inadaptés pour traiter des tâches critiques telles que l’évitement des collisions. Une alternative qui a récemment attiré l’attention des experts (chercheurs et industriels), consiste à utiliser une carte à priori pour localiser la voiture de l’intérieur de celui-ci. En effet, les cartes facilitent le processus de navigation et ajoutent une couche supplémentaire de sécurité et de compréhension. Le véhicule utilise ses capteurs embarqués pour comparer ce qu’il perçoit à un moment donné avec ce qui est stocké dans sa mémoire. Les cartes à priori permettent donc au véhicule de mieux se localiser dans son environnement en lui permettant de focaliser ses capteurs et la puissance de calcul uniquement sur les objets en mouvement. De cette façon, le véhicule peut prédire ce qui devrait arriver et voir ensuite ce qui se passe réellement en temps réel, et donc peut prendre une décision sur ce qu’il faut faire.Cette thèse vise donc à développer des outils permettant une localisation précise d’un véhicule autonome dans un environnement connu à priori. Cette localisation est déterminée par appariement (Map-matching) entre une carte de l’environnement disponible a priori et les données collectées au fur et à mesure que le véhicule se déplace. Pour ce faire, deux phases distinctes sont déployées. La première permet la construction de la carte, avec une précision centimétrique en utilisant des techniques de construction de cartes statiques ou dynamiques. La seconde correspond à la capacité de localiser le véhicule dans cette carte 3D en l’absence d’infrastructures dédiées comprenant le système GPS, les mesures inertielles (IMU) ou des balises.Au cours de ce travail, différentes techniques sont développées pour permettre la réalisation des deux phases mentionnées ci-dessus. Ainsi, la phase de construction de cartes, qui consiste à recaler des nuages de points capturés pour construire une représentation unique et unifiée de l’environnement, correspond au problème de la localisation et de la cartographie simultanée (SLAM). Afin de faire face à ce problème, nous avons testé et comparé différentes méthodes de recalage. Cependant, l’obtention de cartes précises nécessite des nuages de points très denses, ce qui les rend inefficaces pour une utilisation en temps réel. Dans ce contexte, une nouvelle méthode de réduction des points est proposée. (...)
The concept of self-driving vehicles is becoming a happening reality and will soon share our roads with other vehicles –autonomous or not-. For a self-driving car to move around in its environment in a securely, it needs to sense to its immediate environment and most importantly localize itself to be able to plan a safe trajectory to follow. Therefore, to perform tasks suchas trajectory planning and navigation, a precise localization is of upmost importance. This would further allow the vehicle toconstantly plan and predict an optimal path in order to weave through cluttered spaces by avoiding collisions with other agentssharing the same space as the latter. For years, the Global Positioning System (GPS) has been a widespread complementary solution for navigation. The latter allows only a limited precision (range of several meters). Although the Differential GPSand the Real Time Kinematic (RTK) systems have reached considerable accuracy, these systems remain sensitive to signal masking and multiple reflections, offering poor reliability in dense urban areas. All these deficiencies make these systems simply unsuitable to handle hard real time constraints such as collision avoidance. A prevailing alternative that has attracted interest recently, is to use upload a prior map in the system so that the agent can have a reliable support to lean on. Indeed,maps facilitate the navigation process and add an extra layer of security and other dimensions of semantic understanding. The vehicle uses its onboard sensors to compare what it perceives at a given instant to what is stored in the backend memory ofthe system. In this way, the autonomous vehicle can actually anticipate and predict its actions accordingly.The purpose of this thesis is to develop tools allowing an accurate localization task in order to deal with some complex navigation tasks outlined above. Localization is mainly performed by matching a 3D prior map with incoming point cloudstructures as the vehicle moves. Three main objectives are set out leading with two distinct phases deployed (the map building and the localization). The first allows the construction of the map, with centimeter accuracy using static or dynamic laser surveying technique. Explicit details about the experimental setup and data acquisition campaigns thoroughly carried outduring the course of this work are given. The idea is to construct efficient maps liable to be updated in the long run so thatthe environment representation contained in the 3D models are compact and robust. Moreover, map-building invariant on any dedicated infrastructure is of the paramount importance of this work in order to rhyme with the concept of flexible mapping and localization. In order to build maps incrementally, we rely on a self-implementation of state of the art iterative closest point (ICP) algorithm, which is then upgraded with new variants and compared to other implemented versions available inthe literature. However, obtaining accurate maps requires very dense point clouds, which make them inefficient for real-time use. Inthis context, the second objective deals with points cloud reduction. The proposed approach is based on the use of both colorinformation and the geometry of the scene. It aims to find sets of 3D points with the same color in a very small region and replacing each set with one point. As a result, the volume of the map will be significantly reduced, while the proprieties of this map such as the shape and color of scanned objects remain preserved.The third objective resort to efficient, precise and reliable localization once the maps are built and treated. For this purpose, the online data should be accurate, fast with low computational effort whilst maintaining a coherent model of the explored space. To this end, the Velodyne HDL-32 comes into play. (...)

Platooning and cooperative driving can decrease the emissions of greenhouse gases and increase the traffic capacity of the roads. The Grand Cooperative Driving Challenge, GCDC, is a competition that will be held in May 2016 focusing on cooperative driving. A cornerstone in the cooperative driving is autonomous driving. The main objective of this thesis is to design a Model Predictive Control for a truck so it autonomously can perform the following tasks: follow a straight road, make a lane change and make a turn. Constraints are added to the vehicle states and the control signals. Additionally, a constraint is added to make the vehicle keep a safe distance to preceding vehicles. A Linear Time-Varying (LTV) MPC for reference tracking is derived. To use the MPC for reference tracking references for all the states and control signals are derived. For the lane change and turn scenario Bezier curves are used to obtain the position references. The controller is implemented in MATLAB and validated through simulations. For the simulations both a bicycle model of the vehicle and a more complicated four wheel model are used. The latter is implemented in Simulink. The bicycle model is on-line linearised around the references in order to be used as the prediction model for the LTV-MPC. The simulations show that the controller can make the vehicle perform the above mentioned tasks. With a horizon of 18 the time in average to perform one iteration of the control loop is 0.02 s. The maximum deviation in the lateral direction is 0.10 m and occurs for the turn scenario when the four wheel model is used for the simulations. Simulations are also done with a preceding vehicle. The controller is able to make the vehicle keep a safe distance to the preceding vehicle. If the preceding vehicle is driving slower than the controlled vehicle the controller is able to decrease the velocity of the controlled vehicle. In addition to the above mentioned, simulations are also done where disturbances and noise, separately, are added. As a disturbance an error in the start position is used. The vehicle can start 1.3 m from the real start position, in the lateral direction, and still find its way back to the trajectory. The noise is added as white noise to the position updates. The controller can deal with noise with a standard deviation up to 0.3 m.
Kolonnkörning och kooperativkörning kan minska utsläppen av växthusgaser och öka trafikkapaciteten på vägarna. The Grand Cooperative Driving Challenge, GCDC, är en tävling med fokus på kooper-ativkörning som kommer att hållas i maj 2016. En hörnsten i kooperativkörning är autonomkörning. Det huvudsakliga målet med det här examensarbetet ¨ar att designa en MPC för en lastbil så att den autonomt kan genomföra följande: följa en rak väg, göra ett filbyte samt göra en sväng. Begränsningar är lagda på fordonets tillstånd och kontrollsignalerna. Utöver det begränsas avståndetet till framförvarande fordon. En linjär tidsvariant (LTV) MPC för referensföljning tas fram. För att använda MPC:n för referensföljning härleds referenser för fordonets tillstånd och kontrollsignaler. För filbytet och svängen används Bezier kurvor för att få fram positionsreferenserna. Regulatorn implementeras i MATLAB och valideras genom simuleringar. För simuleringarna används både en cykelmodell av fordonet och en mer komplicerad fyrhjuls-modell. Den senare implementeras i Simulink. Cykelmodellen linjäriseras online kring referenserna i syfte att användas som prediktionsmodell för LTV-MPC. Simuleringarna visar att regulatorn kan få fordonet att genomföra de ovan nämnda uppgifterna. Med en horisont på 18 ¨ar tiden det tar att genomföra en iteration av regulator-loopen i genomsnitt 0.02 sekunder. Den maximala avvikelsen i den laterala riktningen ¨ar 0.10 meter och uppstår när simuleringar görs för en sväng med fyrhjuls-modellen som fordonsmodell. Simuleringar görs även med ett framförvarande fordon. Regulatorn kan få fordonet att hallå ett säkerhetsavstånd till fordonet framför. Regulatorn kan vidare få fordonet att sänka hastigheten om det framförvarande fordonet kör långsammare. Utöver de ovan nämnda simuleringarna görs simuleringar där störningar och brus, var för sig, introduceras. Som störning används ett fel i startpositionen. Fordonet kan starta 1.3 meter från den korrekta startpositionen, i den laterala riktningen, och hitta tillbaks till referensbanan. Bruset adderas som vitt brus på positionsuppdateringarna. Regulatorn kan hantera vitt brus med en stan-dardavvikelse på upp till 0.3 meter.

L’anotació manual d’imatges per desenvolupar sistemes basats en visió per computador ha estat un dels punts més problemàtics des que s’utilitza aprenentatge automàtic per a això. Aquesta tesi es centra en aprofitar les dades sintètiques per alleujar el cost de les anotacions manuals en tres tasques de percepció relacionades amb l’assistència a la conducció i la conducció autònoma. En tot moment assumim l’ús de xarxes neuronals convolucionals per al desenvolupament dels nostres models profunds de percepció. La primera tasca planteja el reconeixement de senyals de trànsit, un problema de classificació d’imatges. Assumim que el nombre de classes de senyals de trànsit a reconèixer s’ha d’incrementar sense haver pogut anotar noves imatges amb què realitzar el corresponent reentrenament. Demostrem que aprofitant les dades sintètiques de les noves classes i transformant-les amb una xarxa adversària-generativa (GAN, de les seves sigles en anglès) entrenada amb les classes conegudes (sense usar mostres de les noves classes), és possible reentrenar la xarxa neuronal per classificar tots els senyals en una proporció ~1/4 entre classes noves i conegudes. La segona tasca consisteix en la detecció de vehicles i vianants (objectes) en imatges. En aquest cas, assumim la recepció d’un conjunt d’imatges sense anotar. L’objectiu és anotar automàticament aquestes imatges perquè així es puguin utilitzar posteriorment en l’entrenament del detector d’objectes que desitgem. Per assolir aquest objectiu, vam partir de dades sintètiques anotades i proposem un mètode d’aprenentatge semi-supervisat basat en la idea del co-aprenentatge. A més, utilitzem una GAN per reduir la distància entre els dominis sintètic i real abans d’aplicar el co-aprenentatge. Els nostres resultats quantitatius mostren que el procediment desenvolupat permet anotar el conjunt d’imatges d’entrada amb la precisió suficient per entrenar detectors d’objectes de forma efectiva; és a dir, tan precisos com si les imatges s’haguessin anotat manualment. A la tercera tasca deixem enrere l’espai 2D de les imatges, i ens centrem en processar núvols de punts 3D provinents de sensors LiDAR. El nostre objectiu inicial era desenvolupar un detector d’objectes 3D (vehicles, vianants, ciclistes) entrenat en núvols de punts sintètics estil LiDAR. En el cas de les imatges es podia esperar el problema de canvi de domini degut a les diferències visuals entre les imatges sintètiques i reals. Però, a priori, no esperàvem el mateix en treballar amb núvols de punts LiDAR, ja que es tracta d’informació geomètrica provinent del mostreig actiu del món, sense que l’aparença visual influeixi. No obstant això, a la pràctica, hem vist que també apareixen els problemes d’adaptació de domini. Factors com els paràmetres de mostreig del LiDAR, la configuració dels sensors a bord del vehicle autònom, i l’anotació manual dels objectes 3D, indueixen diferències de domini. A la tesi demostrem aquesta observació mitjançant un exhaustiu conjunt d’experiments amb diferents bases de dades públiques i detectors 3D disponibles. Per tant, en relació amb la tercera tasca, el treball s’ha centrat finalment en el disseny d’una GAN capaç de transformar núvols de punts 3D per portar-los d’un domini a un altre, un tema relativament inexplorat.Finalment, cal esmentar que tots els conjunts de dades sintètiques usats en aquestes tres tasques han estat dissenyats i generats en el context d’aquesta tesi doctoral i es faran públics. En general, considerem que aquesta tesi presenta un avanç en el foment de la utilització de dades sintètiques per al desenvolupament de models profunds de percepció, essencials en el camp de la conducció autònoma.
La anotación manual de imágenes para desarrollar sistemas basados en visión por computador ha sido uno de los puntos más problemáticos desde que se utiliza aprendizaje automático para ello. Esta tesis se centra en aprovechar los datos sintéticos para aliviar el coste de las anotaciones manuales en tres tareas de percepción relacionadas con la asistencia a la conducción y la conducción autónoma. En todo momento asumimos el uso de redes neuronales convolucionales para el desarrollo de nuestros modelos profundos de percepción. La primera tarea plantea el reconocimiento de señales de tráfico, un problema de clasificación de imágenes. Asumimos que el número de clases de señales de tráfico a reconocer se debe incrementar sin haber podido anotar nuevas imágenes con las que realizar el correspondiente reentrenamiento. Demostramos que aprovechando los datos sintéticos de las nuevas clases y transformándolas con una red adversaria-generativa (GAN, de sus siglas en inglés) entrenada con las clases conocidas (sin usar muestras de las nuevas clases), es posible reentrenar la red neuronal para clasificar todas las señales en una proporción de ~1/4 entre clases nuevas y conocidas. La segunda tarea consiste en la detección de vehículos y peatones (objetos) en imágenes. En este caso, asumimos la recepción de un conjunto de imágenes sin anotar. El objetivo es anotar automáticamente esas imágenes para que así se puedan utilizar posteriormente en el entrenamiento del detector de objetos que deseemos. Para alcanzar este objetivo, partimos de datos sintéticos anotados y proponemos un método de aprendizaje semi-supervisado basado en la idea del co-aprendizaje. Además, utilizamos una GAN para reducir la distancia entre los dominios sintético y real antes de aplicar el co-aprendizaje. Nuestros resultados cuantitativos muestran que el procedimiento desarrollado permite anotar el conjunto de imágenes de entrada con la precisión suficiente para entrenar detectores de objetos de forma efectiva; es decir, tan precisos como si las imágenes se hubiesen anotado manualmente. En la tercera tarea dejamos atrás el espacio 2D de las imágenes, y nos centramos en procesar nubes de puntos 3D provenientes de sensores LiDAR. Nuestro objetivo inicial era desarrollar un detector de objetos 3D (vehículos, peatones, ciclistas) entrenado en nubes de puntos sintéticos estilo LiDAR. En el caso de las imágenes cabía esperar el problema de cambio de dominio debido a las diferencias visuales entre las imágenes sintéticas y reales. Pero, a priori, no esperábamos lo mismo al trabajar con nubes de puntos LiDAR, ya que se trata de información geométrica proveniente del muestreo activo del mundo, sin que la apariencia visual influya. Sin embargo, en la práctica, hemos visto que también aparecen los problemas de adaptación de dominio. Factores como los parámetros de muestreo del LiDAR, la configuración de los sensores a bordo del vehículo autónomo, y la anotación manual de los objetos 3D, inducen diferencias de dominio. En la tesis demostramos esta observación mediante un exhaustivo conjunto de experimentos con diferentes bases de datos públicas y detectores 3D disponibles. Por tanto, en relación a la tercera tarea, el trabajo se ha centrado finalmente en el diseño de una GAN capaz de transformar nubes de puntos 3D para llevarlas de un dominio a otro, un tema relativamente inexplorado. Finalmente, cabe mencionar que todos los conjuntos de datos sintéticos usados en estas tres tareas han sido diseñados y generados en el contexto de esta tesis doctoral y se harán públicos. En general, consideramos que esta tesis presenta un avance en el fomento de la utilización de datos sintéticos para el desarrollo de modelos profundos de percepción, esenciales en el campo de la conducción autónoma.
Manually annotating images to develop vision models has been a major bottleneck since computer vision and machine learning started to walk together. This thesis focuses on leveraging synthetic data to alleviate manual annotation for three perception tasks related to driving assistance and autonomous driving. In all cases, we assume the use of deep convolutional neural networks (CNNs) to develop our perception models. The first task addresses traffic sign recognition (TSR), a kind of multi-class classification problem. We assume that the number of sign classes to be recognized must be suddenly increased without having annotated samples to perform the corresponding TSR CNN re-training. We show that leveraging synthetic samples of such new classes and transforming them by a generative adversarial network (GAN) trained on the known classes (i.e., without using samples from the new classes), it is possible to re-train the TSR CNN to properly classify all the signs for a ~1/4 ratio of new/known sign classes. The second task addresses on-board 2D object detection, focusing on vehicles and pedestrians. In this case, we assume that we receive a set of images without the annotations required to train an object detector, i.e., without object bounding boxes. Therefore, our goal is to self-annotate these images so that they can later be used to train the desired object detector. In order to reach this goal, we leverage from synthetic data and propose a semi-supervised learning approach based on the co-training idea. In fact, we use a GAN to reduce the synth-to-real domain shift before applying co-training. Our quantitative results show that co-training and GAN-based image-to-image translation complement each other up to allow the training of object detectors without manual annotation, and still almost reaching the upper-bound performances of the detectors trained from human annotations. While in previous tasks we focus on vision-based perception, the third task we address focuses on LiDAR pointclouds. Our initial goal was to develop a 3D object detector trained on synthetic LiDAR-style pointclouds. While for images we may expect synth/real-to-real domain shift due to differences in their appearance (e.g. when source and target images come from different camera sensors), we did not expect so for LiDAR pointclouds since these active sensors factor out appearance and provide sampled shapes. However, in practice, we have seen that it can be domain shift even among real-world LiDAR pointclouds. Factors such as the sampling parameters of the LiDARs, the sensor suite configuration on-board the ego-vehicle, and the human annotation of 3D bounding boxes, do induce a domain shift. We show it through comprehensive experiments with different publicly available datasets and 3D detectors. This redirected our goal towards the design of a GAN for pointcloud-to-pointcloud translation, a relatively unexplored topic. Finally, it is worth to mention that all the synthetic datasets used for these three tasks, have been designed and generated in the context of this PhD work and will be publicly released. Overall, we think this PhD presents several steps forward to encourage leveraging synthetic data for developing deep perception models in the field of driving assistance and autonomous driving.
Universitat Autònoma de Barcelona. Programa de Doctorat en Informàtica

Bei der Entwicklung neuartiger und innovativer Fahrerassistenzsysteme kommt der Positions- und Ausrichtungsbestimmung von Fahrzeugen eine Schlüsselrolle zu. Dabei entscheidet die Güte der Positionsbestimmung über die Qualität, die Robustheit und den Einsatzbereich des Gesamtsystems. Verbesserungen in der Positionsbestimmung führen zu einer besseren Performanz bzw. sind die Grundvoraussetzung für die Realisierung dieser Fahrerassistenzsysteme. Ein Beispiel für solch ein neuartiges Fahrerassistenzsystem, welches auf eine hochgenaue Positionsbestimmung baut, ist der BMW TrackTrainer. Dieses Assistenzsystem soll den "normalgeübten" Autofahrer beim schnellen Erlernen der Ideallinie auf Rennstrecken unterstützen, indem das Fahrzeug die Rennstrecke völlig autonom auf einer vorher aufgezeichneten Ideallinie umrundet, während der Teilnehmer sich die Strecke aus Fahrerperspektive einprägt. Für die Realisierung eines derartigen Assistenzsystems ist eine hochgenaue Positionsbestimmung im cm-Bereich notwendig. Bisher wurde dafür eine GPS-gestützte Inertialplattform eingesetzt, welche unter guten GPS-Empfangsbedingungen die Anforderungen an die Positionierung erfüllt. Bei schlechten GPS-Empfangsbedingungen, wie sie beispielsweise auf der international bekannten Rennstrecke Nürburgring Nordschleife aufgrund von Verdeckung und Abschattung der Satellitensignale durch stark bebautes oder bewaldetes Gebiet auftreten, liefert das Positionierungssystem keine ausreichend genauen Werte, wodurch das autonome Fahren verhindert wird. Zwar gibt es neben GPS auch weitere Positionsbestimmungssysteme, die aber für den Einsatz auf Rennstrecken entweder zu ungenau sind, oder einen zu hohen Rüstaufwand erfordern würden. Um diese Lücke zu schließen, wurde im Rahmen dieser Arbeit ein hochgenaues Positionsbestimmungssystem entwickelt und evaluiert, welches auch unter schlechten GPS-Empfangsbedingungen den Anforderungen des autonomen Fahren auf Rennstrecken genügt und auf einer Fusion verschiedener Signalquellen in einem Positionsfilter beruht. Folgende Signalquellen wurden hinsichtlich Genauigkeit sowie Praxistauglichkeit für den Einsatz auf Rennstrecken experimentell untersucht: - GPS-gestützte Inertialplattform (GPS/INS) - Fahrzeugsensoren mit erweitertem Fahrzeugmodell - Digitaler Kompass - Laser-Reflexlichtschranken - Servo-Tachymeter - LIDAR-basierte Randbebauungserkennung - Videobasierte Spurerkennung - Digitale Karte. Obwohl eine GPS-gestützte Inertialplattform (GPS/INS) unter schlechten GPS-Empfangsbedingungen keine ausreichend genauen Positionswerte im cm-Bereich liefert, besitzt dieses System dennoch eine hohe Robustheit und Langzeitstabilität und stellt damit eine sehr gute Grundlage für die Positionsbestimmung auf Rennstrecken dar. Fahrzeugsensoren, bestehend aus Raddrehzahl- und Gierratensensor, schreiben die Fahrzeugposition mit Hilfe der Koppelnavigationsgleichung relativ für ca. 10s ohne eine Messung absoluter Positionswerte fort. Um die bestehenden Genauigkeitsanforderungen zu erfüllen, muss jedoch ab einer Geschwindigkeit von 30km/h das Fahrzeugmodell um eine Schwimmwinkelschätzung erweitert werden. Ein digitaler Kompass eignet sich nachweislich nicht für die Positionsbestimmung auf Rennstrecken. Hier treten aufgrund von magnetischen Interferenzen zu große Messfehler der Fahrzeugausrichtung auf, die eine Positionsstützung ungeeignet machen. Bei Referenzmessungen mit einem Servo-Tachymeter konnte die geforderte Genauigkeit dieser Messeinrichtung bei Fahrzeuggeschwindigkeiten kleiner 30km/h nachgewiesen werden. Bei höheren Geschwindigkeiten liefert das System jedoch keine Ergebnisse, was den Einsatz auf Rennstrecken ausschließt. Auf den Boden gerichtete Laser-Reflexlichtschranken können sehr präzise die Überfahrt über eine Bodenmarkierung detektieren. Da diese Überfahrten beim autonomen Fahren auf Rennstrecken nur sehr selten auftreten, ist diese Positionierungsmethode nicht geeignet. Mit Hilfe einer LIDAR-basierten Randbebauungserkennung kann die Fahrzeugposition in Kombination mit einer hochgenauen digitalen Karte der Randbebauung auf ca. 20-30cm genau geschätzt werden. Schwierigkeiten bereiten hier jedoch Unregelmäßigkeiten in der Geometrie der Randbebauung. Während parallel verlaufende Leitplanken neben der Strecke sehr gut erfasst werden können, liefern Sträucher, Erdwälle, etc. ungenaue Messergebnisse. Somit ist die LIDAR-basierte Randbebauungserkennung ein bedingt geeignetes System zur Positionsstützung auf Rennstrecken. Als vielversprechendster Ansatz zur Verbesserung der Positions- und Ausrichtungsbestimmung auf Rennstrecken konnte der Einsatz einer visuellen Spurerkennung in Verbindung mit einer hochgenauen digitalen Karte der Spurmarkierungen identifiziert werden. Hierfür wurde eine sich in Vorserie befindliche Bildverarbeitungseinheit der Firma MobileEye mit einer eigens entwi-ckelten Spurerkennung verglichen. Letztere bietet den Vorteil, Systemwissen über den Verlauf der Fahrspurmarkierung sowie negative Effekte der Fahrzeugeigendynamik mit in den Signalver-arbeitungsprozess einfließen zu lassen. Bei Vergleichsfahrten auf dem BMW eigenem Testgelände in Aschheim konnte der Vorteil der Spurdatenrückführung nachgewiesen werden. Die erwei-terte Spurerkennung hatte nachweislich gegenüber der Vorserienbildverarbeitung eine höhere Verfügbarkeit von gültigen Messwerten. Bei Messfahrten auf der Nordschleife stellte sich jedoch das Vorseriensystem von MobileEye als das deutlich robustere Spurerkennungssystem heraus. Hier führten verschmutzte Fahrbahnmarkierungen, schnell wechselnde Lichtverhältnisse sowie sonstige Straßenbeschriftungen dazu, dass die erweiterte Spurerkennung weitaus weniger gültige Messwerte lieferte als das Vorseriensystem. Aus diesem Grund fiel für Fahrten mit schlechten visuellen Bedingungen die Wahl auf das Vorserienbildverarbeitungssystem. Für den Entwurf des Positionsfilters wurden letztlich folgende Signalquellen verwendet: - GPS-gestützte Inertialplattform (GPS/INS) - Fahrzeugsensoren mit erweitertem Fahrzeugmodell - Videobasierte Spurerkennung in Kombination mit einer selbst aufgezeichneten hochge-nauen Karte der Spurmarkierungen der Teststrecke. Als Fusionsalgorithmus wurde ein erweiterter Kalman-Filter eingesetzt, da sich dieser besonders für die Zusammenführung unterschiedlicher Sensormessdaten eignet. Um eine optimale Zustandsschätzung der Fahrzeugposition und Ausrichtung zu erhalten, mussten die verwendeten Signalquellen zunächst zeitlich synchronisiert sowie auf Plausibilität geprüft werden. Als Synchronisationspunkt wurde der Messzeitpunkt der Signalquelle mit der größten Latenz verwendet. Dieser wurde mit 163ms durch für die videobasierte Spurerkennung bestimmt. Da jedoch eine verzögerte Positionsschätzung für eine stabile Reglung des Fahrzeugs für das autonome Fahren ungenügend ist, wurde die geschätzte Fahrzeugposition am Ausgang des Kalman-Filters mit Hilfe der Koppelnavigationsgleichung sowie der Fahrzeugsensoren auf den aktuellen Zeitpunkt (Latenz = 0s) prädiziert. Für die Detektion systematischer Fehler wie Radschlupf, falsch erkannte Spurmarkierung und GPS-Mehrwegeausbreitung kamen robuste Signalplausibilisierungsalgorithmen zum Einsatz. So erfolgte die Plausibilisierung der Spurerkennung unter anderem über die selbst aufgezeichnete hochgenaue Karte der Spurmarkierungen, da eine Spurerkennung nur da sinnvoll ist, wo Spurmarkierungsstützpunkte in hinterlegt sind. Für die Gültigkeitsüberprüfung der GPS-Messwerte wurde ein GPS-Offset-Beobachter entwickelt und angewendet. Die Evaluierung des entwickelten Positionsfilters wurde im Rahmen der Arbeit am Beispiel des BMW TrackTrainers auf drei ausgewählten Teststrecken mit steigendem Schwierigkeitsniveau (Verschlechterung der GPS-Empfangsbedingungen) durchgeführt. Hierfür wurde die in Echtzeit geschätzte Fahrzeugposition mit einer durch Post-Processing korrigierten Positionslösung referenziert. Die Auswertung der Ergebnisse bewies, dass der entwickelte Positionsfilter durch die Fusion einer GPS-gestützten Inertialplattform, den Fahrzeugsensoren zur Messung von Gierrate und Raddrehzahlen sowie einer visuellen Spurerkennung in Kombination mit einer hochgenauen Karte der Fahrspurmarkierungen die Anforderungen des autonomen Fahrens auch unter schlechten GPS-Empfangsbedingungen erfüllt. Mit diesem, im Rahmen der Arbeit entwickelten, hoch-genauen Positionsbestimmungssystem konnte erstmalig am 21.10.2009 das autonome Fahren auf der Nürburgring Nordschleife nachgewiesen werden.

Le véhicule autonome est un véhicule qui se conduira, à terme, sans aucune intervention du conducteur, quelle que soit la situation de conduite. Ce véhicule comprend une nouvelle fonction, nommée fonction AD, pour Autonomous Driving, en charge de la conduite autonome. Cette fonction peut se trouver dans des états différents (Active, Disponible par exemple) selon l'évolution des conditions environnementales. Le changement de ses états est géré par une fonction de Supervision, nommée Supervision AD. Le principal objet de ces travaux consiste à garantir que la fonction AD se trouve constamment dans un état sûr. Ceci revient à s'assurer que la Supervision AD respecte l'ensemble des exigences fonctionnelles et de sûreté qui spécifient son comportement. Ces deux types d'exigences sont émis par deux métiers distincts : l'Architecte Métier Système (AMS) et le pilote Sûreté de Fonctionnement (SdF). Ces deux disciplines d'ingénierie, bien qu'elles contribuent à la conception d'une même fonction, se distinguent en de nombreux points : objectifs, contraintes, planning, outils... Dans notre cas d'étude, ces différences s'illustrent par les exigences considérées : les exigences fonctionnelles sont allouées à la fonction AD globale, tandis que les exigences de sûreté spécifient le comportement de sous-fonctions locales redondantes assurant une continuité de service en cas de défaillance. La mise en cohérence de ces deux perspectives métier au plus tôt dans le cycle de conception et dans un contexte industriel, est la problématique centrale traitée. Les enjeux de SdF soulevés par le véhicule autonome rendent ce problème primordial pour les constructeurs automobiles. Afin de répondre à ces préoccupations, nous avons proposé une démarche outillée et collaborative de conception sûre de la Supervision AD. Cette démarche est intégrée dans les processus normatifs en vigueur (normes ISO 15288 et ISO 26262) ainsi que dans les processus de conception internes chez Renault. Elle est fondée sur la vérification formelle par model checking, la composition parallèle d'automates finis et l'expertise métier. Cette démarche prône l'utilisation d'un même formalisme (l'automate à états finis) par les deux métiers pour mener à bien des activités partageant un objectif de modélisation commun : la vérification d'exigences de comportement en phase amont de conception. Une méthode pour traduire les exigences en propriétés formelles et construire les modèles d'état a été déployée. Il en résulte une consolidation progressive des exigences traitées, initialement rédigées en langage naturel. Les potentielles ambigüités, incohérences et incomplétudes sont exhibées et traitées
The Autonomous Vehicle is meant to drive itself, without any driver intervention, whatever the driving situation. This vehicle includes a new function, called AD, for Autonomous Driving, function. This function can be in different states (Available, Active for example) according to environmental conditions evolution. This states change is managed by a supervision function, named AD Supervision. The main goal of my works consists in guaranteeing that AD function remains always in a safe state. In other words, the AD Supervision must always respect all the functional and safety requirements that specify its behavior. These two requirements types are produced by two different professions: the System Architect (SA) and the Safety Engineer (SE). These two fields contribute to the design of the same function but distinguish at several aspects: objectives, constraints, planning, tools… In our case study, these differences are illustrated by considered requirements: the functional requirements are allocated to global AD function, while the safety requirements specify the behavior of local redundant sub-functions ensuring a continuous service in case of failure. The consistency of the two perspectives as early as possible in the design phase and in an industrial context, is the central problematic addressed. The safety issues due to Autonomous Vehicle make this topic essential for the automotive manufacturers. To meet these concerns, we proposed a tooled and collaborative approach for safe design of AD Supervision. This approach is integrated in the normative processes (standards ISO 26262 and ISO 15288) as well as in the internal design processes at Renault. It is based on formal verification by model checking, parallel composition of finite sate automata and technical expertise. This approach advocates the utilization of a same formalism (state automata) by the two professions to perform activities sharing a common goal: behavior requirements verification in preliminary design phase. A method to translate requirements into formal properties and to build state models has been deployed. The result is a progressive consolidation of treated requirements, initially expressed in free natural language. The potential ambiguities, inconsistencies and incompleteness are exhibited and treated. Two main contributions are in this way illustrated: highlighting of several formal credible (i.e. validated by expertise) specifications from informal requirements; and precise definition of technical expertise role (milestones, planning). However, this reinforcement – in silos – of the two profession viewpoints does not guarantee that they are mutually consistent. Thus, we proposed a convergence method, relying on expertise and on parallel composition of state automata, for the comparison of local and global views

The master's thesis deals with the design and realization of embedded control system and software of the autonomous mobile robot DACEP. The research section focuses on the selection of sensory equipment. Moreover, the design of the embedded control system and the communication interface between this system and the master PC is described in detail, followed by the design of localization and navigation software that uses ROS framework. The section is written as instructive as possible for the development of robots of similar construction. Finally the development of a graphical interface for robot diagnostics and remote control is depicted.

In 2020, the world suffered from a very severe new pandemic, Covid-19. All industries have been very badly affected. Many people have been forced to isolate themselves at home, working remotely. The streets are sparsely populated, but one benefit is that traffic is not as heavy as it used to be. This has facilitated the testing and trial run of autonomous driving technology in China. Self-driving buses and taxis have been in trial operation in several cities, including Shenzhen, Suzhou and Wuhan, where citizens can ride for free by booking online through a mobile APP. At the same time, unmanned delivery vehicles and sweeper vehicles have also been used in different areas. This paper introduces the basic knowledge of autonomous driving vehicles and the status quo of R&D and testing, analyzes the advantages and disadvantages of its development in China, and gives an example of the latest test results of autonomous driving. Autonomous vehicles will dominate the automotive industry for many years to come, bringing new travel experiences to consumers.

Highly automated driving systems promise increased road traffic safety, as well as positive impacts on sustainable transportation by means of increased traffic efficiency and environmental friendliness. The design and development of such systems require scientific advances in a number of areas. One area is the vehicle's electrical/electronic (E/E) architecture. The E/E architecture can be presented using a number of views, of which an important one is the functional view. The functional view describes the decomposition of the system into its main logical components, along with the hierarchical structure, the component inter-connections, and requirements. When this view captures the principal ideas and patterns that constitute the foundation of a variety of specific architectures, it may be termed as a reference architecture. Two reference architectures for highly automated driving form the principal contribution of this thesis. The first reference architecture is for cooperative driving. In a cooperative driving situation, vehicles and road infrastructure in the vicinity of a vehicle continuously exchange wireless information and this information is then used to control the motion of the vehicle. The second reference architecture is for autonomous driving, wherein the vehicle is capable of driver-less operation even without direct communication with external entities. The description of both reference architectures includes their main components and the rationale for how these components should be distributed across the architecture and its layers. These architectures have been validated via multiple real-world instantiations, and the guidelines for instantiation also form part of the architecture description. A comparison with similar architectures is also provided, in order to highlight the similarities and differences. The comparisons show that in the context of automated driving, the explicit recognition of components for semantic understanding, world modeling, and vehicle platform abstraction are unique to the proposed architecture. These components are not unusual in architectures within the Artificial Intelligence/robotics domains; the proposed architecture shows how they can be applied within the automotive domain. A secondary contribution of this thesis is a description of a lightweight, four step approach for model based systems engineering of highly automated driving systems, along with supporting model classes. The model classes cover the concept of operations, logical architecture, application software components, and the implementation platforms. The thesis also provides an overview of current implementation technologies for cognitive driving intelligence and vehicle platform control, and recommends a specific setup for development and accelerated testing of highly automated driving systems, that includes model- and hardware-in-the-loop techniques in conjunction with a publish/subscribe bus. Beyond the more "traditional" engineering concepts, the thesis also investigates the domain of machine consciousness and computational self-awareness. The exploration indicates that current engineering methods are likely to hit a complexity ceiling, breaking through which may require advances in how safety-critical systems can self-organize, construct, and evaluate internal models to reflect their perception of the world. Finally, the thesis also presents a functional architecture for the brake system of an autonomous truck. This architecture proposes a reconfiguration of the existing brake systems of the truck in a way that provides dynamic, diversified redundancy, and an increase in the system reliability and availability, while meeting safety requirements.

QC 20151216

How can a non-autonomous compaction roller be converted to become temporarily autonomous while it performs a 500hours endurance test? Particularlysince the compaction rollers in question is not built to be autonomous and shall not be autonomous after the endurance test is completed. The autonomous system shall also be adaptable to all compaction rollers which Dynapac is developing and shall be moved to another machine when the endurance test is completed. In this thesis a concept is engineered of how the whole autonomous system will work and a prototype is fabricated of how to convert the current manual mechanical steering to be performed by a computer. The steering prototype has been tested on a Dynapac CC4200 double drumasphalt compaction roller and worked as intended. To develop this, anextensive risk analysis is also established andwith it a requirements list of what's needed to be fulfilled when performing autonomous testing of a compaction roller. The work has been done using the method “design thinking” which is a collection of multiple methods to create new concepts and ideas. The final concept resulted in a navigation system which uses GNSS for path planning and limitation of the operation area. It also uses radar to detect foreign objects in its path to prevent a collision. Multiple systems arealso proposed to be used for malfunction detection of the roller, which is a major part of a human operator’sjob when testing out new machines. The test track for the machine was undefined and also hadto be engineeredas part of the concept. It resultedin closing the area of operation with a mesh fence to prevent access to the area from unauthorised personnel and geo-fence to prevent the machine from escaping. Access to the area is only granted to authorized personnel and only when the autonomous rolleris shut off. Due to the machines in question isn’t fully developed, theycan’t be trusted enough to have people inside the area of operation asthe autonomous machineis operating.
Hur kanen icke-autonom vägvältomvandlas tillatt bli tillfälligt autonom medan den utför ett 500timmar långttidsprov?Särskilt sedanvägvältenifrågainte ärbyggd för att vara autonom och ska intevara autonom efter attlångtidsprovetär slutfört. Det autonoma systemet skaävenkunna anpassas tillalla vältar som Dynapac utvecklar och ska flyttas till en annan maskin närlångtidsprovetär klart. Idenna avhandling konstrueras ett koncept för hur hela det autonoma systemet kommer att fungera ochenprototyp tillverkaspå hur man konverterar den nuvarande manuella mekaniska styrningen till attstyras av en dator. Styrprototypen testades på en Dynapac CC4200 asfaltsvält med dubbla valsar ochfungerade bra. En omfattande riskanalys utvecklades ochlika såen kravlista över vad som behöveruppnås vid autonom testning av en vägvält. Arbetet har gjorts med hjälp av metoden “designthinking”, vilket är en samling av flera metoder för att skapa nya koncept och idéer. Det slutgiltigakonceptet resulterade i ett navigationssystem som använder GNSS för navigering och begränsning avkörområdet. Den använder också radar för att upptäcka främmande föremål i sin vägvilketförhindrarkollision. Flera system föreslås användasförfunktionsfelsdetektering på välten, vilket är en viktig delav en mänskligoperatörs arbetevid provning av nya maskiner. Maskinen kommer att vara i ett slutetområde som är avskilt med ett nätstängsel.Tillträde till området ges endast till behörig personal ochendast när den autonoma välten är avstängd. På grund avmaskinerna ifråga inte är fullt utvecklade,kan de inte litas på tillräckligt för att ha personer inom körområdet medan det autonoma systemet är idrift.