Simultaneous Localisation and Mapping (SLAM) began as a technique to enable real-time robotic navigation on previously unexplored environments. The created maps however were designed for the sole purpose of localising the robot (i.e. what is the position and orientation of the robot in relation to the map) and several systems demonstrated the increasing descriptive power of map representations, which on vision-only SLAM solutions consisted of simple sparse corner-like features as well as edges, planes and most recently fully dense surfaces that abandon the notion of sparse structures altogether. Early sparse representations enjoyed the benefit of being simple to maintain as features could be added, optimised and removed independently while being memory and compute efficient, making them suitable for robust real-time camera tracking that relies on a consistent map. However, sparse representations are limiting when it comes to interaction, as for example, a robot aiming to safely navigate in an environment would need to sense complete surfaces in addition to empty space. Furthermore, sparse features can only be detected on highly-textured areas and during slow motion. Recent dense methods overcome the limitations of sparse methods as they can work in situations where corner features would fail to be detected due to blurry images created during rapid camera motion and also enable to correctly reason about occlusions and complete 3D surfaces, thus raising the interaction capabilities to new levels. This is only possible thanks to the advent of commodity parallel processing power and large amount of memory on Graphic Processing Units (GPUs) that needs careful consideration during algorithm design. However, increasing the map density makes creating consistent structures more challenging due to the vast amount of parameters to optimise and the interdependencies amongst them. More importantly, our interest is in making interaction even more sophisticated by abandoning the idea that an environment is a dense monolithic structure in favour of one composed of discrete detachable objects and bounded regions having physical properties and metadata. This work explores the development of a new type of visual SLAM system representing the map with semantically meaningful objects and planar regions which we call Dense Semantic SLAM, enabling new types of interaction where applications that can go beyond asking the question of "where am I" towards "what is around me and what can I do with it". In a way it can be seen as a return to lightweight sparse-based representations while keeping the predictive power of dense methods with added scene understanding at the object and region levels.

Thesis: Nav. E., Massachusetts Institute of Technology, Department of Mechanical Engineering, May, 2020
Thesis: S.M., Massachusetts Institute of Technology, Department of Mechanical Engineering, May, 2020
Cataloged from the official PDF of thesis.
Includes bibliographical references (pages 75-78).
In a step towards the level of autonomy seen in humans, this work attempts to emulate a high level and low level approach to world representation and short term adaptation. Specifically, this work demonstrates an implementation of robotic perception that transforms stereo camera and LIDAR sensor data into a sparse map of semantic objects and a locally consistent flexible occupancy grid. This provides a topological representation for grouping objects into higher level classes and a geometric map for traditional planning. Additionally, a reactive dynamic window obstacle avoidance system is shown to quickly plan short term trajectories that avoid both static and dynamic objects while progressing towards a goal. To combine computational efficiency with the robust advantages of multimodal inference, this work uses Semantic Max Mixture factors to approximate multimodal belief in a manner compatible to nonlinear least squares solvers. Experimental results are presented using a RACECAR mobile robot operating in several hallways of MIT, using AprilTags as surrogates for objects in the Semantic Max Mixtures Algorithm. Future work will seek to further integrate the components to create a closed-loop active semantic navigation and mapping algorithm.
by David P. Baxter.
Nav. E.
S.M.
Nav.E. Massachusetts Institute of Technology, Department of Mechanical Engineering
S.M. Massachusetts Institute of Technology, Department of Mechanical Engineering

Pour agir de manière autonome et intelligente dans un environnement, un robot mobile doit disposer de cartes. Une carte contient les informations spatiales sur l’environnement. La géométrie 3D ainsi connue par le robot est utilisée non seulement pour éviter la collision avec des obstacles, mais aussi pour se localiser et pour planifier des déplacements. Les robots de prochaine génération ont besoin de davantage de capacités que de simples cartographies et d’une localisation pour coexister avec nous. La quintessence du robot humanoïde de service devra disposer de la capacité de voir comme les humains, de reconnaître, classer, interpréter la scène et exécuter les tâches de manière quasi-anthropomorphique. Par conséquent, augmenter les caractéristiques des cartes du robot à l’aide d’attributs sémiologiques à la façon des humains, afin de préciser les types de pièces, d’objets et leur aménagement spatial, est considéré comme un plus pour la robotique d’industrie et de services à venir. Une carte sémantique enrichit une carte générale avec les informations sur les entités, les fonctionnalités ou les événements qui sont situés dans l’espace. Quelques approches ont été proposées pour résoudre le problème de la cartographie sémantique en exploitant des scanners lasers ou des capteurs de temps de vol RGB-D, mais ce sujet est encore dans sa phase naissante. Dans cette thèse, une tentative de reconstruction sémantisée d’environnement d’intérieur en utilisant une caméra temps de vol qui ne délivre que des informations de profondeur est proposée. Les caméras temps de vol ont modifié le domaine de l’imagerie tridimensionnelle discrète. Elles ont dépassé les scanners traditionnels en termes de rapidité d’acquisition des données, de simplicité fonctionnement et de prix. Ces capteurs de profondeur sont destinés à occuper plus d’importance dans les futures applications robotiques. Après un bref aperçu des approches les plus récentes pour résoudre le sujet de la cartographie sémantique, en particulier en environnement intérieur. Ensuite, la calibration de la caméra a été étudiée ainsi que la nature de ses bruits. La suppression du bruit dans les données issues du capteur est menée. L’acquisition d’une collection d’images de points 3D en environnement intérieur a été réalisée. La séquence d’images ainsi acquise a alimenté un algorithme de SLAM pour reconstruire l’environnement visité. La performance du système SLAM est évaluée à partir des poses estimées en utilisant une nouvelle métrique qui est basée sur la prise en compte du contexte. L’extraction des surfaces planes est réalisée sur la carte reconstruite à partir des nuages de points en utilisant la transformation de Hough. Une interprétation sémantique de l’environnement reconstruit est réalisée. L’annotation de la scène avec informations sémantiques se déroule sur deux niveaux : l’un effectue la détection de grandes surfaces planes et procède ensuite en les classant en tant que porte, mur ou plafond; l’autre niveau de sémantisation opère au niveau des objets et traite de la reconnaissance des objets dans une scène donnée. A partir de l’élaboration d’une signature de forme invariante à la pose et en passant par une phase d’apprentissage exploitant cette signature, une interprétation de la scène contenant des objets connus et inconnus, en présence ou non d’occultations, est obtenue. Les jeux de données ont été mis à la disposition du public de la recherche universitaire
Intelligent autonomous actions in an ordinary environment by a mobile robot require maps. A map holds the spatial information about the environment and gives the 3D geometry of the surrounding of the robot to not only avoid collision with complex obstacles, but also selflocalization and for task planning. However, in the future, service and personal robots will prevail and need arises for the robot to interact with the environment in addition to localize and navigate. This interaction demands the next generation robots to understand, interpret its environment and perform tasks in human-centric form. A simple map of the environment is far from being sufficient for the robots to co-exist and assist humans in the future. Human beings effortlessly make map and interact with environment, and it is trivial task for them. However, for robots these frivolous tasks are complex conundrums. Layering the semantic information on regular geometric maps is the leap that helps an ordinary mobile robot to be a more intelligent autonomous system. A semantic map augments a general map with the information about entities, i.e., objects, functionalities, or events, that are located in the space. The inclusion of semantics in the map enhances the robot’s spatial knowledge representation and improves its performance in managing complex tasks and human interaction. Many approaches have been proposed to address the semantic SLAM problem with laser scanners and RGB-D time-of-flight sensors, but it is still in its nascent phase. In this thesis, an endeavour to solve semantic SLAM using one of the time-of-flight sensors which gives only depth information is proposed. Time-of-flight cameras have dramatically changed the field of range imaging, and surpassed the traditional scanners in terms of rapid acquisition of data, simplicity and price. And it is believed that these depth sensors will be ubiquitous in future robotic applications. In this thesis, an endeavour to solve semantic SLAM using one of the time-of-flight sensors which gives only depth information is proposed. Starting with a brief motivation in the first chapter for semantic stance in normal maps, the state-of-the-art methods are discussed in the second chapter. Before using the camera for data acquisition, the noise characteristics of it has been studied meticulously, and properly calibrated. The novel noise filtering algorithm developed in the process, helps to get clean data for better scan matching and SLAM. The quality of the SLAM process is evaluated using a context-based similarity score metric, which has been specifically designed for the type of acquisition parameters and the data which have been used. Abstracting semantic layer on the reconstructed point cloud from SLAM has been done in two stages. In large-scale higher-level semantic interpretation, the prominent surfaces in the indoor environment are extracted and recognized, they include surfaces like walls, door, ceiling, clutter. However, in indoor single scene object-level semantic interpretation, a single 2.5D scene from the camera is parsed and the objects, surfaces are recognized. The object recognition is achieved using a novel shape signature based on probability distribution of 3D keypoints that are most stable and repeatable. The classification of prominent surfaces and single scene semantic interpretation is done using supervised machine learning and deep learning systems. To this end, the object dataset and SLAM data are also made publicly available for academic research

Con questa dissertazione di tesi miro ad illustrare i risultati della mia ricerca nel campo del Semantic Publishing, consistenti nello sviluppo di un insieme di metodologie, strumenti e prototipi, uniti allo studio di un caso d‟uso concreto, finalizzati all‟applicazione ed alla focalizzazione di Lenti Semantiche (Semantic Lenses).

In the future, robots will expand from industrial and research applications to the home. Domestic service robots will work in the home to perform useful tasks such as object retrieval, cleaning, organization, and security. The tireless support of these systems will not only enable able bodied people to avoid mundane chores; they will also enable the elderly to remain independent from institutional care by providing service, safety, and companionship. Robots will need to understand the relationship between objects and their environments to perform some of these tasks. Structured indoor environments are organized according to architectural guidelines and convenience for their residents. Utilizing this information makes it possible to predict the location of objects. Conversely, one can also predict the function of a room from the detection of a few objects within a given space. This thesis introduces a framework for combining object permanence and context called the probabilistic cognitive model. This framework combines reasoning about spatial extent of places and the identity of objects and their relationships to one another and to the locations where they appear. This type of reasoning takes into account the context in which objects appear to determine their identity and purpose. The probabilistic cognitive model combines a mapping system called OmniMapper with a conditional random field probabilistic model for context representation. The conditional random field models the dependencies between location and identity in a real-world domestic environment. This model is used by mobile robot systems to predict the effects of their actions during autonomous object search tasks in unknown environments.

Although much progress has been made in the field of robotic mapping, many challenges remain including: efficient semantic segmentation using RGB-D sensors, map representations that include complex features (structures and objects), and interfaces for interactive annotation of maps. This thesis addresses how prior knowledge of semi-structured human environments can be leveraged to improve segmentation, mapping, and semantic annotation of maps. We present an organized connected component approach for segmenting RGB-D data into planes and clusters. These segments serve as input to our mapping approach that utilizes them as planar landmarks and object landmarks for Simultaneous Localization and Mapping (SLAM), providing necessary information for service robot tasks and improving data association and loop closure. These features are meaningful to humans, enabling annotation of mapped features to establish common ground and simplifying tasking. A modular, open-source software framework, the OmniMapper, is also presented that allows a number of different sensors and features to be combined to generate a combined map representation, and enabling easy addition of new feature types.

La fusion entre un ensemble de capteurs et de bases de données dont les erreurs sont indépendantes est aujourd’hui la solution la plus fiable et donc la plus répandue de l’état de l’art au problème de la localisation. Les véhicules semi-autonomes et autonomes actuels, ainsi que les applications de réalité augmentée visant les contextes industriels exploitent des graphes de capteurs et de bases de données de tailles considérables, dont la conception, la calibration et la synchronisation n’est, en plus d’être onéreuse, pas triviale. Il est donc important afin de pouvoir démocratiser ces technologies, d’explorer la possibilité de l’exploitation de capteurs et bases de données bas-coûts et aisément accessibles. Cependant, ces sources d’information sont naturellement plus incertaines, et plusieurs obstacles subsistent à leur utilisation efficace en pratique. De plus, les succès récents mais fulgurants des réseaux profonds dans des tâches variées laissent penser que ces méthodes peuvent représenter une alternative peu coûteuse et efficace à certains modules des systèmes de SLAM actuels. Dans cette thèse, nous nous penchons sur la localisation à grande échelle d’un véhicule dans un repère géoréférencé à partir d’un système bas-coût. Celui-ci repose sur la fusion entre le flux vidéo d’une caméra monoculaire, des modèles 3d non-texturés mais géoréférencés de bâtiments,des modèles d’élévation de terrain et des données en provenance soit d’un GPS bas-coût soit de l’odométrie du véhicule. Nos travaux sont consacrés à la résolution de deux problèmes. Le premier survient lors de la fusion par terme barrière entre le VSLAM et l’information de positionnement fournie par un GPS bas-coût. Cette méthode de fusion est à notre connaissance la plus robuste face aux incertitudes du GPS, mais est plus exigeante en matière de ressources que la fusion via des fonctions de coût linéaires. Nous proposons une optimisation algorithmique de cette méthode reposant sur la définition d’un terme barrière particulier. Le deuxième problème est le problème d’associations entre les primitives représentant la géométrie de la scène(e.g. points 3d) et les modèles 3d des bâtiments. Les travaux précédents se basent sur des critères géométriques simples et sont donc très sensibles aux occultations en milieu urbain. Nous exploitons des réseaux convolutionnels profonds afin d’identifier et d’associer les éléments de la carte correspondants aux façades des bâtiments aux modèles 3d. Bien que nos contributions soient en grande partie indépendantes du système de SLAM sous-jacent, nos expériences sont basées sur l’ajustement de faisceaux contraint basé images-clefs. Les solutions que nous proposons sont évaluées sur des séquences de synthèse ainsi que sur des séquence urbaines réelles sur des distances de plusieurs kilomètres. Ces expériences démontrent des gains importants en performance pour la fusion VSLAM/GPS, et une amélioration considérable de la robustesse aux occultations dans la définition des contraintes
The fusion between sensors and databases whose errors are independant is the most re-liable and therefore most widespread solution to the localization problem. Current autonomousand semi-autonomous vehicles, as well as augmented reality applications targeting industrialcontexts exploit large sensor and database graphs that are difficult and expensive to synchro-nize and calibrate. Thus, the democratization of these technologies requires the exploration ofthe possiblity of exploiting low-cost and easily accessible sensors and databases. These infor-mation sources are naturally tainted by higher uncertainty levels, and many obstacles to theireffective and efficient practical usage persist. Moreover, the recent but dazzling successes ofdeep neural networks in various tasks seem to indicate that they could be a viable and low-costalternative to some components of current SLAM systems.In this thesis, we focused on large-scale localization of a vehicle in a georeferenced co-ordinate frame from a low-cost system, which is based on the fusion between a monocularvideo stream, 3d non-textured but georeferenced building models, terrain elevation models anddata either from a low-cost GPS or from vehicle odometry. Our work targets the resolutionof two problems. The first one is related to the fusion via barrier term optimization of VS-LAM and positioning measurements provided by a low-cost GPS. This method is, to the bestof our knowledge, the most robust against GPS uncertainties, but it is more demanding in termsof computational resources. We propose an algorithmic optimization of that approach basedon the definition of a novel barrier term. The second problem is the data association problembetween the primitives that represent the geometry of the scene (e.g. 3d points) and the 3d buil-ding models. Previous works in that area use simple geometric criteria and are therefore verysensitive to occlusions in urban environments. We exploit deep convolutional neural networksin order to identify and associate elements from the map that correspond to 3d building mo-del façades. Although our contributions are for the most part independant from the underlyingSLAM system, we based our experiments on constrained key-frame based bundle adjustment.The solutions that we propose are evaluated on synthetic sequences as well as on real urbandatasets. These experiments show important performance gains for VSLAM/GPS fusion, andconsiderable improvements in the robustness of building constraints to occlusions

La navigation autonome est l'un des plus grands challenges pour un robot autonome. Elle nécessite la capacité à localiser sa position ou celle de l'objectif et à trouver le meilleur chemin connectant les deux en évitant les obstacles. Pour cela, les robots utilisent une carte de l'environnement modélisant sa géométrie ou sa topologie. Cependant la construction d'une telle carte dans des environnements de grande dimension est ardue du fait de la quantité de données à traiter et le problème de la localisation peut devenir insoluble. De plus, un environnement changeant peut conduire à l'obsolescence rapide du modèle. Comme démontré dans cette thèse, l'ajout d'information de nature sémantique dans ces cartes améliore significativement les performances de navigation des robots dans des environnements réels. La labélisation d'image permet de construire des modèles extrêmement compacts qui sont utilisés pour la localisation rapide en utilisant une approche basée comparaison de graphes. Ils sont des outils puissants pour comprendre l'environnement et permettent d'étendre la carte au-delà des limites perceptuelles du robot. L'analyse statistique de ces modèles est utilisée pour construire un embryon de sens commun qui est ensuite utilisé pour détecter des erreurs de labélisation et pour mettre à jour la carte en utilisant des algorithmes conçus pour maintenir une représentation stable en dépits des occlusions créées par les objets dynamiques. Finalement, la sémantique est utilisées pour sélectionner le meilleur chemin vers une position cible en fonction de critères de haut niveau plutôt que métriques, autorisant une navigation intelligente
Utonomous navigation is one of the most challenging tasks for mobile robots. It requires the ability to localize itself or a target and to find the best path linking both positions avoiding obstacles. Towards this goal, robots build a map of the environment that models its geometry or topology. However building such a map in large scale environments is challenging due to the large amount of data to manage and localization could become intractable. Additionally, an ever changing environment leads to fast obsolescence of the map that becomes useless. As shown in this thesis, introducing semantics in those maps dramatically improves navigation performances of robots in realistic environments. Scene parsing allows to build extremely compact semantic models of the scene that are used for fast relocalization using a graph-matching approach. They are powerful tools to understand scene and they are used to extend the map beyond perceptual limits of the robot through reasoning. Statistical analysis of those models is used to build an embryo of common sens which allows to detect labeling errors and to update the map using algorithms designed to maintain a stable model of the world despite occlusions due to dynamic objects. Finally semantics is used to select the best route to a target position according to high level criteria instead of metrical constraints, allowing intelligent navigation

Le Cloud Computing est un nouveau modèle économique hébergeant les applications de la technologie de l’information. Le passage au Cloud devient un enjeu important des entreprises pour des raisons économiques. La nature dynamique et la complexité croissante des architectures de Cloud impliquent plusieurs défis de gestion. Dans ce travail, nous nous intéressons à la gestion des contrats SLA. Vu le manque de standardisation, chaque fournisseur de service décrit les contrats SLA avec son propre langage, ce qui laisse l'utilisateur perplexe concernant le choix de son fournisseur de services. Dans ce travail, nous proposons une architecture de Cloud Broker permettant d’établir et de négocier les contrats SLA entre les fournisseurs et les consommateurs du Cloud. L’objectif de cette architecture est d’aider l’utilisateur à trouver le meilleur fournisseur en utilisant une méthode multi-critère. Cette méthode considère chaque critère comme une fonction d’utilité à intégrer dans une super-fonction d’utilité. Nous proposons d’illustrer chaque fonction d’utilité par une courbe spécifique à lui représentant bien le critère de choix. Nous essayons de cerner la plupart des critères qui contribuent dans le choix du meilleurs service et de les classer en critères fonctionnels et critères non fonctionnels. Les contrats SLA établit par notre broker sont formalisés sous forme d’ontologies qui permettent de masquer l'hétérogénéité et d’assurer l'interopérabilité entre les acteurs du Cloud. En outre, l’utilisation des règles d'inférence nous a permis de détecter les violations dans le contrat SLA établit et de garantir ainsi le respect de la satisfaction client dans le temps
Cloud Computing is a dynamic new technology that has huge potentials in enterprises and markets. The dynamicity and the increasing complexity of Cloud architectures involve several management challenges. In this work, we are interested in Service Level Agreement (SLA) management. Actually, there is no standard to express Cloud SLA, so, providers describe their SLAs in different manner and different languages, which leaves the user puzzled about the choice of its Cloud provider. To overcome these problems, we introduce a Cloud Broker Architecture managing the SLA between providers and consumers. It aims to assist users in establishing and negotiating SLA contracts and to help them in finding the best provider that satisfies their service level expectations. Our broker SLA contracts are formalized as OWL ontologies as they allow hiding the heterogeneity in the distributed Cloud environment and enabling interoperability between Cloud actors. Besides, by combining our ontology with our proposed inference rules, we contribute to detect violations in the SLA contract assuring thereby the sustainability of the user satisfaction. Based on the requirements specified in the SLA contract, our Cloud Broker assists users in selecting the right provider using a multi attribute utility theory method. This method is based on utility functions representing the user satisfaction degree. To obtain accurate results, we have modelled both functional and non functional attributes utilities. We have used personalized utilities for each criterion under negotiation so that our cloud broker satisfies the best consumer requirements from functional and non functional point of view

Sub-Saharan Africa is a multilingual environment and there is a lack of materials available for speech and language assessment in this area (Hartley & Krämer, 2013). The norms for assessment material cannot be used for both monolinguals and bilinguals, since bilinguals may have different levels of knowledge in their languages (Kohnert, 2010). The Spoken Language Assessment Profile – Revised edition (SLAP-R) is an assessment that can be used to evaluate English as a second language (ESL) in Sub-Saharan Africa. The purpose of this instrument is an attempt to fill the gap of suitable speech and language assessment tools that can be used for all those involved in setting up clinics, schools or speech and language assessment tools (Hartley & Krämer, 2013). The aim of the present study was to assess English as a second language for Kenyan children in primary school based on their result on the SLAP-R. The present study consisted of 68 participants with reported typically developed language and hearing that attended first or second grade in a public school in western Kenya. All participants were between six and nine years old, had a Bantu language as their first language and had been exposed to English for less than one year up to eight years. They had also attended preschool at their current school. The independent variables in the present study were grade, age and exposure to English. SLAP-R consists of six subtests that test expressive and receptive phonology, semantics and grammar. These parts constituted the dependent variables. In addition there is a part called ultimate expressive language skill (UELS) that consists of picture sequences where the participant should tell a story of what is happening in the pictures. The result indicated that grade had the largest effect on the participant’s performance in English as a second language. Grade two had significantly higher results regarding receptive phonology as well as expressive and receptive semantics and grammar than the participants in grade one. Most of the incorrect answers were made in the subtest expressive grammar. These answers were mainly incorrect due to other reasons than an answer in Kiswahili.
Sub-Sahara Afrika är en flerspråkig miljö och det finns en brist på material för tal- och språkbedömningar inom detta område (Hartley & Krämer, 2013). Normerna för ett bedömningsinstrument kan inte användas för både enspråkiga och tvåspråkiga barn, eftersom tvåspråkiga barn kan ha varierande kunskapsnivåer inom språken (Kohnert, 2010). Spoken Language Assessment Profile – Revised edition (SLAP-R) är ett bedömningsmaterial som är avsett att utvärdera engelska som andraspråk i Sub-Sahara Afrika. Syftet med detta instrument är att försöka fylla tomrummet av lämpliga tal- och språkbedömningsmaterial som kan användas av samtliga inblandade vid att starta upp kliniker, skolor eller logopedmottagningar (Hartley & Krämer, 2013). Syftet med föreliggande studie var att undersöka engelska som andraspråk för Kenyanska barn i grundskolan baserat på deras resultat i SLAP-R. Föreliggande studie bestod av 68 deltagare med rapporterad typisk hörsel och språkutveckling som gick i klass ett eller två i en kommunal skola i västra Kenya. Alla deltagarna var mellan sex och nio år, hade ett bantuspråk som förstaspråk och hade exponerats till engelska i mindre än ett år upp till åtta år. De hade även gått i den förskolan som tillhörde deras nuvarande skola. De oberoende variablerna i föreliggande studie var klass, ålder och exponeringstid till engelska. SLAP-R består av sex deltest som testar expressiv och receptiv fonologi, semantik och grammatik. De här delarna utgör de beroende variablerna. Det finns ytterligare en del som kallas för ultimate expressive language skill (UELS) som består av sekvensbilder där deltagaren ska berätta en historia om vad som händer på bilderna. Resultatet indikerade att klass var variabeln som hade störst inverkan på deltagarnas prestationer i engelska som andraspråk. Klass två hade signifikant högre resultat gällande receptiv fonologi, såväl som expressiv och receptiv semantik och grammatik än deltagarna i klass ett. De flesta felsvaren gjordes i deltestet expressiv grammatik och var i huvudsak på grund av andra skäl än svar på kiswahili.

This essay reviews vocabulary samples from three different textbooks, which are readers for the basic course in English at an upper secondary school in Sweden. The aim of the study is to determine whether the word samples from the readers’ word lists consist mostly of high- or low frequency words and if the words denote any particular semantic fields. Moreover, the possible use of word frequencies in second language acquisition is also examined. The method used in ascertaining the quality of the words is comparing the word samples to the BNC (the British National Corpus) and analysing how frequently they occur in written and spoken modern English. The results are based on the findings from the analysis made in this study and also compared to current research in the fields of linguistics and language acquisition. The results exhibit both overrepresentation- and absence of words in particular semantic fields. For instance, words from the semantic field concerning ‘food and cooking’ were found to be somewhat predominant. The findings also include support for the use of word frequencies in language acquisition, especially in terms of how words are translated from English into Swedish in the textbooks’ wordlists. The only Swedish synonym given was in some cases item of the least frequent usage in modern English, according to the BNC.  

 

 

 

This thesis explores the problem of visual place recognition and localization for a mobile robot, particularly dealing with the challenges of simultaneous variations in scene appearance and camera viewpoint. The proposed methods draw inspiration from humans and make use of semantic cues to represent places. This approach enables effective place recognition from similar or opposing viewpoints, despite variations in scene appearance caused by different times of day or seasons. The research contributions presented in the thesis advance visual place recognition techniques, making them more useful for deployment in a wide range of robotic and autonomous vehicle scenarios.

In recent decades, the rapid development of intelligent vehicle and 3D scanning tecnologies has led to a growing interest in applications based on 3D point data processing, with many applications such as augmented reality or robot manipulation and obstacle avoidance, scene understanding, robot navigation, tracking and assistive technology among others, requiring an accurate solution for the 3D pose of the recognized objects. Thus object recognition is becoming an important topic in computer vision, where machine vision and robotics techniques are becoming key players. In this thesis work, the main objective is to develop a semantic mapping method by integrating a 3D object recognition pipeline with a feature-based SLAM system, in order to assist autonomous underwater interventions in the near future. To this end, the work proposed in this paper targets three axes. First, it aims to compare the performance of 3D global descriptors within the state of the art, focusing on those based on point clouds and targeted at real-time object recognition applications. For this purpose, we selected a set of test objects representative of Inspection, Maintenance and Repair (IMR) applications and whose shape is usually known a priori. Their CAD models were used to: 1) create a data base of synthetic object views used as a priori knowledge, and 2) simulate the point clouds that would be gathered during the scanning under realistic conditions, with added noise and varying resolution. Extensive experiments were performed with both virtual scans and real data collected with an AUV equipped with a fast laser scanner developed at our research centre. The second goal of our work was to use a real-time laser scanner mounted on an AUV to detect, identify, and locate objects in the robot’s environment, with the aim of allowing an intervention Autonomous Underwater Vehicle (I-AUV) to know what manipulation actions could be performed on each object. This goal was tackled by the design and development of a 3D object recognition method for uncolored point clouds (laser scans) using point features. The algorithm uses a database of partial views of the objects stored as point clouds. The recognition pipeline includes 5 stages: 1) Plane segmentation, 2) Pipe detection, 3) Semantic Object-segmentation, 4) Feature-based Object Recognition and 5) Bayesian estimation. To apply Bayesian estimation, it is necessary to track objects across scans. For this purpose, the Inter-distance Joint Compatibility Branch and Bound (IJCBB) data association algorithm was proposed based on the distances between objects. The performance of the method was tested using a dataset of the inspection of a pipe infrastructure made of PVC objects connected by pipes. The structure is representative of those commonly used by the offshore industry. Experimental results show that Bayesian estimation improves the recognition performance with respect to the case where only the descriptor is used. The inclusion of semantic information about object pipe connectivity further improves recognition performance. The final goal of the thesis, consists of integrating the 3D object recognition system with a feature-based SLAM system to implement a semantic map providing the robot with information about the location and the type of objects in its surroundings. The SLAM improved both the accuracy and reliability of pose estimates of the robot and the objects. This is especially important in challenging scenarios where significant changes in viewpoint and appearance arise
A les darreres dècades, el ràpid desenvolupament de vehicles intel·ligents i de les tecnologies d’escaneig 3D han contribuït a augmentar l’interès en les aplicacions basades en processament de núvols de punts 3D, amb aplicacions com la realitat augmentada, la manipulació robòtica, l’evasió d’obstacles, la comprensió d’escenes, la navegació robòtica, el seguiment d’objectes i la tecnologia d’assistència, etc., que requereixen una soluci´o precisa de la posició 3D i l’orientació d’un objecte. Per tant, el reconeixement d’objectes s’està convertint en un tema, on la visió per computador i les tècniques robòtiques esdevenen protagonistes clau. En aquest treball de tesi, l’objectiu principal és desenvolupar un mètode per a la construcció de mapes semàntic mitjançant la integració d’una cadena de processament per al reconeixement d’objectes 3D, amb un sistema de SLAM basat en característiques, amb l’objectiu d’ajudar a les futures intervencions submarines. Per això, el treball proposat en aquesta tesi es divideix en tres eixos principals. El primer té com a objectiu comparar el rendiment de descriptors globals d’última generació, centrant-se en els basats en núvols de punts 3D i destinats a aplicacions de reconeixement d’objectes en temps real. Per a aquest objectiu, s’ha seleccionat un conjunt d’objectes de prova representatius d’aplicacions d’inspecció, manteniment i reparació (IMR), la forma dels quals es coneix a priori. Els seus models CAD s’han utilitzat per a: 1) crear una base de dades amb les vistes sintètiques dels objectes, i 2) simular els núvols de punts que adquiriria, en condicions realistes, un escàner làser incloent soroll sintètic i simulant diferents resolucions. S’han dut a terme experiments tant a partir d’escaneigs virtuals com de dades reals recopilades amb un AUV equipat amb un escàner làser de temps real desenvolupat al nostre centre de recerca. El segon objectiu del nostre treball va consistir en utilitzar aquest escàner làser, muntat a un AUV per detectar, reconèixer i localitzar objectes a l’entorn del robot, per tal de permetre, a un Vehicle Submarí Autònoms d’Intervenció (IAUV), saber quines accions de manipulació podria fer amb cada objecte. Aquest objectiu es va abordar amb el disseny i el desenvolupament d’un mètode de reconeixement d’objectes 3D en núvols de punts incolors (escanejos làser) utilitzant descriptors dels punts 3D. L’algorisme utilitza una base de dades de vistes parcials dels objectes emmagatzemats en forma de núvols de punts. El procés de reconeixement consta de 5 passos: 1) Segmentació de plànols, 2) Detecció de canonades, 3) Segmentació semàntica d’objectes, 4) Reconeixement d’objectes a partir dels descriptors de punts 3D i 5) Estimació bayesiana. Per aplicar l’estimació bayesiana, cal ser capaços de fer un seguiment dels objectes en escanejos successius. Per fer-ho, s’ha proposat l’algorisme Inter-distance Joint-Compatibility Branch and Bound (IJCBB) d’associació de dades basada en les distancies entre objectes dins del núvol de punts. El rendiment del mètode es va avaluar fent servir dades experimentals relatives a la inspecció d’una infraestructura composta de canonades interconnectades per objectes de PVC. L’estructura ´es representativa de les comunament utilitzades per la indústria offshore. Els resultats experimentals mostren que l’estimació bayesiana millora el rendiment del reconeixement en comparació de l’ús ´únic del descriptor. La inclusió d’informació semàntica sobre la connectivitat d’objectes a canonades millora encara més el rendiment del reconeixement. L’objectiu final de la tesi va abordar la integració del sistema de reconeixement d’objectes 3D basat en descriptors amb un sistema de SLAM basat en característiques, per implementar un mapa semàntic que proporciona al robot informació sobre la ubicació i el tipus d’objectes a l’entorn. La utilització de tècniques de SLAM ha millorat la precisió i la fiabilitat de les estimacions de la postura del robot i els objectes. Això és especialment important en escenaris difícils on es produeixen canvis significatius de perspectiva i aparença
Programa de Doctorat en Tecnologia

Do adult L2 English bilingual speakers have difficulty with cognate words whose meanings are distinct across their two languages? This study explored the extent to which variations in meaning in cross-language cognates affect translation performance in a translation task by L2 English (L1 Spanish) speakers who learned English as adults. A prep-phase experiment was conducted to test native English-speakers’ predicted completions of the study’s stimuli sentences, in order to choose the optimal stimuli for the primary experiment. The method for the primary experiment of this study consisted of a web-based translation task of 120 sentences from Spanish to English, while controlling for polysemy and frequency. The results showed that adult L2 learners of English did experience difficulty when translating cognates in sentences from their L1 to their L2. The interaction of the Spanish word’s polysemous nature, Spanish word frequency, English target frequency and English cognate frequency played a role in the participants’ performance.

碩士
國立臺灣大學
資訊網路與多媒體研究所
107
Typically, simultaneous localization and mapping (SLAM) algorithms are assumed to be performed in the stationary environments only. However, there are ordinarily moving cars and people in the real world. Such a strict assumption restricts its usability especially on the robots or autonomous vehicles. In this work, we present a semantic SLAM for RGB-D and stereo cameras, which can deal with highly dynamic scenes. The visual SLAM system is built on ORB-SLAM2 and the semantic information is acquired from state-of-the-art object detection with high frame rate. Experiments are conducted on two public benchmarks. Our system is faster than DynaSLAM and keeps the similar localization accuracy. The analysis of the computation time is also presented.

碩士
國立中興大學
電機工程學系所
107
In recent years, Simultaneous Localization And Mapping (SLAM) becomes an important topic in the research area of unmanned vehicles. The sensors adopted by SLAM are mainly divided into lidar and camera approaches. A camera is cost-efficeint and easy to obtain. Thus it is extensively used in implementing SLAM. Particularly, ORB-SLAM2 is a real-time visual SLAM method based on feature points, supporting high-precision three-dimensional (3D) maps with monocular cameras, stereo cameras, and RGB-D cameras. However, it is unable to assign semantic labels to objects observed in the environment for the unmanned vehicle to learn the class of a detected object in an ORB-SLAM2 3D map. Although it is possible to use a method like semantic SLAM to make up the above deficiency, the bounding box of a region proposal generated by the aforementioned object detection method can vary depending on factors such as the angle or offset of the proposal itself. 　　In this thesis, we construct a 3D semantic map by integrating ORB-SLAM2 with the deep learning object detection tool YOLOv3 (You Only Look Once v3). More specifically, we obtain the class of detected objects with YOLOv3 at the same time ORB-SLAM2 inserts a new keyframe. Then we project these semantic labels to the point cloud clusters to achieve a 3D semantic map. Particularly, we remove duplicate recognition results by averaging the object size, and also calculate the coordinate of the object in the map according to the camera position. Then we output obtained object labels together with their corresponding coordinates. Finally, based on public datasets, we perform relative experiments to demonstrate the correctness of our proposed method.

This thesis focuses on building an object-centric 3D map given an RGB image sequence, in which the basic elements are object instances. This is fundamentally different from conventional visual Simultaneous Localisation and Mapping (SLAM) that describes the geometry of an environment using geometric entities, such as 3D points, voxels or surfels. Representing an environment at the level of objects captures both the semantic and geometric information of an environment. It is more natural, compact, and closer to how human beings perceive the environment. Specifically, we investigate methods where well-studied geometry and deep learning can be combined to achieve object-centric mapping in general scenes. We first build upon recent advances in deep learning for single-view object reconstruction, a task of recovering full 3D object shape from a single RGB image. An open question when using deep networks to solve this question is how to generate object shape efficiently. To this end, we propose a novel multi-view representation to generate dense point cloud efficiently. Although this pure deep learning paradigm shows impressive results on synthetic data, the lack of a large amount of annotated real images leads to a domain gap when inference on real images. We then introduce a new single-view object reconstruction method by combining well-studied geometry and a deep learned shape prior. This new approach optimises an object’s shape and pose using both 2D image cues, such as object silhouette, and constraints on learned object shape prior at inference time. Although we only address the single-view object reconstruction in this work, the online refinement makes it straightforward to incorporate more observations. We introduce our first object-centric mapping system – FroDO (From Detections to Objects) based on our works on single-view object reconstruction. It takes as input an RGB image sequence and infers object location, pose, and shape in a coarse-to-fine manner, meaning that we reconstruct an object-centric map starting from a set of 2D object detections, through a 3D bounding box, to a sparse point cloud, and a dense mesh progressively. Although FroDO shows promising results on general and cluttered indoor scenes, it is neither an online system nor capable of handling object motions. To address the limitations of FroDO, we subsequently present MO-LTR (Multiple Object Localisation, Tracking, and Reconstruction). It combines a monocular object detector for object pose and scale prediction, a shape embedding network for shape modelling, and an IMM filter for tracking. Although each component’s contribution is relatively incremental, as a system, it achieves dynamic object-centric mapping for both indoor and outdoor scenes.
Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 2021

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics
Extensive recent research has shown the importance of innovation in medical healthcare, with a focus on Pneumonia. It is vital and lifesaving to predict Pneumonia cases as fast as possible and preferably in advance of the symptoms. An online database source managed to gather Pneumonia-specific image data, with not just the presence of the infection, but also the nature of it, divided in bacterial- and viral infection. The first achievement is extracting valuable information from the X-Ray image datasets. Using several ImageNet pre-trained CNNs, knowledge can be gained from images and transferred to numeric arrays. This, both binary and multi-class classification data, requires a sophisticated prediction algorithm that recognizes X-Ray image patterns. Multiple, recently performed experiments show promising results about the innovative Semantic Learning Machine (SLM) that is essentially a geometric semantic hill climber for feedforward Neural Networks. This SLM is based on a derivation of the Geometric Semantic Genetic Programming (GSGP) mutation operator for real-value semantics. To prove the outperformance of the binary and multi-class SLM in general, a selection of commonly used algorithms is necessary in this research. A comprehensive hyperparameter optimization is performed for commonly used algorithms for those kinds of real-life problems, such as: Random Forest, Support Vector Machine, KNearestNeighbors and Neural Networks. The results of the SLM are promising for the Pneumonia application but could be used for all types of predictions based on images in combination with the CNN feature extractions.
Uma extensa pesquisa recente mostrou a importância da inovação na assistência médica, com foco na pneumonia. É vital e salva-vidas prever os casos de pneumonia o mais rápido possível e, de preferência, antes dos sintomas. Uma fonte on-line conseguiu coletar dados de imagem específicos da pneumonia, identificando não apenas a presença da infecção, mas também seu tipo, bacteriana ou viral. A primeira conquista é extrair informações valiosas dos conjuntos de dados de imagem de raios-X. Usando várias CNNs pré-treinadas da ImageNet, é possível obter conhecimento das imagens e transferi-las para matrizes numéricas. Esses dados de classificação binários e multi-classe requerem um sofisticado algoritmo de predição que reconhece os padrões de imagem de raios-X. Vários experimentos realizados recentemente mostram resultados promissores sobre a inovadora Semantic Learning Machine (SLM), que é essencialmente um hill climber semântico geométrico para feedforward neural network. Esse SLM é baseado em uma derivação do operador de mutação da Geometric Semantic Genetic Programming (GSGP) para valor-reais semânticos. Para provar o desempenho superior do SLM binário e multi-classe em geral, é necessária uma seleção de algoritmos mais comuns na pesquisa. Uma otimização abrangente dos hiperparâmetros é realizada para algoritmos comumente utilizados para esses tipos de problemas na vida real, como Random Forest, Support Vector Machine,K-Nearest Neighbors and Neural Networks. Os resultados do SLM são promissores para o aplicativo pneumonia, mas podem ser usados para todos os tipos de previsões baseadas em imagens em combinação com as extrações de recursos da CNN.

A great deal of the research on cross-linguistic phonetic influence demonstrates that a speaker’s knowledge of their first language (L1) significantly affects their ability to perceive and produce sounds in any other language. While current studies show that cross-linguistic transfer occurs at the L3 level, some research suggests that properties of both L1 and L2 are present in the production of L3 (Ionin, Montrul & Santos, 2011). Many studies have addressed perception, production and factors that influence foreign speech in Second Language Acquisition (SLA) (Watkins, Rauber & Baptista, 2009). As the number of multilingual individuals rises, so does the need for studies that investigate not only SLA but also that of additional languages (i.e., Third Language Acquisition). This dissertation examines how cross-linguistic influence (CLI) occurs among English, Spanish, and Brazilian Portuguese (BP), examining instances of vowel reduction, an aspect of phonological production. English and BP are assumed as vowel reducing languages, whereas Spanish displays negligible vowel reduction in comparison. The vowel productions in L3 BP of two multilingual groups, L1English-L2Spanish-L3BP (ESP) and L1 Spanish-L2 English-BP (SEP) were investigated in two tasks: a paragraph reading task (PRT) and a carrier phrase task (CPT). The study sought to determine whether i) a native speaker of a vowel reducing L1 and a non-vowel reducing L2 displays more or less vowel reduction in a vowel reducing L3 than a native speaker of a non-vowel reducing L1 and vowel reducing L2 and ii) how length of exposure to an L3 affects phonological production. Three fixed effects were considered: duration ratio, intensity ratio and height (F1). The goal was to ascertain whether the Typological Primacy Model (TPM) (Rothman 2011, 2015) or the L2 Status Factor Model (Bardel & Falk 2007, 2012; Hammarberg, 2001) would be a better predictor for how vowel reduction would occur in the L3. Results for duration ratio and vowel height showed no significant difference between groups ESP and SEP. Results for intensity ratio suggest L2 Status as a better predictor, as group SEP displayed more phonological transfer than the ESP group. A hybrid approach to L3 acquisition models is proposed.