Littérature scientifique sur le sujet « Next-Best-View planning »

View planning (VP) is a technique that guides the adjustment of the sensor’s postures in multi-view perception tasks. It converts the perception process into active perception, which improves the intelligence and reduces the resource consumption of the robot. We propose a generic VP system for multiple kinds of visual perception. The VP system is built on the basis of the formal description of the visual task, and the next best view is calculated by the system. When dealing with a given visual task, we can simply update its description as the input of the VP system, and obtain the defined best view in real time. Formal description of the perception task includes the task’s status, the objects’ prior information library, the visual representation status and the optimization goal. The task’s status and the visual representation status are updated when data are received at a new view. If the task’s status has not reached its goal, candidate views are sorted based on the updated visual representation status, and the next best view that can minimize the entropy of the model space is chosen as the output of the VP system. Experiments of view planning for 3D recognition and reconstruction tasks are conducted, and the result shows that our algorithm has good performance on different tasks.

Abstract. Structure-from-Motion (SfM) and Multi-View Stereo (MVS) are widely used methods in three dimensional (3D) model reconstruction for an infrastructure maintenance purpose. However, if a set of images is not captured from well-placed positions, the final dense model can contain low-quality regions. Since MVS requires a much longer processing time than SfM as larger amounts of images are provided, it is impossible for surveyors to wait for the SfM–MVS process to complete and evaluate the geometric quality of a final dense model on-site. This challenge results in response inefficiency and the deterioration of dense models in 3D model reconstruction. If the quality of the final dense model can be predicted immediately after SfM, it will be possible to revalidate the images much earlier and to obtain the dense model with better quality than the existing SfM–MVS process. Therefore, we propose a method for reconstructing a more plausible 3D mesh model that accurately approximates the geometry of the final dense model only from sparse point clouds generated from SfM. This approximated mesh model can be generated using Delaunay triangulation for the sparse point clouds and triangle as well as tetrahedron filtering. The approximated model can be used to predict the geometric quality of the final dense model and for an optimization-based view planning. Some experimental results showed that our method is effective in predicting the quality of the final dense model and finding the potentially degraded regions. Moreover, it was confirmed that the average reconstruction errors of the dense model generated by the optimization-based view planning went below tens of millimeters and falls within an acceptable range for an infrastructure maintenance purpose.

Abstract. Path planning for a measuring vehicle requires solving two popular problems from computer science, namely the search for the optimal tour and the search for the optimal viewpoint. Combining both problems results in a new variation of the Traveling Salesman Problem, which we refer to as the Explorational Traveling Salesman Problem. The solution to this problem is the optimal tour with a minimum of observations. In this paper, we formulate the basic problem, discuss it in context of the existing literature and present an iterative solution algorithm. We demonstrate how the method can be applied directly to LiDAR data using an occupancy grid. The ability of our algorithm to generate suitably efficient tours is verified based on two synthetic benchmark datasets, utilizing a ground truth determined by an exhaustive search.

The Next-Best-View (NBV) problem plays an important part in automatic 3D object reconstruction and exploration applications. This thesis presents a novel approach of ray-casting in Occupancy Grid Maps (OGM) in the context of solving the NBV problem in a 3D-exploration setting. The proposed approach utilizes the structure of an octree-based OGM to perform calculations of potential information gain. The computations are significantly faster than current methods, without decreasing mapping quality. Performance, both in terms of mapping quality, coverage and computational complexity, is experimentally verified through a comparison with existing state-of-the-art methods using high-resolution point cloud data generated using time-of-flight laser range scanners. Current methods for viewpoint ranking focus either heavily on mapping performance or computation speed. The results presented in this thesis indicate that the proposed method is able to achieve a mapping performance similar to the performance-oriented approaches while maintaining the same low computation speed as more approximative methods.

Despite the inherent dynamic nature of real world environments, much robotics research focus on applying new concepts and techniques in isolated, static contexts. To further advance the field, it could be useful to explore not only how to compensate for dynamic features, but also how such features could be exploited. In this work, we investigate how next best view planning can be adapted for observation of dynamic scenes. To this purpose, we conduct a thorough review of di˙erent representations of information, based on prece-dent from the well-studied static case. It is found that view planning cannot be directly transferred from static environments to dynamic scenes without accounting for information decay and observational bias. The result is new insight to the time-variant view planning problem.
Mycket av forskningen inom robotik fokuserar på att applicera nya koncept och tekniker i isolerade statiska miljöer, även när det tilltänkta användnings-området är ofrånkomligen dynamiskt. För att föra fältet framåt kan det vara användbart att utforska hur dynamiska egenskaper kan utnyttjas och kompen-seras för. I det här arbetet undersöker vi hur planering av nästa bästa vy kan anpassas till dynamiska scener. För detta syfte genomförs en nogrann under-sökning av hur information bör representeras, baserat på tidigare slutsatser från det välstuderade statiska fallet. Vi finner att vyplanering inte kan direkt över-föras från statiska till dynamiska miljöer utan att hantera informationsbortfall och den partiskhet mot redan observerade regioner som uppstår. Resultatet av undersökningen är en ökad förståelse för det tidsvarianta planeringsproblemet som att välja nästa vy innebär.

Dans un contexte de recherche archéologique ou de préservation du patrimoine, la reconstruction 3D de site à l'aide de capteur de haute qualité, reste un processus lent, complexe et onéreux et chronophage. Avec l'émergence des technologies robotiques, l'utilisation de robots mobiles autonomes permet la production de modèles 3D complets et précis, dans un temps réduit, tout en accédant aux zones les plus difficiles. L'objectif de cette thèse est de mettre au point des méthodes de reconstruction 3D d'environnements inconnus, à l'aide de plusieurs robots mobiles. Les robots, équipés de caméras stéréoscopiques explorent l'environnement et révèlent dans le modèle progressivement reconstruit, des éléments de surface incomplets. Des poses candidates sont alors générées pour les compléter, sont groupées en zones d'intérêt, et sont ensuite assignées aux robots à l'aide d'un algorithme glouton. Cet algorithme est un planificateur de chemin de type Next-Best-View, basé sur un critère surfacique, où le problème du voyageur de commerce est résolu itérativement. Un planificateur probabiliste permet enfin aux robots de trouver leur chemin dans l'espace libre. La méthode a été déclinée en trois architectures : mono-robot dans un cadre préliminaire, multi-robot centralisée pour réduire les temps de reconstruction, puis distribuée pour en augmenter la robustesse. Une validation par simulation et des tests en environnement réel ont été réalisés, pour des robots aériens et roulants, afin de montrer l'efficacité des trois architectures
In archaeology and cultural heritage, the 3D modelling of large-scale structures using high-quality sensors, remains time-consuming, complex, and expansive process. In present age of robotics, a new generation of scanning systems based on mobile robots, could address this challenge, improving efficiency, flexibility and responsiveness. This PhD thesis considers the problem of 3D reconstruction of an unknown environment, with a team of cooperative vehicles. The robots equipped with forward-facing stereo cameras, explore the environment, uncover discrete Incomplete Surface Elements (ISEs) in the volumetric map, and generate candidate viewpoints to scan them. These areas of interest are greedily assigned to the robots using a Next-Best-View approach, where the visit is planned by iteratively solving a Traveling Salesman Problem. Then, a sampling-based planner is used to compute obstacle-free paths using the volumetric map. A single-robot architecture has been first designed, which leverages the 3D surface representation of volumetric map for planning. This architecture has been extended to a multi-robot system with a single base station, in order to accelerate the scanning process. Finally, a distributed architecture has been presented and discussed to increase the robustness of the multi-robot system. Extensive numerical and real-world experiments with multiple aerial and ground robots have been conducted to validate the prosposed architectures in challenging environments

La numérisation 3D telle que pratiquée aujourd'hui repose essentiellement sur les connaissances de l'opérateur qui la réalise. La qualité des résultats reste très sensible à la procédure utilisée et par conséquent aux compétences de l'opérateur. Ainsi, la numérisation manuelle est très coûteuse en ressources humaines et matérielles et son résultat dépend fortement du niveau de technicité de l'opérateur. Les solutions de numérisation les plus avancées en milieu industriel sont basées sur une approche d'apprentissage nécessitant une adaptation manuelle pour chaque pièce. Ces systèmes sont donc semi-automatiques compte tenu de l'importance de la contribution humaine pour la planification des vues.Mon projet de thèse se focalise sur la définition d'un procédé de numérisation 3D automatique et intelligente. Ce procédé est présenté sous forme d'une séquence de processus qui sont la planification de vues, la planification de trajectoires, l'acquisition et les post-traitements des données acquises. L'originalité de notre démarche de numérisation est qu'elle est générique parce qu'elle n'est pas liée aux outils et méthodes utilisés pour la réalisation des tâches liées à chaque processus. Nous avons également développé trois méthodes de planification de vues pour la numérisation d'objets sans connaissance a priori de leurs formes. Ces méthodes garantissent une indépendance des résultats par rapport au savoir-faire de l'opérateur. L'originalité de ces approches est qu'elles sont applicables à tous types de scanners. Nous avons implanté ces méthodes sur une cellule de numérisation robotisée. Nos approches assurent une reconstruction progressive et intelligente d'un large panel d'objets de différentes classes de complexité en déplaçant efficacement le scanner

This thesis deals with active sensing and its use in real exploration tasks under both scene ambiguities and measurement uncertainties. While object modeling is the implicit objective of most of active sensing algorithms, in this work we have explored new strategies to deal with more generic and more complex tasks. Active sensing requires the ability of moving the perceptual system to gather new information. Our approach uses a robot manipulator with a 3D Time-of-Flight (ToF) camera attached to the end-effector. For a complex task, we have focused our attention on plant phenotyping. Plants are complex objects, with leaves that change their position and size along time. Valid viewpoints for a certain plant are hardly valid for a different one, even belonging to the same species. Some instruments, such as chlorophyll meters or disk sampling tools, require being precisely positioned over a particular location of the leaf. Therefore, their use requires the modeling of specific regions of interest of the plant, including also the free space needed for avoiding obstacles and approaching the leaf with tool. It is easy to observe that predefined camera trajectories are not valid here, and that usually with one single view it is very difficult to acquire all the required information. The overall objective of this thesis is to solve complex active sensing tasks by embedding their exploratory goal into a pre-estimated geometrical model, using information-gain as the fundamental guideline for the reward function. The main contributions can be divided in two groups: first, the evaluation of ToF cameras and their calibration to assess the uncertainty of the measurements (presented in Part I); and second, the proposal of a framework capable of embedding the task, modeled as free and occupied space, and that takes into account the modeled sensor's uncertainty to improve the action selection algorithm (presented in Part II). This thesis has given rise to 14 publications, including 5 indexed journals, and its results have been used in the GARNICS European project. The complete framework is based on the Next-Best-View methodology and it can be summarized in the following main steps. First, an initial view of the object (e.g., a plant) is acquired. From this initial view and given a set of candidate viewpoints, the expected gain obtained by moving the robot and acquiring the next image is computed. This computation takes into account the uncertainty from all the different pixels of the sensor, the expected information based on a predefined task model, and the possible occlusions. Once the most promising view is selected, the robot moves, takes a new image, integrates this information intothe model, and evaluates again the set of remaining views. Finally, the task terminates when enough information is gathered. In our examples, this process enables the robot to perform a measurement on top of a leaf. The key ingredient is to model the complexity of the task in a layered representation of free-occupied occupancy grid maps. This allows to naturally encode the requirements of the task, to maintain and update the belief state with the measurements performed, to simulate and compute the expected gains of all potential viewpoints, and to encode the termination condition. During this work the technology of ToF cameras has incredibly evolved. Nowadays it is very popular and ToF cameras are already embedded in some consumer devices. Although the quality of the measurements has been considerably improved, it is still not uniform in the sensor. We believe, as it has been demonstrated in various experiments in this work, that a careful modeling of the sensor's uncertainty is highly beneficial and helps to design better decision systems. In our case, it enables a more realistic computation of the information gain measure, and consequently, a better selection criterion.
Aquesta tesi aborda el tema de la percepció activa i el seu ús en tasques d'exploració en entorns reals tot considerant la ambigüitat en l'escena i la incertesa del sistema de percepció. Al contrari de la majoria d'algoritmes de percepció activa, on el modelatge d'objectes sol ser l'objectiu implícit, en aquesta tesi hem explorat noves estratègies per poder tractar tasques genèriques i de major complexitat. Tot sistema de percepció activa requereix un aparell sensorial amb la capacitat de variar els seus paràmetres de forma controlada, per poder, d'aquesta manera, recopilar nova informació per resoldre una tasca determinada. En tasques d'exploració, la posició i orientació del sensor són paràmetres claus per resoldre la tasca. En el nostre estudi hem fet ús d'un robot manipulador com a sistema de posicionament i d'una càmera de profunditat de temps de vol (ToF), adherida al seu efector final, com a sistema de percepció. Com a tasca final, ens hem concentrat en l'adquisició de mesures sobre fulles dins de l'àmbit del fenotipatge de les plantes. Les plantes son objectes molt complexos, amb fulles que canvien de textura, posició i mida al llarg del temps. Això comporta diverses dificultats. Per una banda, abans de dur a terme una mesura sobre un fulla s'ha d'explorar l'entorn i trobar una regió que ho permeti. A més a més, aquells punts de vista que han estat adequats per una determinada planta difícilment ho seran per una altra, tot i sent les dues de la mateixa espècie. Per un altra banda, en el moment de la mesura, certs instruments, tals com els mesuradors de clorofil·la o les eines d'extracció de mostres, requereixen ser posicionats amb molta precisió. És necessari, doncs, disposar d'un model detallat d'aquestes regions d'interès, i que inclogui no només l'espai ocupat sinó també el lliure. Gràcies a la modelització de l'espai lliure es pot dur a terme una bona evitació d'obstacles i un bon càlcul de la trajectòria d'aproximació de l'eina a la fulla. En aquest context, és fàcil veure que, en general, amb un sol punt de vista no n'hi ha prou per adquirir tota la informació necessària per prendre una mesura, i que l'ús de trajectòries predeterminades no garanteixen l'èxit. L'objectiu general d'aquesta tesi és resoldre tasques complexes de percepció activa mitjançant la codificació del seu objectiu d'exploració en un model geomètric prèviament estimat, fent servir el guany d'informació com a guia fonamental dins de la funció de cost. Les principals contribucions d'aquesta tesi es poden dividir en dos grups: primer, l'avaluació de les càmeres ToF i el seu calibratge per poder avaluar la incertesa de les seves mesures (presentat en la Part I); i en segon lloc, la proposta d'un sistema capaç de codificar la tasca mitjançant el modelatge de l'espai lliure i ocupat, i que té en compte la incertesa del sensor per millorar la selecció de les accions (presentat en la Part II). Aquesta tesi ha donat lloc a 14 publicacions, incloent 5 en revistes indexades, i els resultats obtinguts s'han fet servir en el projecte Europeu GARNICS. La funcionalitat del sistema complet està basada en els mètodes Next-Best-View (següent-millor-vista) i es pot desglossar en els següents passos principals. En primer lloc, s'obté una vista inicial de l'objecte (p. ex., una planta). A partir d'aquesta vista inicial i d'un conjunt de vistes candidates, s'estima, per cada una d'elles, el guany d'informació resultant, tant de moure la càmera com d'obtenir una nova mesura. És rellevant dir que aquest càlcul té en compte la incertesa de cada un dels píxels del sensor, l'estimació de la informació basada en el model de la tasca preestablerta i les possibles oclusions. Un cop seleccionada la vista més prometedora, el robot es mou a la nova posició, pren una nova imatge, integra aquesta informació en el model i torna a avaluar, un altre cop, el conjunt de punts de vista restants. Per últim, la tasca acaba en el moment que es recopila suficient informació.

AbstractAutomated vehicles will have a significant impact on our mobility and the way we travel. Already over the next few years, highly dynamic developments can be expected in the mobility sector, with fundamental changes and upheavals that will present both opportunities and risks. From the point of view of transport, infrastructure and urban planning, strategies are therefore required as to where and how automated vehicles can be best deployed.