Dissertations / Theses on the topic 'Control and learning of soft robots'

Lorsque des changements inattendus de la surface du sol se produisent lors de la marche, le système nerveux central humain doit appliquer des mesures de contrôle appropriées pour assurer une stabilité dynamique. De nombreuses études dans le domaine de la commande moteur ont étudié les mécanismes d'un tel contrôle postural et ont largement décrit comment les trajectoires du centre de masse (COM), le placement des pas et l'activité musculaire s'adaptent pour éviter une perte d'équilibre. Les mesures que nous avons effectuées montrent qu'en arrivant sur un sol mou, les participants ont modulé de façon active les forces de réaction au sol (GRF) sous le pied de support afin d'exploiter les propriétés élastiques et déformables de la surface pour amortir l'impact et probablement dissiper l'énergie mécanique accumulée pendant la ‘chute’ sur la nouvelle surface déformable. Afin de contrôler plus efficacement l'interaction pieds-sol des robots humanoïdes pendant la marche, nous proposons d'ajouter des semelles extérieures souples (c'est-à-dire déformables) aux pieds. Elles absorbent les impacts et limitent les effets des irrégularités du sol pendant le mouvement sur des terrains accidentés. Cependant, ils introduisent des degrés de liberté passifs (déformations sous les pieds) qui complexifient les tâches d'estimation de l'état du robot et ainsi que sa stabilisation globale. Pour résoudre ce problème, nous avons conçu un nouveau générateur de modèle de marche (WPG) basé sur une minimisation de la consommation d'énergie qui génère les paramètres nécessaires pour utiliser conjointement un estimateur de déformation basé sur un modèle éléments finis (FEM) de la semelle souple pour prendre en compte sa déformation lors du mouvement. Un tel modèle FEM est coûteux en temps de calcul et empêche la réactivité en ligne. Par conséquent, nous avons développé une boucle de contrôle qui stabilise les robots humanoïdes lors de la marche avec des semelles souples sur terrain plat et irrégulier. Notre contrôleur en boucle fermée minimise les erreurs sur le centre de masse (COM) et le point de moment nul (ZMP) avec un contrôle en admittance des pieds basé sur un estimateur de déformation simplifié. Nous démontrons son efficacité expérimentalement en faisant marcher le robot humanoïde HRP-4 sur des graviers
When unexpected changes of the ground surface occur while walking, the human central nervous system needs to apply appropriate control actions to assure dynamic stability. Many studies in the motor control field have investigated the mechanisms of such a postural control and have widely described how center of mass (COM) trajectories, step patterns and muscle activity adapt to avoid loss of balance. Measurements we conducted show that when stepping over a soft ground, participants actively modulated the ground reaction forces (GRF) under the supporting foot in order to exploit the elastic and compliant properties of the surface to dampen the impact and to likely dissipate the mechanical energy accumulated during the ‘fall’ onto the new compliant surface.In order to control more efficiently the feet-ground interaction of humanoid robots during walking, we propose adding outer soft (i.e. compliant) soles to the feet. They absorb impacts and cast ground unevenness during locomotion on rough terrains. However, they introduce passive degrees of freedom (deformations under the feet) that complexify the tasks of state estimation and overall robot stabilization. To address this problem, we devised a new walking pattern generator (WPG) based on a minimization of the energy consumption that offers the necessary parameters to be used jointly with a sole deformation estimator based on finite element model (FEM) of the soft sole to take into account the sole deformation during the motion. Such FEM computation is time costly and inhibit online reactivity. Hence, we developed a control loop that stabilizes humanoid robots when walking with soft soles on flat and uneven terrain. Our closed-loop controller minimizes the errors on the center of mass (COM) and the zero-moment point (ZMP) with an admittance control of the feet based on a simple deformation estimator. We demonstrate its effectiveness in real experiments on the HRP-4 humanoid walking on gravels

Soft lightweight robots provide an inherently safe solution to using robots in unmodeled environments by maintaining safety without increasing cost through expensive sensors. Unfortunately, many practical problems still need to be addressed before soft robots can become useful in real world tasks. Unlike traditional robots, soft robot geometry is not constant but can change with deflation and reinflation. Small errors in a robot's kinematic model can result in large errors in pose estimation of the end effector. This error, coupled with the inherent compliance of soft robots and the difficulty of soft robot joint angle sensing, makes it very challenging to accurately control the end effector of a soft robot in task space. However, this inherent compliance means that soft robots lend themselves nicely to coordinated multi-arm manipulation tasks, as deviations in end effector pose do not result in large force buildup in the arms or in the object being manipulated. Coordinated, multi-arm manipulation with soft robots is the focus of this thesis. We first developed two tools enabling multi-arm manipulation with soft robots: (1) a hybrid servoing control scheme for task space control of soft robot arms, and (2) a general base placement optimization for the robot arms in a multi-arm manipulation task. Using these tools, we then developed and implemented a simple multi-arm control scheme. The hybrid servoing control scheme combines inverse kinematics, joint angle control, and task space servoing in order to reduce end effector pose error. We implemented this control scheme on two soft robots and demonstrated its effectiveness in task space control. Having developed a task space controller for soft robots, we then approached the problem of multi-arm manipulation. The placement of each arm for a multi-arm task is non-trivial. We developed an evolutionary optimization that finds the optimal arm base location for any number of user-defined arms in a user-defined task or workspace. We demonstrated the utility of this optimization in simulation, and then used it to determine the arm base locations for two arms in two real world coordinated multi-arm manipulation tasks. Finally, we developed a simple multi-arm control scheme for soft robots and demonstrated its effectiveness using one soft robot arm, and one rigid robot with low-impedance torque control. We placed each arm base in the pose determined by the base placement optimization, and then used the hybrid servoing controller in our multi-arm control scheme to manipulate an object through two desired trajectories.

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2015.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 223-236).
The goal of this thesis is to explore how autonomous robotic systems can be created with soft elastomer bodies powered by fluids. In this thesis we innovate in the design, fabrication, control, and experimental validation of both single and multi-segment soft fluidic elastomer robots. First, this thesis describes an autonomous fluidic elastomer robot that is both self-contained and capable of rapid, continuum body motion. Specifically, the design, modeling, fabrication, and control of a soft fish is detailed, focusing on enabling the robot to perform rapid escape responses. The robot employs a compliant body with embedded actuators emulating the slender anatomical form of a fish. In addition, the robot has a novel fluidic actuation system that drives body motion and has all the subsystems of a traditional robot on-board: power, actuation, processing, and control. At the core of the fish's soft body is an array of Fluidic Elastomer Actuators (FEAs). The fish is designed to emulate escape responses in addition to forward swimming because such maneuvers require rapid body accelerations and continuum body motion. These maneuvers showcase the performance capabilities of this self-contained robot. The kinematics and controllability of the robot during simulated escape response maneuvers are analyzed and compared to studies on biological fish. During escape responses, the soft-bodied robot is shown to have similar input-output relationships to those observed in biological fish. The major implication of this portion of the thesis is that a soft fluidic elastomer robot is shown to be both self-contained and capable of rapid body motion. Next, this thesis provides an approach to planar manipulation using soft fluidic elastomer robots. That is, novel approaches to design, fabrication, kinematic modeling, power, control, and planning as well as extensive experimental evaluations with multiple manipulator prototypes are presented. More specifically, three viable manipulator morphologies composed entirely from soft silicone rubber are explored, and these morphologies are differentiated by their actuator structures, namely: ribbed, cylindrical, and pleated. Additionally, three distinct casting-based fabrication processes are explored: lamination-based casting, retractable-pin-based casting, and lost-wax- based casting. Furthermore, two ways of fabricating a multiple DOF manipulator are explored: casting the complete manipulator as a whole, and casting single DOF segments with subsequent concatenation. An approach to closed-loop configuration control is presented using a piecewise constant curvature kinematic model, real-time localization data, and novel fluidic drive cylinders which power actuation. Multi-segment forward and inverse kinematic algorithms are developed and combined with the configuration controller to provide reliable task-space position control. Building on these developments, a suite of task-space planners are presented to demonstrate new autonomous capabilities from these soft robots such as: (i) tracking a path in free-space, (ii) maneuvering in confined environments, and (iii) grasping and placing objects. Extensive evaluations of these capabilities with physical prototypes demonstrate that manipulation with soft fluidic elastomer robots is viable. Lastly, this thesis presents a robotic manipulation system capable of autonomously positioning a multi-segment soft fluidic elastomer robot in three dimensions while subject to the self-loading effects of gravity. Specifically, an extremely soft robotic manipulator morphology that is composed entirely from low durometer elastomer, powered by pressurized air, and designed to be both modular and durable is presented. To understand the deformation of a single arm segment, a static physics-based model is developed and experimentally validated. Then, to kinematically model the multi-segment manipulator, a piece-wise constant curvature assumption consistent with more traditional continuum manipulators is used. Additionally, a complete fabrication process for this new manipulator is defined and used to make multiple functional prototypes. In order to power the robot's spatial actuation, a high capacity fluidic drive cylinder array is implemented, providing continuously variable, closed-circuit gas delivery. Next, using real-time localization data, a processing and control algorithm is developed that generates realizable kinematic curvature trajectories and controls the manipulator's configuration along these trajectories. A dynamic model for this multi-body fluidic elastomer manipulator is also developed along with a strategy for independently identifying all unknown components of the system: the soft manipulator, its distributed fluidic elastomer actuators, as well as its drive cylinders. Next, using this model and trajectory optimization techniques locally-optimal, open-loop control policies are found. Lastly, new capabilities offered by this soft fluidic elastomer manipulation system are validated with extensive physical experiments. These are: (i) entering and advancing through confined three-dimensional environments, (ii) conforming to goal shape-configurations within a sagittal plane under closed-loop control, and (iii) performing dynamic maneuvers we call grabs.
by Andrew D. Marchese.
Ph. D.

Animals exploit soft structures to move smoothly and effectively in complex natural environments. These capabilities have inspired robotic engineers to incorporate soft actuating technologies into their designs. Developing soft muscle-like actuation technology is one of the grand challenges in the creation of soft-body robots that can move, deform their body, and modulate body stiffness. This paper presents the development of smart pneumatic McKibben muscles woven and reinforced by using conductive insulated wires to equip the muscles with an inherent sensing capability, in which the deformation of the muscles can be effectively measured by calculating the change of wire inductance. Sensing performance of a variety of weaving angles is investigated. The ideal McKibben muscle models are used for analysing muscle performance and sensing accuracy. The experimental results show that the contraction of the muscles is proportional to the measured change of inductance. This relationship is applied to a PID control system to control the contraction of smart muscles in simulation, and good control performance is achieved. The creation of smart muscles with an inherent sensing capability and a good controllability is promising for operation of future soft robots.

La modélisation de robots souples est extrêmement difficile, à cause notamment du nombre théoriquement infini des degrés de liberté. Cette difficulté est accentuée lorsque les robots ont des configurations complexes. Ce problème de modélisation entraîne de nouveaux défis pour la calibration et la conception des commandes des robots, mais également de nouvelles opportunités avec de nouvelles stratégies de détection de force possibles. Cette thèse a pour objectif de proposer des solutions nouvelles et générales utilisant la modélisation et la vision. La thèse présente dans un premier temps un modèle cinématique à temps discret pour les robots souples reposant sur la méthode des éléments finis (FEM) en temps réel. Ensuite, une méthode de calibration basée sur la vision du système de capteur-robot et des actionneurs est étudiée. Deux contrôleurs de position en boucle fermée sont conçus. En outre, pour traiter le problème de la perte d'image, une stratégie de commande commutable est proposée en combinant à la fois le contrôleur à boucle ouverte et le contrôleur à boucle fermée. Deux méthodes (avec et sans marqueur(s)) de détection de force externe pour les robots déformables sont proposées. L'approche est basée sur la fusion de mesures basées sur la vision et le modèle par FEM. En utilisant les deux méthodes, il est possible d'estimer non seulement les intensités, mais également l'emplacement des forces externes. Enfin, nous proposons une application concrète : un robot cathéter dont la flexion à l'extrémité est piloté par des câbles. Le robot est contrôlé par une stratégie de contrôle découplée qui permet de contrôler l’insertion et la flexion indépendamment, tout en se basant sur un modèle FEM
The modeling of soft robots which have, theoretically, infinite degrees of freedom, are extremely difficult especially when the robots have complex configurations. This difficulty of modeling leads to new challenges for the calibration and the control design of the robots, but also new opportunities with possible new force sensing strategies. This dissertation aims to provide new and general solutions using modeling and vision. The thesis at first presents a discrete-time kinematic model for soft robots based on the real-time Finite Element (FE) method. Then, a vision-based simultaneous calibration of sensor-robot system and actuators is investigated. Two closed-loop position controllers are designed. Besides, to deal with the problem of image feature loss, a switched control strategy is proposed by combining both the open-loop controller and the closed-loop controller. Using soft robot itself as a force sensor is available due to the deformable feature of soft structures. Two methods (marker-based and marker-free) of external force sensing for soft robots are proposed based on the fusion of vision-based measurements and FE model. Using both methods, not only the intensities but also the locations of the external forces can be estimated.As a specific application, a cable-driven continuum catheter robot through contacts is modeled based on FE method. Then, the robot is controlled by a decoupled control strategy which allows to control insertion and bending independently. Both the control inputs and the contact forces along the entire catheter can be computed by solving a quadratic programming (QP) problem with a linear complementarity constraint (QPCC)

Q-Learning is a method for solving reinforcement learning problems. Reinforcement learning problems require improvement of behaviour based on received rewards. Q-Learning has the potential to reduce robot programming effort and increase the range of robot abilities. However, most currentQ-learning systems are not suitable for robotics problems: they treat continuous variables, for example speeds or positions, as discretised values. Discretisation does not allow smooth control and does not fully exploit sensed information. A practical algorithm must also cope with real-time constraints, sensing and actuation delays, and incorrect sensor data. This research describes an algorithm that deals with continuous state and action variables without discretising. The algorithm is evaluated with vision-based mobile robot and active head gaze control tasks. As well as learning the basic control tasks, the algorithm learns to compensate for delays in sensing and actuation by predicting the behaviour of its environment. Although the learned dynamic model is implicit in the controller, it is possible to extract some aspects of the model. The extracted models are compared to theoretically derived models of environment behaviour. The difficulty of working with robots motivates development of methods that reduce experimentation time. This research exploits Q-learning’s ability to learn by passively observing the robot’s actions—rather than necessarily controlling the robot. This is a valuable tool for shortening the duration of learning experiments.

This dissertation is focused on the modeling and robust model-based control of high degree-of-freedom (DoF) systems. While most of the contributions are applicable to any difficult-to-model system, this dissertation focuses specifically on applications to large-scale soft robots because their many joints and pressures constitute a high-DoF system and their inherit softness makes them difficult to model accurately. First a joint-angle estimation and kinematic calibration method for soft robots is developed which is shown to decrease the pose prediction error at the end of a 1.5 m robot arm by about 85\%. A novel dynamic modelling approach which can be evaluated within microseconds is then formulated for continuum type soft robots. We show that deep neural networks (DNNs) can be used to approximate soft robot dynamics given training examples from physics-based models like the ones described above. We demonstrate how these machine-learning-based models can be evaluated quickly to perform a form of optimal control called model predictive control (MPC). We describe a method of control trajectory parameterization that enables MPC to be applied to systems with more DoF and with longer prediction horizons than previously possible. We show that this parameterization decreases MPC's sensitivity to model error and drastically reduces MPC solve times. A novel form of MPC is developed based on an evolutionary optimization algorithm that allows the optimization to be parallelized on a computer's graphics processing unit (GPU). We show that this evolutionary MPC (EMPC) can greatly decrease MPC solve times for high DoF systems without large performance losses, especially given a large GPU. We combine the ideas of machine learned DNN models of robot dynamics, with parameterized and parallelized MPC to obtain a nonlinear version of EMPC which can be run at higher rates and find better solutions than many state-of-the-art optimal control methods. Finally we demonstrate an adaptive form of MPC that can compensate for model error or changes in the system to be controlled. This adaptive form of MPC is shown to inherit MPC's robustness to completely unmodeled disturbances and adaptive control's ability to decrease trajectory tracking errors over time.

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (page 47).
Soft robot hands offer numerous advantages over rigid ones for manipulation, including robustness and safety. Yet, compared to rigid robots, soft robots are characterized by continuous mechanics, and finite-element approximations with many degrees of freedom present a significant obstacle for modern control approaches. The central question my thesis explores is whether we can capture the benefits of soft robot hands with relatively simple dynamical models. Specifically, we demonstrate a very simple model of a 2D soft manipulator that uses pulleys and cables to model deformable surfaces. This model captures much of the qualitative behavior of soft membranes, while also proving amenable to modern control techniques. We validate this model physically using a hardware set-up. We then demonstrate a simple quasi-static Jacobian controller which solves a second-order cone program to achieve the task of in-hand object repositioning.
by Geronimo Mirano.
M. Eng.

Soft robotic systems have recently been considered as a new approach that is in principle better suited for tasks where safety and adaptability are important. That is because soft materials are inherently compliant and resilient in the event of collisions. They are also lightweight and can be low-cost; in general, soft robots have the potential to achieve many tasks that were not previously possible with traditional robotic systems. In this paper, we propose a new manufacturing process for creating multi-chambered pneumatic actuators and robots. We focus on using fabric as the primary structural material, but plastic films can be used instead of textiles as well. We introduce two different methods to create layered bellows actuators, which can be made with a heat press machine or in an oven. We also describe origami-like actuators with possible corner structures. Moreover, the fabrication process permits the creation of soft and soft/rigid hybrid robotic systems, and enables the easy integration of sensors into these robots. We analyze various textiles that are possibly used with this method, and model bellows actuators including operating force, restoring force, and estimated geometry with multiple bellows. We then demonstrate the process by showing a bellows actuator with an embedded sensor and other fabricated structures and robots. We next present a new design of a multi-DOF soft/rigid hybrid robotic manipulator. It contains a revolute actuator and several roll-pitch actuators which are arranged in series. To control the manipulator, we use a new variant of the piece-wise constant curvature (PCC) model. The robot can be controlled using forward and inverse kinematics with embedded inertial measurement units (IMUs). A bellows actuator, which is a subcomponent of the manipulator, is modeled with a variable-stiffness spring, and we use the model to predict the behavior of the actuator. With the model, the roll-pitch actuator stiffnesses are measured in all directions through applying forces and torques. The stiffness is used to predict the behavior of the end effector. The robotic system introduced achieved errors of less than 5% when compared to the models, and positioning accuracies of better than 1cm.
Doctor of Philosophy

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mechanical Engineering, 2018.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 247-272).
This thesis describes the creation and control of soft robots made of deformable elastomer materials and powered by fluidics. We embed soft fluidic actuators into self-contained soft robotic systems, such as fish for underwater exploration or soft arms for dynamic manipulation. We present models describing the physical characteristics of these continuously deformable and fully soft robots, and then leverage these models for motion planning and closed-loop feedback control in order to realize quasi-static manipulation, dynamic arm motions, and dynamic interactions with an environment. The design and fabrication techniques for our soft robots include the development of soft actuator morphologies, soft casting techniques, and closed-circuit pneumatic and hydraulic powering methods. With a modular design approach, we combine these soft actuator morphologies into robotic systems. We create a robotic fish for underwater locomotion, as well as multi-finger hands and multi-segment arms for use in object manipulation and interaction with an environment. The robotic fish uses a soft hydraulic actuator as its deformable tail to perform open-loop controlled swimming motions through cyclic undulation. The swimming movement is achieved by a custom-made displacement pump and a custom-made buoyancy control unit, all embedded within the soft robotic fish. The fish robot receives high-level control commands via acoustic signals to move in marine environments. The control of the multi-segment arms is enabled by models describing the geometry, kinematics, impedance, and dynamics. We use the models for quasi-static closed-loop control and dynamic closed-loop control. The quasi-static controllers work in combination with the kinematic models and geometric motion planners to enable the soft arms to move in confined spaces, and to autonomously perform object grasping. Leveraging the models for impedance and dynamics, we also demonstrate dynamic arm motions and end-effector interactions of the arm with an environment. Our dynamic model allows the application of control techniques developed for rigid robots to the dynamic control of soft robots. The resulting model-based closed-loop controllers enable dynamic curvature tracking as well as surface tracing in Cartesian space.
by Robert Kevin Katzschmann.
Ph. D.

La robotique de service est un domaine émergent où il est nécessaire de commander des robots en interaction forte avec leur environnement. Ce travail présente une méthode adaptative de commande combinant de l'apprentissage de modèles physiques et de la commande dans l'espace opérationnel de robots redondants. L'apprentissage des modèles cinématiques est obtenu soit par dérivation de modèles géométriques appris, soit par apprentissage direct. Ces modèles cinématiques, également appelés matrices Jacobiennes, peuvent être utilisés dans le calcul de pseudo-inverse ou de projecteurs pour la commande du robot. Cette combinaison de méthodes permet d'obtenir un contrôleur qui s'adapte à la géométrie du robot commandé. D'une façon similaire, il est également possible d'apprendre un modèle dynamique inverse du robot de manière à commander le robot en couple plutôt qu'en vitesse. Cela a pour avantage de pouvoir s'adapter aux modifications dynamiques qui s'appliquent sur le robot comme par exemple l'application d'une force extérieure ou l'ajout d'un poids. Les expériences menées dans le cadre de cette thèse montrent comment réaliser plusieurs tâches hiérarchiques ou comment s'adapter à des perturbations avec des modèles appris. Des application sur un robots réel ont également été menées afin de rendre compte de la plausibilité de l'approche proposée.

Good models of industrial robots are necessary in a variety of applications, such as mechanical design, performance simulation, control, diagnosis, supervision and offline programming. This motivates the need for good modelling tools. In the first part of this thesis the forward kinematic modelling of serial industrial robots is studied. The first steps towards a toolbox are implemented in the Maple programming language.

A series of possible applications for the toolbox can be mentioned. One example is to estimate the pose of the robot tool using an extended Kalman filter by means of extra sensors mounted on the robot. The kinematic equations and the relations necessary for the extended Kalman filter can be derived in the modelling tool. Iterative learning control, ILC, using an estimate of the tool position can then improve the robot performance.

The second part of the thesis is devoted to ILC, which is a control method that is applicable when the robot performs a repetitive movement starting from the same initial conditions every repetition. The algorithm compensates for repetitive errors by adding a correction signal to the reference. Studies where ILC is applied to a real industrial platform is less common in the literature, which motivates the work in this thesis.

A first-order ILC filter with iteration-independent operators derived using a heuristic design approach is used, which results in a non-causal algorithm. A simulation study is made, where a flexible two-mass model is used as a simplified linear model of a single robot joint and the ILC algorithm applied is based on motor-angle measurements only. It is shown that when a model error is introduced in the relation between the arm and motor reference angle, it is not necessary that the error on the arm side is reduced as much as the error on the motor side, or in fact reduced at all.

In the experiments the ILC algorithm is applied to a large-size commercial industrial robot, performing a circular motion that is relevant for a laser-cutting application. The same ILC design variables are used for all six motors and the learning is stopped after five iterations, which is motivated in practice by experimental results. Performance on the motor side and the corresponding performance on the arm side, using a laser-measurement system, is studied. Even though the result on the motor side is good, it is no guarantee that the errors on the arm side are decreasing. One has to be very careful when dealing with resonant systems when the controlled variable is not directly measured and included in the algorithm. This indicates that the results on the arm side may be improved when an estimate of, for example, the tool position is used in the ILC algorithm.

Bra modeller av industrirobotar behövs i en mängd olika tillämpningar, som till exempel mekanisk design, simulering av prestanda, reglering, diagnos, övervakning och offline-programmering. I första delen av avhandlingen studeras modellering av framåtkinematiken för en seriell robot och implementeringen av ett modelleringsverktyg, en toolbox, för kinematikmodellering i Maple beskrivs ingående.

Ett antal möjliga tillämpningar för toolboxen kan nämnas. Ett exempel är att med hjälp av extra sensorer monterade på roboten och ett så kallat extended Kalmanfilter förbättra skattningen av positionen och orienteringen för robotverktyget. De kinematiska ekvationerna och sambanden som behövs för extended Kalmanfiltret kan beräknas med hjälp av modelleringsverktyget. Reglering genom iterativ inlärning - iterative learning control, ILC - där en skattning av verktygspositionen används, kan sedan förbättra robotens prestanda.

Andra delen av avhandlingen är tillägnad ILC. Det är en reglermetod som är användbar när roboten utför en repetitiv rörelse som startar från samma initialvillkor varje gång. Algoritmen kompenserar för de repetitiva felen genom att addera en korrektionsterm till referenssignalen. Studier där ILC är tillämpad på en verklig industriell plattform är mindre vanligt i litteraturen, vilket motiverar arbetet i avhandlingen.

Ett första ordningens ILC-filter med iterationsoberoende operatorer används. ILC-algoritmen är framtagen enligt ett heuristiskt tillvägagångssätt, vilket resulterar i en ickekausal algoritm. I en simuleringsstudie med en flexibel tvåmassemodell som en förenklad linjär modell av en enskild robotled, används en ILC-algoritm baserad endast på motorvinkelmätningar. Det visar sig att när ett modellfel introduceras i sambandet mellan arm- och motorvinkelreferensen, är det inte säkert att felet på armsidan minskar så mycket som felet på motorsidan, eller minskar överhuvudtaget.

I experiment tillämpas ILC-algoritmen på en stor kommersiell industrirobot som utför en cirkelrörelse som är relevant för en laserskärningstillämpning. Samma designvariabler används för alla sex motorerna och inlärningen stoppas efter fem iterationer, vilket är motiverat i praktiken genom experimentella resultat. Prestanda på motorsidan studeras, och motsvarande prestanda på armsidan mäts med ett lasermätsystem. Trots goda resultat på motorsidan finns det inga garantier för minskande fel på armsidan. Stor försiktighet krävs när experimenten innefattar ett resonant system där den reglerade variabeln inte är mätt explicit och inkluderad i algoritmen. Detta visar på möjligheten att förbättra resultaten på armsidan då en skattning av till exempel verktygspositionen används i ILC-algoritmen.

Report code: LiU-TEK-LIC-2008:1.

Thesis (Elec.E.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998.
Includes bibliographical references (p. 90-94).
Stability and robustness are two important performance requirements for a dynamic walking robot. Learning and adaptation can improve stability and robustness. This thesis explores such an adaptation capability through the use of neural networks. Three neural network models (BP, CMAC and RBF networks) are studied. The RBF network is chosen as best, despite its weakness at covering high dimensional input spaces. To overcome this problem, a self-organizing scheme of data clustering is explored. This system is applied successfully in a biped walking robot system with a supervised learning mode. Generalized Virtual Model Control (GVMC) is also proposed in this thesis, which is inspired by a bio-mechanical model of locomotion, and is an extension of ordinary Virtual Model Control. Instead of adding virtual impedance components to the biped skeletal system in virtual Cartesian space, GVMC uses adaptation to approximately reconstruct the dynamics of the biped. The effectiveness of these approaches is proved both theoretically and experimentally (in simulation).
by Jianjuen Hu.
Elec.E.

Quelle sont les points communs entre un robot humanoïde et un quadrimoteur ? Et bien, pas grand-chose… Cette thèse est donc dédiée au développement d’algorithmes permettant de contrôler un robot de manière dynamique tout en restant générique par rapport au model du robot et à la tâche que l’on cherche à résoudre. Le contrôle optimal numérique est pour cela un bon candidat. Cependant il souffre de plusieurs difficultés comme un nombre important de paramètres à ajuster et des temps de calcul relativement élevés. Ce document présente alors plusieurs améliorations permettant d’atténuer ces difficultés. D’un côté, l’ordonnancement des différentes tâches sous la forme d’une hiérarchie et sa résolution avec un algorithme adapté permet de réduire le nombre de paramètres à ajuster. D’un autre côté, l’utilisation de l’apprentissage automatique afin d’initialiser l’algorithme d’optimisation ou de générer un modèle simplifié du robot permet de fortement diminuer les temps de calcul
What are the common characteristics of humanoid robots and quadrotors? Well, not many… Therefore, this thesis focuses on the development of algorithms allowing to dynamically control a robot while staying generic with respect to the model of the robot and the task that needs to be solved. Numerical optimal control is good candidate to achieve such objective. However, it suffers from several difficulties such as a high number of parameters to tune and a relatively important computation time. This document presents several ameliorations allowing to reduce these problems. On one hand, the tasks can be ordered according to a hierarchy and solved with an appropriate algorithm to lower the number of parameters to tune. On the other hand, machine learning can be used to initialize the optimization solver or to generate a simplified model of the robot, and therefore can be used to decrease the computation time

Ce travail apporte de nouvelles méthodes pour la modélisation cinématique et le contrôle de manipulateurs continus et déformables, fondées sur la méthodes des éléments finis. À la différence des manipulateurs rigides, les manipulateurs continus et déformables engendrent leurs mouvements en se déformant, c'est pourquoi la méthode proposée prend en compte les déformations mécaniques pour mieux décrire la cinématique de ce genre de robots. Cette méthode n'apporte pas de solution analytique, mais une approximation numérique, par des méthodes dérivées de la mécanique numérique. La méthodologie est appliquée à un manipulateur continu, appelé "Compact Bionic Handling Assistant (CBHA)". Une stratégie de commande en boucle fermée, fondée sur l'allocation du contrôle, est également présentée. Les modèles et contrôleurs sont validés expérimentalement
This work provides new methods for the kinematic modeling and control of soft, continuum manipulators based on the Finite Element Method. Contrary to the case of rigid manipulators, soft and continuum manipulators generate their motion by deformation, therefore, the proposed methodology accounts for the deformation mechanics to better describe the kinematics of these type of robots. This methodology does not produce analytic solutions, instead, a numerical approximation is provided by methods derived from Computational Mechanics. The methodology is applied to a continuum manipulator, namely, the Compact Bionic Handling Assistant (CBHA). A closed-loop control scheme based on control allocation is also presented. The models and controller are validated experimentally

Research in soft robot hardware has led to the development of platforms that allow for safer performance when working in uncertain or dynamic environments. The potential of these platforms is limited by the lack of proper dynamic models to describe or controllers to operate them. A common difficulty associated with these soft robots is a representation for torque, the common electromechanical relation seen in motors does not apply. In this thesis, several different torque models are presented and used to construct linear state-space models. The control limitations on soft robots are induced by natural compliance inherent to the hardware. This inherent compliance results in soft robots that are commonly underdamped and present significant oscillations when accelerated quickly. These oscillations can be mitigated through model-based controllers which can anticipate these oscillations. In this thesis, multiple model predictive controllers are implemented with the torque models produced and results are presented for an inflatable single-DoF pneumatically actuated soft robot. Larger, multi-DoF, soft robots present additional issues with control, where flexibility in one joint impacts control in others. In this thesis a preliminary method and results for controlling multiple joints on an inflatable multi-DoF pneumatically actuated soft robot are presented. While model predictive controllers are capable, their control commands are defined by solving an optimization constrained by model dynamics. This optimization relies on minimizing the cost of a user-defined objective function. This objective function contains a series of weights, which allow the user to tune the importance of each component in the objective function. As there are no calculations that can be performed to tune model predictive controllers to achieve superior control performance, they often need to be tuned tediously by a skilled operator. In this thesis, a method for automated discrete performance identification and model predictive controller weight tuning is presented. This thesis constructs multiple state-space models for single- and multi-DoF underdamped, antagonistic, pneumatically actuated soft robots and shows that these models can be used with model predictive control, tuned for performance, to achieve accurate joint position control.

Quadruped robots can traverse a multitude of terrains with greater ease when compared to wheeled robots. Traditional rigid quadruped robots possess severe limitations as they lack structural compliance. Most of the existing soft quadruped robots are tethered and are actuated using pneumatics, which is a low grade energy source and lacks viability for long endurance robots. The work in this thesis proposes the development of a continuum actuator driven quadruped robot which can provide compliance while being un-tethered and electro-mechanically driven. In this work, continuum actuators are developed using mostly 3D printed parts. Additionally, the closed loop control of continuum actuators for walking is developed. Linear Quadratic Regulator (LQR) and pole placement based methods for controller synthesis were evaluated and LQR was determined to be better when minimizing the actuator effort and deviation from set-point. These continuum actuators are composed together to form a quadruped. Gait analyses on the quadruped were conducted and legs of the quadruped were able to trace the gaits for walking and galloping.
Fyrfotarobotar kan lättare korsa en mängd olika terränger jämfört med hjulrobotar. Traditionella styva fyrfotarobotar har kraftiga begränsningar då de saknar strukturell följsamhet. De flesta befintliga mjuka fyrbenta robotar är kopplade till en eller flera kablar och drivs av pneumatik, vilket är en lågkvalitativ energikälla och lämpar sig inte för robotar med lång uthållighet. Arbetet i denna avhandling föreslår utvecklingen av en continuum ställdonsdriven fyrfotarobot, som ger följsamhet samtidigt som den ¨ar frånkopplad och elektromekaniskt driven. I detta arbete framställs continuum ställdon med mestadels 3D-printade delar. Dessutom utvecklas dessa ställdons slutna kontrolloop för gång. Linjärkvadratisk regulator (LQR) och metoder baserade på polplacering utvärderades för styrsyntes, och det fastställdes att LQR presterade bättre när man minimerar ställdonets ansträngning samt avvikelse från referensvärde. Continuum ställdon sammansattes för att bilda en fyrbent robot. Gånganalyser utfördes på roboten och dess ben kunde följa gång- och galopprörelser.

La simulation médicale est un domaine de recherche de plus en plus actif. Malgré les avancées observées ces dernières années, le modèle complet du patient virtuel reste un objectif ambitieux. Il existe encore de nombreuses opportunités de recherche, notamment concernant la modélisation mécanique des conditions aux limites. Jusqu'à présent, la majorité des travaux était consacrée à la simulation d'organes, ces derniers étant généralement simulés seuls. Cette situation pose problème car l'influence des organes environnants sur les conditions aux limites est négligée. Ces interactions peuvent être complexes, impliquant des contacts mais aussi des liaisons mécaniques dues aux tissus conjonctifs. Ainsi, les influences mutuelles entre les structures anatomiques sont souvent simplifiées, diminuant le réalisme des simulations. Cette thèse visé à étudier l'importance des tissus conjonctifs, et plus particulièrement d'une bonne modélisation des conditions aux limites. Dans ce but, le rôle des ligaments lors d'une intervention chirurgicale par laparoscopie a été étudié. Afin d'améliorer le réalisme des simulations, un modèle mécanique dédié aux tissus conjonctifs, basée sur la mécanique des milieux continus et un ensemble de nœuds à 6 degrés de liberté a été développée. En outre, les travaux sur les tissus conjonctifs ont donné lieu à la mise au point d'une méthode de modélisation utilisée dans le cadre des robots déformables. Cette méthode permet un contrôle précis, et temps-réel, d'un bras robotisé déformable. L'utilisation de nœuds orientables a donné lieu à un modèle à nombre de degrés de liberté réduit, permettant de reproduire le comportement de structures plus complexes
Despite the promising advances done in medical simulation, the complete virtual patient’s model is yet to come. There are still many avenues for improvements, especially concerning the mechanical modeling of boundary conditions.So far, most of the work has been dedicated to organs simulation, which are generally simulated alone. This raises a real problem as the role of the surrounding organs in boundary conditions is neglected. However, these interactions can be complex, involving contacts but also mechanical links provided by layers of soft tissues known as connective tissues. As a consequence, the mutual influences between the anatomical structures are generally simplified, weakening realism of simulations.This thesis aims at studying the importance of the connective tissues, and especially of a proper modeling of the boundary conditions. To this end, the role of the ligaments during laparoscopic liver surgery has been investigated. In order to enhance the simulations’ realism, a mechanical model dedicated to the connective tissues has been worked out. This has led to the development of a physically-based method relying on material points that can, not only translate, but also rotate themselves. The goal of this model is to enable the simulation of multiple organs linked by complex interactions.In addition, the work on the connective tissues model has been derived to be used in soft robotics. The principle of relying on orientable material points has been used to developed a reduced model that can reproduce the behavior of more complex structures. The objective of this work is to provide the means to control – in real-time – a soft robot made of a deformable arm

In this document, we propose and examine the novel use of a learning mechanism between the reactive and deliberative layers of a hybrid robot control architecture. Balancing the need to achieve complex goals and meet real-time constraints, many modern mobile robot navigation control systems make use of a hybrid deliberative-reactive architecture. In this paradigm, a high-level deliberative layer plans routes or actions toward a known goal, based on accumulated world knowledge. A low-level reactive layer selects motor commands based on current sensor data and the deliberative layer's plan. The desired system-level effect of this architecture is that the robot is able to combine complex reasoning toward global objectives with quick reaction to local constraints. Implicit in this type of architecture, is the assumption that both layers are using the same model of the robot's capabilities and constraints. It may happen, for example, due to differences in representation of the robot's kinematic constraints, that the deliberative layer creates a plan that the reactive layer cannot follow. This sort of conflict may cause a degradation in system-level performance, if not complete navigational deadlock. Traditionally, it has been the task of the robot designer to ensure that the layers operate in a compatible manner. However, this is a complex, empirical task. Working to improve system-level performance and navigational robustness, we propose introducing a learning mechanism between the reactive layer and the deliberative layer, allowing the deliberative layer to learn a model of the reactive layer's execution of its plans. First, we focus on detecting this inter-layer conflict, and acting based on a corrected model. This is demonstrated on a physical robotic platform in an unstructured outdoor environment. Next, we focus on learning a model to predict instances of inter-layer conflict, and planning to act with respect to this model. This is demonstrated using supervised learning in a physics-based simulation environment. Results and algorithms are presented.

La technologie de fabrication additive a été identifiée comme l'une des innovations numériques majeures qui a révolutionné non seulement le domaine de l'industrie, mais aussi celui de la construction. D'un point de vue de recherche, la fabrication additive reste un sujet d’actualité. C’est un procédé automatisé de dépôt de matériaux couche par couche afin d'imprimer des maisons ou des structures de petites dimensions pour un montage sur site. Dans la fabrication additive, l'étape de dépôt des matériaux est généralement suivie d'une étape de contrôle de la qualité d'impression. Cependant, le contrôle de qualité des objets imprimés ayant des surfaces funiculaires est parfois complexe à réaliser avec des robots rigides, ne pouvant atteindre des zones mortes. Dans cette thèse, un manipulateur souple et hyper-redondant a été modélisé et commandé cinématiquement, placé comme un effecteur d'un manipulateur rigide et mobile, afin d'effectuer une inspection des structures imprimées par des techniques de la fabrication additive. En effet, les manipulateurs souples peuvent fléchir et du coup suivre la forme géométrique de surfaces funiculaires. Ainsi, une approche hybride a été proposée pour modéliser la cinématique du robot souple et hyper-redondant, combinant une approche analytique pour la génération des équations cinématiques et une méthode qualitative à base des réseaux de neurones pour la résolution de ces dernières. Les performances de l'approche proposée sont validées à travers des expériences réalisées sur le robot "compact bionic handling arm" (cbha)
Additive manufacturing technology has been identified as one of the major digital innovations that has revolutionized not only industry, but also building. From a research point of view, additive manufacturing remains a very relevant topic. It is an automated process for depositing materials layer by layer to print houses or small structures for on-site assembly. In additive manufacturing processes, the deposition of materials is generally followed by a printing quality control step. However, the geometry of structures printed with funicular surfaces is sometimes complex, as robots with rigid structures cannot reach certain areas of the structure to be inspected. In this thesis, a flexible and highly redundant manipulator equipped with a camera is attached to the end-effector of a mobile manipulator robot for the quality inspection process of the printed structures. Indeed, soft manipulators can bend along their surounded 3D objects; and this inherent flexibility makes them suitable for navigation in crowded environments. As the number of controlled actuators is greater than the dimension of the workspace, this thesis can be summarized as a trajectory tracking of hyper-redundant robots. In this thesis, a hybrid approach that combines the advantages of model-based approaches and learning-based approaches is developed to model and solve the kinematics of soft and hyper-redundant manipulators. The principle is to develop mathematical models with reasonable assumptions, and to improve their accuracy through learning processes. The performance of the proposed approach is validated by performing a series of simulations and experiments applied to the compact bionic handling arm (cbha) robot

Achieving robust and reliable autonomous operation even in complex unstructured environments is a central goal of field robotics. As the environments and scenarios to which robots are applied have continued to grow in complexity, so has the challenge of properly defining preferences and tradeoffs between various actions and the terrains they result in traversing. These definitions and parameters encode the desired behavior of the robot; therefore their correctness is of the utmost importance. Current manual approaches to creating and adjusting these preference models and cost functions have proven to be incredibly tedious and time-consuming, while typically not producing optimal results except in the simplest of circumstances. This thesis presents the development and application of machine learning techniques that automate the construction and tuning of preference models within complex mobile robotic systems. Utilizing the framework of inverse optimal control, expert examples of robot behavior can be used to construct models that generalize demonstrated preferences and reproduce similar behavior. Novel learning from demonstration approaches are developed that offer the possibility of significantly reducing the amount of human interaction necessary to tune a system, while also improving its final performance. Techniques to account for the inevitability of noisy and imperfect demonstration are presented, along with additional methods for improving the efficiency of expert demonstration and feedback. The effectiveness of these approaches is confirmed through application to several real world domains, such as the interpretation of static and dynamic perceptual data in unstructured environments and the learning of human driving styles and maneuver preferences. Extensive testing and experimentation both in simulation and in the field with multiple mobile robotic systems provides empirical confirmation of superior autonomous performance, with less expert interaction and no hand tuning. These experiments validate the potential applicability of the developed algorithms to a large variety of future mobile robotic systems.

In this thesis, an intelligent controller architecture for gait selection of a serpentine robot intended to be used in search and rescue tasks is designed, developed and simulated. The architecture is independent of the configuration of the robot and the robot is allowed to make different kind of movements, similar to grasping. Moreover, it is applicable to parallel processing in several aspects and it is an implementation of a controller network on robot segment network. In the architecture several behaviors are defined for each of the segments. Every behavior is realized in the form of Fuzzy Actor-Critic Learning agents based on fuzzy networks and reinforcement learning. Each segment controller determines the next suitable position in the sensory space acquired using ultrasound sensors, a genetic algorithm implementation then tries to find the change of the joint angles to achieve the desired movement in a given amount of time. This allows optimization on different criteria, during motion. Simulations are performed and presented to introduce the efficiency of the developed controller architecture. Moreover a simplified mathematical analysis is performed to gain insight of the controller dynamics.

Flexible manufacturing systems (FMS) are production systems consisting of identical multipurpose numerically controlled machines (workstations), automated material handling system, tools, load and unload stations, inspection stations, storage areas and a hierarchical control system. The latter has the task of coordinating and integrating all the components of the whole system for automatic operations. A particular characteristic of FMSs is their complexity along with the difficulties in building analytical models that capture the system in all its important aspects. Thus optimal control strategies, or at least good ones, are hard to find and the full potential of manufacturing systems is not completely exploited.

The complexity of these systems induces a division of the control approaches based on the time frame they are referred to: long, medium and short term. This thesis addresses the short-term control of a FMS. The objective is to define control strategies, based on system state feedback, that fully exploit the flexibility built into those systems. Difficulties arise since the metrics that have to be minimized are often conflicting and some kind of trade-offs must be made using "common sense". The problem constraints are often expressed in a rigid and "crisp" way while their nature is more "fuzzy" and the search for an analytical optimum does not always reflect production needs. Indeed, practical and production oriented approaches are more geared toward a good and robust solution.

This thesis addresses the above mentioned problems proposing a fuzzy scheduler and a reinforcement-learning approach to tune its parameters. The learning procedure is based on evolutionary programming techniques and uses a performance index that contains the degree of satisfaction of multiple and possibly conflicting objectives. This approach addresses the design of the controller by means of language directives coming from the management, thus not requiring any particular interface between management and designers.

The performances of the fuzzy scheduler are then compared to those of commonly used heuristic rules. The results show some improvement offered by fuzzy techniques in scheduling that, along with ease of design, make their applicability promising. Moreover, fuzzy techniques are effective in reducing system congestion as is also shown by slower performance degradation than heuristics for decreasing inter- arrival time of orders. Finally, the proposed paradigm could be extended for on-line adaptation of the scheduler, thus fully responding to the flexibility needs of FMSs.

Master of Science

Over the last few years autonomous driving had an increasingly strong impact on the automotive industry. This created an increased need for artificial intelligence algo- rithms which allow for computers to make human-like decisions. However, a compro- mise between the computational power drawn by these algorithms and their subsequent performance must be found to fulfil production requirements. In this thesis incremental deep learning strategies are used for the control of a mobile robot such as a four wheel steering vehicle. This strategy is similar to the human approach of learning. In many small steps the vehicle learns to achieve a specific goal. The usage of incremental training leads to growing knowledge-base within the system. It also provides the opportunity to use older training achievements to improve the system, when more training data is available. To demonstrate the capabilities of such an algorithm, two different models have been formulated. First, a more simple model with counter wheel steering, and second, a more complex, nonlinear model with independent steering. These two models are trained incrementally to follow different types of trajectories. Therefore an algorithm was established to generate useful initial points. The incremental steps allow the robot to be positioned further and further away from the desired trajectory in the environ- ment. Afterwards, the effects of different trajectory types on model behaviour are investigated by over one thousand simulation runs. To do this, path planning for straight lines and circles are introduced. This work demonstrates that even simulations with simple network structures can have high performance.

Aquesta tesi proposa l'ús d'un seguit de tècniques pel control a alt nivell d'un robot autònom i també per l'aprenentatge automàtic de comportaments. L'objectiu principal de la tesis fou el de dotar d'intel·ligència als robots autònoms que han d'acomplir unes missions determinades en entorns desconeguts i no estructurats. Una de les premisses tingudes en compte en tots els passos d'aquesta tesis va ser la selecció d'aquelles tècniques que poguessin ésser aplicades en temps real, i demostrar-ne el seu funcionament amb experiments reals. El camp d'aplicació de tots els experiments es la robòtica submarina.
En una primera part, la tesis es centra en el disseny d'una arquitectura de control que ha de permetre l'assoliment d'una missió prèviament definida. En particular, la tesis proposa l'ús de les arquitectures de control basades en comportaments per a l'assoliment de cada una de les tasques que composen la totalitat de la missió. Una arquitectura d'aquest tipus està formada per un conjunt independent de comportaments, els quals representen diferents intencions del robot (ex.: "anar a una posició", "evitar obstacles",...). Es presenta una recerca bibliogràfica sobre aquest camp i alhora es mostren els resultats d'aplicar quatre de les arquitectures basades en comportaments més representatives a una tasca concreta. De l'anàlisi dels resultats se'n deriva que un dels factors que més influeixen en el rendiment d'aquestes arquitectures, és la metodologia emprada per coordinar les respostes dels comportaments. Per una banda, la coordinació competitiva és aquella en que només un dels comportaments controla el robot. Per altra banda, en la coordinació cooperativa el control del robot és realitza a partir d'una fusió de totes les respostes dels comportaments actius. La tesis, proposa un esquema híbrid d'arquitectura capaç de beneficiar-se dels principals avantatges d'ambdues metodologies.
En una segona part, la tesis proposa la utilització de l'aprenentatge per reforç per aprendre l'estructura interna dels comportaments. Aquest tipus d'aprenentatge és adequat per entorns desconeguts i el procés d'aprenentatge es realitza al mateix temps que el robot està explorant l'entorn. La tesis presenta també un estat de l'art d'aquest camp, en el que es detallen els principals problemes que apareixen en utilitzar els algoritmes d'aprenentatge per reforç en aplicacions reals, com la robòtica. El problema de la generalització és un dels que més influeix i consisteix en permetre l'ús de variables continues sense augmentar substancialment el temps de convergència. Després de descriure breument les principals metodologies per generalitzar, la tesis proposa l'ús d'una xarxa neural combinada amb l'algoritme d'aprenentatge per reforç Q_learning. Aquesta combinació proporciona una gran capacitat de generalització i una molt bona disposició per aprendre en tasques de robòtica amb exigències de temps real. No obstant, les xarxes neurals són aproximadors de funcions no-locals, el que significa que en treballar amb un conjunt de dades no homogeni es produeix una interferència: aprendre en un subconjunt de l'espai significa desaprendre en la resta de l'espai. El problema de la interferència afecta de manera directa en robòtica, ja que l'exploració de l'espai es realitza sempre localment. L'algoritme proposat en la tesi té en compte aquest problema i manté una base de dades representativa de totes les zones explorades. Així doncs, totes les mostres de la base de dades s'utilitzen per actualitzar la xarxa neural, i per tant, l'aprenentatge és homogeni.
Finalment, la tesi presenta els resultats obtinguts amb la arquitectura de control basada en comportaments i l'algoritme d'aprenentatge per reforç. Els experiments es realitzen amb el robot URIS, desenvolupat a la Universitat de Girona, i el comportament après és el seguiment d'un objecte mitjançant visió per computador. La tesi detalla tots els dispositius desenvolupats pels experiments així com les característiques del propi robot submarí. Els resultats obtinguts demostren la idoneïtat de les propostes en permetre l'aprenentatge del comportament en temps real. En un segon apartat de resultats es demostra la capacitat de generalització de l'algoritme d'aprenentatge mitjançant el "benchmark" del "cotxe i la muntanya". Els resultats obtinguts en aquest problema milloren els resultats d'altres metodologies, demostrant la millor capacitat de generalització de les xarxes neurals.

With current state-of-the-art approaches, development of a single mobile manipulation capability can be a labor-intensive process that presents an impediment to the creation of general purpose household robots. At the same time, we expect that involving a larger community of non-roboticists can accelerate the creation of new novel behaviors. We introduce the use of a software authoring environment called ROS Commander (ROSCo) allowing end-users to create, refine, and reuse robot behaviors with complexity similar to those currently created by roboticists. Akin to Photoshop, which provides end-users with interfaces for advanced computer vision algorithms, our environment provides interfaces to mobile manipulation algorithmic building blocks that can be combined and configured to suit the demands of new tasks and their variations. As our system can be more demanding of users than alternatives such as using kinesthetic guidance or learning from demonstration, we performed a user study with 11 able-bodied participants and one person with quadriplegia to determine whether computer literate non-roboticists will be able to learn to use our tool. In our study, all participants were able to successfully construct functional behaviors after being trained. Furthermore, participants were able to produce behaviors that demonstrated a variety of creative manipulation strategies, showing the power of enabling end-users to author robot behaviors. Additionally, we introduce how using autonomous robot learning, where the robot captures its own training data, can complement human authoring of behaviors by freeing users from the repetitive task of capturing data for learning. By taking advantage of the robot's embodiment, our method creates classifiers that predict using visual appearances 3D locations on home mechanisms where user constructed behaviors will succeed. With active learning, we show that such classifiers can be learned using a small number of examples. We also show that this learning system works with behaviors constructed by non-roboticists in our user study. As far as we know, this is the first instance of perception learning with behaviors not hand-crafted by roboticists.

Thesis (Ph. D.)--Mechanical Engineering, Georgia Institute of Technology, 2008.
Committee Chair: Steve M. Potter; Committee Member: Eric Schumacher; Committee Member: Robert J. Butera; Committee Member: Stephan P. DeWeerth; Committee Member: Thomas D. DeMarse.

Cette thèse s’intéresse à la modélisation et à la commande de robots déformables, c’est à dire de robots dont le mouvement se fait par déformation. Nous nous intéressons à la conception de lois de contrôle en boucle fermée répondant aux besoins spécifiques du contrôle dynamique de robots déformables, sans restrictions fortes sur leur géométrie. La résolution de ce défi soulève des questions théoriques qui nous amènent au deuxième objectif de cette thèse: développer de nouvelles stratégies pour étudier les systèmes de grandes dimensions. Ce manuscrit couvre l’ensemble du développement des lois de commandes, de l’étape de modélisation à la validation expérimentale. Outre les études théoriques, différentes plateformes expérimentales sont utilisées pour valider les résultats. Des robots déformables actionnés par câble et par pression sont utilisés pour tester les algorithmes de contrôle. A travers ces différentes plateformes, nous montrons que la méthode peut gérer différents types d’actionnement, différentes géométries et propriétés mécaniques. Cela souligne l’un des intérêts de la méthode, sa généricité. D’un point de vue théorique, les systèmes dynamiques à grande dimensions ainsi que les algorithmes de réduction de modèle sont étudiés. En effet, modéliser des structures déformables implique de résoudre des équations issues de la mécanique des milieux continus, qui sont résolues à l’aide de la méthode des éléments finis (FEM). Ceci fournit un modèle précis des robots mais nécessite de discrétiser la structure en un maillage composé de milliers d’éléments, donnant lieu à des systèmes dynamiques de grandes dimensions. Cela conduit à travailler avec des modèles de grandes dimensions, qui ne conviennent pas à la conception d’algorithmes de contrôle. Une première partie est consacrée à l’étude du modèle dynamique à grande dimension et de son contrôle, sans recourir à la réduction de modèle. Nous présentons un moyen de contrôler le système à grande dimension en utilisant la connaissance d’une fonction de Lyapunov en boucle ouverte. Ensuite, nous présentons des algorithmes de réduction de modèle afin de concevoir des contrôleurs de dimension réduite et des observateurs capables de piloter ces robots déformables. Les lois de contrôle validées sont basées sur des modèles linéaires, il s’agit d’une limitation connue de ce travail car elle contraint l’espace de travail du robot. Ce manuscrit se termine par une discussion qui offre un moyen d’étendre les résultats aux modèles non linéaires. L’idée est de linéariser le modèle non linéaire à grande échelle autour de plusieurs points de fonctionnement et d’interpoler ces points pour couvrir un espace de travail plus large
This thesis focuses on the design of closed-loop control laws for the specific needs of dynamic control of soft robots, without being too restrictive regarding the robots geometry. It covers the entire development of the controller, from the modeling step to the practical experimental validation. In addition to the theoretical studies, different experimental setups are used to illustrate the results. A cable-driven soft robot and a pressurized soft arm are used to test the control algorithms. Through these different setups, we show that the method can handle different types of actuation, different geometries and mechanical properties. This emphasizes one of the interests of the method, its genericity. From a theoretical point a view, large-scale dynamical systems along with model reduction algorithms are studied. Indeed, modeling soft structures implies solving equations coming from continuum mechanics using the Finite Element Method (FEM). This provides an accurate model of the robots but it requires to discretize the structure into a mesh composed of thousands of elements, yielding to large-scale dynamical systems. This leads to work with models of large dimensions, that are not suitable to design control algorithms. A first part is dedicated to the study of the large-scale dynamic model and its control, without using model reduction. We present a way to control the large-scale system using the knowledge of an open-loop Lyapunov function. Then, this work investigates model reduction algorithms to design low order controllers and observers to drive soft robots. The validated control laws are based on linear models. This is a known limitation of this work as it constrains the guaranteed domain of the controller. This manuscript ends with a discussion that offers a way to extend the results towards nonlinear models. The idea is to linearize the large-scale nonlinear model around several operating points and interpolate between these points to cover a wider workspace

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

Récemment, il a été possible d’observer l’accélération du déploiement de robots dans des domaines dépassant l’habituel cadre industriel et manufacturier. Cependant, pour la majorité des tâches autonomes, la définition d’un modèle analytique ou la recherche d’une solution optimale requiert des ressources rarement accessibles en temps-réel, favorisant par conséquent des techniques basées sur l’apprentissage. Ces dernières, présentant l’avantage de ne pas nécessiter de modèle ainsi que de présenter un temps de calcul en exécution relativement constant, permettent d’appréhender des configurations et tâches hautement complexes. Les techniques basées sur les données font cependant état de temps d’entraînement considérables, nécessitant fréquemment des millions d’exemples et d’interaction avec leur environnement pour construire des politiques de contrôle admissibles. Le transfert de connaissance entre modèles est crucial pour le déploiement à grande échelle des méthodes d’apprentissage et bien que des stratégies de transmission aient été au coeur des récentes préoccupations, elles sont essentiellement dirigées vers les domaines de vision ou de compréhension du langage et ne sont pas directement applicables à des problématiques de transfert de compétences entre robots présentant des structures cinématique différentes. Les travaux présentés dans ce manuscrit de thèse se focalisent précisément sur ce point et visent à déterminer dans quelle mesure la compréhension entre deux entités morphologiquement distinctes est possible. Cette question est explorée via l’introduction de deux paradigmes distincts: Task-Centered et Teacher-Centered. La famille de techniques dite Task-Centered est basée sur l’idée de la séparation du savoir-faire relatif àune tâche des stratégies de contrôle du robot. A la manière d’une notice d’instruction, un tel noyau indépendant peut par conséquent être passé à d’autres robots de morphologie différentes et idéalement rendre possible la réalisation de la tâche par ce nouvel agent. Dans ce contexte, plusieurs procédures de création de ce noyau sont proposées et évaluées sur un large panel d’environnements simulés. Cependant, en dépit des perspectives attractives de cette formulation, le caractère "onesize-fits-all" des techniques Task-Centered n’est pas exempte de limitations qui sont extensivement discutées. C’est dans ce contexte que les approches Teacher-Centered sont introduites. Poursuivant le même objectif, ces démarches innovantes font intervenir un agent expert à partir duquel le savoir relatif à la tâche doit être distillé dans l’agent cible. Pour ce faire, une métrique originale est utilisée pour contourner la différence de structure entre l’agent cible et l’agent expert et permettre,malgré cette distinction, la rétro-propagation de l’erreur afin d’optimiser l’agent
Recently, it has been possible to observe the acceleration of robot deployment in domains beyondthe usual industrial and manufacturing framework. However, for the majority of autonomoustasks, the definition of an analytical model or the search for an optimal (or acceptable) solutionrequires resources that are seldom available in real-time, thus favoring learning-based techniques.Indeed, learned models present the advantage of being model-free as well as having a constantexecution time, consequently enabling the realization of highly complex trajectories and tasks.Data-driven techniques, however, are hindered by considerable training time, frequently requiringmillions of examples and interactions with their environment to build acceptable control policies.As such, knowledge transfer, also known as transfer learning, between models is crucial for largescaledeployment of learned policies. Although transmission strategies have been the focus ofrecent concerns, they are mainly directed towards the fields of vision or language understandingand are not directly applicable to control environments where skill transfer is likely to happenbetween robots with different kinematic structures. The works presented in this thesis manuscriptfocus precisely on this point and aims at determining to what extent understanding between twomorphologically distinct entities is possible. This question is explored through the introductionof two distinct paradigms: Task-Centered and Teacher-Centered. The Task-Centered family oftechniques is based on the idea of the separation of task-related know-how from robot control policy.Such an independent kernel can therefore be passed on to other robots of different morphology andideally make it possible for the new agent to perform the task. In this context, several blueprintsfor creating this kernel are proposed and evaluated on a wide range of simulated environments.However, despite the attractive prospects of this formulation, the "one-size-fits-all" character ofTask-Centered techniques is not free of limitations which are extensively discussed. It is in thiscontext that Teacher-Centered approaches are introduced. Pursuing the same objective, theseinnovative procedure involve an expert agent from which the knowledge related to the task mustbe distilled into the target agent. To do this, an original metric is used to circumvent the structuraldifferences between the target agent and the expert agent and allow, despite this distinction, theerror to be back-propagated in order to optimize the agent

[EN] This thesis deals with two characteristic problems in visual feedback robot control: 1) sensor latency; 2) providing suitable trajectories for the robot and for the measurement in the image. All the approaches presented in this work are analyzed and implemented on a 6 DOF industrial robot manipulator or/and a wheeled robot. Focusing on the sensor latency problem, this thesis proposes the use of dual-rate high order holds within the control loop of robots. In this sense, the main contributions are: - Dual-rate high order holds based on primitive functions for robot control (Chapter 3): analysis of the system performance with and without the use of this multi-rate technique from non-conventional control. In addition, as consequence of the use of dual-rate holds, this work obtains and validates multi-rate controllers, especially dual-rate PIDs. - Asynchronous dual-rate high order holds based on primitive functions with time delay compensation (Chapter 3): generalization of asynchronous dual-rate high order holds incorporating an input signal time delay compensation component, improving thus the inter-sampling estimations computed by the hold. It is provided an analysis of the properties of such dual-rate holds with time delay compensation, comparing them with estimations obtained by the equivalent dual-rate holds without this compensation, as well as their implementation and validation within the control loop of a 6 DOF industrial robot manipulator. - Multi-rate nonlinear high order holds (Chapter 4): generalization of the concept of dual-rate high order holds with nonlinear estimation models, which include information about the plant to be controlled, the controller(s) and sensor(s) used, obtained from machine learning techniques. Thus, in order to obtain such a nonlinear hold, it is described a methodology non dependent of the machine technique used, although validated using artificial neural networks. Finally, an analysis of the properties of these new holds is carried out, comparing them with their equivalents based on primitive functions, as well as their implementation and validation within the control loop of an industrial robot manipulator and a wheeled robot. With respect to the problem of providing suitable trajectories for the robot and for the measurement in the image, this thesis presents the novel reference features filtering control strategy and its generalization from a multi-rate point of view. The main contributions in this regard are: - Reference features filtering control strategy (Chapter 5): a new control strategy is proposed to enlarge significantly the solution task reachability of robot visual feedback control. The main idea is to use optimal trajectories proposed by a non-linear EKF predictor-smoother (ERTS), based on Rauch-Tung-Striebel (RTS) algorithm, as new feature references for an underlying visual feedback controller. In this work it is provided both the description of the implementation algorithm and its implementation and validation utilizing an industrial robot manipulator. - Dual-rate Reference features filtering control strategy (Chapter 5): a generalization of the reference features filtering approach from a multi-rate point of view, and a dual Kalman-smoother step based on the relation of the sensor and controller frequencies of the reference filtering control strategy is provided, reducing the computational cost of the former algorithm, as well as addressing the problem of the sensor latency. The implementation algorithms, as well as its analysis, are described.
[ES] La presente tesis propone soluciones para dos problemas característicos de los sistemas robóticos cuyo bucle de control se cierra únicamente empleando sensores de visión artificial: 1) la latencia del sensor; 2) la obtención de trayectorias factibles tanto para el robot así como para las medidas obtenidas en la imagen. Todos los métodos propuestos en este trabajo son analizados, validados e implementados utilizando brazo robot industrial de 6 grados de libertad y/o en un robot con ruedas. Atendiendo al problema de la latencia del sensor, esta tesis propone el uso de retenedores bi-frequencia de orden alto dentro de los lazos de control de robots. En este aspecto las principales contribuciones son: -Retenedores bi-frecuencia de orden alto basados en funciones primitivas dentro de lazos de control de robots (Capítulo 3): análisis del comportamiento del sistema con y sin el uso de esta técnica de control no convencional. Además, como consecuencia del empleo de los retenedores, obtención y validación de controladores multi-frequencia, concretamente de PIDs bi-frecuencia. -Retenedores bi-frecuencia asíncronos de orden alto basados en funciones primitivas con compensación de retardos (Capítulo 3): generalización de los retenedores bi-frecuencia asíncronos de orden alto incluyendo una componente de compensación del retardo en la señal de entrada, mejorando así las estimaciones inter-muestreo calculadas por el retenedor. Se proporciona un análisis de las propiedades de los retenedores con compensación del retardo, comparándolas con las obtenidas por sus predecesores sin compensación, así como su implementación y validación en un brazo robot de 6 grados de libertad. -Retenedores multi-frecuencia no lineales de orden alto (Capítulo 4): generalización del concepto de retenedor bi-frecuencia de orden alto con modelos de estimación no lineales, los cuales incluyen información tanto de la planta a controlar, como del controlador(es) y sensor(es) empleado(s), obtenida a partir de técnicas de aprendizaje. Así pues, para obtener dicho retenedor no lineal, se describe una metodología independiente de la herramienta de aprendizaje utilizada, aunque validada con el uso de redes neuronales artificiales. Finalmente se realiza un análisis de las propiedades de estos nuevos retenedores, comparándolos con sus predecesores basados en funciones primitivas, así como su implementación y validación en un brazo robot de 6 grados de libertad y en un robot móvil con ruedas. Por lo que respecta al problema de generación de trayectorias factibles para el robot y para la medida en la imagen, esta tesis propone la nueva estrategia de control basada en el filtrado de la referencia y su generalización desde el punto de vista multi-frecuencial. -Estrategia de control basada en el filtrado de la referencia (Capítulo 5): una nueva estrategia de control se propone para ampliar significativamente el espacio de soluciones de los sistemas robóticos realimentados con sensores de visión artificial. La principal idea es utilizar las trayectorias óptimas obtenidas por una trayectoria predicha por un filtro de Kalman seguido de un suavizado basado en el algoritmo Rauch-Tung-Striebel (RTS) como nuevas referencias para un controlador dado. En este trabajo se proporciona tanto la descripción del algoritmo como su implementación y validación empleando un brazo robótico industrial. -Estrategia de control bi-frecuencia basada en el filtrado de la referencia (Capítulo 5): generalización de la estrategia de control basada en filtrado de la referencia desde un punto de vista multi-frecuencial, con un filtro de Kalman multi-frecuencia y un Kalman-smoother dual basado en la relación existente entre las frecuencias del sensor y del controlador, reduciendo así el coste computacional del algoritmo y, al mismo tiempo, dando solución al problema de la latencia del sensor. La validación se realiza utilizando un barzo robot industria asi
[CAT] La present tesis proposa solucions per a dos problemes característics dels sistemes robòtics el els que el bucle de control es tanca únicament utilitzant sensors de visió artificial: 1) la latència del sensor; 2) l'obtenció de trajectòries factibles tant per al robot com per les mesures en la imatge. Tots els mètodes proposats en aquest treball son analitzats, validats e implementats utilitzant un braç robot industrial de 6 graus de llibertat i/o un robot amb rodes. Atenent al problema de la latència del sensor, esta tesis proposa l'ús de retenidors bi-freqüència d'ordre alt a dins del llaços de control de robots. Al respecte, les principals contribucions son: - Retenidors bi-freqüència d'ordre alt basats en funcions primitives a dintre dels llaços de control de robots (Capítol 3): anàlisis del comportament del sistema amb i sense l'ús d'aquesta tècnica de control no convencional. A més a més, com a conseqüència de l'ús dels retenidors, obtenció i validació de controladors multi-freqüència, concretament de PIDs bi-freqüència. - Retenidors bi-freqüència asíncrons d'ordre alt basats en funcions primitives amb compensació de retards (Capítol 3): generalització dels retenidors bi-freqüència asíncrons d'ordre alt inclouen una component de compensació del retràs en la senyal d'entrada al retenidor, millorant així les estimacions inter-mostreig calculades per el retenidor. Es proporciona un anàlisis de les propietats dels retenidors amb compensació del retràs, comparant-les amb les obtingudes per el seus predecessors sense la compensació, així com la seua implementació i validació en un braç robot industrial de 6 graus de llibertat. - Retenidors multi-freqüència no-lineals d'ordre alt (Capítol 4): generalització del concepte de retenidor bi-freqüència d'ordre alt amb models d'estimació no lineals, incloent informació tant de la planta a controlar, com del controlador(s) i sensor(s) utilitzat(s), obtenint-la a partir de tècniques d'aprenentatge. Així doncs, per obtindre el retenidor no lineal, es descriu una metodologia independent de la ferramenta d'aprenentatge utilitzada, però validada amb l'ús de rets neuronals artificials. Finalment es realitza un anàlisis de les propietats d'aquestos nous retenidors, comparant-los amb els seus predecessors basats amb funcions primitives, així com la seua implementació i validació amb un braç robot de 6 graus de llibertat i amb un robot mòbil de rodes. Per el que respecta al problema de generació de trajectòries factibles per al robot i per la mesura en la imatge, aquesta tesis proposa la nova estratègia de control basada amb el filtrat de la referència i la seua generalització des de el punt de vista multi-freqüència. - Estratègia de control basada amb el filtrat de la referència (Capítol 5): una nova estratègia de control es proposada per ampliar significativament l'espai de solucions dels sistemes robòtics realimentats amb sensors de visió artificial. La principal idea es la d'utilitzar les trajectòries optimes obtingudes per una trajectòria predita per un filtre de Kalman seguit d'un suavitzat basat en l'algoritme Rauch-Tung-Striebel (RTS) com noves referències per a un control donat. En aquest treball es proporciona tant la descripció del algoritme així com la seua implementació i validació utilitzant un braç robòtic industrial de 6 graus de llibertat. - Estratègia de control bi-freqüència basada en el filtrat (Capítol 5): generalització de l'estratègia de control basada am filtrat de la referència des de un punt de vista multi freqüència, amb un filtre de Kalman multi freqüència i un Kalman-Smoother dual basat amb la relació existent entre les freqüències del sensor i del controlador, reduint així el cost computacional de l'algoritme i, al mateix temps, donant solució al problema de la latència del sensor. L'algoritme d'implementació d'aquesta tècnica, així com la seua validaci
Solanes Galbis, JE. (2015). MULTI-RATE VISUAL FEEDBACK ROBOT CONTROL [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/57951
TESIS

L’utilisation de systèmes robotiques pour la navigation dans des zones confinées pose des défis intéressants sur les thèmes de conception, de modélisation et de commande, particulièrement complexes pour les applications médicales. Dans ce contexte, nous introduisons un nouveau concept de robots continus, fortement prometteurs pour des applications biomédicales, dont la forme complexe, la dextérité et la capacité de miniaturisation constituent des avantages majeurs pour la navigation intra corporelle. Parmi cette classe, les robots à tubes concentriques (RTC), qui constituent notre point de départ, sont améliorés grâce à un actionnement embarqué innovant. Nos travaux s’articulent autour de deux thématiques aux frontières de l’état de l’art. D’une part, nous avons proposé une modélisation générique et conduit une analyse cinématique approfondie de robots continus basés sur l’architecture des RTC standards et ceux avec changement de courbure de leurs tubes dans deux variantes : courbures unidirectionnelle et bidirectionnelle. D’autre part, leur commande cartésienne en pose complète est introduite avec une validation expérimentale sur un prototype développé de RTC standard, ainsi que les simulations numériques d’une loi de commande comprenant la gestion de la redondance des RTC à changement de courbure. D’autre part, nous avons effectué la synthèse, la caractérisation et la mise en œuvre de micro-actionneurs souples basés sur les polymères électro-actifs (PEA), intégrés pour la première fois dans un robot continu.Ainsi, l’asservissement visuel d’un prototype de robot télescopique souple est proposé avec des précisions atteignant 0.21 mm sur différentes trajectoires
Major challenges need to be risen in order to perform navigation in confined spaces with robotic systems in terms of design, modeling, and control, particularly for biomedical applications. Indeed,the complex shape, dexterity, and miniaturization ability of continuum robots can help solving intracorporeal navigation problems. Within this class, we introduce a novel concept in order to augment the concentric tube robots (CTR) with embedded actuation. Our works hinge on two majorcutting-edge thematics. On the one hand, we address modeling and kinematics analysis of standard CTR as well as variable curvature CTR with their two varieties : single and double bending directions.Furthermore, we perform the experimental validation of Cartesian control of a CTR prototype, anda task hierarchy based control law for redundancy resolution of CTR with variable curvatures. Onthe other hand, we develop the synthesis, the characterization, and the integration of soft microactuatorsbased on electro-active polymers (EAP) for the first time in a continuum robot. Thus, thevisual servoing of a telescopic soft robot is performed with precisions down to 0.21 mm following different trajectories

L’environnement marin est un cadre très hostile pour la robotique. Il est fortement non-structuré, très incertain et inclut beaucoup de perturbations externes qui ne peuvent pas être facilement prédites ou modélisées. Dans ce travail, nous allons essayer de contrôler un véhicule sous-marin autonome (AUV) afin d’effectuer une tâche de suivi de points de cheminement, en utilisant un contrôleur basé sur de l’apprentissage automatique. L’apprentissage automatique a permis de faire des progrès impressionnants dans de nombreux domaines différents ces dernières années, et le sous-domaine de l’apprentissage profond par renforcement a réussi à concevoir plusieurs algorithmes très adaptés au contrôle continu de systèmes dynamiques. Nous avons choisi d’implémenter l’algorithme du Soft Actor-Critic (SAC), un algorithme d’apprentissage profond par renforcement régularisé en entropie permettant de simultanément remplir une tâche d’apprentissage et d’encourager l’exploration de l’environnement. Nous avons comparé un contrôleur basé sur le SAC avec un contrôleur Proportionnel-Intégral-Dérivé (PID) sur une tâche de suivi de points de cheminement et en utilisant des métriques de performance spécifiques. Tous ces tests ont été effectués en simulation grâce à l’utilisation de l’UUV Simulator. Nous avons décidé d’appliquer ces deux contrôleurs au RexROV 2, un véhicule sous-marin téléguidé (ROV) de forme cubique et à six degrés de liberté converti en AUV. Grâce à ces tests, nous avons réussi à proposer plusieurs contributions intéressantes telles que permettre au SAC d’accomplir un contrôle de l’AUV de bout en bout, surpasser le contrôleur PID en terme d’économie d’énergie, et réduire la quantité d’informations dont l’algorithme du SAC a besoin. De plus nous proposons une méthodologie pour l’entraînement d’algorithmes d’apprentissage profond par renforcement sur des tâches de contrôle, ainsi qu’une discussion sur l’absence d’algorithmes de guidage pour notre contrôleur d’AUV de bout en bout
The marine environment is a very hostile setting for robotics. It is strongly unstructured, very uncertain and includes a lot of external disturbances which cannot be easily predicted or modelled. In this work, we will try to control an autonomous underwater vehicle (AUV) in order to perform a waypoint tracking task, using a machine learning-based controller. Machine learning allowed to make impressive progress in a lot of different domain in the recent years, and the subfield of deep reinforcement learning managed to design several algorithms very suitable for the continuous control of dynamical systems. We chose to implement the Soft Actor-Critic (SAC) algorithm, an entropy-regularized deep reinforcement learning algorithm allowing to fulfill a learning task and to encourage the exploration of the environment simultaneously. We compared a SAC-based controller with a Proportional-Integral-Derivative (PID) controller on a waypoint tracking task and using specific performance metrics. All the tests were performed in simulation thanks to the use of the UUV Simulator. We decided to apply these two controllers to the RexROV 2, a six degrees of freedom cube-shaped remotely operated underwater vehicle (ROV) converted in an AUV. Thanks to these tests, we managed to propose several interesting contributions such as making the SAC achieve an end-to-end control of the AUV, outperforming the PID controller in terms of energy saving, and reducing the amount of information needed by the SAC algorithm. Moreover we propose a methodology for the training of deep reinforcement learning algorithms on control tasks, as well as a discussion about the absence of guidance algorithms for our end-to-end AUV controller

Les robots intégrés aux chaînes de production sont généralement isolés des ouvriers et ne prévoient pas d'interaction physique avec les humains. Dans le futur, le robot humanoïde deviendra un partenaire pour vivre ou travailler avec les êtres humains. Cette coexistence prévoit l'interaction physique et sociale entre le robot et l'être humain. En robotique humanoïde les futurs progrès dépendront donc des connaissances dans les mécanismes cognitifs présents dans les interactions interpersonnelles afin que les robots interagissent avec les humains physiquement et socialement. Un bon exemple d'interaction interpersonnelle est l'acte de la poignée de la main qui possède un rôle social très important. La particularité de cette interaction est aussi qu'elle est basée sur un couplage physique et social qui induit une synchronisation des mouvements et des efforts. L'intérêt d'étudier la poignée de main pour les robots consiste donc à élargir leurs propriétés comportementales pour qu'ils interagissent avec les humains de manière plus habituelle.Cette thèse présente dans un premier chapitre un état de l'art sur les travaux dans les domaines des sciences humaines, de la médecine et de la robotique humanoïde qui sont liés au phénomène de la poignée de main. Le second chapitre, est consacré à la nature physique du phénomène de poignée de main chez l'être humain par des mesures quantitatives des mouvements. Pour cela un système de mesures a été construit à l'Université Nationale Technique de Donetsk (Ukraine). Il est composé d'un gant instrumenté par un réseau de capteurs portés qui permet l'enregistrement des vitesses et accélérations du poignet et les forces aux points de contact des paumes, lors de l'interaction. Des campagnes de mesures ont permis de montrer la présence d'un phénomène de synchronie mutuelle précédé d'une phase de contact physique qui initie cette synchronie. En tenant compte de cette nature rythmique, un contrôleur à base de neurones rythmiques de Rowat-Selverston, intégrant un mécanisme d'apprentissage de la fréquence d'interaction, est proposé et etudié dans le troisième chapitre pour commander un bras robotique. Le chapitre quatre est consacré aux expériences d'interaction physique homme/robot. Des expériences avec un bras robotique Katana montrent qu'il est possible d'apprendre à synchroniser la rythmicité du robot avec celle imposée par une per-sonne lors d'une poignée de main grâce à ce modèle de contrôleur bio-inspiré. Une conclusion générale dresse le bilan des travaux menés et propose des perspectives
Automated production lines integrate robots which are isolated from workers, so there is no physical interaction between a human and robot. In the near future, a humanoid robot will become a part of the human environment as a companion to help or work with humans. The aspects of coexistence always presuppose physical and social interaction between a robot and a human. In humanoid robotics, further progress depends on knowledge of cognitive mechanisms of interpersonal interaction as robots physically and socially interact with humans. An illustrative example of interpersonal interaction is an act of a handshake that plays a substantial social role. The particularity of this form of interpersonal interaction is that it is based on physical and social couplings which lead to synchronization of motion and efforts. Studying a handshake for robots is interesting as it can expand their behavioral properties for interaction with a human being in more natural way. The first chapter of this thesis presents the state of the art in the fields of social sciences, medicine and humanoid robotics that study the phenomenon of a handshake. The second chapter is dedicated to the physical nature of the phenomenon between humans via quantitative measurements. A new wearable system to measure a handshake was built in Donetsk National Technical University (Ukraine). It consists of a set of several sensors attached to the glove for recording angular velocities and gravitational acceleration of the hand and forces in certain points of hand contact during interaction. The measurement campaigns have shown that there is a phenomenon of mutual synchrony that is preceded by the phase of physical contact which initiates this synchrony. Considering the rhythmic nature of this phenomenon, the controller based on the models of rhythmic neuron of Rowat-Selverston, with learning the frequency during interaction was proposed and studied in the third chapter. Chapter four deals with the experiences of physical human-robot interaction. The experimentations with robot arm Katana show that it is possible for a robot to learn to synchronize its rhythm with rhythms imposed by a human during handshake with the proposed model of a bio-inspired controller. A general conclusion and perspectives summarize and finish this work

One of the main tasks for an industrial robot is to move the end-effector in a predefined path with a specified velocity and acceleration. Different applications have different requirements of the performance. For some applications it is essential that the tracking error is extremely small, whereas other applications require a time optimal tracking. Independent of the application, the controller is a crucial part of the robot system. The most common controller configuration uses only measurements of the motor angular positions and velocities, instead of the position and velocity of the end-effector. The development of new cost optimised robots has introduced unwanted flexibilities in the joints and the links. The consequence is that it is no longer possible to get the desired performance and robustness by only measuring the motor angular positions.  This thesis investigates if it is possible to estimate the end-effector position using Bayesian estimation methods for state estimation, here represented by the extended Kalman filter and the particle filter. The arm-side information is provided by an accelerometer mounted at the end-effector. The measurements consist of the motor angular positions and the acceleration of the end-effector. In a simulation study on a realistic flexible industrial robot, the angular position performance is shown to be close to the fundamental Cramér-Rao lower bound. The methods are also verified in experiments on an ABB IRB4600 robot, where the dynamic performance of the position for the end-effector is significantly improved. There is no significant difference in performance between the different methods. Instead, execution time, model complexities and implementation issues have to be considered when choosing the method. The estimation performance depends strongly on the tuning of the filters and the accuracy of the models that are used. Therefore, a method for estimating the process noise covariance matrix is proposed. Moreover, sampling methods are analysed and a low-complexity analytical solution for the continuous-time update in the Kalman filter, that does not involve oversampling, is proposed.  The thesis also investigates two types of control problems. First, the norm-optimal iterative learning control (ILC) algorithm for linear systems is extended to an estimation-based norm-optimal ILC algorithm where the controlled variables are not directly available as measurements. The algorithm can also be applied to non-linear systems. The objective function in the optimisation problem is modified to incorporate not only the mean value of the estimated variable, but also information about the uncertainty of the estimate. Second, H∞ controllers are designed and analysed on a linear four-mass flexible joint model. It is shown that the control performance can be increased, without adding new measurements, compared to previous controllers. Measuring the end-effector acceleration increases the control performance even more. A non-linear model has to be used to describe the behaviour of a real flexible joint. An H∞-synthesis method for control of a flexible joint, with non-linear spring characteristic, is therefore proposed.
En av de viktigaste uppgifterna för en industrirobot är att förflytta verktyget i en fördefinierad bana med en specificerad hastighet och acceleration. Exempel på användningsområden för en industrirobot är bland annat bågsvetsning eller limning. För dessa typer av applikationer är det viktigt att banföljningsfelet är extremt litet, men även hastighetsprofilen måste följas så att det till exempel inte appliceras för mycket eller för lite lim. Andra användningsområden kan vara punktsvetsning av bilkarosser och paketering av olika varor. För dess applikationer är banföljningen inte det viktiga, istället kan till exempel en tidsoptimal banföljning krävas eller att svängningarna vid en inbromsning minimeras. Oberoende av applikationen är regulatorn en avgörande del av robotsystemet. Den vanligaste regulatorkonfigurationen använder bara mätningar av motorernas vinkelpositioner och -hastigheter, istället för positionen och hastigheten för verktyget, som är det man egentligen vill styra.  En del av utvecklingsarbetet för nya generationers robotar är att reducera kostnaden men samtidigt förbättra prestandan. Ett sätt att minska kostnaden kan till exempel vara att minska dimensionerna på länkarna eller köpa in billigare växellådor. Den här utvecklingen av kostnadsoptimerade robotar har infört oönskade flexibiliteter i leder och länkar. Det är därför inte längre möjligt att få den önskade prestandan och robustheten genom att bara mäta motorernas vinkelpositioner och -hastigheter. Istället krävs det omfattande matematiska modeller som beskriver dessa oönskade flexibiliteter. Dessa modeller kräver mycket arbete att dels ta fram men även för att identifiera parametrarna. Det finns automatiska metoder för att beräkna modellparametrarna men oftast krävs det en manuell justering för att få bra prestanda.  Den här avhandlingen undersöker möjligheterna att beräkna verktygspositionen med hjälp av bayesianska metoder för tillståndsskattning. De bayesianska skattningsmetoderna beräknar tillstånden för ett system iterativt. Med hjälp av en matematisk modell över systemet predikteras vad tillståndet ska vara vid nästa tidpunkt. Efter att mätningar av systemet vid den nya tidpunkten har genomförts justeras skattningen med hjälp av dessa mätningar. De metoder som har använts i avhandlingen är det så kallade extended Kalman filtret samt partikelfiltret.  Informationen på armsidan av växellådan ges av en accelerometer som är monterad på verktyget. Med hjälp av accelerationen för verktyget och motorernas vinkelpositioner kan en skattning av verktygspositionen beräknas. I en simuleringsstudie för en realistisk vek robot har det visats att skattningsprestandan ligger nära den teoretiska undre gränsen, känd som Raooch mätstörningar som påverkar roboten. För att underlätta trimningen så har en metod för att skatta processbrusets kovariansmatris föreslagits. En annan viktig del som påverkar prestandan är modellerna som används i filtren. Modellerna för en industrirobot är vanligtvis framtagna i kontinuerlig tid medan filtren använder modeller i diskret tid. För att minska felen som uppkommer då de tidskontinuerliga modellerna överförs till diskret tid har olika samplingsmetoder studerats. Vanligtvis används enkla metoder för att diskretisera vilket innebär problem med prestanda och stabilitet. För att hantera dessa problem införs översampling vilket innebär att tidsuppdateringen sker med en mycket kortare sampeltid än vad mätuppdateringen gör. För att undvika översampling kan det motsvarande tidskontinuerliga filtret användas för att prediktera tillstånden vid nästa diskreta tidpunkt. En analytisk lösning med låg beräkningskomplexitet till detta problem har föreslagits.  Vidare innehåller avhandlingen två typer av reglerproblem relaterade till industrirobotar. För det första har den så kallade norm-optimala iterative learning control styrlagen utökats till att hantera fallet då en skattning av den önskade reglerstorheten används istället för en mätning. Med hjälp av skattningen av systemets tillståndsvektor kan metoden nu även användas till olinjära system vilket inte är fallet med standardformuleringen. Den föreslagna metoden utökar målfunktionen i optimeringsproblemet till att innehålla inte bara väntevärdet av den skattade reglerstorheten utan även skattningsfelets kovariansmatris. Det innebär att om skattningsfelet är stort vid en viss tidpunkt ska den skattade reglerstorheten vid den tidpunkten inte påverka resultatet mycket eftersom det finns en stor osäkerhet i var den sanna reglerstorheten befinner sig.  För det andra har design och analys av H∞-regulatorer för en linjär modell av en vek robotled, som beskrivs med fyra massor, genomförts. Det visar sig att reglerprestandan kan förbättras, utan att lägga till fler mätningar än motorns vinkelposition, jämfört med tidigare utvärderade regulatorer. Genom att mäta verktygets acceleration kan prestandan förbättras ännu mer. Modellen över leden är i själva verket olinjär. För att hantera detta har en H∞-syntesmetod föreslagits som kan hantera olinjäriteten i modellen.
Vinnova Excellence Center LINK-SIC

L'objectif de cette thèse s'inscrit dans la cadre général du développement d'une stratégie de contrôle intelligent distribué en robotique collective. En effet, dans un avenir proche, de nombreux robots vont progressivement intégrer notre environnent aussi bien dans les milieux industriel que domestique. L'objectif de ces robots sera de fournir, de manière autonome, des services aux êtres humains afin de leurs faciliter la vie quotidienne comme par exemple dans le cas de robots compagnons. Ces services pourront être le résultat du travail d'un robot ou bien la conséquence de la coopération de plusieurs robots homogènes et/ou hétérogènes regroupés au sein d'un réseau. Dans ce contexte, si les progrès technologiques permettent sans problème de communiquer et d'échanger des données entre deux agents artificiels distants, la conception de stratégies de contrôle permettant l'auto-organisation de plusieurs robots dans le but de réaliser une tâche précise est encore aujourd'hui un verrou scientifique important. Cette thèse a donc pour but de proposer des pistes pour élaborer des stratégies de contrôle intelligent pour des systèmes multi-robots dans le cadre plus particulier de la logistique industrielle. En effet, le domaine de logistique industrielle nécessite l'utilisation de nombreux robots mobiles comme par exemple des AGV (Automatic Guided Vehicles) pour transporter et stoker des marchandises. Dans ce contexte, nous pensons que le domaine de la logistique peut tirer bénéfice de l'utilisation de systèmes multi-robots. Dans un premier temps, cette thèse aborde donc la problématique de transport d'objet volumineux et encombrant par une formation de robot. Effectivement, il semble que la solution qui consiste à utiliser un ensemble de robots identiques pour transporter des charges de grandes envergures soit, d'une part, très intéressante d'un point de vue économique et, d'autre part, plus robuste et flexible d'un point vue technologique. Dans un deuxième temps, cette thèse aborde l'utilisation d'un réseau de robots hétérogènes qui sont capables de s'organiser afin de réaliser une tâche précise dans un milieu dynamique. Les travaux effectués dans le cadre de la présente thèse doctorale ont donc abouti à la proposition des stratégies viables de contrôle intelligent pour des systèmes multi-robots. Une étude d'application des concepts étudiés a été réalisée, implantée et validée dans le cadre plus particulier de la logistique industrielle. Elle a concerné d'abord le contexte d'un groupe multi-robots homogène, puis a été étendue au contexte d'un système multi-robots hétérogènes. Les points forts des travaux réalisés peuvent être résumés comme ceci :- Proposition, conception, réalisation et validation expérimentale d'une stratégie de contrôle adaptatif par l'apprentissage artificiel pour un robot non-holonomique. Quatre publications internationales ont valorisé cette partie des travaux.- Proposition, conception, réalisation et validation expérimentale d'une stratégie de contrôle hybridant la vision artificielle et l'apprentissage artificiel pour un groupe de robots homogènes. Deux publications internationales ont valorisé cette partie des travaux.- Proposition, conception, réalisation et validation expérimentale d'une stratégie de contrôle hybridant la vision artificielle et l'apprentissage artificiel pour un groupe de robots hétérogènes. Deux publications internationales ont valorisé cette partie des travaux. Il est pertinent de souligner que les travaux relatifs aux aspects précités ont été couronnés par le prix : ″Innovation Award 2011″ de Industrial Robot
In this thesis, it concentrated the multi robot team navigating in an unknown environment. In our multi robot team, there is a humanoid robot as a leader and a team of two-wheel nonholonomic robots which form a vertical formation. Besides, a top camera and a computer which is a supervisor are the auxiliary robots in the multi robot team. The main purpose of the thesis is to propose an online and an offline navigation strategy for the closed and open area respectively. The core of navigation strategies is the same and it included path planning part and control part. Both the two parts constructed on the virtual structure of the formation robot team. In the former part, it improved the path planning part by the reinforcement Q learning and the image processing to acknowledge the unknown environment. And it applied the Adaptive Neural Fuzzy Inference System (ANFIS) algorithm to control of both the single nonholonomic robot and formation robot team. Furthermore, the strategies are applied to the formation robot team and the multi robot team in both closed and open environment. Simulations and real experiments are provided in the detail in the thesis. The strong points of the contribution are :- Proposition, conception, realization and experimental validation of machine learning based adaptive control for a nonholonomic single robot (in a group of robots). Four international publications have valorized this part of the doctoral Works. - Proposition, conception, realization and experimental validation of an adaptive intelligent control strategy hybridizing Artificial vision and Machine Learning for a group of nonholonomic homogeneous robots. Two international publications have valorized this part of the doctoral Works.- Proposition, conception, realization and experimental validation of an adaptive intelligent control strategy hybridizing Artificial vision and Machine Learning for a group of heterogeneous robots. Two international publications have valorized this part of the doctoral Works. It is pertinent to emphasize the investigations relative to the above-mentioned works have been awarded by: ″Innovation Award 2011″ of Industrial Robot

Industrial robots are delivering more and more manipulation services in manufacturing. However, when the task is complex, it is difficult to programme a robot to fulfil all the requirements because even a relatively simple task such as a peg-in-hole insertion contains many uncertainties, e.g. clearance, initial grasping position and insertion path. Humans, on the other hand, can deal with these variations using their vision and haptic feedback. Although humans can adapt to uncertainties easily, most of the time, the skilled based performances that relate to their tacit knowledge cannot be easily articulated. Even though the automation solution may not fully imitate human motion since some of them are not necessary, it would be useful if the skill based performance from a human could be firstly interpreted and modelled, which will then allow it to be transferred to the robot. This thesis aims to reduce robot programming efforts significantly by developing a methodology to capture, model and transfer the manual manufacturing skills from a human demonstrator to the robot. Recently, Learning from Demonstration (LfD) is gaining interest as a framework to transfer skills from human teacher to robot using probability encoding approaches to model observations and state transition uncertainties. In close or actual contact manipulation tasks, it is difficult to reliabley record the state-action examples without interfering with the human senses and activities. Therefore, wearable sensors are investigated as a promising device to record the state-action examples without restricting the human experts during the skilled execution of their tasks. Firstly to track human motions accurately and reliably in a defined 3-dimensional workspace, a hybrid system of Vicon and IMUs is proposed to compensate for the known limitations of the individual system. The data fusion method was able to overcome occlusion and frame flipping problems in the two camera Vicon setup and the drifting problem associated with the IMUs. The results indicated that occlusion and frame flipping problems associated with Vicon can be mitigated by using the IMU measurements. Furthermore, the proposed method improves the Mean Square Error (MSE) tracking accuracy range from 0.8˚ to 6.4˚ compared with the IMU only method. Secondly, to record haptic feedback from a teacher without physically obstructing their interactions with the workpiece, wearable surface electromyography (sEMG) armbands were used as an indirect method to indicate contact feedback during manual manipulations. A muscle-force model using a Time Delayed Neural Network (TDNN) was built to map the sEMG signals to the known contact force. The results indicated that the model was capable of estimating the force from the sEMG armbands in the applications of interest, namely in peg-in-hole and beater winding tasks, with MSE of 2.75N and 0.18N respectively. Finally, given the force estimation and the motion trajectories, a Hidden Markov Model (HMM) based approach was utilised as a state recognition method to encode and generalise the spatial and temporal information of the skilled executions. This method would allow a more representative control policy to be derived. A modified Gaussian Mixture Regression (GMR) method was then applied to enable motions reproduction by using the learned state-action policy. To simplify the validation procedure, instead of using the robot, additional demonstrations from the teacher were used to verify the reproduction performance of the policy, by assuming human teacher and robot learner are physical identical systems. The results confirmed the generalisation capability of the HMM model across a number of demonstrations from different subjects; and the reproduced motions from GMR were acceptable in these additional tests. The proposed methodology provides a framework for producing a state-action model from skilled demonstrations that can be translated into robot kinematics and joint states for the robot to execute. The implication to industry is reduced efforts and time in programming the robots for applications where human skilled performances are required to cope robustly with various uncertainties during tasks execution.