Academic literature on the topic 'Knowledge related to joint task and motion planning'

Various path planning algorithms have been used to deduce optimal manipulator joint trajectories for spacecraft attitude regulation using arm motion. However, few papers have considered the unexpected factors when applying those planned path into actual situations. Even though conventional feedback control would drive the arm dynamics to the desired one, this only appears when time evolves to infinity which means during some time the actual joint paths deviate from the desired ones. However, the spacecraft attitude change is related to the entire process of arm motion. So, even a small deviation of the actual joint movement from the desired one would cause the failure of spacecraft attitude regulation task. In this paper, sliding mode control technique is adopted to force the actual joint moves along the desired trajectory from the start. Further, saturation function is used to eliminate the chattering phenomenon. Moreover, the relation between attitude regulation accuracy and controller parameters is deduced which gives instructions in tuning the controller parameters. In the end, numerical simulation is conducted to show the feasibility of the proposed controllers.

Although extensive efforts have been dedicated to investigating the risk factors of low back pain in the workplace, a clear knowledge of their effects on the facet joint (FJ) mechanics is lacking. In this study, fourteen healthy participants performed dynamic lifting task with varying external load while a dynamic stereo-radiography system captured their lumbar motion continuously. The FJ kinematics in the lumbar spine were ascertained using a volumetric model-based tracking method. The FJ kinematics data from seven participants were processed and analyzed using non-parametric statistical test. The results indicated significant (p<0.05) effects of external load on the FJ flexion and superior-inferior translation at all segments, showing more consistent trends at the L2-L3, L3-L4, and L4-L5 joints during trunk flexion angles of approximately 20° and 40°. Findings of this study provide a preliminary but important foundation in elucidating facet-related injury mechanism due to strenuous exertions in workplaces.

Background The objective kinematic assessments of activities of daily living are desired. However, the limited knowledge regarding age- and sex-related differences prevents the adaptation of these measurements to clinical settings and in-home exercises. Therefore, this study aimed to determine the effects of age and sex on joint and endpoint kinematics during a common activity of daily living, specifically, drinking from a glass. Methods In total, 32 healthy adults (18 males and 14 females) aged 22–77 years performed a drinking task comprising reaching for a glass, bringing it forward and sipping, returning it, and placing the hand back to the starting position, which was recorded using a three-dimensional motion-capturing system. A two-way analysis of variance was used to statistically compare joint angles at five different time points and endpoint kinematic variables in the four drinking phases between older and younger age groups and sexes. Results Wrist radial deviation was greater in older adults than in younger participants at all five different time points (F = 5.16–7.34, p ≤ 0.03, η2 = 0.14–0.21). Moreover, lesser shoulder abduction and greater shoulder internal rotation and forearm pronation when moving and returning the hand to the starting position were observed in the female group than in the male group (F = 4.21–20.03, p ≤ 0.0497, η2 = 0.13–0.41). Trunk flexion was lower in the female group than in the male group at all time points (F = 4.25–7.13, p ≤ 0.0485, η2 = 0.12–0.19). Regarding endpoint kinematics, the performance time in the reaching phase was longer in older adults than in younger individuals (F = 4.96, p = 0.03, η2 = 0.14). Furthermore, a shorter time while returning the hand to the starting position was observed in the female group than in the male group (F = 9.55, p < 0.01, η2 = 0.22). Conclusions The joint kinematics of drinking were partially characterized by an age effect, whereas endpoint kinematics were scattered in all drinking phases. Sex-related effects in most upper-body motions and postures may cause rapid motions in females. Therefore, clinicians could use this knowledge for precise assessments and to suggest feasible in-home exercises.

Humanoid robots are equipped with humanoid arms to make them more acceptable to the general public. Humanoid robots are a great challenge in robotics. The concept of digital twin technology complies with the guiding ideology of not only Industry 4.0, but also Made in China 2025. This paper proposes a scheme that combines deep reinforcement learning (DRL) with digital twin technology for controlling humanoid robot arms. For rapid and stable motion planning for humanoid robots, multitasking-oriented training using the twin synchro-control (TSC) scheme with DRL is proposed. For switching between tasks, the robot arm training must be quick and diverse. In this work, an approach for obtaining a priori knowledge as input to DRL is developed and verified using simulations. Two simple examples are developed in a simulation environment. We developed a data acquisition system to generate angle data efficiently and automatically. These data are used to improve the reward function of the deep deterministic policy gradient (DDPG) and quickly train the robot for a task. The approach is applied to a model of the humanoid robot BHR-6, a humanoid robot with multiple-motion mode and a sophisticated mechanical structure. Using the policies trained in the simulations, the humanoid robot can perform tasks that are not possible to train with existing methods. The training is fast and allows the robot to perform multiple tasks. Our approach utilizes human joint angle data collected by the data acquisition system to solve the problem of a sparse reward in DRL for two simple tasks. A comparison with simulation results for controllers trained using the vanilla DDPG show that the designed controller developed using the DDPG with the TSC scheme have great advantages in terms of learning stability and convergence speed.

ABSTRACT In 2010, the American Petroleum Institute coordinated the establishment of an Oil Spill Preparedness and Response Joint Industry Task Force (JITF) comprised of experts from industry and government. The JITF was established to examine the efforts implemented during recent oil spill events, assess the current state of the industry's ability to respond to significant spills, and identify potential learnings. A remote sensing technical working group (TWG) was formed under the Oil Sensing and Tracking program to document current remote sensing technology and future research and development needs related to tracking oil on the water's surface. Since its inception, the Remote Sensing TWG has held face-to-face meetings, engaged with other joint industry groups, and completed a planning guide, “Remote Sensing in Support of Oil Spill Response,” to assist in the identification of surveillance technologies, sensors, and platforms that could enhance response efforts. The document provides information related to both strategic and tactical response activities. The guide was designed as a practical “Users Guide” as well as a planning and preparedness tool for response personnel that may be required direct surveillance activities. The group also identified key areas for industry and academia for future research and development, including:Quantifying the aerial extent and concentration of oil on the water's surface; andEnhancing ways to provide quality-assessed data in near real-time to both field resources and command centers. In addition, the TWG is currently researching the development of a web-based tool that would aid in the selection of appropriate satellite sensors for a particular response scenario. The group is working with the National Aeronautics and Space Administration (NASA) to customize and expand an existing NASA satellite tool for use during oil spill response. The TWG plans to continue collaboration and knowledge-sharing with other joint industry projects, and to continue to hold regular meetings to share recent experiences, collaborate on potential technology areas that need development, and disseminate scientific findings in the field of remote sensing that will enhance the capabilities and readiness for response.

Highly articulated systems are capable of executing a variety of behaviors by coordinating their many internal degrees of freedom to help them move more effectively in complex terrains. However, this inherent variety poses significant challenges that have been the subject of a great deal of previous work: What are the most effective or most efficient methods for achieving the intrinsic coordination necessary to produce desired global objectives? This work takes these questions one step further, asking how different levels of coordination, which we quantify in terms of kinematic coupling, affect articulated locomotion in environments with different degrees of underlying structure. We introduce shape functions as the analytical basis for specifying kinematic coupling relationships that constrain the relative motion among the internal degrees of freedom for a given system during its nominal locomotion. Furthermore, we show how shape functions are used to derive shape-based controllers (SBCs) that manage the compliant interaction between articulated bodies and the environment while explicitly preserving the inter-joint coupling defined by shape functions. Initial experimental evidence provides a comparison of the benefits of different levels of coordination for two separate platforms in environments with different degrees of inherent structure. The experimental results show that decentralized implementations, where there is relatively little inter-joint coupling, perform well across a spectrum of different terrains but that there are potential benefits to higher degrees of coupling in structured terrains. We discuss how this observation has implications related to future planning and control approaches that actively “tune” their underlying structure by dynamically varying the assumed level of coupling as a function of task specification and local environmental conditions.

Unlike a purely reactive system where the motor output is exclusively controlled by the actual sensory input, a cognitive system must be capable of running mental processes which virtually simulate action sequences aimed at achieving a goal. The mental process either attempts to find a feasible course of action compatible with a number of constraints (Internal, Environmental, Task Specific etc) or selects it from a repertoire of previously learned actions, according to the parameters of the task. If neither reasoning process succeeds, a typical backup strategy is to look for a tool that might allow the operator to match all the task constraints. This further necessitates having the capability to alter ones own goal structures to generate sub-goals which must be successfully accomplished in order to achieve the primary goal. In this paper, we introduce a forward/inverse motor control architecture (FMC/IMC) that relaxes an internal model of the overall kinematic chain to a virtual force field applied to the end effector, in the intended direction of movement. This is analogous to the mechanism of coordinating the motion of a wooden marionette by means of attached strings. The relaxation of the FMC/IMC pair provides a general solution for mentally simulating an action of reaching a target position taking into consideration a range of geometric constraints (range of motion in the joint space, internal and external constraints in the workspace) as well as effort-related constraints (range of torque of the actuators, etc.). In case, the forward simulation is successful, the movement is executed; otherwise the residual "error" or measure of inconsistency is taken as a starting point for breaking the action plan into a sequence of sub actions. This process is achieved using a recurrent neural network (RNN) which coordinates the overall reasoning process of framing and issuing goals to the forward inverse models, searching for alternatives tools in solution space and formation of sub-goals based on past context knowledge and present inputs. The RNN + FMC/IMC system is able to successfully reason and coordinate a diverse range of reaching and grasping sequences with/without tools. Using a simple robotic platform (5 DOF Scorbot arm + Stereo vision) we present results of reasoning and coordination of arm/tool movements (real and mental simulation) specifically directed towards solving the classical 2-stick paradigm from animal reasoning at a non linguistic level.

Abstract More precise knowledge concerning gait patterns of movement in stroke patients incorporating modern diagnostic tools is necessary. Exact information about qualitative and quantitative changes during the process of rehabilitation based on reeducation of functions and relating it to possible changes of motor deficit will possibly lead to better physiotherapy planning. Goal of this study is to assess some components and changes of gait pattern and motor deficit after stroke to develop better physiotherapy this group of patients Materials and methods Consecutively admitting to rehabilitation unit first stroke patients who met inclusion criteria ( time from the onset of stroke between 1st and 3th month, independent walking skills on the distance of 10m, motor deficit, self awareness of disease, signed inform consent form) were enrolled to the study. Patients were participating in physiotherapy five days a week for 12 weeks. Rivermead Motor Assessment (Lower Extremity Section), Fugl-Meyer Motor Deficit Scale, Nottingham Extended Activities of Daily Living Scale and three-dimension motion analysis system Vicon 460 (amplitude of hip joint range of motion, walking speed, cadence, steps length) were used. All measurement procedures were carried out three times: before therapy, after 6 weeks and after 12 weeks of treatment. Results: Twelve patients were enrolled to the study (4 female, 8 man, mean age 58 years old, 5 persons suffering from right side paresis). All patients improved after 6 and 12 weeks. Alterations were related both to clinical assessment and to objective movement analysis and they were positively correlated. Mean motor deficit (FM) stated 140,148, 161p, ADL -30, 42, 47p., walking speed: 0,4m/s, 0,5m/s, 0,6m/s, cadence (steps/min):70, 81, 89. Conclusions: Results indicate relatively constant progress of motor abilities in this group of patients however it decreased during second period. Clinical improvements were accompanied by similar progress of walking speed and cadence. Further studies are necessary

It is difficult to introduce highly versatile automation using robots to handling deformable objects such as thread, cloth, wire, long beams, and thin plates in plant production processes, compared to the handling of rigid objects. Office equipment handles deformable objects such as paper and plastic. Problems unique to these objects is caused by speeding up such equipment and demand for upgrading its accuracy. In agriculture and medical care, automatic, intelligent handling of deformable objects such as fruit and animals has long been desired and practical systems sought. Deformable objects whose handling should be versatiley and accurately automated are classified into two groups based on handling: (A) Flexible, mostly thin, fine objects capable of elastic deformation (B) Soft objects easily crushed, such as soft fruits or animals The problem in handling the first group is controlling object deformation of an infinite degree of freedom with a finite number of manipulated variables. In contrast, a significant problem in handling the second group is often how to handle them without exerting excessive stress and how to handle them safely and reliably. The handling of these two groups differ greatly in mechanics and control theory, and this special issue focuses on the first group — flexible objects — mechanical collection and transport studies, control, and software. Recent studies on their handling are classified into four groups for convenience based on handled objects and types of handling task: (a) Control of deformation, internal force, and vibration or path planning of flexible objects (mainly thin plates and beams) using single or multiple manipulators. (b) Task understanding in insertion of elastic into rigid parts and vice versa, and the study of human skills to help robots accomplish these task. (c) Approaches on improved accuracy, intelligent control, and vibration damping in handling and transfer of sheets and strings with low flexural rigidity, represented by paper or wire. (d) Strategies for grasping and unfolding sheets such as cloth whose flexural rigidity is almost nil. For (a), studies are active on deformation control by two robot hands attempting to grasp cloth. 1-3) In the automobile industry, so-called flexible fixtureless assembly systems are advancing in which two robots process or assemble parts in mid-air without a fixed table to reduce lead time and cost. These systems are mostly developed assuming handled parts are rigid. Nguyen et al. work assuming parts such as sheet metal whose deformation must be taken into consideration.1) Nakagaki et al. propose form estimation that considers even plastic deformation in wire handling by robots, in connection with the development of robots for electric wire installation.4) Many studies cover flexible wire as elastic beams,3-9) but comparatively few focus on bending deformation of thin plates. This special edition includes a paper by Kosuge et al. on thin-plate deformation control. Vibration control of grasped objects becomes important as speed increases. Matsuno kindly contributed his paper on optimum path planning in elastic plate handling. In controlling the deformation of elastic bodies, the mechanics of objects handled is often unknown. This special issue features a paper by Kojima et al. on an approach to this problem by adaptive feed-forward control. For (b), we consider three cases: (1) A cylindrical rigid body inserted into a hole on an elastic plate. (2) An elastic bar inserted into a hole on a rigid body. (3) A tubular elastic body put on a cylindrical rigid body. This special issue carries papers on these problems by Brata et al., Matsuno et al., and Hirai. For (2), a paper by Nakagaki et al.10) covers electric wire installation. For (3), the paper by Shima et al.11) covers insertion of a rigid axis into an elastic hose. Robot skill acquisition is an important issue in robotics in general, and the above papers should prove highly interesting and information because they treat studies by comparing robot and human skills in accomplishing work and acquiring concrete skills knowledge. For (c), attempts are made to theoretically analyze sheet handling mechanisms and control developed based on trial and error, and to structure design theory based on such analysis. These attempts are related to the increased accuracy and speed and enhanced intelligence of sheet-handling office automation equipment such as printers, facsimile machines, copiers, and automated teller machines. Yoshida et al. conducted a series of studies on the effects of guides forming paper feed paths and of inertia force of paper by approximating sheets with a chain of discrete masses and springs.12-14) This special edition also features a study on sheet sticking and jamming. Okuna et al. handles a system of similar nature, mechanical studying the form of paper guides.15) Introducing mechanisms to control the positioning of sheets is effective in raising sheet transfer accuracy. Feedback control that regulates feed roller skew angle as a manipulated variable is proposed.16) Increased reliability in separating single sheets from stacked effectively reduces the malfunction rate in sheet-handling equipment. Ways of optimizing the form of sheet-separation rollers17) and estimating frictional force between separation gates and sheets 18) are also proposed. This special issue contains a proposal by Nakazawa et al. of a mechanism that uses reactive sheet buckling force, made in connection with development of a newspaper page turner for the disabled as technology for separating single sheets. Dry frictional force is most widely used for transporting sheets, but is not stable and may even act as an obstacle to improving accuracy. Niino et al. propose a sheet transfer mechanism that uses electrostatic force.19) For improving the accuracy of flexible wire transmission, this special issue carries a study on transporting flexible thin wire through tension control at multiple points, from a study by Morimitsu et al. on optical fiber installation. The thickness of wire used in equipment is becoming increasingly slim and flexible, along with the equipment it is used in. Tension control in the production process is an important factor in the manufacture of such thin wire. Production efficiency constantly calls for increased transfer speed. It has thus become important to estimate air resistance and inertia and to measure and control the tension of running wire. Studies20,21) by Batra, Fraser, et al. which deal the motion of string in the spinning process provide good examples for learning analytical techniques for air drag and inertia. In string vibration where inertia dominates, attempts are made to control vibration by boundary shaking22,23) and feed-forward/back control.24) For (d), highly versatile robots for handling cloth are being developed, and the software technology for automatic cloth selection and unfolding by robot hands is a popular topic.25-27) Ono et al. comment on the nature of problems in developing intelligent systems for handling cloth and similar objects whose bending rigidity is low and which readily fold and overlap—a paper that will prove a good reference in basic approaches in this field. Mechanical analyses are indispensable to studies on (a) through (c). In contrast, information technology such as characteristic variable measurement, image processing, and discrimination, rather than mechanical analyses, play an important roles in studies on (d). This special issue features a study by Hamashima, Uraya et al. on cloth unfolding as an example of such studies. Studies up to now largely assumed that properties of grasped objects did not change environmental influences such as temperature and humidity. Such influence is often, however, a major factor in handling fiber thread and cloth. This special issue has a paper contributed by Taylor, who studies handling method to prevent influence by such environmental factors. The objective of this special issue will have been achieved if it aids those studying the handling of flexible objects by providing approaches and methodologies of researchers whose target objects differ and if it aids those planning to take up study in this field by providing a general view of this field. References: 1) Nguyen, W. and Mills, J., ""Multi-Robot Control For Plexible Fixtureless Assembly of Flexible Sheet Metal Auto Body Parts,"" Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 2340-2345, (1996). 2) Sun, D. and Shi, X. and Liu, Y., ""Modeling and Cooperation of Two-Arm Robotic System Manipulating a Deformable Object,"" Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 2346-2351, (1996). 3) Kosuge, K., Sakaki, M., Kanitani, K., Yoshida, H. and Fukuda, T., ""Manipulation of a Flexible Object by Dual Manipulators,"" IEEE International Conference on Robotics and Automation, 318-323, (1995). 4) Nakagaki, H., Kitagaki, K., Ogasawara, T. and Tukune H., ""Handling of a Flexible Wire -Detecting a Deformed Shape of the Wire by Vision and a Force Sensor,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'96), 207-210, (1996). 5) Wakamatsu, H., Hirai, S. and Iwata, K., ""Static Analysis of Deformable Object Grasping Based on Bounded Force Closure,"" Trans. of JSML, 84-618 (C), 508-515, (1998). 6) Katoh, R. and Fujmoto, T., ""Study on Deformation of Elastic Object By Manipulator -Path Planning of End -Effector-,"" J. of the Robotics Society of Japan, 13-1, 157-160, (1995). 7) Yukawa, T., Uohiyama, M. and Inooka, M., ""Stability of Control System in Handling a Flexible Object by Rigid Arm Robots,"" JSME Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 169-172, (1995). 8) Yukawa, T., Uohiyama, M. and Cbinata, G., ""Handling of a Vibrating Flexible Structure by a Robot,"" Trans. JSME, 61-583, 938-943, (1995). 9) Sun, D. and Liu, Y., ""Modeling and Impedance Control of a Two-Manipulator System Handling a Flexible Beam,"" Trans. of the ASME, 119, 736-742, (1997). 10) Nakagaki, H., Kitagaki, K. and Tukune, H., ""Contact Motion in Inserting a Flexible Wire into a Hole,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 175-178, (1995). 11) Shimaji, S., Brata, A. and Hattori, H., ""Robot Skill in Assembling a Cylinder into an Elastic Hose,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 752-755, (1995). 12) Yoshida, K. and Kawauchi, M., ""The Analysis of Deformation and Behavior of Flexible Materials (1st Reprt, Study of Spring-Mass Beam Model of the Sheet,"" Trans. of JSME, 58-552, 1474-1480, (1992). 13) Yoshida, K., ""Analysis of Deformation and Behavior of Flexible Materials (2nd Report, Static Analysis for Deformation of the Sheet in the Space Formed by Guide Plates),"" Trans. JSME, 60-570, 501-507, (1994). 14) Yoshida, K., ""Dynamic Analysis of Sheet Defofmation Using Spring-Mass-Beam Model,"" Trans. JSME, 63-615, 3926-3932 (1997). 15) Okuna, K., Nishigaito, T. and Shina, Y., ""Analysis of Paper Deformation Considering Guide Friction (Improvement of Paper Path for Paper-Feeding Mechanism),"" Trans. JSME, 60-575, 2279-2284, (1994). 16) Fujimura, H. and Ono, K., ""Analysis of Paper Motion Driven by Skew-Roll Paper Feeding System,"" Trans. JSME, 62-596, 1354-1360, (1996). 17) Shima, Y., Hattori, S., Kobayashi, Y. and Ukai, M., ""Optimum of Gate-Roller Shape in Paper Isolating Methods,"" Conference of Information, Intelligence and Precision Equipment (IIP'96), 61-62, (1996). 18) Suzuki, Y, Hattori, S., Shima, Y. and Ukai, M., ""Contact Analysis of Paper in Gate-Roller Handling Method"", Conference on Information, Intelligence and Precision Equipment (IIP'95), 19-20, (1995). 19) Niino, T., Egawa, S. and Higuchi, T., ""An Electrostatic Paper Feeder,"" J. of the Japan Society for Precision Engineering, 60-12,1761-1765, (1994). 20) Batra, S., Ghosh, T. and Zeidman, M., ""An Integrated Approach to Dynamic Analysis of the Ring Spinning Process , PartII: With Air Drag,"" Textile Research Journal, 59, 416-424, (1989). 21) Fraser, W., Ghosh, T. and Batra, S., ""On Unwinding Yarn from a Cylindrical Package,"" Proceedings of Royal Society of London, A, 436, 479-438, (1992). 22) Jacob, S., ""Control of Vibrating String Using Impedance Matching,"" Proceedings of the American Control Conference (San Francisco),468-472, (1993). 23) Lee, S. and Mote, C., ""Vibration Control of an Axially Moving String by Boundary Control,"" Trans. of the ASME, J. of Dynamic Systems, Measurement, and Control, 118, 66-74, (1996). 24) Ying, S. and Tan, C., ""Active Vibration Control of the Axially Moving String Using Space Feedforward and Feedback Controllers,"" Trans. ASME, J. of Vibration and Acoustics, 118, 306-312, (1996). 25) Ono, E., Ichijo, H. and Aisaka, N., ""Flexible Robotic Hand for Handling Fabric Pieces in Garment Manufacture,"" International Journal of Clothing Science and Technology, 4-5,18-23, (1992). 26) Paraschidis, K., Fahantidis, N, Petridis, V., Doulgeri, Z., Petrou, L. and Hasapis, G, ""A Robotic System for Handling Textile and Non Rigid Flat Materials,"" Computers in Industry, 26, 303-313, (1995). 27) Fahantidis, N., Paraschidis, K, Petridis, V., Doulgeri, Z., Petrou, L. and Hasapis, G., ""Robot Handling of Flat Textile Materials,"" IEEE Robotics & Automation Magazine, 4-1, 34-41, (1997).

IntroductionReinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks.MethodsThe proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent.Results and discussionThe effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks.

Pour rester compétitifs, les industriels doivent réduire de plus en plus les coûts et les temps de développement de leurs nouveaux produits. Ceux-ci sont aujourd'hui de plus en plus intégrés, plus petits, plus légers et moins gourmands en énergie. Ils sont plus difficiles à concevoir et doivent être assemblés, maintenus et désassemblés sous de très fortes contraintes géométriques. Traditionnellement, en phase de conception, on établit le modèle CAO du produit, puis on fabrique les différentes parties physiques de celui-ci pour s’apercevoir trop souvent ensuite, que tout ou partie des tâches associées au cycle de vie du produit sont difficiles ou impossibles à réaliser. Une détection tardive de ces problèmes nécessite alors de remettre en cause la conception du produit. Les travaux de cette thèse s'intéressent à valider, dès la phase de conception et par simulation numérique avant la fabrication des prototypes physiques, l'ensemble des tâches associées au PLM, ce qui permettrait de réduire les temps et coûts de développement et de viser des procédés de fabrication plus respectueux de l’environnement en réduisant le nombre de prototypes physiques fabriqués. Une étape clef dans la validation par simulation des tâches du PLM consiste à trouver une trajectoire réalisable et sans collision afin de prouver leur faisabilité. La communauté robotique a, depuis les années 80, mis en oeuvre des méthodes de planification automatiques de trajectoires pour résoudre cette problématique. Toutefois, ces méthodes ont des limites, principalement liées à la complexité des modèles de l'environnement, traditionnellement purement géométriques. Dans des environnements très complexes, les planificateurs de trajectoires peuvent proposer des trajectoires peu pertinentes, dans des temps pouvant être très longs, voire échouer. Pour répondre à ces limites, des travaux ont considéré des approches collaboratives homme - planificateur mais qui ne permettent que rarement une interaction continue. Par ailleurs, les techniques de RV permettent la simulation avec un opérateur humain dans la boucle, en immersion dans l’environnement virtuel et en interaction avec celui-ci. Une approche originale liant planification automatique de trajectoires et RV a ainsi été développée au LGP permettant de profiter de la puissance de calcul des ordinateurs et des capacités cognitives d'un opérateur humain. Toutefois, dans cette approche, l'assistance proposée à l'opérateur n'est pas orientée vers le métier et la tâche à réaliser. Pour pouvoir raisonner au niveau de la tâche à réaliser il faut considérer conjointement planification de tâches et planification de trajectoires et s’intéresser à la capacité de modéliser des informations relatives à cette tâche et de raisonner sur celles-ci ; les ontologies sont un outil prometteur. L'objectif de cette thèse concerne l'élaboration d'une méthodologie pour le couplage sémantique des planificateurs de trajectoires et de tâches pour l’assistance à la manipulation en RV ou la robotique. Dans ce cadre, nous proposons deux contributions principales : La première contribution de ce travail propose deux ontologies originales. La première, ENVOn-2, concerne la modélisation de l'environnement dans lequel se déroule une tâche. La seconde, TAMPO, est une ontologie développée pour le planification conjointe de tâches et de trajectoires. La seconde contribution porte sur l'élaboration d'une méthodologie pour le couplage sémantique des planificateurs de tâches et de trajectoires. Cette méthodologie, par l'utilisation conjointe des deux ontologies, permet d'améliorer la planification de trajectoires d'une action primitive tout en proposant un plan (ou des plans) de tâche (s) pertinent(s) pour la manipulation effectuée. Ces développements ont ensuite été validés à l'aide de scénarios variés et de complexités croissantes. Les résultats obtenus montrent la pertinence de l'approche
To remain competitive, manufacturers need to reduce the costs and development times of their new products. Current products are increasingly integrated, smaller, lighter and more energyefficient. They are more difficult to design and have to be assembled, maintained and disassembled under very high geometric constraints. Traditionally, during the design phase, the CAD model of the product is established, then the physical parts of the product are manufactured, to discover all too often that some or all of the tasks associated with the product's life cycle are difficult or impossible to carry out. If these problems are detected too late, the product design has to be reconsidered. The aim of this thesis is to validate, at the design stage, all the tasks associated to the PLM using digital simulation before the physical prototypes are manufactured. This would make it possible to reduce development times and costs and to aim for more environmentally-friendly manufacturing processes by reducing the number of physical prototypes manufactured. A key step in the simulation-based validation of PLM tasks is to find a feasible collision-free trajectory in order to prove their feasibility. Since the 1980s, the robotics community has been using automatic path planning methods to solve this problem. However, these methods have limitations, mainly linked to the complexity of the environment models, which are traditionally purely geometric. In very complex environments, path planners can propose trajectories that are not very relevant, in times that can be very long, or even fail. To overcome these limitations, some works has considered collaborative human-planner approaches, but these rarely enable continuous interaction. On the other hand, VR techniques allow simulation with a human operator in the loop, immersed in the virtual environment and interacting with it. An original approach linking automatic path planning and VR has been developed at LGP, taking advantage of the computing power of computers and the cognitive abilities of a human operator. However, in this approach, the assistance offered to the operator is not oriented towards the task to be carried out. In order to be able to reason at the level of the task to be carried out, task planning and path planning must be considered together, and attention must be paid to the ability to model information relating to the task and to reason about these information; ontologies are a promising tool. The aim of this thesis is to develop a common framework for the semantic coupling of path and task planners for manipulation assistance in VR or robotics. Within this framework, we propose two main contributions: The first contribution of this work proposes two original ontologies. The first, ENVOn-2, concerns the modelling of the environment in which a manipulation task takes place. The second, TAMPO, is an ontology developed for jointly use path and task planning. The second contribution concerns the development of a methodology for the semantic coupling of task and trajectory planners. This methodology, through the joint use of the two ontologies, makes it possible to improve the path planning of a primitive action while proposing a task plan (or plans) that is (are) relevant to the manipulation being carried out. These developments were then validated using a variety of scenarios of increasing complexity. The results obtained demonstrate the relevance of the approach

Ce travail traite de la simulation et de la validation de tâches de manipulation complexes sous de fortes contraintes géométriques dans des environnements virtuels. Les applications visées sont liées au Framework industriel 4.0 ; à mesure que les produits s’intègrent de plus en plus et que la concurrence économique s'intensifie, les industriels expriment le besoin de valider, dès la conception, non seulement les modèles CAO statiques de leurs produits mais aussi les tâches (ex. : assemblage ou maintenance) liées à leur Product Lifecycle Management (PLM). La communauté scientifique s'est penchée sur cette question sous deux angles : - La planification des tâches décompose une tâche de manipulation à réaliser en une séquence d'actions primitives (c.-à-d. un plan de tâches). - La planification de trajectoire calcule des trajectoires sans collision, notamment pour les objets manipulés. Elle utilise traditionnellement des données purement géométriques, ce qui conduit à des limitations classiques. Les approches conjointes de planification des tâches et des trajectoires que l'on trouve dans la littérature effectuent habituellement une étape classique de planification des tâches, puis vérifient la faisabilité des demandes de planification de trajectoire associées aux actions primitives de ce plan de tâches. Le lien entre la planification des tâches et des trajectoires doit être amélioré, notamment en raison de l'absence de bouclage entre le niveau de planification des trajectoires et le niveau de planification des tâches : - L'information de planification de trajectoire utilisée pour remettre en question le plan de tâches se limite habituellement à la faisabilité du mouvement lorsqu'une information plus riche telle que la pertinence ou la complexité de la trajectoire proposée serait nécessaire ; - Les requêtes de planification de trajectoire utilisent traditionnellement des données purement géométriques et/ou des méthodes de planification de trajectoire "aveugles" (par exemple, RRT), et aucune information liée aux tâches n'est utilisée au niveau de la planification de trajectoire. Notre travail se concentre sur l'utilisation de l'information au niveau des tâches au niveau de la planification de la trajectoire. L'algorithme de planification de trajectoire RRT est considéré, parce que nous considérons la simulation de tâches complexes sous fortes contraintes géométriques. Nous proposons une approche basée sur l'ontologie pour utiliser les informations au niveau des tâches afin de spécifier les requêtes de planification de trajectoire pour les actions primitives d'un plan de tâches. Tout d'abord, nous proposons une ontologie pour conceptualiser les connaissances sur l'environnement 3D dans lequel la tâche simulée se déroule. L'originalité de l'ontologie proposée réside dans le fait qu'elle conceptualise des connaissances hétérogènes (données sémantiques, topologiques et géométriques) tant sur les obstacles que sur les modèles d'espace libre. Deuxièmement, nous exploitons cette ontologie pour générer automatiquement une requête de planification de trajectoire associée à chaque action primitive donnée d'un plan de tâches. Grâce à un processus de raisonnement impliquant les actions primitives instanciées dans l'ontologie, nous sommes capables de déduire les configurations de départ et d'objectif, ainsi que les contraintes géométriques liées aux tâches. Enfin, un planificateur de trajet multi-niveaux est appelé pour générer la trajectoire. Les contributions de ce travail ont été validées par la simulation de plusieurs tâches de manipulation sous de fortes contraintes géométriques. Les résultats obtenus démontrent que l'utilisation de l'information liée aux tâches permet un meilleur contrôle sur l'algorithme de planification de trajectoire RRT pour vérifier la faisabilité du mouvement des actions primitives d'un plan de tâches, ce qui entraîne une réduction du temps de calcul et des trajectoires plus pertinentes pour les actions primitives
This work deals with the simulation and the validation of complex manipulation tasks under strong geometric constraints in virtual environments. The targeted applications relate to the industry 4.0 framework; as up-to-date products are more and more integrated and the economic competition increases, industrial companies express the need to validate, from design stage on, not only the static CAD models of their products but also the tasks (e.g., assembly or maintenance) related to their Product Lifecycle Management (PLM). The scientific community looked at this issue from two points of view: - Task planning decomposes a manipulation task to be realized into a sequence of primitive actions (i.e., a task plan) - Path planning computes collision-free trajectories, notably for the manipulated objects. It traditionally uses purely geometric data, which leads to classical limitations (possible high computational processing times, low relevance of the proposed trajectory concerning the task to be performed, or failure); recent works have shown the interest of using higher abstraction level data. Joint task and path planning approaches found in the literature usually perform a classical task planning step, and then check out the feasibility of path planning requests associated with the primitive actions of this task plan. The link between task and path planning has to be improved, notably because of the lack of loopback between the path planning level and the task planning level: - The path planning information used to question the task plan is usually limited to the motion feasibility where richer information such as the relevance or the complexity of the proposed path would be needed; - path planning queries traditionally use purely geometric data and/or “blind” path planning methods (e.g., RRT), and no task-related information is used at the path planning level Our work focuses on using task level information at the path planning level. The path planning algorithm considered is RRT; we chose such a probabilistic algorithm because we consider path planning for the simulation and the validation of complex tasks under strong geometric constraints. We propose an ontology-based approach to use task level information to specify path planning queries for the primitive actions of a task plan. First, we propose an ontology to conceptualize the knowledge about the 3D environment in which the simulated task takes place. The environment where the simulated task takes place is considered as a closed part of 3D Cartesian space cluttered with mobile/fixed obstacles (considered as rigid bodies). It is represented by a digital model relying on a multilayer architecture involving semantic, topologic and geometric data. The originality of the proposed ontology lies in the fact that it conceptualizes heterogeneous knowledge about both the obstacles and the free space models. Second, we exploit this ontology to automatically generate a path planning query associated to each given primitive action of a task plan. Through a reasoning process involving the primitive actions instantiated in the ontology, we are able to infer the start and the goal configurations, as well as task-related geometric constraints. Finally, a multi-level path planner is called to generate the corresponding trajectory. The contributions of this work have been validated by full simulation of several manipulation tasks under strong geometric constraints. The results obtained demonstrate that using task-related information allows better control on the RRT path planning algorithm involved to check the motion feasibility for the primitive actions of a task plan, leading to lower computational time and more relevant trajectories for primitive actions

A mobile manipulator typically consists of a mobile platform and a robotic manipulator mounted on the platform. The base placement of the platform has a great influence on whether the manipulator can perform a given task. In view of the issue, a new approach to optimize the base placement for a specified task is proposed in this paper. Firstly, the workspace of a redundant manipulator is investigated. The manipulation capability of the redundant manipulator is maximized based on the manipulability index through the joint self-motion of the redundant manipulator. Then the maximum manipulation capability in the specified work point is determined. Next, the relative manipulability index (RMI) is defined for analyzing manipulation capability of the manipulator in its workspace, and the global manipulability map (GMM) is presented based on the above measure. Moreover, the optimal base placement related to the given task is obtained, and the motion planning is implemented by an improved rapidly-exploring random tree (RRT) algorithm with the RMI, which can enhance the manipulation capability from the initial point to the target point. Finally, the feasibility of the proposed algorithm is illustrated with numerical simulations and experiments on the mobile manipulator.

Abstract Decreasing user effort and automating subtasks such as obstacle avoidance and user guidance has shown to increase the effectiveness and utility of teleoperation. Extending the capabilities of teleoperation remains a critical research topic for tasks that need to leverage user knowledge, or for unstructured environments that autonomous solutions are not robust enough to handle. Previous methods have focused individually on joint space tasks, regression or training based user intention recognition and intervention, or application specific solutions. To overcome the limitations of these methods, this paper proposes the use of path planning based gross motion assistance with a projection based user intention recognition method, for improving task execution in semi-autonomous teleoperation. The proposed solution synthesizes an assistive architecture that leverages the benefit of supervisory level task identification with semi-autonomous trajectory tracking. With the proposed method, continuous and more immersive teleoperation is achieved, as control states are user selected and task execution is informed from the operator’s motion. The effectiveness of the proposed method is validated with a user study.