Journal articles on the topic 'Robotic Manipulation of Deformable Objects (RMDO)'

In this article, we present a review on the recent advancement in flexible deformable object modeling for dexterous manipulation in robotic system. Flexible deformable object is one of the most research topics in computer graphic, computer vision, and robotic literature. The deformable models are known as the construction of object with material parameters in virtual environment to describe the deformation behavior. Existing modeling techniques and different types of deformable model are described. Various approaches of deformable object modeling have been used in robotic recognition and manipulation in order to reduce the time and cost to obtain more accurate result. In robotic manipulation, object detection, classification, and recognition of deformable objects are always a challenging problem and required as a first step to imbue the robot to able handle these deformable objects. Furthermore, the dexterity of robot control is also another essential key in handling of deformable object which its manipulation strategies need to plan intelligently for each sequence process. We also discuss some deserving direction for further research based on most current contribution.

This paper reviews the robotic manipulation of deformable objects in caregiving scenarios. Deformable objects like clothing, food, and medical supplies are ubiquitous in care tasks, yet pose modeling, control, and sensing challenges. This paper categorises caregiving deformable objects and analyses their distinct properties influencing manipulation. Key sections examine progress in simulation, perception, planning, control, and system designs for deformable object manipulation, along with end-to-end deep learning’s potential. Hybrid analytical data-driven modeling shows promise. While laboratory successes have been achieved, real-world caregiving applications lag behind. Enhancing safety, speed, generalisation, and human compatibility is crucial for adoption. The review synthesises critical technologies, capabilities, and limitations, while also pointing to open challenges in deformable object manipulation for robotic caregiving. It provides a comprehensive reference for researchers tackling this socially valuable domain. In conclusion, multi-disciplinary innovations combining analytical and data-driven methods are needed to advance real-world robot performance and safety in deformable object manipulation for patient care.

Deformable objects such as ropes, wires, and clothing are omnipresent in society and industry but are little researched in robotics research. This is due to the infinite amount of possible state configurations caused by the deformations of the deformable object. Engineered approaches try to cope with this by implementing highly complex operations in order to estimate the state of the deformable object. This complexity can be circumvented by utilizing learning-based approaches, such as reinforcement learning, which can deal with the intrinsic high-dimensional state space of deformable objects. However, the reward function in reinforcement learning needs to measure the state configuration of the highly deformable object. Vision-based reward functions are difficult to implement, given the high dimensionality of the state and complex dynamic behavior. In this work, we propose the consideration of concepts beyond vision and incorporate other modalities which can be extracted from deformable objects. By integrating tactile sensor cells into a textile piece, proprioceptive capabilities are gained that are valuable as they provide a reward function to a reinforcement learning agent. We demonstrate on a low-cost dual robotic arm setup that a physical agent can learn on a single CPU core to fold a rectangular patch of textile in the real world based on a learned reward function from tactile information.

In this paper, methodologies are presented for the development of intelligent robot systems for the manipulation of linear and sheet like objects with low and/or very low bending rigidity. In the introduction the non-rigid objects are defined and classified considering their shape, bending rigidity and extensibility. The industrial and service applications of these systems are presented and the state of the art approaches for the manipulation of various categories of the non-rigid objects are presented. A brief State-of the-Art on the manipulation of the deformable objects with relatively low bending rigidity and presenting elastic behavior like foam, sheet metal is presented as well.The main part of the paper is devoted to the robotic manipulation of the sheet-like objects with very low rigidity such as fabrics and leather. Laboratory demonstrators accompany the presentation of the developed intelligent robotic systems for manipulation of non-rigid objects and the paper concludes with hints for the future directions of the research and development in robotic systems for handling non-rigid objects.

In the grasping and manipulation of 3D deformable objects by robotic hands, the physical contact constraints between the fingers and the object have to be considered in order to validate the robustness of the task. Nevertheless, previous works rarely establish contact interaction models based on these constraints that enable the precise control of forces and deformations during the grasping process. This paper considers all steps of the grasping process of deformable objects in order to implement a complete grasp planning pipeline by computing the initial contact points (pregrasp strategy), and later, the contact forces and local deformations of the contact regions while the fingers close over the grasped object (grasp strategy). The deformable object behavior is modeled using a nonlinear isotropic mass-spring system, which is able to produce potential deformation. By combining both models (the contact interaction and the object deformation) in a simulation process, a new grasp planning method is proposed in order to guarantee the stability of the 3D grasped deformable object. Experimental grasping experiments of several 3D deformable objects with a Barrett hand (3-fingered) and a 6-DOF industrial robotic arm are executed. Not only will the final stable grasp configuration of the hand + object system be obtained, but an arm + hand approaching strategy (pregrasp) will also be computed.

We present a survey of recent work on robot manipulation and sensing of deformable objects, a field with relevant applications in diverse industries such as medicine (e.g. surgical assistance), food handling, manufacturing, and domestic chores (e.g. folding clothes). We classify the reviewed approaches into four categories based on the type of object they manipulate. Furthermore, within this object classification, we divide the approaches based on the particular task they perform on the deformable object. Finally, we conclude this survey with a discussion of the current state-of-the-art approaches and propose future directions within the proposed classification.

Although deformable linear objects (DLOs), such as cables, are widely used in the majority of life fields and activities, the robotic manipulation of these objects is considerably more complex compared to the rigid-body manipulation and still an open challenge. In this paper, we introduce a new framework using two robotic arms cooperatively manipulating a DLO from an initial shape to a desired one. Based on visual servoing and computer vision techniques, a perception approach is proposed to detect and sample the DLO as a set of virtual feature points. Then a manipulation planning approach is introduced to map between the motion of the manipulators end effectors and the DLO points by a Jacobian matrix. To avoid excessive stretching of the DLO, the planning approach generates a path for each DLO point forming profiles between the initial and desired shapes. It is guaranteed that all these intershape profiles are reachable and maintain the cable length constraint. The framework and the aforementioned approaches are validated in real-life experiments.

Abstract Object manipulation usually requires dexterity, encoded as the ability to roll, which is very difficult to achieve with robotic hands based on point contact models (subject to holonomic constraints). As an alternative for dexterous manipulation, deformable contact with hemispherical shape fingertips has been proposed to yield naturally a rolling constraint. It entails dexterity at the expense of dealing with normal and tangential forces, as well as more elaborated models and control schemes. Furthermore, the essential feature of the quality of grasp can be addressed with this type of robot hands, but it has been overlooked for deformable contact. In this paper, a passivity-based controller that considers an optimal grasping measure is proposed for robotic hands with hemispherical deformable fingertips, to manipulate circular dynamic objects. Optimal grasping that minimizes the contact wrenches is achieved through fingertip rolling until normal forces pass through the center of mass of the object, aligning the relative angle between these normal forces. The case of a circular object is developed in detail, though our proposal can be extended to objects with an arbitrary shape that admit a local decomposition by a circular curvature. Simulation and experimental results show convergence under various conditions, wherein rolling and tangent forces become instrumental to achieve such a quality of grasp.

Manipulating cloth-like deformable objects (CDOs) is a long-standing problem in the robotics community. CDOs are flexible (non-rigid) objects that do not show a detectable level of compression strength while two points on the article are pushed towards each other and include objects such as ropes (1D), fabrics (2D) and bags (3D). In general, CDOs’ many degrees of freedom (DoF) introduce severe self-occlusion and complex state–action dynamics as significant obstacles to perception and manipulation systems. These challenges exacerbate existing issues of modern robotic control methods such as imitation learning (IL) and reinforcement learning (RL). This review focuses on the application details of data-driven control methods on four major task families in this domain: cloth shaping, knot tying/untying, dressing and bag manipulation. Furthermore, we identify specific inductive biases in these four domains that present challenges for more general IL and RL algorithms.

This paper presents the design of a new force/tactile sensor for robotic applications. The sensor is suitably designed to provide the robotic grasping device with a sensory system mimicking the human sense of touch, namely, a device sensitive to contact forces, object slip and object geometry. This type of perception information is of paramount importance not only in dexterous manipulation but even in simple grasping task, especially when objects are fragile and deformable, such that only a minimum amount of grasping force can be applied to hold the object without damaging it.

At present, the tactile perception is essential for robotic applications when performing complex manipulation tasks, e.g., grasping objects of different shapes and sizes, distinguishing between different textures, and avoiding slips by grasping an object with a minimal force. Considering Deformable Linear Object manipulation applications, this paper presents an efficient and straightforward method to allow robots to autonomously work with thin objects, e.g., wires, and to recognize their features, i.e., diameter, by relying on tactile sensors developed by the authors. The method, based on machine learning algorithms, is described in-depth in the paper to make it easily reproducible by the readers. Experimental tests show the effectiveness of the approach that is able to properly recognize the considered object’s features with a recognition rate up to 99.9%. Moreover, a pick and place task, which uses the method to classify and organize a set of wires by diameter, is presented.

In robotic grasping and manipulation, the knowledge of a precise object pose represents a key issue. The point acquires even more importance when the objects and, then, the grasping areas become smaller. This is the case of Deformable Linear Object manipulation application where the robot shall autonomously work with thin wires which pose and shape estimation could become difficult given the limited object size and possible occlusion conditions. In such applications, a vision-based system could not be enough to obtain accurate pose and shape estimation. In this work the authors propose a Time-of-Flight pre-touch sensor, integrated with a previously designed tactile sensor, for an accurate estimation of thin wire pose and shape. The paper presents the design and the characterization of the proposed sensor. Moreover, a specific object scanning and shape detection algorithm is presented. Experimental results support the proposed methodology, showing good performance. Hardware design and software applications are freely accessible to the reader.

Abstract This paper discusses the design and development of a biomimetic robotic arm, elaborating on the experiments conducted with the developed arm to handle objects of diverse geometries, as well as evaluating its agility during grasping tasks. When automating fruit harvesting, it is crucial to minimize damage to leaves, as they play an essential role in the photosynthesis process. Thus, a versatile prehensile design is imperative for grasping fruits with various shapes. Existing technologies for harvesting fruit meant for processing are limited to soft, fresh fruit due to the risk of mechanical damage. As an alternative, a robotic system that emulates human fruit picking can improve fruit quality while maintaining efficiency. Consequently, a robotic hand with deformable fingers inspired by the human arm is developed. The robotic system must also be cost-effective. A single-gear motor is utilized to control the arm’s functions and ensure agile responsiveness when grasping objects with different shapes, incorporating a self-adaptive mechanism. During the development process, several grasping tests are conducted to evaluate the arm’s ability to handle basic shape primitives such as spheres and cylinders. The goal is to offer an alternative to manual fruit picking by creating a system capable of identifying, locating, and detaching fruit without causing damage to the fruit or tree. The robot is also equipped with A. In technology such as object detection and manipulation, the model is trained using a convolutional neural network for grasping the objects with appropriate pressures.

The manipulation of delicate objects remains a key challenge in the development of industrial robotic grippers. Magnetic force sensing solutions, which provide the required sense of touch, have been demonstrated in previous work. The sensors feature a magnet embedded within a deformable elastomer, which is mounted on top of a magnetometer chip. A key drawback of these sensors lies in the manufacturing process, which relies on the manual assembly of the magnet–elastomer transducer, impacting both the repeatability of measurements across sensors and the potential for a cost-effective solution through mass-manufacturing. In this paper, a magnetic force sensor solution is presented with an optimized manufacturing process that will facilitate mass production. The elastomer–magnet transducer was fabricated using injection molding, and the assembly of the transducer unit, on top of the magnetometer chip, was achieved using semiconductor manufacturing techniques. The sensor enables robust differential 3D force sensing within a compact footprint (5 mm × 4.4 mm × 4.6 mm). The measurement repeatability of these sensors was characterized over multiple samples and 300,000 loading cycles. This paper also showcases how the 3D high-speed sensing capabilities of these sensors can enable slip detection in industrial grippers.

It is difficult to introduce highly versatile automation using robots to handling deformable objects such as thread, cloth, wire, long beams, and thin plates in plant production processes, compared to the handling of rigid objects. Office equipment handles deformable objects such as paper and plastic. Problems unique to these objects is caused by speeding up such equipment and demand for upgrading its accuracy. In agriculture and medical care, automatic, intelligent handling of deformable objects such as fruit and animals has long been desired and practical systems sought. Deformable objects whose handling should be versatiley and accurately automated are classified into two groups based on handling: (A) Flexible, mostly thin, fine objects capable of elastic deformation (B) Soft objects easily crushed, such as soft fruits or animals The problem in handling the first group is controlling object deformation of an infinite degree of freedom with a finite number of manipulated variables. In contrast, a significant problem in handling the second group is often how to handle them without exerting excessive stress and how to handle them safely and reliably. The handling of these two groups differ greatly in mechanics and control theory, and this special issue focuses on the first group — flexible objects — mechanical collection and transport studies, control, and software. Recent studies on their handling are classified into four groups for convenience based on handled objects and types of handling task: (a) Control of deformation, internal force, and vibration or path planning of flexible objects (mainly thin plates and beams) using single or multiple manipulators. (b) Task understanding in insertion of elastic into rigid parts and vice versa, and the study of human skills to help robots accomplish these task. (c) Approaches on improved accuracy, intelligent control, and vibration damping in handling and transfer of sheets and strings with low flexural rigidity, represented by paper or wire. (d) Strategies for grasping and unfolding sheets such as cloth whose flexural rigidity is almost nil. For (a), studies are active on deformation control by two robot hands attempting to grasp cloth. 1-3) In the automobile industry, so-called flexible fixtureless assembly systems are advancing in which two robots process or assemble parts in mid-air without a fixed table to reduce lead time and cost. These systems are mostly developed assuming handled parts are rigid. Nguyen et al. work assuming parts such as sheet metal whose deformation must be taken into consideration.1) Nakagaki et al. propose form estimation that considers even plastic deformation in wire handling by robots, in connection with the development of robots for electric wire installation.4) Many studies cover flexible wire as elastic beams,3-9) but comparatively few focus on bending deformation of thin plates. This special edition includes a paper by Kosuge et al. on thin-plate deformation control. Vibration control of grasped objects becomes important as speed increases. Matsuno kindly contributed his paper on optimum path planning in elastic plate handling. In controlling the deformation of elastic bodies, the mechanics of objects handled is often unknown. This special issue features a paper by Kojima et al. on an approach to this problem by adaptive feed-forward control. For (b), we consider three cases: (1) A cylindrical rigid body inserted into a hole on an elastic plate. (2) An elastic bar inserted into a hole on a rigid body. (3) A tubular elastic body put on a cylindrical rigid body. This special issue carries papers on these problems by Brata et al., Matsuno et al., and Hirai. For (2), a paper by Nakagaki et al.10) covers electric wire installation. For (3), the paper by Shima et al.11) covers insertion of a rigid axis into an elastic hose. Robot skill acquisition is an important issue in robotics in general, and the above papers should prove highly interesting and information because they treat studies by comparing robot and human skills in accomplishing work and acquiring concrete skills knowledge. For (c), attempts are made to theoretically analyze sheet handling mechanisms and control developed based on trial and error, and to structure design theory based on such analysis. These attempts are related to the increased accuracy and speed and enhanced intelligence of sheet-handling office automation equipment such as printers, facsimile machines, copiers, and automated teller machines. Yoshida et al. conducted a series of studies on the effects of guides forming paper feed paths and of inertia force of paper by approximating sheets with a chain of discrete masses and springs.12-14) This special edition also features a study on sheet sticking and jamming. Okuna et al. handles a system of similar nature, mechanical studying the form of paper guides.15) Introducing mechanisms to control the positioning of sheets is effective in raising sheet transfer accuracy. Feedback control that regulates feed roller skew angle as a manipulated variable is proposed.16) Increased reliability in separating single sheets from stacked effectively reduces the malfunction rate in sheet-handling equipment. Ways of optimizing the form of sheet-separation rollers17) and estimating frictional force between separation gates and sheets 18) are also proposed. This special issue contains a proposal by Nakazawa et al. of a mechanism that uses reactive sheet buckling force, made in connection with development of a newspaper page turner for the disabled as technology for separating single sheets. Dry frictional force is most widely used for transporting sheets, but is not stable and may even act as an obstacle to improving accuracy. Niino et al. propose a sheet transfer mechanism that uses electrostatic force.19) For improving the accuracy of flexible wire transmission, this special issue carries a study on transporting flexible thin wire through tension control at multiple points, from a study by Morimitsu et al. on optical fiber installation. The thickness of wire used in equipment is becoming increasingly slim and flexible, along with the equipment it is used in. Tension control in the production process is an important factor in the manufacture of such thin wire. Production efficiency constantly calls for increased transfer speed. It has thus become important to estimate air resistance and inertia and to measure and control the tension of running wire. Studies20,21) by Batra, Fraser, et al. which deal the motion of string in the spinning process provide good examples for learning analytical techniques for air drag and inertia. In string vibration where inertia dominates, attempts are made to control vibration by boundary shaking22,23) and feed-forward/back control.24) For (d), highly versatile robots for handling cloth are being developed, and the software technology for automatic cloth selection and unfolding by robot hands is a popular topic.25-27) Ono et al. comment on the nature of problems in developing intelligent systems for handling cloth and similar objects whose bending rigidity is low and which readily fold and overlap—a paper that will prove a good reference in basic approaches in this field. Mechanical analyses are indispensable to studies on (a) through (c). In contrast, information technology such as characteristic variable measurement, image processing, and discrimination, rather than mechanical analyses, play an important roles in studies on (d). This special issue features a study by Hamashima, Uraya et al. on cloth unfolding as an example of such studies. Studies up to now largely assumed that properties of grasped objects did not change environmental influences such as temperature and humidity. Such influence is often, however, a major factor in handling fiber thread and cloth. This special issue has a paper contributed by Taylor, who studies handling method to prevent influence by such environmental factors. The objective of this special issue will have been achieved if it aids those studying the handling of flexible objects by providing approaches and methodologies of researchers whose target objects differ and if it aids those planning to take up study in this field by providing a general view of this field. References: 1) Nguyen, W. and Mills, J., ""Multi-Robot Control For Plexible Fixtureless Assembly of Flexible Sheet Metal Auto Body Parts,"" Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 2340-2345, (1996). 2) Sun, D. and Shi, X. and Liu, Y., ""Modeling and Cooperation of Two-Arm Robotic System Manipulating a Deformable Object,"" Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 2346-2351, (1996). 3) Kosuge, K., Sakaki, M., Kanitani, K., Yoshida, H. and Fukuda, T., ""Manipulation of a Flexible Object by Dual Manipulators,"" IEEE International Conference on Robotics and Automation, 318-323, (1995). 4) Nakagaki, H., Kitagaki, K., Ogasawara, T. and Tukune H., ""Handling of a Flexible Wire -Detecting a Deformed Shape of the Wire by Vision and a Force Sensor,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'96), 207-210, (1996). 5) Wakamatsu, H., Hirai, S. and Iwata, K., ""Static Analysis of Deformable Object Grasping Based on Bounded Force Closure,"" Trans. of JSML, 84-618 (C), 508-515, (1998). 6) Katoh, R. and Fujmoto, T., ""Study on Deformation of Elastic Object By Manipulator -Path Planning of End -Effector-,"" J. of the Robotics Society of Japan, 13-1, 157-160, (1995). 7) Yukawa, T., Uohiyama, M. and Inooka, M., ""Stability of Control System in Handling a Flexible Object by Rigid Arm Robots,"" JSME Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 169-172, (1995). 8) Yukawa, T., Uohiyama, M. and Cbinata, G., ""Handling of a Vibrating Flexible Structure by a Robot,"" Trans. JSME, 61-583, 938-943, (1995). 9) Sun, D. and Liu, Y., ""Modeling and Impedance Control of a Two-Manipulator System Handling a Flexible Beam,"" Trans. of the ASME, 119, 736-742, (1997). 10) Nakagaki, H., Kitagaki, K. and Tukune, H., ""Contact Motion in Inserting a Flexible Wire into a Hole,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 175-178, (1995). 11) Shimaji, S., Brata, A. and Hattori, H., ""Robot Skill in Assembling a Cylinder into an Elastic Hose,"" Annual Conference on Robotics and Mechatronics (ROBOMEC'95), 752-755, (1995). 12) Yoshida, K. and Kawauchi, M., ""The Analysis of Deformation and Behavior of Flexible Materials (1st Reprt, Study of Spring-Mass Beam Model of the Sheet,"" Trans. of JSME, 58-552, 1474-1480, (1992). 13) Yoshida, K., ""Analysis of Deformation and Behavior of Flexible Materials (2nd Report, Static Analysis for Deformation of the Sheet in the Space Formed by Guide Plates),"" Trans. JSME, 60-570, 501-507, (1994). 14) Yoshida, K., ""Dynamic Analysis of Sheet Defofmation Using Spring-Mass-Beam Model,"" Trans. JSME, 63-615, 3926-3932 (1997). 15) Okuna, K., Nishigaito, T. and Shina, Y., ""Analysis of Paper Deformation Considering Guide Friction (Improvement of Paper Path for Paper-Feeding Mechanism),"" Trans. JSME, 60-575, 2279-2284, (1994). 16) Fujimura, H. and Ono, K., ""Analysis of Paper Motion Driven by Skew-Roll Paper Feeding System,"" Trans. JSME, 62-596, 1354-1360, (1996). 17) Shima, Y., Hattori, S., Kobayashi, Y. and Ukai, M., ""Optimum of Gate-Roller Shape in Paper Isolating Methods,"" Conference of Information, Intelligence and Precision Equipment (IIP'96), 61-62, (1996). 18) Suzuki, Y, Hattori, S., Shima, Y. and Ukai, M., ""Contact Analysis of Paper in Gate-Roller Handling Method"", Conference on Information, Intelligence and Precision Equipment (IIP'95), 19-20, (1995). 19) Niino, T., Egawa, S. and Higuchi, T., ""An Electrostatic Paper Feeder,"" J. of the Japan Society for Precision Engineering, 60-12,1761-1765, (1994). 20) Batra, S., Ghosh, T. and Zeidman, M., ""An Integrated Approach to Dynamic Analysis of the Ring Spinning Process , PartII: With Air Drag,"" Textile Research Journal, 59, 416-424, (1989). 21) Fraser, W., Ghosh, T. and Batra, S., ""On Unwinding Yarn from a Cylindrical Package,"" Proceedings of Royal Society of London, A, 436, 479-438, (1992). 22) Jacob, S., ""Control of Vibrating String Using Impedance Matching,"" Proceedings of the American Control Conference (San Francisco),468-472, (1993). 23) Lee, S. and Mote, C., ""Vibration Control of an Axially Moving String by Boundary Control,"" Trans. of the ASME, J. of Dynamic Systems, Measurement, and Control, 118, 66-74, (1996). 24) Ying, S. and Tan, C., ""Active Vibration Control of the Axially Moving String Using Space Feedforward and Feedback Controllers,"" Trans. ASME, J. of Vibration and Acoustics, 118, 306-312, (1996). 25) Ono, E., Ichijo, H. and Aisaka, N., ""Flexible Robotic Hand for Handling Fabric Pieces in Garment Manufacture,"" International Journal of Clothing Science and Technology, 4-5,18-23, (1992). 26) Paraschidis, K., Fahantidis, N, Petridis, V., Doulgeri, Z., Petrou, L. and Hasapis, G, ""A Robotic System for Handling Textile and Non Rigid Flat Materials,"" Computers in Industry, 26, 303-313, (1995). 27) Fahantidis, N., Paraschidis, K, Petridis, V., Doulgeri, Z., Petrou, L. and Hasapis, G., ""Robot Handling of Flat Textile Materials,"" IEEE Robotics & Automation Magazine, 4-1, 34-41, (1997).

Modeling deformable objects is an important preliminary step for performing robotic manipulation tasks with more autonomy and dexterity. Currently, generalization capabilities in unstructured environments using analytical approaches are limited, mainly due to the lack of adaptation to changes in the object shape and properties. Therefore, this paper proposes the design and implementation of a data-driven approach, which combines machine learning techniques on graphs to estimate and predict the state and transition dynamics of deformable objects with initially undefined shape and material characteristics. The learned object model is trained using RGB-D sensor data and evaluated in terms of its ability to estimate the current state of the object shape, in addition to predicting future states with the goal to plan and support the manipulation actions of a robotic hand.

Deformable linear objects (DLOs), such as rods, cables, and ropes, play important roles in daily life. However, manipulation of DLOs is challenging as large geometrically nonlinear deformations may occur during the manipulation process. This problem is made even more difficult as the different deformation modes (e.g., stretching, bending, and twisting) may result in elastic instabilities during manipulation. In this paper, we formulate a physics-guided data-driven method to solve a challenging manipulation task—accurately deploying a DLO (an elastic rod) onto a rigid substrate along various prescribed patterns. Our framework combines machine learning, scaling analysis, and physical simulations to develop a physics-based neural controller for deployment. We explore the complex interplay between the gravitational and elastic energies of the manipulated DLO and obtain a control method for DLO deployment that is robust against friction and material properties. Out of the numerous geometrical and material properties of the rod and substrate, we show that only three non-dimensional parameters are needed to describe the deployment process with physical analysis. Therefore, the essence of the controlling law for the manipulation task can be constructed with a low-dimensional model, drastically increasing the computation speed. The effectiveness of our optimal control scheme is shown through a comprehensive robotic case study comparing against a heuristic control method for deploying rods for a wide variety of patterns. In addition to this, we also showcase the practicality of our control scheme by having a robot accomplish challenging high-level tasks such as mimicking human handwriting, cable placement, and tying knots.

Abstract Deformable linear objects (DLOs) such as ropes, cables, and surgical sutures have a wide variety of uses in automotive engineering, surgery, and electromechanical industries. Therefore, modeling of DLOs as well as a computationally efficient way to predict the DLO behavior is of great importance, in particular to enable robotic manipulation of DLOs. The main motivation of this work is to enable efficient prediction of the DLO behavior during robotic manipulation. In this paper, the DLO is modeled by a multivariate dynamic spline, while a symplectic integration method is used to solve the model iteratively by interpolating the DLO shape during the manipulation process. Comparisons between the symplectic, Runge–Kutta, and Zhai integrators are reported. The presented results show the capabilities of the symplectic integrator to overcome other integration methods in predicting the DLO behavior. Moreover, the results obtained with different sets of model parameters integrated by means of the symplectic method are reported to show how they influence the DLO behavior estimation.

AbstractService robotics is a fast-developing sector, requiring embedded intelligence into robotic platforms to interact with the humans and the surrounding environment. One of the main challenges in the field is robust and versatile manipulation in everyday life activities. An appealing opportunity is to exploit compliant end-effectors to address the manipulation of deformable objects. However, the intrinsic compliance of such grippers results in increased difficulties in grasping control. Within the described context, this work addresses the problem of optimizing the grasping of deformable objects making use of a compliant, under-actuated, sensorless robotic hand. The main aim of the paper is, therefore, finding the best position and joint configuration for the mentioned robotic hand to grasp an unforeseen deformable object based on collected RGB image and partial point cloud. Due to the complex grasping dynamics, learning-from-simulations approaches (e.g., Reinforcement Learning) are not effective in the faced context. Thus, trial-and-error-based methodologies have to be exploited. In order to save resources, a samples-efficient approach has to be employed. Indeed, a Bayesian approach to address the optimization of the grasping strategy is proposed, enhancing it with transfer learning capabilities to exploit the acquired knowledge to grasp (partially) new objects. A PAL Robotics TIAGo (a mobile manipulator with a 7-degrees-of-freedom arm and an anthropomorphic underactuated compliant hand) has been used as a test platform, executing a pouring task while manipulating plastic (i.e., deformable) bottles. The sampling efficiency of the data-driven learning is shown, compared to an evenly spaced grid sampling of the input space. In addition, the generalization capability of the optimized model is tested (exploiting transfer learning) on a set of plastic bottles and other liquid containers, achieving a success rate of the 88%.

AbstractOver the past decades, robotic automation has expanded in numerous industrial sectors; however, manufacturing operations involving the manipulation of non-rigid products are mostly preserved manual. The dynamic distortion of flexible objects underlines limitations in robot cognition and dexterity. Inspired by the gaps in composites industry automation, this paper presents a novel multifunctional robot end-effector for the robotic automation of composites layup. Aiming a holistic confrontation of layup challenges, the proposed end-effector incorporates tools for (a) the manipulation of sheet materials, like composite fabrics of variable dimensions; (b) the grasping of core materials, like foam blocks; (c) the handling of peripheral tools; (d) the exertion of fitting forces; and (e) the application of resin in diverse mold geometry cavities. The meticulous design of the end-effector’s configuration allows for consistent and collision-free tool usage with no excess robot tool changes, granting even higher productivity and efficiency. The enhancement in robot dexterity and proficiency for layup operations is validated within an automotive case, where semi-automation is intended for the improvement in overall workstation’s efficiency, quality, ergonomics, and well-being.

Compliant flexure joints have been widely used for cable-driven soft robotic hands and grippers due to their safe interaction with humans and objects. This paper presents a soft and compliant revolute flexure joint based on the auxetic cellular mechanical metamaterials with a heterogeneous structure. The heterogeneous architecture of the proposed metamaterial flexure joint (MFJ), which is inspired by the human finger joints, provides mechanically tunable multi-stiffness bending motion and large range of bending angle in comparison to conventional flexure joints. The multi-level variation of the joint stiffness over the range of bending motion can be tuned through the geometrical parameters of the cellular mechanical metamaterial unit cells. The proposed flexure joints are 3D printed with single flexible material in monolithic fashion using a standard benchtop 3D printer. The application of the MFJ is demonstrated in robotic in-hand manipulation and grasping thin and deformable objects such as wires and cables. The results show the capability and advantages of the proposed MFJ in soft robotic grippers and highly functional bionic hands.

The National Institute of Standards and Technology is developing performance tests and associated artifacts to benchmark research in the area of robotic assembly. Sets of components consistent with mechanical assemblies including screws, gears, electrical connectors, wires, and belts are configured for assembly or disassembly using a task board concept. Test protocols accompany the task boards and are designed to mimic low-volume, high-mixture assembly challenges typical to small and medium sized manufacturers. In addition to the typical rigid components found in assembled products, the task boards include many non-rigid component operations representative of wire harness and belt drive assemblies to support research in the area of grasping and manipulation of deformable objects, an area still considered to be an emerging research problem in robotics. A set of four primary task boards as well as competition task boards are presented as benchmarks along with scoring metrics and a method to compare robot system assembly times with human performance. Competitions are used to raise awareness to these benchmarks. Tools to progress and compare research are described along with emphasis placed on system competition-based solutions to grasp and manipulate deformable task board components.

Deformable object manipulation (DOM) with point clouds has great potential as nonrigid 3D shapes can be measured without detecting and tracking image features. However, robotic shape control of deformable objects with point clouds is challenging due to: the unknown point correspondences and the noisy partial observability of raw point clouds; the modeling difficulties of the relationship between point clouds and robot motions. To tackle these challenges, this paper introduces a novel modal-graph framework for the model-free shape servoing of deformable objects with raw point clouds. Unlike the existing works studying the object’s geometry structure, we propose a modal graph to describe the low-frequency deformation structure of the DOM system, which is robust to the measurement irregularities. The modal graph enables us to directly extract low-dimensional deformation features from raw point clouds without extra processing of registrations, refinements, and occlusion removal. It also preserves the spatial structure of the DOM system to inverse the feature changes into robot motions. Moreover, as the framework is built with unknown physical and geometric object models, we design an adaptive robust controller to deform the object toward the desired shape while tackling the modeling uncertainties, noises, and disturbances online. The system is proved to be input-to-state stable (ISS) using Lyapunov-based methods. Extensive experiments are conducted to validate our method using linear, planar, tubular, and volumetric objects under different settings.

This paper addresses the general problem of deformable linear object manipulation. The main application we consider is in the field of agriculture, for plant grasping, but may have interests in other tasks such as human daily activities and industrial production. We specifically consider an elastic linear object where one of its endpoints is fixed, and another point can be grasped by a robotic arm. To deal with the mentioned problem, we propose a model-free method to control the state of an arbitrary point that can be at any place along the object’s length. Our approach allows the robot to manipulate the object without knowing any model parameters or offline information of the object’s deformation. An adaptive control strategy is proposed for regulating the state of any point automatically deforming the object into the desired location. A control law is developed to regulate the object’s shape thanks to the adaptive estimation of the system parameters and its states. This method can track a desired manipulation trajectory to reach the target point, which leads to a smooth deformation without drastic changes. A Lyapunov-based argument is presented for the asymptotic convergence of the system that shows the process’s stability and convergence to desired state values. To validate the controller, numerical simulations involving two different deformation models are conducted, and performances of the proposed algorithm are investigated through full-scale experiments.