Academic literature on the topic 'Soft tissue simulation, Robotic surgery, Autonomous surgery'

SUMMARYA brief history of robotic surgery is provided, which describes the transition from autonomous robots to hands-on systems that are under the direct control of the surgeon. An example of the latter is the Acrobot (for active-constraint robot) system used in orthopaedics, whilst soft-tissue surgery is illustrated by the daVinci telemanipulator system. Non-technological aspects of robotic surgery have often been a major impediment to their widespread clinical use. These are discussed in detail, together with the role of navigation systems, which are considered a major competitor to surgical robots. A detailed description is then given of a registration method for robots to achieve improved accuracy. Registration is a major source of error in robotic surgery, particularly in orthopaedics. The paper describes the design and clinical implementation of a novel method, coined the bounded registration method, applied to minimally invasive registration of the femur. Results of simulations which compare the performance of bounded registration with a standard implementation of the iterative closest point algorithm are also presented, alongside a description of their application in the Acrobot hands-on robot, used clinically for uni-condylar knee arthroplasty.

Minimally invasive robotic surgery has gained wide acceptance recently. Computer-aided features to assist the surgeon during these interventions may help to develop safer, faster, and more accurate procedures. Especially physiological motion compensation of the beating heart and online soft tissue modelling are promising features that were developed recently. This paper presents the integration of these new features into the minimally invasive robotic surgery platform MiroSurge. A central aim of this research platform is to enable evaluation and comparison of new functionalities for minimally invasive robotic surgery. The system structure of MiroSurge is presented as well as the interfaces for the new functionalities. Some details about the modules for motion tracking and for soft tissue simulation are given. Results are shown with an experimental setup that includes a heart motion simulator and dedicated silicone organ models. Both features are integrated seamlessly and work reliably in the chosen setup. The MiroSurge platform thus shows the potential to provide valuable results in evaluating new functionalities for minimally invasive robotic surgery.

The rise and advancement of minimally invasive surgery (MIS) has significantly improved patient outcomes, yet its technical challenges—such as tissue manipulation and tissue retraction—are not yet overcome. Robotic surgery offers some compensation for the ergonomic challenges, as retraction typically requires an extra robotic arm, which makes the complete system more costly. Our research aimed to explore the potential of rapidly deployable structures for soft tissue actuation and retraction, developing clinical and technical requirements and putting forward a critically evaluated concept design. With systematic measurements, we aimed to assess the load capacities and force tolerance of different magnetic constructions. Experimental and simulation work was conducted on the magnetic coupling technology to investigate the conditions where the clinically required lifting force of 11.25 N could be achieved for liver retraction. Various structure designs were investigated and tested with N52 neodymium magnets to create stable mechanisms for tissue retraction. The simplified design of a new MIS laparoscopic instrument was developed, including a deployable structure connecting the three internal rod magnets with joints and linkages that could act as an actuator for liver retraction. The deployable structure was designed to anchor strings or bands that could facilitate the lifting or sideways folding of the liver creating sufficient workspace for the target upper abdominal procedures. The critical analysis of the project concluded a notable potential of the developed solution for achieving improved liver retraction with minimal tissue damage and minimal distraction of the surgeon from the main focus of the operation, which could be beneficial, in principle, even at robot-assisted procedures.

Background How do surgical residents learn to operate? What is a surgical plane? How does one learn to see and dissect the plane? How do surgical residents learn tissue handling and suturing (sewing)? One method to learn and practice performing surgery is through the use of simulation training. Surgical training models include laparoscopic box trainers (a plastic box with holes for instruments) with synthetic materials inside to simulate tissues, or computer-based virtual reality simulation for laparoscopic, endoscopic, and robotic techniques. These methods, however, do not use real tissues. They lack the haptic and kinesthetic feedback of real tissue. These simulations fail to recreate the fidelity of soft tissues, do not foster the ability to accurately see surgical planes, do not accurately mimic the act of dissecting surgical planes, do not allow for complex surgical procedures, and do not provide accurate experience to learn tissue handling and suturing. Despite their poor performance, these plastic and virtual trainers are extremely costly to purchase, maintain, and keep up to date - with prices starting at $700 for basic plastic training boxes to thousands of dollars for virtual simulation. Also, there are additional costs of maintenance and software curriculum. Despite the cost of software, virtual simulators do not include a simulation for every surgery. Our aim was to create a life-like surgical simulation as close to real world as possible that allows trainees to learn how to see and dissect surgical planes, learn how soft tissues move, and learn the dynamics of soft tissue manipulation. We created a laparoscopic simulator using porcine tissues for gallbladder removal, acid reflux surgery, and surgery to treat swallowing difficulties (cholecystectomy, Nissen fundoplication, and Heller myotomy, respectively). Second year general surgery residents were able to practice these procedures on real tissues, enabling them to learn the steps of each procedure, increase manual dexterity, improve use of laparoscopic equipment, all while maintaining life-like haptic, soft-tissue feedback and enabling them to develop the ability to see real surgical planes. Methods The abdomen was recreated by purchasing intact porcine liver, gallbladder, (Cholecystectomy simulation) and intact esophagus, stomach, and diaphragm (Nissen and Heller simulation) from a packing supplier. Each organ system was placed into a laparoscopic trainer box with the ability to re-create laparoscopic ports. Surgical residents were then able to perform the procedures using real laparoscopic instruments, laparoscopic camera/video imaging, and real-time electrocautery. The simulation included all critical steps of each procedure such as obtaining the critical view of safety and removing the gallbladder from the liver bed (cholecystectomy), wrapping the stomach around the esophagus and laparoscopic suturing (Nissen fundoplication), and dissecting the muscular portion of the esophageal wall (Heller myotomy). Because these porcine tissues were readily available, several stations were set-up to teach multiple residents during each session (10-12 residents / session). Discussion Surgeons develop haptic perception of soft tissues by cutaneous or tactile feedback and kinesthetic feedback (Okamura, 2009). Kinesthetic feedback is the force and pressure transmitted by the soft tissues along the shaft of the laparoscopic instruments (Okamura, 2009). This soft tissue simulation re-creates the ability to experience what soft tissue feedback feels like, outside a normal operative environment. Real tissue learning allows trainees to learn how to see surgical planes, learn how soft tissues feel and move, develop proficiency in surgical dissection, and learn how to suture laparoscopically. This is the only model that recreates the movement of soft tissues and visualization of dissection planes outside the operative environment. Because this model utilizes the laparoscopic instruments used in the operating room, residents also develop familiarity with laparoscopic instruments, thus, flattening another learning curve. A literature review found that this is the only real tissue simulation being performed for foregut procedures used specifically for resident training. By building a realistic, anatomical model with inherent accurate soft tissue surgical planes, surgical trainees can have a more realistic surgical experience and develop skills in a safe, low pressure environment without sacrificing the hepatic learning and surgical visualization that is critical to performing safe laparoscopic surgery. All residents that participated in the stimulation reported positive feedback and felt that is contributed to their surgical education.

Accurate kinematic models are essential for effective control of surgical robots. For tendon driven robots, which are common for minimally invasive surgery, the high nonlinearities in the transmission make modelling complex. Machine learning techniques are a preferred approach to tackle this problem. However, surgical environments are rarely structured, due to organs being very soft and deformable, and unpredictable, for instance, because of fluids in the system, wear and break of the tendons that lead to changes of the system’s behaviour. Therefore, the model needs to quickly adapt. In this work, we propose a method to learn the kinematic model of a redundant surgical robot and control it to perform surgical tasks both autonomously and in teleoperation. The approach employs Feedforward Artificial Neural Networks (ANN) for building the kinematic model of the robot offline, and an online adaptive strategy in order to allow the system to conform to the changing environment. To prove the capabilities of the method, a comparison with a simple feedback controller for autonomous tracking is carried out. Simulation results show that the proposed method is capable of achieving very small tracking errors, even when unpredicted changes in the system occur, such as broken joints. The method proved effective also in guaranteeing accurate tracking in teleoperation.

Összefoglaló. A technika folyamatos fejlődésével a robotikának és az adattudományoknak minden bizonnyal hasonló hatása lesz az invazív medicina egyes ágaira a következő 20 évben, mint a gyártástechnikára volt az elmúlt évtizedekben. A korai kép által vezetett sebészeti rendszereket és sebészrobotokat elsősorban pontosságuk és megbízhatóságuk miatt alkalmazzák, mivel segítségükkel kisebb szöveti sérülés mellett gyorsabban és biztonságosabban végezhetők el a beavatkozások, különösen az ortopéd- és idegsebészetben, ugyanakkor az igazi, globális áttörést a teleoperációs irányítás elven működő da Vinci Sebészeti Rendszernek köszönhetjük. A da Vinci neve egybeforrt a robotsebészettel, annak ellenére, hogy tucatnyi más rendszer is létezik ma már. Habár a teleoperációs Robot-asszisztált Minimál Invazív Sebészeti rendszerek esetében az eszközök irányítását mindvégig a sebész végzi a konzolon keresztül, az elmúlt években itt is egyre komolyabb szerepet kapott a preoperatív adatok integrációja, a műtéti navigációra épülő adatfúzió és a hibakompenzáció. A sebészeti döntéstámogatás és az esetleges hibák kiküszöbölése egyre nagyobb jelentőséget kap a távsebészeti alkalmazások esetén is. Alapvető fontosságúak a megfelelő algoritmusok a kommunikáció során fellépő torzítások, késleltetés és egyéb, akár rosszindulatú zavarjelek kezeléséhez. A robotos távsebészet koncepciója az amerikai NASA űrügynökségtől ered, és mind a mai napig aktívan kutatják a technológia nyújtotta további lehetőségeket, mivel a milliós számban végzett műtétekből származó adatok ma már teljesen más adattudományi módszerekkel dolgozhatók fel, így esély nyílt arra, hogy egy nap akár a lágyszöveti beavatkozásokat is autonóm sebészeti robotok hajtsák majd végre. A cikk célja megismertetni az olvasót e modern interdiszciplináris terület alapvető fogalmaival, bemutatni a fontosabb részterületeket és rendszereket. Áttekintést nyújtunk a távsebészet különböző formáiról, és képet adunk az adatvezérelt beavatkozások összetettségéről. Summary. With the continuous development of information technology, robotics and data science will certainly have a similar impact on invasive medicine over the next 20 years as it has had on manufacturing technology in the recent decades. Early image-guided systems and surgical robots were employed in the operating room primarily for their accuracy and reliability, as they allowed for faster and safer interventions with minimal tissue damage, targeting especially orthopedics and neurosurgery. On the other hand, a real global breakthrough came with the teleoperated da Vinci Surgical System, ideal for soft tissue procedures. The success and dominance of the da Vinci has dimmed the dozens of other surgical robots already on the market. It partially originated from the teleoperation concept of Robot-Assisted Minimally Invasive Surgery, where the full control of the robotic tools is always maintained by the human operator via the console. Nevertheless, the availability of data at large brings new possibilities, e.g., the in-view integration of preoperative data, data fusion based on surgical navigation, and error compensation have become increasingly available in prototypes. Surgical decision support and the elimination/eviction of potential errors also became increasingly important in telesurgical applications. Appropriate algorithms for handling distortions, delays, and other, even malicious, interference attempts during communication are essential. The concept of robotic telesurgery originates from NASA, and even as of today they are actively exploring the additional possibilities offered by cutting-edge technology to improve surgical systems using data science methods. In the not so distant future, even soft tissue interventions will be performed by autonomous robots. The aim of this article is to present the reader the basic concepts of this modern interdisciplinary field named Computer-Integrated Surgery, and to introduce the most important robots and robotic systems. We provide an overview of the different forms of telesurgery and describe the idea and the complexity of data-driven interventions.

Background. The revolutions in AI hold tremendous capacity to augment human achievements in surgery, but robust integration of deep learning algorithms with high-fidelity surgical simulation remains a challenge. We present a novel application of reinforcement learning (RL) for automating surgical maneuvers in a graphical simulation. Methods. In the Unity3D game engine, the Machine Learning-Agents package was integrated with the NVIDIA FleX particle simulator for developing autonomously behaving RL-trained scissors. Proximal Policy Optimization (PPO) was used to reward movements and desired behavior such as movement along desired trajectory and optimized cutting maneuvers along the deformable tissue-like object. Constant and proportional reward functions were tested, and TensorFlow analytics was used to informed hyperparameter tuning and evaluate performance. Results. RL-trained scissors reliably manipulated the rendered tissue that was simulated with soft-tissue properties. A desirable trajectory of the autonomously behaving scissors was achieved along 1 axis. Proportional rewards performed better compared to constant rewards. Cumulative reward and PPO metrics did not consistently improve across RL-trained scissors in the setting for movement across 2 axes (horizontal and depth). Conclusion. Game engines hold promising potential for the design and implementation of RL-based solutions to simulated surgical subtasks. Task completion was sufficiently achieved in one-dimensional movement in simulations with and without tissue-rendering. Further work is needed to optimize network architecture and parameter tuning for increasing complexity.

Autonomous robotic surgery has the potential to provide efficacy, safety, and consistency independent of individual surgeon’s skill and experience. Autonomous anastomosis is a challenging soft-tissue surgery task because it requires intricate imaging, tissue tracking, and surgical planning techniques, as well as a precise execution via highly adaptable control strategies often in unstructured and deformable environments. In the laparoscopic setting, such surgeries are even more challenging because of the need for high maneuverability and repeatability under motion and vision constraints. Here we describe an enhanced autonomous strategy for laparoscopic soft tissue surgery and demonstrate robotic laparoscopic small bowel anastomosis in phantom and in vivo intestinal tissues. This enhanced autonomous strategy allows the operator to select among autonomously generated surgical plans and the robot executes a wide range of tasks independently. We then use our enhanced autonomous strategy to perform in vivo autonomous robotic laparoscopic surgery for intestinal anastomosis on porcine models over a 1-week survival period. We compared the anastomosis quality criteria—including needle placement corrections, suture spacing, suture bite size, completion time, lumen patency, and leak pressure—of the developed autonomous system, manual laparoscopic surgery, and robot-assisted surgery (RAS). Data from a phantom model indicate that our system outperforms expert surgeons’ manual technique and RAS technique in terms of consistency and accuracy. This was also replicated in the in vivo model. These results demonstrate that surgical robots exhibiting high levels of autonomy have the potential to improve consistency, patient outcomes, and access to a standard surgical technique.

An Autonomous Robotic Surgical System (ARSS) has to interact with the complex anatomical environment, which is deforming and whose properties are often uncertain. Within this context, an ARSS can benefit from the availability of patient-specific simulation of the anatomy. For example, simulation can provide a safe and controlled environment for the design, test and validation of the autonomous capabilities. Moreover, it can be used to generate large amounts of patient-specific data that can be exploited to learn models and/or tasks. The aim of this Thesis is to investigate the different ways in which simulation can support an ARSS and to propose solutions to favor its employability in robotic surgery. We first address all the phases needed to create such a simulation, from model choice in the pre-operative phase based on the available knowledge to its intra-operative update to compensate for inaccurate parametrization. We propose to rely on deep neural networks trained with synthetic data both to generate a patient-specific model and to design a strategy to update model parametrization starting directly from intra-operative sensor data. Afterwards, we test how simulation can assist the ARSS, both for task learning and during task execution. We show that simulation can be used to efficiently train approaches that require multiple interactions with the environment, compensating for the riskiness to acquire data from real surgical robotic systems. Finally, we propose a modular framework for autonomous surgery that includes deliberative functions to handle real anatomical environments with uncertain parameters. The integration of a personalized simulation proves fundamental both for optimal task planning and to enhance and monitor real execution. The contributions presented in this Thesis have the potential to introduce significant step changes in the development and actual performance of autonomous robotic surgical systems, making them closer to applicability to real clinical conditions.

yes
Most of surgical simulators employ a linear elastic model to simulate soft tissue material properties due to its computational efficiency and the simplicity. However, soft tissues often have elaborate nonlinearmaterial characteristics. Most prominently, soft tissues are soft and compliant to small strains, but after initial deformations they are very resistant to further deformations even under large forces. Such material characteristic is referred as the nonlinear material incompliant which is computationally expensive and numerically difficult to simulate. This paper presents a constraint-based finite-element algorithm to simulate the nonlinear incompliant tissue materials efficiently for interactive simulation applications such as virtual surgery. Firstly, the proposed algorithm models the material stiffness behavior of soft tissues with a set of 3-D strain limit constraints on deformation strain tensors. By enforcing a large number of geometric constraints to achieve the material stiffness, the algorithm reduces the task of solving stiff equations of motion with a general numerical solver to iteratively resolving a set of constraints with a nonlinear Gauss–Seidel iterative process. Secondly, as a Gauss–Seidel method processes constraints individually, in order to speed up the global convergence of the large constrained system, a multiresolution hierarchy structure is also used to accelerate the computation significantly, making interactive simulations possible at a high level of details . Finally, this paper also presents a simple-to-build data acquisition system to validate simulation results with ex vivo tissue measurements. An interactive virtual reality-based simulation system is also demonstrated.

The main goal of surgical oncology is to achieve complete resection of cancerous tissue with minimal iatrogenic injury to the adjacent healthy structures. Brain tumour surgery is particularly demanding due to the eloquence of the tissue involved. There is evidence that increasing the extent of tumour resection substantially improves overall and progression-free survival. Realtime intraoperative tools which inform of residual disease are invaluable. Intraoperative Ultrasound (iUS) has been established as an efficient tool for tissue characterisation during brain tumour resection in neurosurgery [1]. The integration of iUS into the operating theatre is characterised by significant challenges related to the interpretation and quality of the US data. The capturing of high-quality US images requires substantial experi- ence and visuo-tactile skills during manual operation. Recently, robotically-controlled US scanning systems have been proposed (see e.g. [2]) but the scanning of brain tissue poses major challenges to robotic systems because of the safety-critical nature of the procedure, the very low and precise contact forces required, the narrow access space and the large variety of tissue properties (hard scull, soft brain structure). The aim of this paper is to introduce a robotic platform for autonomous iUS tissue scanning to optimise intraop- erative diagnosis and improve surgical resection during robot-assisted operations. To guide anatomy specific robotic scanning and generate a representation of the robot task space, fast and accurate techniques for the recovery of 3D morphological structures of the surgical cavity are developed. The prototypic DLR MIRO surgi- cal robotic arm [3] is used to control the applied force and the in-plane motion of the US transducer. A key application of the proposed platform is the scanning of brain tissue to guide tumour resection.