Interfacing Soft and Hard: A Spring Reinforced Actuator
[Video: Interfacing Soft and Hard: A Spring Reinforced Actuator.]
Muscular hydrostats have long been a source of inspiration for soft robotic designs. With their inherent compliance, they excel in unpredictable environments and can gently manipulate objects with ease. However, their performance lacks where high force or a fast-dynamic response is needed. In this project, we propose a novel spring reinforced actuator (SRA) that explores the intermediate state between muscular hydrostats and endoskeletal mechanisms. The result is that we dramatically enhance the robot dynamic performance, which is unprecedented in similar kinds of soft robots, while retaining compliant omni-directional bending. Analytical modelling of the flexible backbone was built and experimentally validated. This is also the first attempt to perform detailed finite element analysis (FEA) to investigate the strain-stress behavior of the constraining braided bellow tube. The braided interweaving threads are modeled, wherein complex thread-to-thread contacts occurs. Experimental evaluation of SRAs was performed for actuation force, stiffness, and dynamic response. We showcase the enhanced actuator’s performance in several applications such as locomotion and heavy object manipulation.
Real-time Surface Shape Sensing for Soft and Flexible Structures
[Video: Real-time Surface Shape Sensing for Soft and Flexible Structures.]
We present a new soft and flexible sensor which can reconstruct its surface shape in real-time. A single-core optical fiber with fiber Bragg gratings (FBGs) is capable of detecting sparse local strains at high bandwidth using wavelength-division multiplexing (WDM). The fiber was embedded into an elastomeric substrate to reconstruct its global surface morphology. Finite element analysis (FEA) was used to determine the design parameters, and also to validate the unique mapping from sparse strain measurements to the continuum shape of the sensor. To simplify the fabrication and error compensation process without precise/prior knowledge of the FBG locations in the sensor, machine learning-based modelling was applied. This enables real-time, robust and reliable shape reconstruction. It is demonstrated to outperform various applications of electronics-based sensors which require sophisticated electrode wiring and noise reduction.
Large-scale surface shape sensing with learning-based computational mechanics
[Video: Large-scale surface shape sensing with learning-based computational mechanics
In a step forward to realize robots with proprioception, we propose a flexible sensor framework that incorporates a novel hybrid modeling strategy, taking advantage of computational mechanics and machine learning. We implement the sensor framework on a large, thin and flexible sensor that transforms sparsely distributed strains into continuous surface shape. Finite element (FE) analysis is utilized to determine sensor design parameters, while an FE model is built to enrich the morphological data used in the supervised training to achieve continuous surface reconstruction. A mapping between the local strain data and the enriched surface data is subsequently trained using ensemble learning. This hybrid approach enables real-time, robust and high-order surface shape reconstruction, which has not been demonstrated and reported on such a large-scale (A4-paper-size) sensor before.
Vision-based Online Learning Kinematic Control for Soft Robots
[Video: Vision-based Online Learning Kinematic Control for Soft Robots.]
Soft robots, owing to their elastomeric material, ensure safe interaction with their surroundings. These robot compliance properties inevitably impose a trade-off against precise motion control, as to which conventional model-based methods were proposed to approximate the robot kinematics. However, too many parameters, regarding robot deformation and external disturbance, are difficult to obtain, even if possible, which could be very nonlinear. Sensors self-contained in the robot are required to compensate modelling uncertainties and external disturbances. Camera (eye) integrated at the robot end-effector (hand) is a common setting. We are investigating an eye-in-hand visual servo that incorporates with learning-based controller to accomplish more precise robotic tasks. Our work first demonstrates vision-based path following for a hyper-elastic robot with heavy variable loading (up to 105% of the robot weight). The enhanced accuracy and adaptability can bring new opportunities to minimally invasive surgical applications, such as soft robotic endoscopy and laparoscopy, as they can take advantage of existing camera feedback.
Soft Robot Visual Servoing Enhanced with Sparse Strain Measurement
[Video: Soft Robot Visual Servoing Enhanced with Sparse Strain Measurement.]
In the feature/object tracking of eye-in-hand visual servoing, 2D motion estimation relying only on image plane feedback is easily affected by vision occlusion, blurring, or poor lighting. For the commonly-used template matching method, tracking performance greatly depends on the image quality. Fiber Bragg gratings (FBGs), a type of high-frequency flexible strain sensor, can be used as an assistant device for soft robot control. We propose a method to enhance motion estimation in soft robotic visual servoing by fusing the results from template matching and FBG feedback. Sparse strain measurement of the single-core FBG fiber could be trained as an independent motion sensor, and combined with image processing to improve tracking accuracy. Path following performance is validated in a simulated laparoscopic scene and LEGO-constructed scene, demonstrating significant improvement to feature tracking, even under external forces. The enhanced estimation method increased tracking accuracy by 82.3% when capturing the dark and feature-deficient liver surface. It also showed stable performance under imitated endoscopic irrigation.
Online Learning Control for Effective Endoscopic Navigation
[Video: Online learning control for effective endoscopic navigation.]
Bio-inspired robotic structures composed of soft actuation units have attracted increasing research interest. Taking advantage of its inherent compliance, soft robots can assure safe interaction with external environments, provided that precise and effective manipulation could be achieved. However, previous model-based control approaches often require simplified geometric assumptions on the soft manipulator, but which could be very inaccurate in the presence of unmodeled external interaction forces. We are investigating model-free control methods that do not require prior knowledge of the robot’s structural parameters. A generic control framework based on a nonparametric online learning technique is developed, which in the inverse model is acquired directly. As a result, a soft continuum robot can precisely follow a 3D trajectory, even under dynamical external disturbance (e.g. push). Such enhanced control accuracy and adaptability would therefore facilitate effective manipulation in complex and changing environments such as endoscopic navigation. Furthermore, finite element analysis (FEA) using advanced element formulations is also adopted. We study the dynamical response by considering different materials, actuation methods, and external loadings. Based on the FEA realization, not only the geometrical and structural design parameters can be optimized, but also the simulation data can be used to initialize the model-free control policy, hence eliminating the need for random exploration in the robot’s workspace.
 T.L.T. Lun, K. Wang, J.D.L. Ho, K.H. Lee, K.Y. Sze, K.W. Kwok,"Real-time Surface Shape Sensing for Soft and Flexible Structures using Fiber Bragg Gratings," IEEE Robotics and Automation Letters (RA-L) 4(2):1454-1461, 2019 Detail
 G. Fang, X. Wang, K. Wang, K.H. Lee, J.D.L. Ho, H.C. Fu, D.K.C. Fu, K.W. Kwok,"Vision-based Online Learning Kinematic Control for Soft Robots using Local Gaussian Process Regression," IEEE Robotics and Automation Letters (RA-L) 4(2):1194-1201, 2019 Detail
 K.H. Lee, M.C.W. Leong, M.C.K. Chow, H.C. Fu, D.K.C. Fu, Wayne Luk, K.Y. Sze, C.K. Yeung, K.W. Kwok, "FEM-based Soft Robotic Control Framework for Intracavitary Navigation," IEEE International Conference on Real-time Computing and Robotics (RCAR), pp.11-16,2017. Detail
 K.H. Lee, D.K. Fu, M.C. Leong, M. Chow, H.C. Fu, K. Althoefer, K.Y. Sze, C.K. Yeung, and K.W. Kwok, "Nonparametric Online Learning Control for Soft Continuum Robot: An Enabling Technique for Effective Endoscopic Navigation," Soft Robotics, vol. 4, no. 4, pp. 324-337, 2017. Detail
 H.C. Fu, J.D.L. Ho, K.H. Lee, Y.C. Hu, K.W. Au, K.J. Cho, K.Y. Sze, K.W. Kwok, "Interfacing soft and hard: a spring reinforced actuator," Soft Robotics, 2020;7(1):44-58. Detail
 K. Wang, C.H. Mak, J.D.L. Ho, Z.Y, Liu, K. Y. Sze, K.K.Y. Wong, K. Althoefer, Y.H. Liu, T. Fukuda, K.W. Kwok, "Large-scale surface shape sensing with learning-based computational mechanics", Advanced Intelligent Systems. (under review).
1. Best Conference Paper Award in the IEEE International Conference on Real-time Computing and Robotics (RCAR) 2017.
Authors and title: K.H. Lee, M.C.W. Leong, M.C.K Chow, H.C. Fu, W. Luk, K.Y. Sze, C.K. Yeung and K.W. Kwok, "FEM-based Soft Robotic Control Framework for Intracavitary Navigation."
1. PCT Patent: Endoscopic Systems, Devices, and Methods for Performing In Vivo Procedures, App No. PCT/CN2016/070906 (Filed on Jan 14, 2016.)
2. PRC Patent: Endoscopic Systems, Devices, and Methods for Performing In Vivo Procedures, App No. 201610147810.9 (Filed on March 15, 2016.)
3. US Provisional Patent: Endoscopic Systems, Devices, and Methods for Performing In Vivo Procedures, App No. US 14/985,587 (Filed on Dec 31, 2015.)