A Stereovision-based Approach for Retrieving Variable Force Feedback in Robotic-Assisted Surgery Using Modified Inception ResNet V2 Networks

Abstract

AbstractThe surge of haptic technology has greatly impacted Robotic-assisted surgery in recent years due to its inspirational advancement in the field. Delivering tactile feedback to the surgeon has a significant role in improving the user experience in RAMIS. This work proposes a Modified inception ResNet network along with dimensionality reduction to regenerate the variable force produced during the surgical intervention. This work collects the relevant dataset from two ex vivo porcine skins and one ex vivo artificial skin for the validation of the results. The proposed framework is used to model both spatial and temporal data collected from the sensors, tissue, manipulators, and surgical tools. The evaluations are based on three distinct datasets with modest variations in tissue properties. The results of the proposed framework show an improvement of force prediction accuracy by 10.81% over RNN, 6.02% over RNN + LSTM, and 3.81% over the CNN + LSTM framework, and torque prediction accuracy by 12.41% over RNN, 5.75% over RNN + LSTM, and 3.75% over CNN + LSTM. The sensitivity study demonstrates that features such as torque (96.93%), deformation (94.02%), position (93.98%), vision (92.12%), stiffness (87.95%), tool diameter (89.24%), rotation (65.10%), and orientation (62.51%) have respective influences on the anticipated force. It was observed that the quality of the predicted force improved by 2.18% when performing feature selection and dimensionality reduction on features collected from tool, manipulator, tissue, and vision data and processing them simultaneously in all four architectures. The method has potential applications for online surgical tasks and surgeon training.