Robotic surgery has been proven to offer clear advantages during surgical procedures, however, one of the major limitations is obtaining haptic feedback. Since it is often challenging to devise a hardware solution with accurate force feedback, we propose the use of "visual cues" to infer forces from tissue deformation. Endoscopic video is a passive sensor that is freely available, in the sense that any minimally-invasive procedure already utilizes it. To this end, we employ deep learning to infer forces from video as an attractive low-cost and accurate alternative to typically complex and expensive hardware solutions. First, we demonstrate our approach in a phantom setting using the da Vinci Surgical System affixed with an OptoForce sensor. Second, we then validate our method on an ex vivo liver organ. Our method results in a mean absolute error of 0.814 N in the ex vivo study, suggesting that it may be a promising alternative to hardware based surgical force feedback in endoscopic procedures.
@article{arxiv.1808.00057,
title = {Learning to See Forces: Surgical Force Prediction with RGB-Point Cloud Temporal Convolutional Networks},
author = {Cong Gao and Xingtong Liu and Michael Peven and Mathias Unberath and Austin Reiter},
journal= {arXiv preprint arXiv:1808.00057},
year = {2019}
}
Comments
MICCAI 2018 workshop, CARE(Computer Assisted and Robotic Endoscopy)