PERSEUS: Perception with Semantic Endoscopic Understanding and SLAM
Abstract
Purpose: Natural orifice surgeries minimize the need for incisions and reduce the recovery time compared to open surgery; however, they require a higher level of expertise due to visualization and orientation challenges. We propose a perception pipeline for these surgeries that allows semantic scene understanding. Methods: We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together to create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images, and allow the use of semantically informed real-time reconstructions in robotic surgeries. Results: We achieve sub-milimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Conclusion: We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance.
Cite
@article{arxiv.2509.13541,
title = {PERSEUS: Perception with Semantic Endoscopic Understanding and SLAM},
author = {Ayberk Acar and Fangjie Li and Susheela Sharma Stern and Lidia Al-Zogbi and Hao Li and Kanyifeechukwu Jane Oguine and Dilara Isik and Brendan Burkhart and Jesse F. d'Almeida and Robert J. Webster and Ipek Oguz and Jie Ying Wu},
journal= {arXiv preprint arXiv:2509.13541},
year = {2026}
}
Comments
13 pages, 6 figures, 2 tables. Under review for The 17th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI 2026)