We propose a framework enabling mobile manipulators to reliably complete pick-and-place tasks for assembling structures from construction blocks. The picking uses an eye-in-hand visual servoing controller for object tracking with Control Barrier Functions (CBFs) to ensure fiducial markers in the blocks remain visible. An additional robot with an eye-to-hand setup ensures precise placement, critical for structural stability. We integrate human-in-the-loop capabilities for flexibility and fault correction and analyze robustness to camera pose errors, proposing adapted barrier functions to handle them. Lastly, experiments validate the framework on 6-DoF mobile arms.
Cite
@article{arxiv.2504.12506,
title = {Robust Visual Servoing under Human Supervision for Assembly Tasks},
author = {Victor Nan Fernandez-Ayala and Jorge Silva and Meng Guo and Dimos V. Dimarogonas},
journal= {arXiv preprint arXiv:2504.12506},
year = {2025}
}
Comments
This work has been submitted to the IEEE for possible publication