The robot manipulation ecosystem currently faces issues with integrating open-source components and reproducing results. This limits the ability of the community to benchmark and compare the performance of different solutions to one another in an effective manner, instead relying on largely holistic evaluations. As part of the COMPARE Ecosystem project, we are developing modular grasping and manipulation pipeline infrastructure in order to streamline performance benchmarking. The infrastructure will be used towards the establishment of standards and guidelines for modularity and improved open-source development and benchmarking. This paper provides a high-level overview of the architecture of the pipeline infrastructure, experiments conducted to exercise it during development, and future work to expand its modularity.
@article{arxiv.2504.06819,
title = {Developing Modular Grasping and Manipulation Pipeline Infrastructure to Streamline Performance Benchmarking},
author = {Brian Flynn and Kostas Bekris and Berk Calli and Aaron Dollar and Adam Norton and Yu Sun and Holly Yanco},
journal= {arXiv preprint arXiv:2504.06819},
year = {2025}
}
Comments
IEEE International Conference on Robotics and Automation (ICRA) 2025, Workshop on Robot Software Architectures (RSA25), Atlanta, Georgia, USA, May 2025