MotionBenchMaker: A Tool to Generate and Benchmark Motion Planning Datasets
Abstract
Recently, there has been a wealth of development in motion planning for robotic manipulation new motion planners are continuously proposed, each with their own unique strengths and weaknesses. However, evaluating new planners is challenging and researchers often create their own ad-hoc problems for benchmarking, which is time-consuming, prone to bias, and does not directly compare against other state-of-the-art planners. We present MotionBenchMaker, an open-source tool to generate benchmarking datasets for realistic robot manipulation problems. MotionBenchMaker is designed to be an extensible, easy-to-use tool that allows users to both generate datasets and benchmark them by comparing motion planning algorithms. Empirically, we show the benefit of using MotionBenchMaker as a tool to procedurally generate datasets which helps in the fair evaluation of planners. We also present a suite of 40 prefabricated datasets, with 5 different commonly used robots in 8 environments, to serve as a common ground to accelerate motion planning research.
Keywords
Cite
@article{arxiv.2112.06402,
title = {MotionBenchMaker: A Tool to Generate and Benchmark Motion Planning Datasets},
author = {Constantinos Chamzas and Carlos Quintero-Peña and Zachary Kingston and Andreas Orthey and Daniel Rakita and Michael Gleicher and Marc Toussaint and Lydia E. Kavraki},
journal= {arXiv preprint arXiv:2112.06402},
year = {2022}
}
Comments
accepted in IEEE Robotics and Automation Letters (RAL), 2022. Supplementary video: https://youtu.be/t96Py0QX0NI Code: https://github.com/KavrakiLab/motion_bench_maker