This paper presents a novel concept for intuitive end-user programming of robots, inspired by natural interaction between humans. Natural language and supportive gestures are translated into robot programs using large language models (LLMs) and computer vision (CV). Through equally natural system feedback in the form of clarification questions and visual representations, the generated program can be reviewed and adjusted, thereby ensuring safety, transparency, and user acceptance.
@article{arxiv.2604.05978,
title = {Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics},
author = {Bijan Kavousian and Petar Tesic and Oliver Petrovic and Christian Brecher},
journal= {arXiv preprint arXiv:2604.05978},
year = {2026}
}
Comments
This submission contains both an English translation and the original German version. The German version was originally published in the Proceedings of the 71st GfA Conference (2025)