Behavioral cloning becomes difficult when the same observation admits several valid actions. We study this problem for action-chunking policies and show that different multimodal parameterizations fail in different ways. For latent-variable policies, posterior-prior regularization makes deployment-time sampling more reliable, but excessive regularization removes the action-conditioned information needed to distinguish demonstrated modes. Reducing this regularization can preserve mode information, but then success depends on whether the prior covers the relevant latent regions. For action-space generative policies, multimodality is constrained by the smoothness of the base-to-action transport: a map with small Lipschitz constant cannot assign substantial probability to many well-separated modes. Covering many modes therefore requires either sharp transitions in base space or off-support bridge regions in action space. Experiments on synthetic multimodal tasks and robotic simulation benchmarks support these mechanisms.
Cite
@article{arxiv.2605.22493,
title = {Understanding Multimodal Failure in Action-Chunking Behavioral Cloning},
author = {Lorenzo Mazza and Massimiliano Datres and Ariel Rodriguez and Sebastian Bodenstedt and Gitta Kutyniok and Stefanie Speidel},
journal= {arXiv preprint arXiv:2605.22493},
year = {2026}
}