Recent motion-aware large language models have demonstrated promising potential in unifying motion comprehension and generation. However, existing approaches primarily focus on coarse-grained motion-text modeling, where text describes the overall semantics of an entire motion sequence in just a few words. This limits their ability to handle fine-grained motion-relevant tasks, such as understanding and controlling the movements of specific body parts. To overcome this limitation, we pioneer MG-MotionLLM, a unified motion-language model for multi-granular motion comprehension and generation. We further introduce a comprehensive multi-granularity training scheme by incorporating a set of novel auxiliary tasks, such as localizing temporal boundaries of motion segments via detailed text as well as motion detailed captioning, to facilitate mutual reinforcement for motion-text modeling across various levels of granularity. Extensive experiments show that our MG-MotionLLM achieves superior performance on classical text-to-motion and motion-to-text tasks, and exhibits potential in novel fine-grained motion comprehension and editing tasks. Project page: CVI-SZU/MG-MotionLLM
@article{arxiv.2504.02478,
title = {MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularities},
author = {Bizhu Wu and Jinheng Xie and Keming Shen and Zhe Kong and Jianfeng Ren and Ruibin Bai and Rong Qu and Linlin Shen},
journal= {arXiv preprint arXiv:2504.02478},
year = {2025}
}