English

Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing

Computer Vision and Pattern Recognition 2025-12-30 v1 Robotics

Abstract

Text-based 3D motion generation aims to automatically synthesize diverse motions from natural-language descriptions to extend user creativity, whereas motion editing modifies an existing motion sequence in response to text while preserving its overall structure. Pose-code-based frameworks such as CoMo map quantifiable pose attributes into discrete pose codes that support interpretable motion control, but their frame-wise representation struggles to capture subtle temporal dynamics and high-frequency details, often degrading reconstruction fidelity and local controllability. To address this limitation, we introduce pose-guided residual refinement for motion (PGR2^2M), a hybrid representation that augments interpretable pose codes with residual codes learned via residual vector quantization (RVQ). A pose-guided RVQ tokenizer decomposes motion into pose latents that encode coarse global structure and residual latents that model fine-grained temporal variations. Residual dropout further discourages over-reliance on residuals, preserving the semantic alignment and editability of the pose codes. On top of this tokenizer, a base Transformer autoregressively predicts pose codes from text, and a refine Transformer predicts residual codes conditioned on text, pose codes, and quantization stage. Experiments on HumanML3D and KIT-ML show that PGR2^2M improves Fr\'echet inception distance and reconstruction metrics for both generation and editing compared with CoMo and recent diffusion- and tokenization-based baselines, while user studies confirm that it enables intuitive, structure-preserving motion edits.

Keywords

Cite

@article{arxiv.2512.22464,
  title  = {Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing},
  author = {Sukhyun Jeong and Yong-Hoon Choi},
  journal= {arXiv preprint arXiv:2512.22464},
  year   = {2025}
}
R2 v1 2026-07-01T08:42:23.125Z