English

SpatialPrompt: XR-Based Spatial Intent Expression as Executable Constraints for AI Generative 3D Design

Human-Computer Interaction 2026-05-11 v1

Abstract

We present SpatialPrompt, an Extended Reality(XR) system that turns spatial sketches into executable constraints for controllable 3D generation. Users draw rough structures with a 3D pen and add voice prompts for semantic and stylistic intent. The system supports iterative refinement and synchronous co-creation in shared space with color-coded contributions. Implemented on Apple Vision Pro with Logitech Muse and Meshy, a heuristic evaluation suggests that the workflow is intuitive and supports shared understanding in collaborative creation, while revealing needs for faster generation and clearer feedback.

Keywords

Cite

@article{arxiv.2605.07894,
  title  = {SpatialPrompt: XR-Based Spatial Intent Expression as Executable Constraints for AI Generative 3D Design},
  author = {Yichen Andy Yu and Wanru Li and Qiaoran Wang and Jymon Ross and Gavin Johnson and Mandy Lui and Qiao Jin},
  journal= {arXiv preprint arXiv:2605.07894},
  year   = {2026}
}
R2 v1 2026-07-01T12:58:01.619Z