We introduce a latent 3D representation that models 3D surfaces as probability density functions in 3D, i.e., p(x,y,z), with flow-matching. Our representation is specifically designed for consumption by machine learning models, offering continuity and compactness by construction while requiring only point clouds and minimal data preprocessing. Despite being a data-driven method, our use of flow matching in the 3D space enables interesting geometry properties, including the capabilities to perform zero-shot estimation of surface normal and deformation field. We evaluate with several machine learning tasks, including 3D-CLIP, unconditional generative models, single-image conditioned generative model, and intersection-point estimation. Across all experiments, our models achieve competitive performance to existing baselines, while requiring less preprocessing and auxiliary information from training data.
@article{arxiv.2412.15618,
title = {3D Shape Tokenization via Latent Flow Matching},
author = {Jen-Hao Rick Chang and Yuyang Wang and Miguel Angel Bautista Martin and Jiatao Gu and Xiaoming Zhao and Josh Susskind and Oncel Tuzel},
journal= {arXiv preprint arXiv:2412.15618},
year = {2025}
}