Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics
Abstract
Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the core algorithm, VHGM-MAE, a masked autoencoder (MAE) tailored for handling high-dimensional, sparse healthcare data. VHGM-MAE tackles four key technical challenges: (1) heterogeneity of healthcare data types, (2) probability distribution modeling, (3) systematic missingness in the training dataset arising from multiple data sources, and (4) the high-dimensional, small--large- problem. To address these challenges, VHGM-MAE employs a likelihood-based approach to model distributions with heterogeneous types, a transformer-based MAE to capture complex dependencies among observed and missing attributes, and a novel training scheme that effectively leverages available samples with diverse missingness patterns to mitigate the small-n-large-p problem. Experimental results demonstrate that VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation.
Keywords
Cite
@article{arxiv.2306.10656,
title = {Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics},
author = {Kenta Oono and Nontawat Charoenphakdee and Kotatsu Bito and Zhengyan Gao and Hideyoshi Igata and Masashi Yoshikawa and Yoshiaki Ota and Hiroki Okui and Kei Akita and Shoichiro Yamaguchi and Yohei Sugawara and Shin-ichi Maeda and Kunihiko Miyoshi and Yuki Saito and Koki Tsuda and Hiroshi Maruyama and Kohei Hayashi},
journal= {arXiv preprint arXiv:2306.10656},
year = {2025}
}