Multimodal data integration and label scarcity pose significant challenges for machine learning in medical settings. To address these issues, we conduct an in-depth evaluation of the newly proposed Multimodal Variational Mixture-of-Experts (MMVM) VAE on the challenging MIMIC-CXR dataset. Our analysis demonstrates that the MMVM VAE consistently outperforms other multimodal VAEs and fully supervised approaches, highlighting its strong potential for real-world medical applications.
@article{arxiv.2411.10356,
title = {Weakly-Supervised Multimodal Learning on MIMIC-CXR},
author = {Andrea Agostini and Daphné Chopard and Yang Meng and Norbert Fortin and Babak Shahbaba and Stephan Mandt and Thomas M. Sutter and Julia E. Vogt},
journal= {arXiv preprint arXiv:2411.10356},
year = {2024}
}
Comments
Findings paper presented at Machine Learning for Health (ML4H) symposium 2024, December 15-16, 2024, Vancouver, Canada, 13 pages. arXiv admin note: text overlap with arXiv:2403.05300