Effectively modeling multimodal longitudinal data is a pressing need in various application areas, especially biomedicine. Despite this, few approaches exist in the literature for this problem, with most not adequately taking into account the multimodality of the data. In this study, we developed multiple configurations of a novel multimodal and longitudinal learning framework, Longitudinal Ensemble Integration (LEI), for sequential classification. We evaluated LEI's performance, and compared it against existing approaches, for the early detection of dementia, which is among the most studied multimodal sequential classification tasks. LEI outperformed these approaches due to its use of intermediate base predictions arising from the individual data modalities, which enabled their better integration over time. LEI's design also enabled the identification of features that were consistently important across time for the effective prediction of dementia-related diagnoses. Overall, our work demonstrates the potential of LEI for sequential classification from longitudinal multimodal data.
@article{arxiv.2411.05983,
title = {Longitudinal Ensemble Integration for sequential classification with multimodal data},
author = {Aviad Susman and Rupak Krishnamurthy and Yan Chak Li and Mohammad Olaimat and Serdar Bozdag and Bino Varghese and Nasim Sheikh-Bahaei and Gaurav Pandey},
journal= {arXiv preprint arXiv:2411.05983},
year = {2025}
}
Comments
Accepted to IEEE ICDH 2025. This is the author's accepted manuscript (AAM). The final version will appear in the IEEE ICDH 2025 proceedings on IEEE Xplore