As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memory (OoM) errors occur. It is well known that OoM interrupts the whole training itself and wastes substantial computational resources. Therefore, to prevent OoM, accurate prediction of GPU memory usage is essential. However, previous studies focus only on unimodal architectures and fail to generalize to multimodal models, even though the multimodal models are a common choice in agentic AI systems. To address this limitation, we propose a framework that predicts the peak GPU memory usage by analyzing the model architecture and training behavior of multimodal models. Specifically, the framework decomposes the multimodal model into its constituent layers and applies factorization to estimate the memory usage of each layer. Our evaluation shows that our framework achieves high prediction accuracy of ~8.7% average MAPE.
@article{arxiv.2512.07853,
title = {GPU Memory Prediction for Multimodal Model Training},
author = {Jinwoo Jeong and Minchul Kang and Younghun Go and Changyong Shin and Hyunho Lee and Junho Yoon and Gyeongsik Yang and Chuck Yoo},
journal= {arXiv preprint arXiv:2512.07853},
year = {2025}
}
Comments
1st Workshop on Systems for Agentic AI (SAA '25), co-located with SOSP 2025