Continual learning (CL) enables models to adapt to evolving data streams without catastrophic forgetting, a fundamental requirement for real-world AI systems. However, the current methods often depend on large replay buffers or heavily annotated datasets which are impractical due to storage, privacy, and cost constraints. We propose CLTS (Continual Learning via Text-Image Synergy), a novel class-incremental framework that mitigates forgetting without storing real task data. CLTS leverages pre-trained vision-language models, BLIP (Bootstrapping Language-Image Pre-training) for caption generation and stable diffusion for sample generation. Each task is handled by a dedicated Task Head, while a Task Router learns to assign inputs to the correct Task Head using the generated data. On three benchmark datasets, CLTS improves average task accuracy by up to 54% and achieves 63 times better memory efficiency compared to four recent continual learning baselines, demonstrating improved retention and adaptability. CLTS introduces a novel perspective by integrating generative text-image augmentation for scalable continual learning.
@article{arxiv.2409.17806,
title = {Caption, Create, Continue: Continual Learning with Pre-trained Generative Vision-Language Models},
author = {Indu Solomon and Aye Phyu Phyu Aung and Uttam Kumar and Senthilnath Jayavelu},
journal= {arXiv preprint arXiv:2409.17806},
year = {2025}
}
Comments
This is the revised and peer-reviewed version of our paper, accepted and published in the Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM 2025)