English

Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization

Computer Vision and Pattern Recognition 2026-03-16 v1

Abstract

Generalizing image classification across domains remains challenging in critical tasks such as fundus image-based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multimodal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively.

Keywords

Cite

@article{arxiv.2603.12369,
  title  = {Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization},
  author = {Ayan Banerjee and Kuntal Thakur and Sandeep Gupta},
  journal= {arXiv preprint arXiv:2603.12369},
  year   = {2026}
}
R2 v1 2026-07-01T11:17:29.754Z