English

Brain3D: Brain Report Automation via Inflated Vision Transformers in 3D

Computer Vision and Pattern Recognition 2026-02-26 v1

Abstract

Current medical vision-language models (VLMs) process volumetric brain MRI using 2D slice-based approximations, fragmenting the spatial context required for accurate neuroradiological interpretation. We developed \textbf{Brain3D}, a staged vision-language framework for automated radiology report generation from 3D brain tumor MRI. Our approach inflates a pretrained 2D medical encoder into a native 3D architecture and progressively aligns it with a causal language model through three stages: contrastive grounding, supervised projector warmup, and LoRA-based linguistic specialization. Unlike generalist 3D medical VLMs, \textbf{Brain3D} is tailored to neuroradiology, where hemispheric laterality, tumor infiltration patterns, and anatomical localization are critical. Evaluated on 468 subjects (BraTS pathological cases plus healthy controls), our model achieves a Clinical Pathology F1 of 0.951 versus 0.413 for a strong 2D baseline while maintaining perfect specificity on healthy scans. The staged alignment proves essential: contrastive grounding establishes visual-textual correspondence, projector warmup stabilizes conditioning, and LoRA adaptation shifts output from verbose captions to structured clinical reports\footnote{Our code is publicly available for transparency and reproducibility

Keywords

Cite

@article{arxiv.2602.22098,
  title  = {Brain3D: Brain Report Automation via Inflated Vision Transformers in 3D},
  author = {Mariano Barone and Francesco Di Serio and Giuseppe Riccio and Antonio Romano and Marco Postiglione and Antonino Ferraro and Vincenzo Moscato},
  journal= {arXiv preprint arXiv:2602.22098},
  year   = {2026}
}
R2 v1 2026-07-01T10:52:23.165Z