Segmentation architectures are typically benchmarked on single imaging modalities, obscuring deployment-relevant performance variations: an architecture optimal for one modality may underperform on another. We present a cross-modal evaluation framework for materials image segmentation spanning SEM, AFM, XCT, and optical microscopy. Our evaluation of six encoder-decoder combinations across seven datasets reveals that optimal architectures vary systematically by context: UNet excels for high-contrast 2D imaging while DeepLabv3+ is preferred for the hardest cases. The framework also provides deployment feedback via out-of-distribution detection and counterfactual explanations that reveal which microstructural features drive predictions. Together, the architecture guidance, reliability signals, and interpretability tools address a practical gap in materials characterization, where researchers lack tools to select architectures for their specific imaging setup or assess when models can be trusted on new samples.
@article{arxiv.2602.04154,
title = {Context Determines Optimal Architecture in Materials Segmentation},
author = {Mingjian Lu and Pawan K. Tripathi and Mark Shteyn and Debargha Ganguly and Roger H. French and Vipin Chaudhary and Yinghui Wu},
journal= {arXiv preprint arXiv:2602.04154},
year = {2026}
}