Counterfactual image generation enables controlled data augmentation, bias mitigation, and disease modeling. However, existing methods guided by external classifiers or regressors are limited to subject-level factors (e.g., age) and fail to produce localized structural changes, often resulting in global artifacts. Pixel-level guidance using segmentation masks has been explored, but requires user-defined counterfactual masks, which are tedious and impractical. Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT) addressed this by using segmentation-derived measurements to supervise structure-specific variables, yet it remains restricted to global interventions. We propose Positional Seg-CFT, which subdivides each structure into regional segments and derives independent measurements per region, enabling spatially localized and anatomically coherent counterfactuals. Experiments on coronary CT angiography show that Pos-Seg-CFT generates realistic, region-specific modifications, providing finer spatial control for modeling disease progression.
@article{arxiv.2603.21213,
title = {Positional Segmentor-Guided Counterfactual Fine-Tuning for Spatially Localized Image Synthesis},
author = {Tian Xia and Matthew Sinclair and Andreas Schuh and Fabio De Sousa Ribeiro and Raghav Mehta and Rajat Rasal and Esther Puyol-Antón and Samuel Gerber and Kersten Petersen and Michiel Schaap and Ben Glocker},
journal= {arXiv preprint arXiv:2603.21213},
year = {2026}
}