English

When Experts Disagree: Characterizing Annotator Variability for Vessel Segmentation in DSA Images

Image and Video Processing 2025-08-15 v1 Computer Vision and Pattern Recognition
Authors: M. Geshvadi , G. So , D. D. Chlorogiannis , C. Galvin , E. Torio , A. Azimi , Y. Tachie-Baffour , N. Haouchine , A. Golby , M. Vangel , W. M. Wells , Y. Epelboym , R. Du , F. Durupinar , S. Frisken
View on arXiv ↗ PDF ↗

Abstract

We analyze the variability among segmentations of cranial blood vessels in 2D DSA performed by multiple annotators in order to characterize and quantify segmentation uncertainty. We use this analysis to quantify segmentation uncertainty and discuss ways it can be used to guide additional annotations and to develop uncertainty-aware automatic segmentation methods.

Keywords

medical image segmentation

Cite

@article{arxiv.2508.10797,
  title  = {When Experts Disagree: Characterizing Annotator Variability for Vessel Segmentation in DSA Images},
  author = {M. Geshvadi and G. So and D. D. Chlorogiannis and C. Galvin and E. Torio and A. Azimi and Y. Tachie-Baffour and N. Haouchine and A. Golby and M. Vangel and W. M. Wells and Y. Epelboym and R. Du and F. Durupinar and S. Frisken},
  journal= {arXiv preprint arXiv:2508.10797},
  year   = {2025}
}

Related papers

View all related →
Computer Vision and Pattern Recognition · Computer Science
Diversified and Personalized Multi-rater Medical Image Segmentation
Yicheng Wu, Xiangde Luo, Zhe Xu, Xiaoqing Guo +4
2024-03-21
Image and Video Processing · Electrical Eng. & Systems
Simulation-Based Segmentation of Blood Vessels in Cerebral 3D OCTA Images
Bastian Wittmann, Lukas Glandorf, Johannes C. Paetzold, Tamaz Amiranashvili +3
2024-03-13
Computer Vision and Pattern Recognition · Computer Science
Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures
Xiaofeng Liu, Fangxu Xing, Thibault Marin, Georges El Fakhri +1
2022-01-19
Image and Video Processing · Electrical Eng. & Systems
Guidelines for Cerebrovascular Segmentation: Managing Imperfect Annotations in the context of Semi-Supervised Learning
Pierre Rougé, Pierre-Henri Conze, Nicolas Passat, Odyssée Merveille
2024-04-03
Computer Vision and Pattern Recognition · Computer Science
Partial Vessels Annotation-based Coronary Artery Segmentation with Self-training and Prototype Learning
Zheng Zhang, Xiaolei Zhang, Yaolei Qi, Guanyu Yang
2023-07-11
Image and Video Processing · Electrical Eng. & Systems
PHiSeg: Capturing Uncertainty in Medical Image Segmentation
Christian F. Baumgartner, Kerem C. Tezcan, Krishna Chaitanya, Andreas M. Hötker +5
2019-07-29
Image and Video Processing · Electrical Eng. & Systems
D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation
Zahra Mirikharaji, Kumar Abhishek, Saeed Izadi, Ghassan Hamarneh
2021-05-04
Computer Vision and Pattern Recognition · Computer Science
What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?
Kumar Abhishek, Jeremy Kawahara, Ghassan Hamarneh
2025-08-14
Image and Video Processing · Electrical Eng. & Systems
Spatiotemporal Disentanglement of Arteriovenous Malformations in Digital Subtraction Angiography
Kathleen Baur, Xin Xiong, Erickson Torio, Rose Du +5
2024-02-16
Image and Video Processing · Electrical Eng. & Systems
Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models
Tomer Amit, Shmuel Shichrur, Tal Shaharabany, Lior Wolf
2023-06-16
Image and Video Processing · Electrical Eng. & Systems
Learning Robust Medical Image Segmentation from Multi-source Annotations
Yifeng Wang, Luyang Luo, Mingxiang Wu, Qiong Wang +1
2023-11-14
Machine Learning · Computer Science
Supervised Uncertainty Quantification for Segmentation with Multiple Annotations
Shi Hu, Daniel Worrall, Stefan Knegt, Bas Veeling +2
2022-05-31
Image and Video Processing · Electrical Eng. & Systems
3D Vessel Segmentation with Limited Guidance of 2D Structure-agnostic Vessel Annotations
Huai Chen, Xiuying Wang, Lisheng Wang
2023-02-08
Image and Video Processing · Electrical Eng. & Systems
The RETA Benchmark for Retinal Vascular Tree Analysis
Xingzheng Lyu, Li Cheng, Sanyuan Zhang
2021-11-24
Computer Vision and Pattern Recognition · Computer Science
A small note on variation in segmentation annotations
Silas Nyboe Ørting
2020-12-04
Image and Video Processing · Electrical Eng. & Systems
Adaptive Semi-Supervised Segmentation of Brain Vessels with Ambiguous Labels
Fengming Lin, Yan Xia, Nishant Ravikumar, Qiongyao Liu +2
2023-08-08
Computer Vision and Pattern Recognition · Computer Science
Aleatoric Uncertainty Medical Image Segmentation Estimation via Flow Matching
Phi Van Nguyen, Ngoc Huynh Trinh, Duy Minh Lam Nguyen, Phu Loc Nguyen +1
2026-04-08
Computer Vision and Pattern Recognition · Computer Science
Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation (CURVAS) challenge results
Meritxell Riera-Marin, Sikha O K, Julia Rodriguez-Comas, Matthias Stefan May +28
2025-10-15
Computer Vision and Pattern Recognition · Computer Science
Segmentation Style Discovery: Application to Skin Lesion Images
Kumar Abhishek, Jeremy Kawahara, Ghassan Hamarneh
2024-08-07
Image and Video Processing · Electrical Eng. & Systems
Give me (un)certainty -- An exploration of parameters that affect segmentation uncertainty
Katharina Hoebel, Ken Chang, Jay Patel, Praveer Singh +1
2019-11-18
Image and Video Processing · Electrical Eng. & Systems
Uncertainty Quantification in Medical Image Segmentation with Multi-decoder U-Net
Yanwu Yang, Xutao Guo, Yiwei Pan, Pengcheng Shi +2
2021-09-16
Computer Vision and Pattern Recognition · Computer Science
Using Soft Labels to Model Uncertainty in Medical Image Segmentation
João Lourenço Silva, Arlindo L. Oliveira
2021-09-28
Computer Vision and Pattern Recognition · Computer Science
Disentangling Human Error from the Ground Truth in Segmentation of Medical Images
Le Zhang, Ryutaro Tanno, Mou-Cheng Xu, Chen Jin +4
2020-10-26
Image and Video Processing · Electrical Eng. & Systems
Explaining Uncertainty in Multiple Sclerosis Cortical Lesion Segmentation Beyond Prediction Errors
Nataliia Molchanova, Pedro M. Gordaliza, Alessandro Cagol, Mario Ocampo--Pineda +13
2026-04-21
Computer Vision and Pattern Recognition · Computer Science
Variability Matters : Evaluating inter-rater variability in histopathology for robust cell detection
Cholmin Kang, Chunggi Lee, Heon Song, Minuk Ma +1
2022-10-12
R2 v1 2026-07-01T04:50:14.173Z