Silent data corruption (SDC) threatens the reliability of large-scale GPU clusters used for training large language models, yet its rarity and lack of explicit error signals make accurate high-level modeling challenging. To address this gap, we conducted a large-scale gate-level stuck-at fault injection on a production-class data-center GPU, consuming over three million simulator hours across 63 CUDA micro-benchmarks. We extracted GPU SDC characteristics in terms of corruption types, bit-flip behavior, and warp-aligned spatial correlation. Our results show that NaN/+INF/-INF account for only 1.01% of SDC outcomes, that single-bit flips constitute less than 40% of bit-flip events, and that corruption addresses exhibit periodicity. These statistics motivate distribution-aware high-level fault modeling and realistic software-based fault injection for resilience evaluation of production-class GPU architectures.
@article{arxiv.2605.04213,
title = {The Anatomy of Silent Data Corruption: GPU Error Pattern Study and Modeling Guidance},
author = {Chung-Hsuan Tung and Yanxiang Huang and Nirmal Saxena and Philip Shirvani and Saurabh Hukerikar and Twinkle Jain and Abhishek Tyagi and Sanjay Gongalore},
journal= {arXiv preprint arXiv:2605.04213},
year = {2026}
}
Comments
Accepted for publication in IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) 2026 (Industry Track)