Non-determinism and non-reproducibility present significant challenges in deep learning, leading to inconsistent results across runs and platforms. These issues stem from two origins: random number generation and floating-point computation. While randomness can be controlled through deterministic configurations, floating-point inconsistencies remain largely unresolved. To address this, we introduce RepDL, an open-source library that ensures deterministic and bitwise-reproducible deep learning training and inference across diverse computing environments. RepDL achieves this by enforcing correct rounding and order invariance in floating-point computation. The source code is available at https://github.com/microsoft/RepDL .
@article{arxiv.2510.09180,
title = {RepDL: Bit-level Reproducible Deep Learning Training and Inference},
author = {Peichen Xie and Xian Zhang and Shuo Chen},
journal= {arXiv preprint arXiv:2510.09180},
year = {2025}
}