Emerging Deep Learning (DL) applications introduce heavy I/O workloads on computer clusters. The inherent long lasting, repeated, and random file access pattern can easily saturate the metadata and data service and negatively impact other users. In this paper, we present FanStore, a transient runtime file system that optimizes DL I/O on existing hardware/software stacks. FanStore distributes datasets to the local storage of compute nodes, and maintains a global namespace. With the techniques of system call interception, distributed metadata management, and generic data compression, FanStore provides a POSIX-compliant interface with native hardware throughput in an efficient and scalable manner. Users do not have to make intrusive code changes to use FanStore and take advantage of the optimized I/O. Our experiments with benchmarks and real applications show that FanStore can scale DL training to 512 compute nodes with over 90\% scaling efficiency.
@article{arxiv.1809.10799,
title = {FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning},
author = {Zhao Zhang and Lei Huang and Uri Manor and Linjing Fang and Gabriele Merlo and Craig Michoski and John Cazes and Niall Gaffney},
journal= {arXiv preprint arXiv:1809.10799},
year = {2018}
}