Easy Acceleration with Distributed Arrays
Abstract
High level programming languages and GPU accelerators are powerful enablers for a wide range of applications. Achieving scalable vertical (within a compute node), horizontal (across compute nodes), and temporal (over different generations of hardware) performance while retaining productivity requires effective abstractions. Distributed arrays are one such abstraction that enables high level programming to achieve highly scalable performance. Distributed arrays achieve this performance by deriving parallelism from data locality, which naturally leads to high memory bandwidth efficiency. This paper explores distributed array performance using the STREAM memory bandwidth benchmark on a variety of hardware. Scalable performance is demonstrated within and across CPU cores, CPU nodes, and GPU nodes. Horizontal scaling across multiple nodes was linear. The hardware used spans decades and allows a direct comparison of hardware improvements for memory bandwidth over this time range; showing a 10x increase in CPU core bandwidth over 20 years, 100x increase in CPU node bandwidth over 20 years, and 5x increase in GPU node bandwidth over 5 years. Running on hundreds of MIT SuperCloud nodes simultaneously achieved a sustained bandwidth 1 PB/s.
Cite
@article{arxiv.2508.17493,
title = {Easy Acceleration with Distributed Arrays},
author = {Jeremy Kepner and Chansup Byun and LaToya Anderson and William Arcand and David Bestor and William Bergeron and Alex Bonn and Daniel Burrill and Vijay Gadepally and Ryan Haney and Michael Houle and Matthew Hubbell and Hayden Jananthan and Michael Jones and Piotr Luszczek and Lauren Milechin and Guillermo Morales and Julie Mullen and Andrew Prout and Albert Reuther and Antonio Rosa and Charles Yee and Peter Michaleas},
journal= {arXiv preprint arXiv:2508.17493},
year = {2025}
}
Comments
8 pages, 4 figures, 2 tables, 2 algorithm listings, 2 code listings, to appear in IEEE HPEC 2025