We characterize the GPU energy usage of two widely adopted exascale-ready applications representing two classes of particle and mesh solvers: (i) QMCPACK, a quantum Monte Carlo package, and (ii) AMReXCastro, an adaptive mesh astrophysical code. We analyze power, temperature, utilization, and energy traces from double-/single (mixed)-precision benchmarks on NVIDIA's A100 and H100 and AMD's MI250X GPUs using queries in NVML and rocm_smi_lib, respectively. We explore application-specific metrics to provide insights on energy vs. performance trade-offs. Our results suggest that mixed-precision energy savings range between 6-25% on QMCPACK and 45% on AMReX-Castro. Also, we found gaps in the AMD tooling used on Frontier GPUs that need to be understood, while query resolutions on NVML have little variability between 1 ms-1 s. Overall, application level knowledge is crucial to define energy-cost/science-benefit opportunities for the codesign of future supercomputer architectures in the post-Moore era.
@article{arxiv.2505.05623,
title = {Characterizing GPU Energy Usage in Exascale-Ready Portable Science Applications},
author = {William F. Godoy and Oscar Hernandez and Paul R. C. Kent and Maria Patrou and Kazi Asifuzzaman and Narasinga Rao Miniskar and Pedro Valero-Lara and Jeffrey S. Vetter and Matthew D. Sinclair and Jason Lowe-Power and Bobby R. Bruce},
journal= {arXiv preprint arXiv:2505.05623},
year = {2025}
}
Comments
13 pages, 8 figures, 3 tables. Accepted at the Energy Efficiency with Sustainable Performance: Techniques, Tools, and Best Practices, EESP Workshop, in conjunction with ISC High Performance 2025