Scalable Climate Data Analysis: Balancing Petascale Fidelity and Computational Cost
Abstract
The growing resolution and volume of climate data from remote sensing and simulations pose significant storage, processing, and computational challenges. Traditional compression or subsampling methods often compromise data fidelity, limiting scientific insights. We introduce a scalable ecosystem that integrates hierarchical multiresolution data management, intelligent transmission, and ML-assisted reconstruction to balance accuracy and efficiency. Our approach reduces storage and computational costs by 99\%, lowering expenses from $100,000 to $24 while maintaining a Root Mean Square (RMS) error of 1.46 degrees Celsius. Our experimental results confirm that even with significant data reduction, essential features required for accurate climate analysis are preserved. Validated on petascale NASA climate datasets, this solution enables cost-effective, high-fidelity climate analysis for research and decision-making.
Cite
@article{arxiv.2507.08006,
title = {Scalable Climate Data Analysis: Balancing Petascale Fidelity and Computational Cost},
author = {Aashish Panta and Amy Gooch and Giorgio Scorzelli and Michela Taufer and Valerio Pascucci},
journal= {arXiv preprint arXiv:2507.08006},
year = {2025}
}
Comments
Presented at The CCGRID International Scalable Computing Challenge (SCALE), 2025