ClimSim-Online: A Large Multi-scale Dataset and Framework for Hybrid ML-physics Climate Emulation
Abstract
Modern climate projections lack adequate spatial and temporal resolution due to computational constraints, leading to inaccuracies in representing critical processes like thunderstorms that occur on the sub-resolution scale. Hybrid methods combining physics with machine learning (ML) offer faster, higher fidelity climate simulations by outsourcing compute-hungry, high-resolution simulations to ML emulators. However, these hybrid ML-physics simulations require domain-specific data and workflows that have been inaccessible to many ML experts. As an extension of the ClimSim dataset (Yu et al., 2024), we present ClimSim-Online, which also includes an end-to-end workflow for developing hybrid ML-physics simulators. The ClimSim dataset includes 5.7 billion pairs of multivariate input/output vectors, capturing the influence of high-resolution, high-fidelity physics on a host climate simulator's macro-scale state. The dataset is global and spans ten years at a high sampling frequency. We provide a cross-platform, containerized pipeline to integrate ML models into operational climate simulators for hybrid testing. We also implement various ML baselines, alongside a hybrid baseline simulator, to highlight the ML challenges of building stable, skillful emulators. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res) and code (https://leap-stc.github.io/ClimSim and https://github.com/leap-stc/climsim-online) are publicly released to support the development of hybrid ML-physics and high-fidelity climate simulations.
Cite
@article{arxiv.2306.08754,
title = {ClimSim-Online: A Large Multi-scale Dataset and Framework for Hybrid ML-physics Climate Emulation},
author = {Sungduk Yu and Zeyuan Hu and Akshay Subramaniam and Walter Hannah and Liran Peng and Jerry Lin and Mohamed Aziz Bhouri and Ritwik Gupta and Björn Lütjens and Justus C. Will and Gunnar Behrens and Julius J. M. Busecke and Nora Loose and Charles I. Stern and Tom Beucler and Bryce Harrop and Helge Heuer and Benjamin R. Hillman and Andrea Jenney and Nana Liu and Alistair White and Tian Zheng and Zhiming Kuang and Fiaz Ahmed and Elizabeth Barnes and Noah D. Brenowitz and Christopher Bretherton and Veronika Eyring and Savannah Ferretti and Nicholas Lutsko and Pierre Gentine and Stephan Mandt and J. David Neelin and Rose Yu and Laure Zanna and Nathan Urban and Janni Yuval and Ryan Abernathey and Pierre Baldi and Wayne Chuang and Yu Huang and Fernando Iglesias-Suarez and Sanket Jantre and Po-Lun Ma and Sara Shamekh and Guang Zhang and Michael Pritchard},
journal= {arXiv preprint arXiv:2306.08754},
year = {2024}
}
Comments
This manuscript is an expanded version of our paper that received the Outstanding Paper Award at the NeurIPS 2023 conference