MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces
Abstract
The fast pace of artificial intelligence~(AI) innovation demands an agile methodology for observation, reproduction and optimization of distributed machine learning~(ML) workload behavior in production AI systems and enables efficient software-hardware~(SW-HW) co-design for future systems. We present Chakra, an open and portable ecosystem for performance benchmarking and co-design. The core component of Chakra is an open and interoperable graph-based representation of distributed AI/ML workloads, called Chakra execution trace~(ET). These ETs represent key operations, such as compute, memory, and communication, data and control dependencies, timing, and resource constraints. Additionally, Chakra includes a complementary set of tools and capabilities to enable the collection, analysis, generation, and adoption of Chakra ETs by a broad range of simulators, emulators, and replay tools. We present analysis of Chakra ETs collected on production AI clusters and demonstrate value via real-world case studies. Chakra has been adopted by MLCommons and has active contributions and engagement across the industry, including but not limited to NVIDIA, AMD, Meta, Keysight, HPE, and Scala, to name a few.
Cite
@article{arxiv.2605.11333,
title = {MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces},
author = {Srinivas Sridharan and Theodor-Adrian Badea and Andy Balogh and Bradford M. Beckmann and Brian Coutinho and Louis Feng and Sheng Fu and Sanshan Gao and Mehryar Garakani and Taekyung Heo and David Kanter and Josh Ladd and Ziwei Li and Winston Liu and Changhai Man and Dan Mihailescu and Spandan More and Joongun Park and Ashwin Ramachandran and Vinay Ramakrishnaiah and Saeed Rashidi and Vijay Janapa Reddi and Puneet Sharma and Phio Tian and William Won and Hanjiang Wu and Huan Xu and Jinsun Yoo and Tushar Krishna},
journal= {arXiv preprint arXiv:2605.11333},
year = {2026}
}
Comments
Accepted at the 9th Conference on Machine Learning and Systems (MLSys 2026)