We introduce the AI Productivity Index for Software Engineering (APEX-SWE), a benchmark for assessing whether frontier AI models can execute economically valuable software engineering work. Unlike existing evaluations that focus on narrow, well-defined tasks, APEX-SWE assesses two novel task types that reflect real-world software engineering: (1) Integration tasks (n=100), which require constructing end-to-end systems across heterogeneous cloud primitives, business applications, and infrastructure-as-code services, and (2) Observability tasks (n=100), which require debugging production failures using telemetry signals such as logs and dashboards, as well as unstructured context. We evaluated eleven frontier models for the APEX-SWE leaderboard. Claude Opus 4.6 leads the APEX-SWE leaderboard with 40.5% Pass@1, followed by Claude Opus 4.5 at 38.7%. Our analysis shows that strong performance is primarily driven by epistemic discipline, defined as the capacity to distinguish between assumptions and verified facts. It is often combined with systematic verification prior to acting. We open-source the APEX-SWE evaluation harness and a dev set (n=50).
@article{arxiv.2601.08806,
title = {APEX-SWE},
author = {Abhi Kottamasu and Chirag Mahapatra and Sam Lee and Ben Pan and Aakash Barthwal and Akul Datta and Anurag Gupta and Pranav Mehta and Ajay Arun and Silas Alberti and Adarsh Hiremath and Brendan Foody and Bertie Vidgen},
journal= {arXiv preprint arXiv:2601.08806},
year = {2026}
}