English

EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

Computation and Language 2026-07-06 v1 Machine Learning
Authors: Deyao Zhu , Xin Zhou , Shengling Qin , Xuekai Zhu , Hangliang Ding , Shu Zhong , Zixin Wen , Zhonglin Xie , Chenhui Gou , Linxuan Ren , Yueyang Wang , Junfeng Zhong , Rui Liu , Tian Gao , Yangguang Lin , Jingyuan Zhang , Maojia Song , Xuan Qi , Jinhong Wu , Chenyang Zhang , Yinzhu Piao , Ziru Niu , Hongbin Lin , Lingxiang Meng , Peng Tang , Chengyao Tang , Shanyu Wu , Huanyu Zheng , Yu Liu , Liya Zhu , He Wang , Ming Ding , Ziyu Wan , Hao Liu , Sibo Wang , Haotian Zhu , Xintian Zhang , Nan Chai , Yipeng Liu , Panhao Lai , Sihang Yuan , Zixin Su , Ge Zhang , Wangchunshu Zhou , Yantao Du , Wenhao Huang , Guang Shi
View on arXiv ↗ PDF ↗

Abstract

Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours of agent interaction with the environment across 134 real world tasks, we find, to the best of our knowledge, the first evidence that overall performance during environment learning follows a log-sigmoid scaling law with remarkably high precision, reaching R^2 = 0.998. Across model generations, we also find that agent learning speed roughly doubles every three months. This discovery stems from EdgeBench, a suite of 134 real world tasks with ultra-long horizons, spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and interactive games. Each task sustains at least 12 hours of continuous agent operation under rich, multilevel feedback, and is built through substantial expert effort. We publicly release 51 tasks and our full evaluation framework to accelerate the study of how agents learn from real world experience.

Keywords

scaling laws real-world environments agent learning long-horizon tasks benchmark suite empirical study

Cite

@article{arxiv.2607.05155,
  title  = {EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments},
  author = {Deyao Zhu and Xin Zhou and Shengling Qin and Xuekai Zhu and Hangliang Ding and Shu Zhong and Zixin Wen and Zhonglin Xie and Chenhui Gou and Linxuan Ren and Yueyang Wang and Junfeng Zhong and Rui Liu and Tian Gao and Yangguang Lin and Jingyuan Zhang and Maojia Song and Xuan Qi and Jinhong Wu and Chenyang Zhang and Yinzhu Piao and Ziru Niu and Hongbin Lin and Lingxiang Meng and Peng Tang and Chengyao Tang and Shanyu Wu and Huanyu Zheng and Yu Liu and Liya Zhu and He Wang and Ming Ding and Ziyu Wan and Hao Liu and Sibo Wang and Haotian Zhu and Xintian Zhang and Nan Chai and Yipeng Liu and Panhao Lai and Sihang Yuan and Zixin Su and Ge Zhang and Wangchunshu Zhou and Yantao Du and Wenhao Huang and Guang Shi},
  journal= {arXiv preprint arXiv:2607.05155},
  year   = {2026}
}