How AI settled the complexity of the oldest SGD algorithm
Machine Learning
2026-06-28 v1 Artificial Intelligence
Numerical Analysis
Optimization and Control
Machine Learning
Abstract
In 1937, Stefan Kaczmarz proposed a simple algorithm for solving systems of linear equations. This algorithm turned out to be the earliest known example of stochastic gradient descent, a ubiquitous computing paradigm that drives the training of modern AI models such as ChatGPT and Gemini. Now, those AI models have joined forces to discover the worst-case complexity of the Kaczmarz algorithm. This paper tells the story of how it happened.
Cite
@article{arxiv.2606.29593,
title = {How AI settled the complexity of the oldest SGD algorithm},
author = {Michał Dereziński and Xiaoyu Dong},
journal= {arXiv preprint arXiv:2606.29593},
year = {2026}
}