Sharp Minima Can Generalize: A Loss Landscape Perspective On Data
Machine Learning
2025-11-10 v1
Abstract
The volume hypothesis suggests deep learning is effective because it is likely to find flat minima due to their large volumes, and flat minima generalize well. This picture does not explain the role of large datasets in generalization. Measuring minima volumes under varying amounts of training data reveals sharp minima which generalize well exist, but are unlikely to be found due to their small volumes. Increasing data changes the loss landscape, such that previously small generalizing minima become (relatively) large.
Cite
@article{arxiv.2511.04808,
title = {Sharp Minima Can Generalize: A Loss Landscape Perspective On Data},
author = {Raymond Fan and Bryce Sandlund and Lin Myat Ko},
journal= {arXiv preprint arXiv:2511.04808},
year = {2025}
}