English

Time is Not Compute: Scaling Laws for Wall-Clock Constrained Training on Consumer GPUs

Performance 2026-04-01 v1 Artificial Intelligence

Abstract

Scaling laws relate model quality to compute budget (FLOPs), but practitioners face wall-clock time constraints, not compute budgets. We study optimal model sizing under fixed time budgets from 5 minutes to 24 hours on consumer GPUs (RTX 4090). Across 70+ runs spanning 50M--1031M parameters, we find: (1)~at each time budget a U-shaped curve emerges where too-small models overfit and too-large models undertrain; (2)~optimal model size follows Nt0.60N^* \propto t^{0.60}, growing \emph{faster} than Chinchilla's NC0.50N^* \propto C^{0.50}, with α=0.60±0.07\alpha = 0.60 \pm 0.07 robustly exceeding compute-optimal across all sensitivity analyses; (3)~a \emph{dual U-shape mechanism}: short-budget U-curves arise from compute bottlenecks, while long-budget U-curves emerge from data bottlenecks (overfitting), with an intermediate regime where the U-curve temporarily disappears. These findings have immediate implications for researchers training on consumer hardware, where wall-clock time -- not FLOPs -- is the binding constraint. We release all code, logs, and 70+ experimental configurations.

Keywords

Cite

@article{arxiv.2603.28823,
  title  = {Time is Not Compute: Scaling Laws for Wall-Clock Constrained Training on Consumer GPUs},
  author = {Yi Liu},
  journal= {arXiv preprint arXiv:2603.28823},
  year   = {2026}
}
R2 v1 2026-07-01T11:44:42.152Z