Not Every AI Problem is a Data Problem: We Should Be Intentional About Data Scaling
Machine Learning
2025-06-04 v2 Artificial Intelligence
Abstract
While Large Language Models require more and more data to train and scale, rather than looking for any data to acquire, we should consider what types of tasks are more likely to benefit from data scaling. We should be intentional in our data acquisition. We argue that the shape of the data itself, such as its compositional and structural patterns, informs which tasks to prioritize in data scaling, and shapes the development of the next generation of compute paradigms for tasks where data scaling is inefficient, or even insufficient.
Keywords
Cite
@article{arxiv.2501.13779,
title = {Not Every AI Problem is a Data Problem: We Should Be Intentional About Data Scaling},
author = {Tanya Rodchenko and Natasha Noy and Nino Scherrer},
journal= {arXiv preprint arXiv:2501.13779},
year = {2025}
}