Why the Third Axis Is Freedom
Abstract
In generative training, a model produces an output and is penalised for its difference from an example. With one output per comparison, a model that produces one common answer can outperform a model retaining a broader repertoire. Explorative Modeling (XM) produces outputs per comparison and updates on the closest, claiming exploration as a "third pretraining axis" associated with generative expressivity. Here I show the third axis is actually freedom, meaning the weakness of the constraint implied by a model's behaviour. Previous work showed freedom is a property of function rather than form. Parameters, architecture, minimum-description-length (MDL), and data can vary while the behavioural constraint remains unchanged. It was formally proved that weakest models are likeliest to generalise, and freedom selection beat MDL by 110-500\% in induction experiments. I prove average XM loss depends on the chance a candidate misses an acceptable region, with exploration raising miss probability to power . For , match probability rises with freedom. I then demonstrate empirically that XM optimises for freedom. In a Forward XM experiment, larger increased or saturated measured freedom, and increased freedom at every tested value under context-dependent targets. I trained XM candidate pools and compared validation selection with a freedom selector that read unlabelled parent contexts. Freedom won in 29 of 30 cases. Generative expressivity is a mode-count proxy for freedom, that discards the extension structure that gives freedom its generalisation significance. XM is a means, freedom an end, and selecting for freedom improved XM under distribution shift.
Cite
@article{arxiv.2608.05423,
title = {Why the Third Axis Is Freedom},
author = {Michael Timothy Bennett},
journal= {arXiv preprint arXiv:2608.05423},
year = {2026}
}
Comments
Experiment code available on GitHub, in the papers folder: https://github.com/ViscousLemming/Technical-Appendices