Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods like VCReg address this by regularizing first and second-order feature statistics, which cannot fully achieve maximum entropy. We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution-a defining property of high-dimensional Gaussians. We prove that Radial-VCReg transforms a broader class of distributions towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance by reducing higher-order dependencies and promoting more diverse and informative representations.
Cite
@article{arxiv.2602.14272,
title = {Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization},
author = {Yilun Kuang and Yash Dagade and Deep Chakraborty and Erik Learned-Miller and Randall Balestriero and Tim G. J. Rudner and Yann LeCun},
journal= {arXiv preprint arXiv:2602.14272},
year = {2026}
}
Comments
Published in the Unifying Representations in Neural Models (UniReps) and Symmetry and Geometry in Neural Representations (NeurReps) Workshops at NeurIPS 2025