Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal.
@article{arxiv.2407.07664,
title = {A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry},
author = {Martin Lindström and Borja Rodríguez-Gálvez and Ragnar Thobaben and Mikael Skoglund},
journal= {arXiv preprint arXiv:2407.07664},
year = {2025}
}
Comments
Changes in version 2: Minor formatting changes. Published in the Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM), PMLR 251. Available at: https://proceedings.mlr.press/v251/lindstrom24a.html 14 pages: 9 of the main paper, 2 of references, and 3 of appendices.. Code is available at: https://github.com/martinlindstrom/coding_theoretic_hpl