HomeMachine LearningarXiv:2605.29987

MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment

Abstract

Although multi-scales representation learning enables elastic-dimension embeddings, nested subspaces often suffer from dimensional redundancy and spectral collapse. To address this, we introduce MIC, a framework that optimizes the geometric landscape of multi-granular embeddings through isotropic subspace alignment. MIC employs Soft Collapse Regularization (SCR) to mitigate redundancy between prefix and residual subspaces via cross-correlation penalties, alongside Spectral Isotropy Regularization (SIR) to ensure hyper-spherical uniformity in low-dimensional prefixes. By unifying these strategies through a self-distillation objective, MIC generates semantically dense representations that maintain high discriminative power. Our experiments demonstrate that MIC significantly outperforms standard baselines, particularly in high-compression scenarios where maintaining informational capacity is most critical.

Comments: Accepted at the GlobalSouthML Workshop at ICML 2026. 13 pages, 2 figures

Cite

@article{arxiv.2605.29987,
  title  = {MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment},
  author = {Dang Hong Nguyen and Nhi Ngoc-Yen Nguyen and Huy-Hieu Pham},
  journal= {arXiv preprint arXiv:2605.29987},
  year   = {2026}
}