Space filling positionality and the Spiroformer
Machine Learning
2025-07-14 v1 Artificial Intelligence
Differential Geometry
Dynamical Systems
Symplectic Geometry
Abstract
Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the -sphere.
Cite
@article{arxiv.2507.08456,
title = {Space filling positionality and the Spiroformer},
author = {M. Maurin and M. Á. Evangelista-Alvarado and P. Suárez-Serrato},
journal= {arXiv preprint arXiv:2507.08456},
year = {2025}
}
Comments
9 pages, 5 figures. To appear in Geometric Science of Information 2025