Transformer models are gauge invariant: A mathematical connection between AI and particle physics
Machine Learning
2024-12-20 v1 High Energy Physics - Theory
Abstract
In particle physics, the fundamental forces are subject to symmetries called gauge invariance. It is a redundancy in the mathematical description of any physical system. In this article I will demonstrate that the transformer architecture exhibits the same properties, and show that the default representation of transformers has partially, but not fully removed the gauge invariance.
Keywords
Cite
@article{arxiv.2412.14543,
title = {Transformer models are gauge invariant: A mathematical connection between AI and particle physics},
author = {Leo van Nierop},
journal= {arXiv preprint arXiv:2412.14543},
year = {2024}
}
Comments
10 pages, 2 figures, 1 table