Previous work has shown that the languages recognized by average-hard attention transformers (AHATs) and softmax-attention transformers (SMATs) are within the circuit complexity class TC0. However, these results assume limited-precision arithmetic: using floating-point numbers with O(log n) bits (where n is the length of the input string), Strobl showed that AHATs can be approximated in L-uniform TC0, and Merrill and Sabharwal showed that SMATs can be approximated in DLOGTIME-uniform TC0. Here, we improve these results, showing that AHATs with no approximation, SMATs with O(poly(n)) bits of floating-point precision, and SMATs with at most 2−O(poly(n)) absolute error are all in DLOGTIME-uniform TC0.
Cite
@article{arxiv.2409.13629,
title = {Transformers in Uniform TC$^0$},
author = {David Chiang},
journal= {arXiv preprint arXiv:2409.13629},
year = {2025}
}