Related papers: Expressivity of Transformers: A Tropical Geometry …
We explain how to tropicalize scalar quantum field theory and show that tropicalized massive scalar quantum field theory is exactly solvable. This exact solution manifests as a non-linear recursion equation fulfilled by the expansion…
This paper studies how Transformer models with Rotary Position Embeddings (RoPE) develop emergent, wavelet-like properties that compensate for the positional encoding's theoretical limitations. Through an analysis spanning model scales,…
Since its inception in "Attention Is All You Need", transformer architecture has led to revolutionary advancements in NLP. The attention layer within the transformer admits a sequence of input tokens $X$ and makes them interact through…
We investigate the combinatorial complexity of geodesic Voronoi diagrams on polyhedral terrains using a probabilistic analysis. Aronov etal [ABT08] prove that, if one makes certain realistic input assumptions on the terrain, this complexity…
With the use of mathematical techniques of tropical geometry, it was shown by Mikhalkin some twenty years ago that certain Gromov-Witten invariants associated with topological quantum field theories of pseudoholomorphic maps can be computed…
In this paper we initiate the study of tropical Voronoi diagrams. We start out with investigating bisectors of finitely many points with respect to arbitrary polyhedral norms. For this more general scenario we show that bisectors of three…
Previous research has explored the computational expressivity of Transformer models in simulating Boolean circuits or Turing machines. However, the learnability of these simulators from observational data has remained an open question. Our…
Extremality and irreducibility constitute fundamental concepts in mathematics, particularly within tropical geometry. While extremal decomposition is typically computationally hard, this article presents a fast algorithm for identifying the…
Machine learning (ML) is shaping our exploration of topological matter, whose existence is inherently tied to the geometry of quantum states or energy spectra. In non-Hermitian systems, distinctive spectral geometry can lead to topological…
Previous work on the learnability of transformers \textemdash\ focused primarily on examining their ability to approximate specific algorithmic patterns through training \textemdash\ has largely been data-driven, offering only probabilistic…
What computational structures emerge in transformers trained on next-token prediction? In this work, we provide evidence that transformers implement constrained Bayesian belief updating -- a parallelized version of partial Bayesian…
We establish a characterization of the vertices of a tropical polyhedron defined as the intersection of finitely many half-spaces. We show that a point is a vertex if, and only if, a directed hypergraph, constructed from the…
Sometimes it is possible to extend some using Newton polyhedra computations in algebraic geometry from polynomials to exponential sums. For this purpose it is useful to consider analogues of tropical varieties in complex space. These…
We propose multiloop vacuum amplitudes as the optimal building blocks for efficiently assembling theoretical predictions at high-energy colliders. This hypothesis is strongly supported by the manifestly causal properties of the loop-tree…
Transformers provide a class of expressive architectures that are extremely effective for sequence modeling. However, the key limitation of transformers is their quadratic memory and time complexity $\mathcal{O}(L^2)$ with respect to the…
We introduce tropical Newton-Puiseux polynomials admitting rational exponents. A resolution of a tropical hypersurface is defined by means of a tropical Newton-Puiseux polynomial. A polynomial complexity algorithm for resolubility of a…
Recent advancements in large language models (LLMs) based on transformer architectures have sparked significant interest in understanding their inner workings. In this paper, we introduce a novel approach to modeling transformer…
A realisation of a graph in the plane as a bar-joint framework is rigid if there are finitely many other realisations, up to isometries, with the same edge lengths. Each of these finitely-many realisations can be seen as a solution to a…
Thermoelectric (TE) generators can directly convert heat into electricity, but their performance is often constrained by limited temperature gradients. Here it is shown that width-modulated metamaterials with constrictions and expansions…
The shellability of the boundary complex of an unbounded polyhedron is investigated. To this end, it is necessary to pass to a suitable compactification, e.g., by one point. This observation can be exploited to prove that any tropical…