Related papers: From Universal Approximation Theorem to Tropical G…
Deep ReLU Networks can be decomposed into a collection of linear models, each defined in a region of a partition of the input space. This paper provides three results extending this theory. First, we extend this linear decompositions to…
With the rapid development of geometric deep learning techniques, many mesh-based convolutional operators have been proposed to bridge irregular mesh structures and popular backbone networks. In this paper, we show that while convolutions…
Language models have emerged as a critical area of focus in artificial intelligence, particularly with the introduction of groundbreaking innovations like ChatGPT. Large-scale Transformer networks have quickly become the leading approach…
Feed-forward, fully-connected Artificial Neural Networks (ANNs) or the so-called Multi-Layer Perceptrons (MLPs) are well-known universal approximators. However, their learning performance varies significantly depending on the function or…
Generalizing supertropical algebras, we present a "layered" structure, "sorted" by a semiring which permits varying ghost layers, and indicate how it is more amenable than the "standard" supertropical construction in factorizations of…
We state the fundamental theorem of projective geometry for semimodules over semirings, which is facilitated by recent work in the study of bases in semimodules defined over semirings. In the process we explore in detail the linear algebra…
Neural networks are important tools in machine learning. Representing piecewise linear activation functions with tropical arithmetic enables the application of tropical geometry. Algorithms are presented to compute regions where the neural…
Affine Bruhat--Tits buildings are geometric spaces extracting the combinatorics of algebraic groups. The building of $\mathrm{PGL}$ parametrizes flags of subspaces/lattices in or, equivalently, norms on a fixed finite-dimensional vector…
Previous work has demonstrated that attention mechanisms are Turing complete. More recently, it has been shown that a looped 9-layer Transformer can function as a universal programmable computer. In contrast, the multi-layer perceptrons…
Machine learning interatomic potentials (MLIPs) are one of the main techniques in the materials science toolbox, able to bridge ab initio accuracy with the computational efficiency of classical force fields. This allows simulations ranging…
Tropical mathematics often is defined over an ordered cancellative monoid $\tM$, usually taken to be $(\RR, +)$ or $(\QQ, +)$. Although a rich theory has arisen from this viewpoint, cf. [L1], idempotent semirings possess a restricted…
This document is a slightly expanded version of a series of talks given by J. Giansiracusa at the workshop `Geometry over semirings' at Universitat Aut\`{o}noma de Barcelona in July 2025. In the first lecture we introduce tropical…
We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and…
This theoretical paper is devoted to developing a rigorous theory for demystifying the global convergence phenomenon in a challenging scenario: learning over-parameterized Rectified Linear Unit (ReLU) nets for very high dimensional dataset…
We propose an alternating minimization heuristic for regression over the space of tropical rational functions with fixed exponents. The method alternates between fitting the numerator and denominator terms via tropical polynomial…
Conventional continual pretraining (CPT) for large language model (LLM) domain adaptation often suffers from catastrophic forgetting and limited domain capacity. Existing strategies adopt layer expansion, introducing additional trainable…
We examine the incidence geometry of lines in the tropical plane. We prove tropical analogs of the Sylvester-Gallai and Motzkin-Rabin theorems in classical incidence geometry. This study leads naturally to a discussion of the realizability…
This paper is about the combinatorics of finite point configurations in the tropical projective space or, dually, of arrangements of finitely many tropical hyperplanes. Moreover, arrangements of finitely many tropical halfspaces can be…
Recently proposed Gated Linear Networks present a tractable nonlinear network architecture, and exhibit interesting capabilities such as learning with local error signals and reduced forgetting in sequential learning. In this work, we…
The dominant paradigm in modern neural networks relies on simple, monotonically-increasing activation functions like ReLU. While effective, this paradigm necessitates large, massively-parameterized models to approximate complex functions.…