Related papers: TokenBlowUp: Resolving Representational Singularit…
Real blow-ups and more refined "zooms" play a key role in the analysis of singularities of complex-analytic differential modules. They do not change the underlying topology, but the uniform structure. This suggests to revisit the cohomology…
Sampling-based decoding underlies complex reasoning in large language models (LLMs), where decoding strategies critically shape model behavior. Temperature- and truncation-based methods reshape the next-token distribution through global…
Fulton and MacPherson famously constructed a configuration space that encodes infinitesimal collision data by blowing up the diagonals. We observe that when generalizing their approach to configuration spaces of filtered manifolds (e.g. jet…
GL is a verified tool for proving ACL2 theorems using Boolean methods such as BDD reasoning and satisfiability checking. In its typical operation, GL recursively traverses a term, computing a symbolic object representing the value of each…
We investigate the relationship between the geometry of token embeddings and their role in the next token prediction within transformer models. An important aspect of this connection uses the notion of empirical measure, which encodes the…
Tokenization is the first - and often underappreciated - layer of computation in language models. While Chain-of-Thought (CoT) prompting enables transformer models to approximate recurrent computation by externalizing intermediate steps, we…
We provide a procedure for resolving, in characteristic 0, singularities of a variety $X$ embedded in a smooth variety $Y$ by repeatedly blowing up the worst singularities, in the sense of stack-theoretic weighted blowings up. No history,…
Despite their empirical success, pushing Transformer architectures to extreme depth often leads to a paradoxical failure: representations become increasingly redundant, lose rank, and ultimately collapse. Existing explanations largely…
A widely cited result by Dong et al. (2021) showed that Transformers built from self-attention alone, without skip connections or feed-forward layers, suffer from rapid rank collapse: all token representations converge to a single…
In terms of the gauged nonlinear $\sigma$-models, we describe some results and implications of solving the following problem: Given a smooth symplectic manifold as target space with a quasi-free Hamiltonian group action, perform the…
In this paper we report on numerical studies of formation of singularities for the semilinear wave equations with a focusing power nonlinearity $u_{tt} - \Delta u = u^{p}$ in three space dimensions. We show that for generic large initial…
Blowup equations and holomorphic anomaly equations are two universal yet completely different approaches to solve refined topological string theory on local Calabi-Yau threefolds corresponding to A- and B-model respectively. The former…
Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to…
Multimodal large language models (MLLMs) have exhibited remarkable performance in various visual tasks, yet still struggle with spatial reasoning. Recent efforts mitigate this by injecting geometric features from 3D foundation models, but…
Compactification of the heterotic string on toroidal orbifolds is a promising set-up for the construction of realistic unified models of particle physics. The target space dynamics of such models, however, drives them slightly away from the…
It is shown that, for any reduced algebraic variety in characteristic zero, one can resolve all but simple normal crossings (snc) singularities by a finite sequence of blowings-up with smooth centres which, at every step, avoids points…
Numerical interactions leading to users sharing textual content published by others are naturally represented by a network where the individuals are associated with the nodes and the exchanged texts with the edges. To understand those…
Manifold learning aims to discover and represent low-dimensional structures underlying high-dimensional data while preserving critical topological and geometric properties. Existing methods often fail to capture local details with global…
Let $M$ be a real-analytic connected CR-hypersurface of CR-dimension $n>0$ having a point of Levi-nondegeneracy. The following alternative is demonstrated for both the symmetry algebra $s$ and the automorphism group $G$ of $M$. Denote by…
To drive progress in science and engineering, large language models (LLMs) must be able to process large amounts of numerical data and solve long calculations efficiently. This is currently only possible through the use of external tools or…