Related papers: Pebble-Depth
Diffusion Transformers have achieved state-of-the-art performance in class-conditional and multimodal generation, yet the structure of their learned conditional embeddings remains poorly understood. In this work, we present the first…
We answer an open question in the theory of transducer degrees initially posed in [3], on the structure of polynomial transducer degrees, in particular the question of what degrees, if any, lie below the degree of $n^3$. Transducer degrees…
We construct a minimal dynamical system of mean dimension equal to $1$, which can be embedded in the shift action on the Hilbert cube $[0,1]^\mathbb{Z}$. This clarifies a seemingly plausible impression about embedding possibility in…
The concept of depth has proved very important for multivariate and functional data analysis, as it essentially acts as a surrogate for the notion a ranking of observations which is absent in more than one dimension. Motivated by the rapid…
The learning dynamics of deep neural networks are not well understood. The information bottleneck (IB) theory proclaimed separate fitting and compression phases. But they have since been heavily debated. We comprehensively analyze the…
We develop a numerical method for solving a free boundary problem which describes shape relaxation, by surface tension, of a long and thin bubble of an inviscid fluid trapped inside a viscous fluid in a Hele-Shaw cell. The method of…
This paper introduces a new automata-theoretic class of string-to-string functions with polynomial growth. Several equivalent definitions are provided: a machine model which is a restricted variant of pebble transducers, and a few inductive…
Deep residual architectures, such as ResNet and the Transformer, have enabled models of unprecedented depth, yet a formal understanding of why depth is so effective remains an open question. A popular intuition, following Veit et al.…
In this paper we develop the theory of the depth of a simple algebraic extension of valued fields $(L/K,v)$. This is defined as the minimal number of augmentations appearing in some Mac Lane-Vaqui\'e chain for the valuation on $K[x]$…
Based on a natural connection between ResNet and transport equation or its characteristic equation, we propose a continuous flow model for both ResNet and plain net. Through this continuous model, a ResNet can be explicitly constructed as a…
The blow-up lemma states that a system of super-regular pairs contains all bounded degree spanning graphs as subgraphs that embed into a corresponding system of complete pairs. This lemma has far-reaching applications in extremal…
Prototype-based classification learning methods are known to be inherently interpretable. However, this paradigm suffers from major limitations compared to deep models, such as lower performance. This led to the development of the so-called…
We develop some basic results about full amalgamation classes with intrinsic trascendentals. These classes have generics whose models may have finite subsets whose intrinsic closure is not contained in its algebraic closure. We will show…
We propose a principle for exploring context in machine learning models. Starting with a simple assumption that each observation may or may not depend on its context, a conditional probability distribution is decomposed into two parts:…
Counting logics with a bounded number of variables form one of the central concepts in descriptive complexity theory. Although they restrict the number of variables that a formula can contain, the variables can be nested within scopes of…
Semiclassical methods are extremely important in the subjects of wave packet and coherent state dynamics. Unfortunately, these essentially saddle point approximations are considered nearly impossible to carry out in detail for systems with…
Token embeddings play a crucial role in language modeling but, despite this practical relevance, their theoretical understanding remains limited. Our paper addresses the gap by characterizing the structure of embeddings obtained via…
The low-frequency modal and non-modal stability characteristics of an incompressible, pressure-gradient-induced turbulent separation bubble (TSB) are investigated with the objective of studying the mechanism responsible for the…
The notion of a normal bit sequence was introduced by Borel in 1909; it was the first definition of an individual random object. Normality is a weak notion of randomness requiring only that all $2^n$ factors (substrings) of arbitrary…
Random sequences attain the highest entropy rate. The estimation of entropy rate for an ergodic source can be done using the Lempel Ziv complexity measure yet, the exact entropy rate value is only reached in the infinite limit. We prove…