Related papers: Flowing Through Layers: A Continuous Dynamical Sys…
Formally capturing the transition from a continuous model to a discrete model is investigated using model based refinement techniques. A very simple model for stopping (eg. of a train) is developed in both the continuous and discrete…
Incremental flow-based denoising models have reshaped generative modelling, but their empirical advantage still lacks a rigorous approximation-theoretic foundation. We show that incremental generation is necessary and sufficient for…
The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide a novel perspective towards understanding the architecture:…
Under general assumptions on the target distribution $p^\star$, we establish a sharp Lipschitz regularity theory for flow-matching vector fields and diffusion-model scores, with optimal dependence on time and dimension. As applications, we…
Despite the widespread success of Transformers across various domains, their optimization guarantees in large-scale model settings are not well-understood. This paper rigorously analyzes the convergence properties of gradient flow in…
Parallel shear flows come with continuous symmetries of translation in the downstream and spanwise direction. As a consequence, flow states that differ in their spanwise or downstream location but are otherwise identical are dynamically…
Standard Transformers have a fixed computational depth, fundamentally limiting their ability to generalize to tasks requiring variable-depth reasoning, such as multi-hop graph traversal or nested logic. We propose a depth-recurrent…
In this paper, we aim to investigate the synchronization problem of dynamical systems, which can be of generic linear or Lipschitz nonlinear type, communicating over directed switching network topologies. A mild connectivity assumption on…
Deep learning employs multi-layer neural networks trained via the backpropagation algorithm. This approach has achieved success across many domains and relies on adaptive gradient methods such as the Adam optimizer. Sequence modeling…
In this paper, we introduce Exact Flow Linear Attention~(EFLA), an exact-flow formulation of delta-rule linear attention. We show that the delta-rule update can be interpreted as an explicit Euler discretization of an underlying…
Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tracking -- the iterative updating of latent variables…
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate the main cause of this phenomenon. In this work, we make the…
In the present contribution we investigate some features of dynamical lattice systems near periodic traveling waves. First, following the formal averaging method of Whitham, we derive modulation systems expected to drive at main order the…
In this paper we present a continuation method which transforms spatially distributed ODE systems into continuous PDE. We show that this continuation can be performed both for linear and nonlinear systems, including multidimensional, space-…
An implicit Euler finite-volume scheme for a degenerate cross-diffusion system describing the ion transport through biological membranes is analyzed. The strongly coupled equations for the ion concentrations include drift terms involving…
We extend beyond the Euler scales the hydrodynamic theory for quantum and classical integrable models developed in recent years, accounting for diffusive dynamics and local entropy production. We review how the diffusive scale can be…
Attention-based transformers have been remarkably successful at modeling generative processes across various domains and modalities. In this paper, we study the behavior of transformers on data drawn from \kth Markov processes, where the…
This work deals with a number of questions relative to the discrete and continuous adjoint fields associated with the compressible Euler equations and classical aerodynamic functions. The consistency of the discrete adjoint equations with…
Domain decomposition has been shown to be a computationally efficient distributed method for solving large scale entropic optimal transport problems. However, a naive implementation of the algorithm can freeze in the limit of very fine…
We extend slow manifolds near a transcritical singularity in a fast-slow system given by the explicit Euler discretization of the corresponding continuous-time normal form. The analysis uses the blow-up method and direct trajectory-based…