相关论文: How Long Does Infinite Width Last? Signal Propagat…

The study of temporal networks in discrete time has yielded numerous insights into time-dependent networked systems in a wide variety of applications. For many complex systems, however, it is useful to develop continuous-time models of…

How finite-sized material lines stretch in chaotic (mono-scale) and turbulent (multi-scale) flows remains a central but unresolved problem that governs mixing, transport and reaction. We show elongation is controlled by a finite-sampling…

To understand the training dynamics of neural networks (NNs), prior studies have considered the infinite-width mean-field (MF) limit of two-layer NN, establishing theoretical guarantees of its convergence under gradient flow training as…

We study numerically finite-size corrections in scaling relations for roughness distributions of various interface growth models. The most common relation, which considers the average roughness $<w_2>$ as scaling factor, is not obeyed in…

The prediction of rail crack length propagation plays a crucial role in the maintenance and safety assessment of materials and structures. Traditional methods rely on physical models and empirical equations such as Paris law, which often…

This paper is the second in the series Commutative Scaling of Width and Depth (WD) about commutativity of infinite width and depth limits in deep neural networks. Our aim is to understand the behaviour of neural functions (functions that…

It is widely believed that deep neural networks contain layer specialization, wherein neural networks extract hierarchical features representing edges and patterns in shallow layers and complete objects in deeper layers. Unlike common…

We present the first analytic study of finite-size effects on critical diffusion above and below T_c of three-dimensional Ising-like systems whose order parameter is coupled to a conserved density. We also calculate the finite-size…

This work addresses the superdiffusive motion of a discrete time random walker on ordered discrete substrates and complex networks with the presence of long-range interactions (LRIs). In ordered regular lattices, where LRIs have a clear…

It is shown that the dispersion and attenuation functions in a linear viscoelastic medium with a positive relaxation spectrum have a sublinear growth rate at very high frequencies. A local dispersion relation in parametric form is found.…

Diffusion of information in networks is at the core of many problems in AI. Common examples include the spread of ideas and rumors as well as marketing campaigns. Typically, information diffuses at a non-linear rate, for example, if markets…

In this work, we analyze various scaling limits of the training dynamics of transformer models in the feature learning regime. We identify the set of parameterizations that admit well-defined infinite width and depth limits, allowing the…

We study higher-order small-noise fluctuation expansions for the overdamped Langevin dynamics in a quartic double-well potential. Assuming that the initial data admits a suitable expansion structure, we obtain a strong dynamical expansion…

Pinning models are built from discrete renewal sequences by rewarding (or penalizing) the trajectories according to their number of renewal epochs up to time $N$, and $N$ is then sent to infinity. They are statistical mechanics models to…

In data analysis problems where we are not able to rely on distributional assumptions, what types of inference guarantees can still be obtained? Many popular methods, such as holdout methods, cross-validation methods, and conformal…

Multi-layer feedforward networks have been used to approximate a wide range of nonlinear functions. An important and fundamental problem is to understand the learnability of a network model through its statistical risk, or the expected…

We prove that for an $L$-layer fully-connected linear neural network, if the width of every hidden layer is $\tilde\Omega (L \cdot r \cdot d_{\mathrm{out}} \cdot \kappa^3 )$, where $r$ and $\kappa$ are the rank and the condition number of…

Scalable sequence models, such as Transformer variants and structured state-space models, often trade expressivity power for sequence-level parallelism, which enables efficient training. Here we examine the bounds on error and how error…

The open problem of calculating the limiting spectrum (or its Shannon transform) of increasingly large random Hermitian finite-band matrices is described. In general, these matrices include a finite number of non-zero diagonals around their…

The derivation of ultimate limits to communication over certain quantum repeater networks have provided extremely valuable benchmarks for assessing near-term quantum communication protocols. However, these bounds are usually derived in the…