Related papers: Spectral Thresholds for Identifiability and Stabil…
Linear probes are widely used to interpret and evaluate neural representations, yet their reliability remains unclear, as probes may appear accurate in some regimes but collapse unpredictably in others. We uncover a spectral mechanism…
Consider two $p$-variate populations, not necessarily Gaussian, with covariance matrices $\Sigma_1$ and $\Sigma_2$, respectively, and let $S_1$ and $S_2$ be the sample covariances matrices from samples of the populations with degrees of…
Existing analyses of neural network training often operate under the unrealistic assumption of an extremely small learning rate. This lies in stark contrast to practical wisdom and empirical studies, such as the work of J. Cohen et al.…
Despite the widespread use and success of machine-learning techniques for detecting phase transitions from data, their working principle and fundamental limits remain elusive. Here, we explain the inner workings and identify potential…
An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. For example, the maximum likelihood estimator has a bias that can result in a significant inferential loss. This problem is…
Identification of the parameters of stable linear dynamical systems is a well-studied problem in the literature, both in the low and high-dimensional settings. However, there are hardly any results for the unstable case, especially…
Graph-structured data arise in a variety of real-world context ranging from sensor and transportation to biological and social networks. As a ubiquitous tool to process graph-structured data, spectral graph filters have been used to solve…
Equilibrium statistical physics is applied to layered neural networks with differentiable activation functions. A first analysis of off-line learning in soft-committee machines with a finite number (K) of hidden units learning a perfectly…
Whether singularities can form in fluids remains a foundational unanswered question in mathematics. This phenomenon occurs when solutions to governing equations, such as the 3D Euler equations, develop infinite gradients from smooth initial…
We develop a finite-dimensional sensitivity framework for studying stability in learning systems whose states include representations, parameters, and update variables. The central object is the \emph{Learning Stability Profile}, a…
Linear stability of inviscid, parallel, and stably stratified shear flow is studied under the assumption of smooth strictly monotonic profiles of shear flow and density, so that the local Richardson number is positive everywhere. The…
Precision measurement has been an important research area in sensing and metrology. In classical physics, the Fisher information determines the maximum extractable information from statistically unknown signals, based on a joint probability…
Numerical experiments demonstrate that deep neural network classifiers progressively separate class distributions around their mean, achieving linear separability on the training set, and increasing the Fisher discriminant ratio. We explain…
A necessary and sufficient condition for linear stability of inviscid parallel shear flow is formulated by developing a novel variational principle, where the velocity profile is assumed to be monotonic and analytic. It is shown that…
We study the scalar curvature of the Fisher information metric on the microscopic coupling-parameter manifold of lattice spin models at criticality. For a $d$-dimensional lattice with periodic boundary conditions and $n = L^d$ sites, the…
The problem of reconstructing a function from the magnitudes of its frame coefficients has recently been shown to be never uniformly stable in infinite-dimensional spaces [5]. This result also holds for frames that are possibly continuous…
The incompressible smoothed particle hydrodynamics method (ISPH) is a numerical method widely used for accurately and efficiently solving flow problems with free surface effects. However, to date there has been little mathematical…
The Rayleigh--Taylor instability of two immiscible fluids in the limit of small Atwood numbers is studied by means of a phase-field description. In this method the sharp fluid interface is replaced by a thin, yet finite, transition layer…
Softmax feedback systems are a common mathematical core of entropy-regularized reinforcement learning, logit game dynamics, population choice, and mean-field variational updates. Their central stability question is simple: when does a…
Gradient Descent (GD) is a powerful workhorse of modern machine learning thanks to its scalability and efficiency in high-dimensional spaces. Its ability to find local minimisers is only guaranteed for losses with Lipschitz gradients, where…