Related papers: Higher-order asymptotics for the parametric comple…
Parametric complexity is a central concept in MDL model selection. In practice it often turns out to be infinite, even for quite simple models such as the Poisson and Geometric families. In such cases, MDL model selection as based on NML…
In the Minimum Description Length (MDL) principle, learning from the data is equivalent to an optimal coding problem. We show that the codes that achieve optimal compression in MDL are critical in a very precise sense. First, when they are…
We study neural network compressibility by using singular learning theory to extend the minimum description length (MDL) principle to singular models like neural networks. Through extensive experiments on the Pythia suite with quantization,…
We introduce a new family of paraxial asymptotic models that approximate the Vlasov-Maxwell equations in non-relativistic cases. This formulation is $n$-th order accurate in a parameter $\eta$, which denotes the ratio between the…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
I define a natural measure of the complexity of a parametric distribution relative to a given true distribution called the {\it razor} of a model family. The Minimum Description Length principle (MDL) and Bayesian inference are shown to…
In this paper, we study inference for high-dimensional data characterized by small sample sizes relative to the dimension of the data. In particular, we provide an infinite-dimensional framework to study statistical models that involve…
This article includes a short survey of selected averaging and dimension reduction techniques for deterministic fast-slow systems. This survey includes, among others, classical techniques, such as the WKB approximation or the averaging…
We compute the asymptotic temporal behavior of the dynamical complexity associated with the maximum probability trajectories on Gaussian statistical manifolds in presence of correlations between the variables labeling the macrostates of the…
The behavior of many critical phenomena at large distances is expected to be invariant under the full conformal group, rather than only isometries and scale transformations. When studying critical phenomena, approximations are often…
We are interested in the asymptotic behavior of orthogonal polynomials of the generalized Jacobi type as their degree $n$ goes to $\infty$. These are defined on the interval $[-1,1]$ with weight function…
Hyperparameters greatly impact models' capabilities; however, modern models are too large for extensive search. Instead, researchers design recipes that train well across scales based on their understanding of the hyperparameters. Despite…
We present improved methods for calculating confidence intervals and $p$-values in situations where standard asymptotic approaches fail due to small sample sizes. We apply these techniques to a specific class of statistical model that can…
Bessel and modified Bessel functions of imaginary order $i\nu$ ($\nu >0$) are studied. Asymptotic expansions are derived as $\nu \to \infty$ that are uniformly valid in unbounded complex domains, with error bounds provided. Coupled with…
A famous result by Hammersley and Versik-Kerov states that the length $L_n$ of the longest increasing subsequence among $n$ iid continuous random variables grows like $2\sqrt{n}$. We investigate here the asymptotic behavior of $L_n$ for…
Three-dimensional random tensor models are a natural generalization of the celebrated matrix models. The associated tensor graphs, or 3D maps, can be classified with respect to a particular integer or half-integer, the degree of the…
We model the development of the linear complexity of multisequences by a stochastic infinite state machine, the Battery-Discharge-Model, BDM. The states s in S of the BDM have asymptotic probabilities or mass Pr(s)=1/(P(q,M) q^K(s)), where…
We consider linear regression in the high-dimensional regime where the number of observations $n$ is smaller than the number of parameters $p$. A very successful approach in this setting uses $\ell_1$-penalized least squares (a.k.a. the…
In Bayesian nonparametric inference, random discrete probability measures are commonly used as priors within hierarchical mixture models for density estimation and for inference on the clustering of the data. Recently, it has been shown…
A general method is presented for deriving the limiting behavior of estimators that are defined as the values of parameters optimizing an empirical criterion function. The asymptotic behavior of such estimators is typically deduced from…