Related papers: Universality, optimality, and randomness deficienc…
We consider effective versions of two classical theorems, the Lebesgue density theorem and the Denjoy-Young-Saks theorem. For the first, we show that a Martin-Loef random real $z\in [0,1]$ is Turing incomplete if and only if every…
Raptor codes have been widely used in many multimedia broadcast/multicast applications. However, our understanding of Raptor codes is still incomplete due to the insufficient amount of theoretical work on the performance analysis of Raptor…
Linear regression is a classical paradigm in statistics. A new look at it is provided via the lens of universal learning. In applying universal learning to linear regression the hypotheses class represents the label $y\in {\cal R}$ as a…
Model-agnostic meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior, while the outer level searches for the optimal shared prior by optimizing its…
We characterize some major algorithmic randomness notions via differentiability of effective functions. (1) As the main result we show that a real number z in [0,1] is computably random if and only if each nondecreasing computable function…
Efficient sampling from complex and high dimensional target distributions turns out to be a fundamental task in diverse disciplines such as scientific computing, statistics and machine learning. In this paper, we propose a new kind of…
Let $X_N$ be an $N$-dimensional subspace of $L_2$ functions on a probability space $(\Omega, \mu)$ spanned by a uniformly bounded Riesz basis $\Phi_N$. Given an integer $1\leq v\leq N$ and an exponent $1\leq q\leq 2$, we obtain universal…
Kucera and Gacs independently showed that every infinite sequence is Turing reducible to a Martin-Lof random sequence. This result is extended by showing that every infinite sequence S is Turing reducible to a Martin-Lof random sequence R…
The reusability of state-of-the-art Pre-trained Language Models (PLMs) is often limited by their generalization problem, where their performance drastically decreases when evaluated on examples that differ from the training dataset, known…
We provide an efficient unified plug-in approach for estimating symmetric properties of distributions given $n$ independent samples. Our estimator is based on profile-maximum-likelihood (PML) and is sample optimal for estimating various…
We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…
Slot and van Emde Boas' weak invariance thesis states that reasonable machines can simulate each other within a polynomially overhead in time. Is lambda-calculus a reasonable machine? Is there a way to measure the computational complexity…
We prove a universality theorem for learning with random features. Our result shows that, in terms of training and generalization errors, a random feature model with a nonlinear activation function is asymptotically equivalent to a…
In this paper, we study the problem of sampling from a given probability density function that is known to be smooth and strongly log-concave. We analyze several methods of approximate sampling based on discretizations of the (highly…
We present Universal Conditional Logic (UCL), a mathematical framework for prompt optimization that transforms prompt engineering from heuristic practice into systematic optimization. Through systematic evaluation (N=305, 11 models, 4…
We prove the universality for the eigenvalue gap statistics in the bulk of the spectrum for band matrices, in the regime where the band width is comparable with the dimension of the matrix, $W\sim N$. All previous results concerning…
For a given distribution, learning algorithm, and performance metric, the rate of convergence (or data-scaling law) is the asymptotic behavior of the algorithm's test performance as a function of number of train samples. Many learning…
Outlier hypothesis testing is studied in a universal setting. Multiple sequences of observations are collected, a small subset of which are outliers. A sequence is considered an outlier if the observations in that sequence are distributed…
We study the computational power of randomized computations on infinite objects, such as real numbers. In particular, we introduce the concept of a Las Vegas computable multi-valued function, which is a function that can be computed on a…
We show that for each computable ordinal $\alpha>0$ it is possible to find in each Martin-L\"of random $\Delta^0_2$ degree a sequence $R$ of Cantor-Bendixson rank $\alpha$, while ensuring that the sequences that inductively witness $R$'s…