Related papers: K-trivial, K-low and MLR-low sequences: a tutorial
In the theory of algorithmic randomness, several notions of random sequence are defined via a game-theoretic approach, and the notions that received most attention are perhaps Martin-Loef randomness and computable randomness. The latter…
Every K-trivial set is computable from an incomplete Martin-L\"of random set, i.e., a Martin-L\"of random set that does not compute 0'.
One of the main lines of research in algorithmic randomness is that of lowness notions. Given a randomness notion R, we ask for which sequences A does relativization to A leave R unchanged (i.e., R^A = R)? Such sequences are call low for R.…
In this position paper, I first describe a new perspective on machine learning (ML) by four basic problems (or levels), namely, "What to learn?", "How to learn?", "What to evaluate?", and "What to adjust?". The paper stresses more on the…
Bennett's notion of depth is usually considered to describe the usefulness and internal organization of the information encoded into an object such as an infinite binary sequence. We consider a natural way to relativize the notion of depth…
Large language models display remarkable capabilities in logical and mathematical reasoning, allowing them to solve complex tasks. Interestingly, these abilities emerge in networks trained on the simple task of next-token prediction. In…
Algorithmic information theory studies description complexity and randomness and is now a well known field of theoretical computer science and mathematical logic. There are several textbooks and monographs devoted to this theory where one…
We investigate enumerability properties for classes of sets which permit recursive, lexicographically increasing approximations, or left-r.e. sets. In addition to pinpointing the complexity of left-r.e. Martin-L\"{o}f, computably, Schnorr,…
Algorithmic randomness theory starts with a notion of an individual random object. To be reasonable, this notion should have some natural properties; in particular, an object should be random with respect to image distribution if and only…
We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…
The main goal of this paper is to put some known results in a common perspective and to simplify their proofs. We start with a simple proof of a result from (Vereshchagin, 2002) saying that $\limsup_n\KS(x|n)$ (here $\KS(x|n)$ is…
Few-shot learning refers to understanding new concepts from only a few examples. We propose an information retrieval-inspired approach for this problem that is motivated by the increased importance of maximally leveraging all the available…
The purpose of this paper is to introduce a concept of equivalence between machine learning algorithms. We define two notions of algorithmic equivalence, namely, weak and strong equivalence. These notions are of paramount importance for…
Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view. We explore the interface between AIT and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on the…
Many latent-variable applications, including community detection, collaborative filtering, genomic analysis, and NLP, model data as generated by low-rank matrices. Yet despite considerable research, except for very special cases, the number…
Many algorithms in scientific computing and data science take advantage of low-rank approximation of matrices and kernels, and understanding why nearly-low-rank structure occurs is essential for their analysis and further development. This…
We prove two theorems related to the Central Limit Theorem (CLT) for Martin-L\"of Random (MLR) sequences. Martin-L\"of randomness attempts to capture what it means for a sequence of bits to be "truly random". By contrast, CLTs do not make…
Martin-Lof's definition of random sequences of cbits as those not belonging to any set of constructive zero Lebesgue measure is reformulated in the language of Algebraic Probability Theory. The adoption of the Pour-El Richards theory of…
Kobayashi introduced a uniform notion of compressibility of infinite binary sequences in terms of relative Turing computations with sub-identity use of the oracle. Kobayashi compressibility has remained a relatively obscure notion, with the…
Low-rank approximation of a matrix by means of random sampling has been consistently efficient in its empirical studies by many scientists who applied it with various sparse and structured multipliers, but adequate formal support for this…