Related papers: Kolmogorov's Structure Functions and Model Selecti…
For each partition of a data set into a given number of parts there is a partition such that every part is as much as possible a good model (an "algorithmic sufficient statistic") for the data in that part. Since this can be done for every…
The information in an individual finite object (like a binary string) is commonly measured by its Kolmogorov complexity. One can divide that information into two parts: the information accounting for the useful regularity present in the…
While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing…
We consider the problem of finding an optimal statistical model for a given binary string. Following Kolmogorov, we use structure functions. In order to get concrete results, we replace Turing machines by finite automata and Kolmogorov…
For a finite word $w$ we define and study the Kolmogorov structure function $h_w$ for nondeterministic automatic complexity. We prove upper bounds on $h_w$ that appear to be quite sharp, based on numerical evidence.
Given a reference computer, Kolmogorov complexity is a well defined function on all binary strings. In the standard approach, however, only the asymptotic properties of such functions are considered because they do not depend on the…
Randomness and regularities in Finance are usually treated in probabilistic terms. In this paper, we develop a completely different approach in using a non-probabilistic framework based on the algorithmic information theory initially…
Kolmogorov suggested to measure quality of a statistical hypothesis $P$ for a data $x$ by two parameters: Kolmogorov complexity $C(P)$ of the hypothesis and the probability $P(x)$ of $x$ with respect to $P$. P. G\'acs, J. Tromp, P.M.B.…
In an attempt to demonstrate that local hidden variables are mathematically possible, Pitowsky constructed "spin-$\frac12$ functions" and later "Kolmogorovian models", which employs a nonstandard notion of probability. We describe…
Algorithmic statistics has two different (and almost orthogonal) motivations. From the philosophical point of view, it tries to formalize how the statistics works and why some statistical models are better than others. After this notion of…
Kolmogorov complexity theory is used to tell what the algorithmic informational content of a string is. It is defined as the length of the shortest program that describes the string. We present a programming language that can be used to…
We summarize our recent findings, where we proposed a framework for learning a Kolmogorov model, for a collection of binary random variables. More specifically, we derive conditions that link outcomes of specific random variables, and…
The mission of statistics is to provide adequate statistical hypotheses (models) for observed data. But what is an "adequate" model? To answer this question, one needs to use the notions of algorithmic information theory. It turns out that…
Link prediction in graphs is an important task in the fields of network science and machine learning. We investigate a flexible means of regularization for link prediction based on an approximation of the Kolmogorov complexity of graphs…
Kolmogorov complexity measures the algorithmic complexity of a finite binary string $\sigma$ in terms of the length of the shortest description $\sigma^*$ of $\sigma$. Traditionally, the length of a string is taken to measure the amount of…
Classical versions of Kolmogorov complexity are incomputable. Nevertheless, in 1975 Solovay showed that there are computable functions $f > K+O(1)$ such that for infinitely many strings $\sigma$, $f(\sigma)=K(\sigma)+O(1)$, where $K$…
The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description…
The Kolmogorov-Arnold representation is a proven adequate replacement of a continuous multivariate function by an hierarchical structure of multiple functions of one variable. The proven existence of such representation inspired many…
Estimating the joint probability mass function (PMF) of a set of random variables lies at the heart of statistical learning and signal processing. Without structural assumptions, such as modeling the variables as a Markov chain, tree, or…
In this paper we prove a theorem about regression, in that the shortest description of a function consistent with a finite sample of data is less than the combined conditional Kolmogorov complexities over the data in the sample.