Related papers: A Computable Measure of Algorithmic Probability by…
We study the approximation of arbitrary distributions $P$ on $d$-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback--Leibler-type functional. We show that such an approximation exists if…
The present work explores the theoretical limits of Machine Learning (ML) within the framework of Kolmogorov's theory of Algorithmic Probability, which clarifies the notion of entropy as Expected Kolmogorov Complexity and formalizes other…
We study the empirical meaning of randomness with respect to a family of probability distributions $P_\theta$, where $\theta$ is a real parameter, using algorithmic randomness theory. In the case when for a computable probability…
The paper is concerned with the equilibrium distributions of continuous-time density dependent Markov processes on the integers. These distributions are known typically to be approximately normal, and the approximation error, as measured in…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
The linear complexity (LC) of a sequence has been used as a convenient measure of the randomness of a sequence. Based on the theories of linear complexity, $k$-error linear complexity, the minimum error and the $k$-error linear complexity…
In this monograph, we prove an asymptotic approximation for integrals of probability densities over sets in finite dimensional euclidean space, which are far away from the origin (asymptotic sets). We use this approximation to investigate…
This paper studies best finitely supported approximations of one-dimensional probability measures with respect to the $L^r$-Kantorovich (or transport) distance, where either the locations or the weights of the approximations' atoms are…
This paper is concerned with algorithms for prediction of discrete sequences over a finite alphabet, using variable order Markov models. The class of such algorithms is large and in principle includes any lossless compression algorithm. We…
The approximation of a discrete probability distribution $\mathbf{t}$ by an $M$-type distribution $\mathbf{p}$ is considered. The approximation error is measured by the informational divergence $\mathbb{D}(\mathbf{t}\Vert\mathbf{p})$, which…
We present a new similarity measure based on information theoretic measures which is superior than Normalized Compression Distance for clustering problems and inherits the useful properties of conditional Kolmogorov complexity. We show that…
We investigate the complexity of short symbolic sequences of chaotic dynamical systems by using lossless compression algorithms. In particular, we study Non-Sequential Recursive Pair Substitution (NSRPS), a lossless compression algorithm…
For a measurable map $T$ and a sequence of $T$-invariant probability measures $\mu_n$ that converges in some sense to a $T$-invariant probability measure $\mu$, an estimate from below for the Kolmogorov--Sinai entropy of $T$ with respect to…
The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These…
After reviewing unnormalized and normalized information distances based on incomputable notions of Kolmogorov complexity, we discuss how Kolmogorov complexity can be approximated by data compression algorithms. We argue that optimal…
The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong…
In a variety of applications it is important to extract information from a probability measure $\mu$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time…
In this paper, we developed a novel method of nonparametric relative entropy (RlEn) for modelling loss of complexity in intermittent time series. The method consists of two steps. We first fit a nonlinear autoregressive model to each…
Kolmogorov complexity of a finite binary word reflects both algorithmic structure and the empirical distribution of symbols appearing in the word. Words with symbol frequencies far from one half have smaller combinatorial richness and…
Langevin algorithms are gradient descent methods augmented with additive noise, and are widely used in Markov Chain Monte Carlo (MCMC) sampling, optimization, and machine learning. In recent years, the non-asymptotic analysis of Langevin…