Related papers: Occam Bound on Lowest Complexity of Elements
Symmetry of information establishes a relation between the information that x has about y (denoted I(x : y)) and the information that y has about x (denoted I(y : x)). In classical information theory, the two are exactly equal, but in…
Consider the case where consecutive blocks of N letters of a semi-infinite individual sequence X over a finite-alphabet are being compressed into binary sequences by some one-to-one mapping. No a-priori information about X is available at…
A word on $q$ symbols is a sequence of letters from a fixed alphabet of size $q$. For an integer $k\ge 1$, we say that a word $w$ is $k$-universal if, given an arbitrary word of length $k$, one can obtain it by removing entries from $w$. It…
Symmetry of information states that $C(x) + C(y|x) = C(x,y) + O(\log C(x))$. We show that a similar relation for online Kolmogorov complexity does not hold. Let the even (online Kolmogorov) complexity of an n-bitstring $x_1x_2... x_n$ be…
Mutual information I in infinite sequences (and in their finite prefixes) is essential in theoretical analysis of many situations. Yet its right definition has been elusive for a long time. I address it by generalizing Kolmogorov Complexity…
G. Edelman, O. Sporns, and G. Tononi have introduced the neural complexity of a family of random variables, defining it as a specific average of mutual information over subfamilies. We show that their choice of weights satisfies two natural…
The complexity of a system, in general, makes it difficult to determine some or almost all matrix elements of its operators. The lack of accuracy acts as a source of randomness for the matrix elements which are also subjected to an external…
The Element Distinctness problem is to decide whether each character of an input string is unique. The quantum query complexity of Element Distinctness is known to be $\Theta(N^{2/3})$; the polynomial method gives a tight lower bound for…
An infinite permutation is a linear ordering of the set of non-negative integers. Generally, the properties of infinite permutations analogous to those of infinite words show some resemblances and some differences between permutations and…
We consider a learning problem of identifying a dictionary matrix D (M times N dimension) from a sample set of M dimensional vectors Y = N^{-1/2} DX, where X is a sparse matrix (N times P dimension) in which the density of non-zero entries…
In this paper we analyze the notion of "stopping time complexity", informally defined as the amount of information needed to specify when to stop while reading an infinite sequence. This notion was introduced by Vovk and Pavlovic (2016). It…
Randomized higher-order computation can be seen as being captured by a lambda calculus endowed with a single algebraic operation, namely a construct for binary probabilistic choice. What matters about such computations is the probability of…
A composite likelihood is a combination of low-dimensional likelihood objects useful in applications where the data have complex structure. Although composite likelihood construction is a crucial aspect influencing both computing and…
We consider the distinct elements problem, where the goal is to estimate the number of distinct colors in an urn containing $ k $ balls based on $n$ samples drawn with replacements. Based on discrete polynomial approximation and…
We present an information-theoretic approach to lower bound the oracle complexity of nonsmooth black box convex optimization, unifying previous lower bounding techniques by identifying a combinatorial problem, namely string guessing, as a…
A lower bound on the probability $P(0<X<\delta)$ for all real $\delta>0$ and all random variables $X$ with log-concave p.d.f.'s such that $EX=0$ and $EX^2=1$ is obtained.
Concentration bounds are given for throwing balls into bins independently according to a distribution $p$. The probability of a $k$-loaded bin after $m$ balls is shown to be controlled on both sides by $\rho_{m,k} := m \|p\|_k / k$. This…
We consider two quantities that measure complexity of binary strings: $\mathit{KA}(x)$ is defined as the minus logarithm of continuous a priori probability on the binary tree, and $\mathit{KP}(x)$ denotes prefix complexity of a binary…
We study the approximability of Max Ones when the number of variable occurrences is bounded by a constant. For conservative constraint languages (i.e., when the unary relations are included) we give a complete classification when the number…
An infinite binary sequence has randomness rate at least $\sigma$ if, for almost every $n$, the Kolmogorov complexity of its prefix of length $n$ is at least $\sigma n$. It is known that for every rational $\sigma \in (0,1)$, on one hand,…