Related papers: Facticity as the amount of self-descriptive inform…
Strictness analysis is critical to efficient implementation of languages with non-strict evaluation, mitigating much of the performance overhead of laziness. However, reasoning about strictness at the source level can be challenging and…
The complexity of a system description is a function of the entropy of its symbolic description. Prior to computing the entropy of the system description, an observation scale has to be assumed. In natural language texts, typical scales are…
Symmetry of information establishes a relation between the information that x has about y (denoted I(x : y)) and the information that y has about x (denoted I(y : x)). In classical information theory, the two are exactly equal, but in…
Machine learning systems regularly deal with structured data in real-world applications. Unfortunately, such data has been difficult to faithfully represent in a way that most machine learning techniques would expect, i.e. as a real-valued…
A meta-complexity assumption, Feasible Chaitin Incompleteness (FCI), asserts the hardness of ruling out length $t$ proofs that string $x$ is Kolmogorov random (e.g. $x{\in}R$), by analogy to Chaitin's result that proving $x{\in}R$ is…
Topological models of empirical and formal inquiry are increasingly prevalent. They have emerged in such diverse fields as domain theory [1, 16], formal learning theory [18], epistemology and philosophy of science [10, 15, 8, 9, 2],…
Depth is a complexity measure for natural systems of the kind studied in statistical physics and is defined in terms of computational complexity. Depth quantifies the length of the shortest parallel computation required to construct a…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
The concept of effective complexity of an object as the minimal description length of its regularities has been initiated by Gell-Mann and Lloyd. The regularities are modeled by means of ensembles, that is probability distributions on…
Identity theorem for analytic complex functions says that a function is uniquely defined by its values on a set that contains a density point. The paper presents sufficient conditions for classes of real analytic functions that ensures…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
As is the case of many signals produced by complex systems, language presents a statistical structure that is balanced between order and disorder. Here we review and extend recent results from quantitative characterisations of the degree of…
Algorithmic statistics considers the following problem: given a binary string $x$ (e.g., some experimental data), find a "good" explanation of this data. It uses algorithmic information theory to define formally what is a good explanation.…
Satisfiability solvers are increasingly playing a key role in software verification, with particularly effective use in the analysis of security vulnerabilities. String processing is a key part of many software applications, such as…
The weighted entropy $H^{\rm w}_\phi (X)=H^{\rm w}_\phi (f)$ of a random variable $X$ with values $x$ and a probability-mass/density function $f$ is defined as the mean value ${\mathbb E} I^{\rm w}_\phi(X)$ of the weighted information…
We study the structure of the partial order induced by the definability relation on definitions of truth for the language of arithmetic. Formally, a definition of truth is any sentence $\alpha$ which extends a weak arithmetical theory…
Statistical data by their very nature are indeterminate in the sense that if one repeats the process of collecting the data the new data set will be different from the original. But two data sets generated in the same way should ``tell the…
String theory provides an example of the kind of apparent inconsistency that the {\it Principle of Black Hole Complementarity\/} deals with. To a freely infalling observer a string falling through a black hole horizon appears to be a Planck…
We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s…
In 1974 Kolmogorov proposed a non-probabilistic approach to statistics and model selection. Let data be finite binary strings and models be finite sets of binary strings. Consider model classes consisting of models of given maximal…