Related papers: Information-related complexity: a problem-oriented…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
This is a chapter in the Encyclopedia of Robotics. It is devoted to the study of complexity of complete (or exact) algorithms for robot motion planning. The term ``complete'' indicates that an approach is guaranteed to find the correct…
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two…
We define the complexity of a continuous-time linear system to be the minimum number of bits required to describe its forward increments to a desired level of fidelity, and compute this quantity using the rate distortion function of a…
We initiate the study of a quantity that we call coordination complexity. In a distributed optimization problem, the information defining a problem instance is distributed among $n$ parties, who need to each choose an action, which jointly…
The emergence of a complex, large-scale organisation of cosmic matter into the Cosmic Web is a beautiful exemplification of how complexity can be produced by simple initial conditions and simple physical laws. In the epoch of Big Data in…
The fine approach to measure information dependence is based on the total conditional complexity CT(y|x), which is defined as the minimal length of a total program that outputs y on the input x. It is known that the total conditional…
Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise…
The problem of defining and studying complexity of a time series has interested people for years. In the context of dynamical systems, Grassberger has suggested that a slow approach of the entropy to its extensive asymptotic limit is a sign…
The theory of computational complexity focuses on functions and, hence, studies programs whose interactive behavior is reduced to a simple question/answer pattern. We propose a broader theory whose ultimate goal is expressing and analyzing…
I discuss several aspects of information theory and its relationship to physics and neuroscience. The unifying thread of this somewhat chaotic essay is the concept of Kolmogorov or algorithmic complexity (Kolmogorov Complexity, for short).…
While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing…
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
When additional information sources are available in decision making problems that allow stochastic optimization formulations, an important question is how to optimally use the information the sources are capable of providing. A framework…
We explore a definition of complexity based on logic functions, which are widely used as compact descriptions of rules in diverse fields of contemporary science. Detailed numerical analysis shows that (i) logic complexity is effective in…
Our technologies complexify our environments. Thus, new technologies need to deal with more and more complexity. Several efforts have been made to deal with this complexity using the concept of self-organization. However, in order to…
In this paper we give a definition for the Kolmogorov complexity of a pure quantum state. In classical information theory the algorithmic complexity of a string is a measure of the information needed by a universal machine to reproduce the…
Data complexity is an important concept in the natural sciences and related areas, but lacks a rigorous and computable definition. In this paper, we focus on a particular sense of complexity that is high if the data is structured in a way…
In the first part of this paper, we present a unified framework for analyzing the algorithmic complexity of any optimization problem, whether it be continuous or discrete in nature. This helps to formalize notions like "input", "size" and…
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as…