Related papers: Individual codewords
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
A physical field has an infinite number of degrees of freedom since it has a field value at each location of a continuous space. Therefore, it is impossible to know a field from finite measurements alone and prior information on the field…
We introduce Ideograph, a language for expressing and manipulating structured data. Its types describe kinds of structures, such as natural numbers, lists, multisets, binary trees, syntax trees with variable binding, directed multigraphs,…
The problem of ranking can be described as follows. We have a set of combinatorial objects $S$, such as, say, the k-subsets of n things, and we can imagine that they have been arranged in some list, say lexicographically, and we want to…
The sequential structure of language, and the order of words in a sentence specifically, plays a central role in human language processing. Consequently, in designing computational models of language, the de facto approach is to present…
This paper proposes an alternative language for expressing results of the algorithmic theory of randomness. The language is more precise in that it does not involve unspecified additive or multiplicative constants, making mathematical…
In this short communication it is discussed the relation between disentangled states and algorithmic information theory aiming to construct an irreducible sentence whose length increases in a non-polynomial way when the number of qubits…
Interpretability is a pressing issue for machine learning. Common approaches to interpretable machine learning constrain interactions between features of the input, rendering the effects of those features on a model's output comprehensible…
I propose here a new concept of information based on two relevant aspects of its expression. The first related to the undeniable fact that the expression of information modifies the physical state of its receiver. The second to the…
We define a notion of randomness for individual and collections of formal languages based on automatic martingales acting on sequences of words from some underlying domain. An automatic martingale bets if the incoming word belongs to the…
Information theory is a powerful tool to express principles to drive autonomous systems because it is domain invariant and allows for an intuitive interpretation. This paper studies the use of the predictive information (PI), also called…
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…
Using the theory of Kolmogorov complexity the notion of facticity {\phi}(x) of a string is defined as the amount of self-descriptive information it contains. It is proved that (under reasonable assumptions: the existence of an empty machine…
The notion of transducer integer sequences is considered through a series of examples. By definition, transducer integer sequences are integer sequences produced, under a suitable interpretation, by finite automata encoding tree morphisms…
In this work, we attempt to explain the prediction of any black-box classifier from an information-theoretic perspective. For each input feature, we compare the classifier outputs with and without that feature using two…
In combinatorics, the probabilistic method is a very powerful tool to prove the existence of combinatorial objects with interesting and useful properties. Explicit constructions of objects with such properties are often very difficult, or…
Information Geometry has been used to inspire efficient algorithms for stochastic optimization, both in the combinatorial and the continuous case. We give an overview of the authors' research program and some specific contributions to the…
Life depends as much on the flow of information as on the flow of energy. Here we review the many efforts to make this intuition precise. Starting with the building blocks of information theory, we explore examples where it has been…
We study links between first-order formulas and arbitrary properties for families of theories, classes of structures and their isomorphism types. Possibilities for ranks and degrees for formulas and theories with respect to given properties…
A theoretical framework is presented allowing the treatment of quantum messages with components of variable length. To this aim a many-letter space, similiar to the Fock space, is constructed, generalizing the standard quantum information…