Related papers: Individual codewords
We characterize the algorithmic dimensions (i.e., the lower and upper asymptotic densities of information) of infinite binary sequences in terms of the inability of learning functions having an algorithmic constraint to detect patterns in…
A practical notation can convey valuable intuitions and concisely express new ideas. Information theory is of importance to machine learning, but the notation for information-theoretic quantities is sometimes opaque. We propose a practical…
Complex systems, from the human brain to the global economy, are made of multiple elements that interact in such ways that the behaviour of the `whole' often seems to be more than what is readily explainable in terms of the `sum of the…
Information personalization is fertile ground for application of AI techniques. In this article I relate personalization to the ability to capture partial information in an information-seeking interaction. The specific focus is on…
A large body of work in psycholinguistics has focused on the idea that online language comprehension can be shallow or `good enough': given constraints on time or available computation, comprehenders may form interpretations of their input…
Our understanding of the visual world goes beyond naming objects, encompassing our ability to parse objects into meaningful parts, attributes, and relations. In this work, we leverage natural language descriptions for a diverse set of 2K…
Decision trees are simple, yet powerful, classification models used to classify categorical and numerical data, and, despite their simplicity, they are commonly used in operations research and management, as well as in knowledge mining.…
Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help…
Presents a history of the evolution of the author's ideas on program-size complexity and its applications to metamathematics over the course of more than four decades. Includes suggestions for further work.
In this short communication it is proposed the general form of a n-qubit disentangled state as a irreducible sentence, in the sense explained by the algorithmic information theory, whose length increases in a non-polynomial way when the…
Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures and point out some of…
Algorithmic idealism represents a transformative approach to understanding reality, emphasizing the informational structure of self-states and their algorithmic transitions over traditional notions of an external, objective universe. Rooted…
We give a non-technical introduction of the basic concepts of Quantum Information Theory along the distinction between possible and impossible machines. We then proceed to describe the mathematical framework of Quantum Information Theory.…
A principal who values an object allocates it to one or more agents. Agents learn private information (signals) from an information designer about the allocation payoff to the principal. Monetary transfer is not available but the principal…
We introduce a new combinatorial object called tower diagrams and prove fundamental properties of these objects. We also introduce an algorithm that allows us to slide words to tower diagrams. We show that the algorithm is well-defined only…
Categorization is an important topic both for biological and artificial neural networks. Here, we take an information theoretic approach to assess the efficiency of the representations induced by category learning. We show that one can…
Machine language acquisition is often presented as a problem of imitation learning: there exists a community of language users from which a learner observes speech acts and attempts to decode the mappings between utterances and situations.…
A self-replicating system where the elements belonging to a solution category can replicate themselves by copying their own informations, is considered. The information carried by each element is defined by an element of all the n multiple…
Clones are specializations of operads forming powerful instruments to describe varieties of algebras wherein repeating variables are allowed in their equations. They allow us in this way to realize and study a large range of algebraic…
Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…