Related papers: Estimating Information-Theoretic Quantities
We provide an information-theoretic analysis of Thompson sampling that applies across a broad range of online optimization problems in which a decision-maker must learn from partial feedback. This analysis inherits the simplicity and…
Information-theoretic limits of {\it cognitive radio} networks have been under exploration for more than a decade. Although such limits are unknown for many networks, including the simplest case with two pairs of transmitter-receiver, there…
This paper consists of two parts. In the first part, we develop a new information theory, in which it is not a coincidence that information and physical entropy share the same mathematical formula. It is an adaptation of mind to help search…
In this article a notion of information is presented which stresses the contextuality of quantum objects and their measurement. Mathematically this is reached by a quantification of the quantum mechanical surplus knowledge which has been…
An information-theoretic framework known as integrated information theory (IIT) has been introduced recently for the study of the emergence of consciousness in the brain [D. Balduzzi and G. Tononi, PLoS Comput. Biol. 4, e1000091 (2008)].…
Many of today's most pressing issues require a more robust understanding of how information spreads in populations. Current models of information spread can be thought of as falling into one of two varieties: epidemiologically-inspired…
Criticality is hypothesized as a physical mechanism underlying efficient transitions between cortical states and remarkable information processing capacities in the brain. While considerable evidence generally supports this hypothesis,…
We consider probabilistic theories in which the most elementary system, a two-dimensional system, contains one bit of information. The bit is assumed to be contained in any complete set of mutually complementary measurements. The…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
Neuroscience is undergoing dramatic progress because of the vast data streams derived from the new technologies product of the BRAIN initiative and other enterprises. As any other scientific field, neuroscience benefits from having clear…
Information theory gives rise to a novel method for causal skeleton discovery by expressing associations between variables as tensors. This tensor-based approach reduces the dimensionality of the data needed to test for conditional…
Estimating mutual correlations between random variables or data streams is essential for intelligent behavior and decision-making. As a fundamental quantity for measuring statistical relationships, mutual information has been extensively…
Information flow framed in a computational and complexity context is relevant to the understanding of cognitive processes and awareness. In this paper, we begin with analyzing an information theory framework developed in recent years under…
In this paper, we prove the existence of fundamental relations between information theory and estimation theory for network-coded flows. When the network is represented by a directed graph G=(V, E) and under the assumption of uncorrelated…
An information-theoretic framework is introduced to analyze last-layer embedding, focusing on learned representations for regression tasks. We define representation-rate and derive limits on the reliability with which input-output…
We develop a general formalism for representing and understanding structure in complex systems. In our view, structure is the totality of relationships among a system's components, and these relationships can be quantified using information…
In machine learning or scientific computing, model performance is measured with an objective function. But why choose one objective over another? Information theory gives one answer: To maximize the information in the model, select the…
Resource theories are broad frameworks that capture how useful objects are in performing specific tasks. In this paper we devise a formal resource theory quantum measurements, focusing on the ability of a measurement to acquire information.…
The informational synthesis of neural structures, processes, parameters and characteristics that allow a unified description and modeling as neural machines of natural and artificial neural systems is presented. The general informational…
Quantum mechanics, information theory, and relativity theory are the basic foundations of theoretical physics. The acquisition of information from a quantum system is the interface of classical and quantum physics. Essential tools for its…