Related papers: A Logarithmic Decomposition for Information
Shannon mutual information provides a measure of how much information is, on average, contained in a set of neural activities about a set of stimuli. It has been extensively used to study neural coding in different brain areas. To apply a…
We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…
Proving linear inequalities and identities of Shannon's information measures, possibly with linear constraints on the information measures, is an important problem in information theory. For this purpose, ITIP and other variant algorithms…
A linear transformation f(S) of configurational entropy with length scale dependent coefficients as a measure of spatial inhomogeneity is considered. When a final pattern is formed with periodically repeated initial arrangement of point…
Any coded subshift X defined by a set C of code words contains a subshift, which we call L, consisting of limits of single code words. We show that when C satisfies a unique decomposition property, the topological entropy h(X) of X is…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
When the von Neumann entropy (VNE) of a system increases due to measurements, certain information is lost, some of which may be recoverable. We define information retrievability (IR) and information loss (IL) as functions of the density…
The notion of a partition on a set is mathematically dual to the notion of a subset of a set, so there is a logic of partitions dual to Boole's logic of subsets (Boolean subset logic is usually mis-specified as the special case of…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
We show that a newly proposed Shannon-like entropic measure of shape complexity applicable to spatially-localized or periodic mathematical functions known as configurational entropy (CE) can be used as a predictor of spontaneous decay rates…
We present a validation of the asdf method, an information-theoretic framework for computing thermodynamic entropy from molecular configurations. The method reformulates entropy estimation as the Shannon entropy of a residual mapping…
Collective organization in matter plays a significant role in its expressed physical properties. Typically, it is detected via an order parameter, appropriately defined for each given system's observed emergent patterns. Recent developments…
We present graphs of information versus disturbance for general quantum measurements of completely unknown states. Each piece of information and disturbance is quantified by two measures: (i) the Shannon entropy and estimation fidelity for…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
The problem of how to properly quantify redundant information is an open question that has been the subject of much recent research. Redundant information refers to information about a target variable S that is common to two or more…
We establish a precise connection between statistical significance in dependence testing and information-theoretic dependence as quantified by Shannon mutual information (MI). In the absence of prior distributional information, we consider…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…