Related papers: Transfer Entropy: where Shannon meets Turing
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
We investigate correlations in information carriers, e.g. texts and pieces of music, which are represented by strings of letters. For information carrying strings generated by one source (i.e. a novel or a piece of music) we find…
The ultimate purpose of the statistical analysis of ordinal patterns is to characterize the distribution of the features they induce. In particular, knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class…
Multivariate fluctuation relations are established in three stochastic models of transistors, which are electronic devices with three ports and thus two coupled currents. In the first model, the transistor has no internal state variable and…
The paper is devoted to the analysis of relationships between principal objects of the spectral theory of dynamical systems (transfer and weighted shift operators) and basic characteristics of information theory and thermodynamic formalism…
The entropic way of formulating Heisenberg's uncertainty principle not only plays a fundamental role in applications of quantum information theory but also is essential for manifesting genuine nonclassical features of quantum systems. In…
Classification is a machine learning method used in many practical applications: text mining, handwritten character recognition, face recognition, pattern classification, scene labeling, computer vision, natural langage processing. A…
Partial teleportation of entanglement is to teleport one particle of an entangled pair through a quantum channel. This is conceptually equivalent to quantum swapping. We consider the partial teleportation of entanglement in the noisy…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
A general theory for computing information transfers in nonlinear stochastic systems driven by deterministic forcing and additive and/or multiplicative noises, is presented. It extends the Liang-Kleeman framework of causality inference to…
Different observations of a relation between inputs ("sources") and outputs ("targets") are often reported in terms of histograms (discretizations of the source and the target densities). Transporting these densities to each other provides…
Causal inference is perhaps one of the most fundamental concepts in science, beginning originally from the works of some of the ancient philosophers, through today, but also weaved strongly in current work from statisticians, machine…
Dynamical systems can be analyzed via their Frobenius-Perron transfer operator and its estimation from data is an active field of research. Recently entropic transfer operators have been introduced to estimate the operator of deterministic…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
Granger causality is a statistical notion of causal influence based on prediction via vector autoregression. Developed originally in the field of econometrics, it has since found application in a broader arena, particularly in neuroscience.…
We introduce the concept of entropic nonsignaling correlations, i.e., entropies arising from probabilistic theories that are compatible with the fact that we cannot transmit information instantaneously. We characterize and show the…
We present a universal relation between the flow of a Renyi entropy and the full counting statistics of energy transfers. We prove the exact relation for a flow to a system in thermal equilibrium that is weakly coupled to an arbitrary…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…