Related papers: Measuring Information Transfer
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
No consensus seems to exist as to what constitutes a measurement which is still considered somewhat mysterious in many respects in quantum mechanics. At successive stages mathematical theory of measure, metrology and measurement theory…
Dynamics, the study of change, is normally the subject of mechanics. Whether the chosen mechanics is ``fundamental'' and deterministic or ``phenomenological'' and stochastic, all changes are described relative to an external time. Here we…
The evident contrast between the time symmetry of fundamental microscopic laws and the time asymmetry of macroscopic processes is a challenging physical problem. The observation of unitary evolution of a general physical system by an…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
Machine learning theory has mostly focused on generalization to samples from the same distribution as the training data. Whereas a better understanding of generalization beyond the training distribution where the observed distribution…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
We discuss the information entropy for a general open pointer-based simultaneous measurement and show how it is bound from below. This entropic uncertainty bound is a direct consequence of the structure of the entropy and can be obtained…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
Interpreting partial information collected from systems subject to noise is a key problem across scientific disciplines. Theoretical frameworks often focus on the dynamics of variables that result from coarse-graining the internal states of…
We perform an information-theoretical analysis of quantum measurement processes and obtain the global information balance in quantum measurements, in the form of a closed chain equation for quantum mutual entropies. Our balance provides a…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
The amount of information generated by a discrete time stochastic processes in a single step can be quantified by the entropy rate. We investigate the differences between two discrete time walk models, the discrete time quantum walk and the…
We propose to examine the predictability and the complexity characteristics of the Standard&Poor500 dynamics behaviors in a coarse-grained way using the symbolic dynamics method and under the prism of the Information theory through the…
We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…
The two-time measurement scheme is well studied in the context of quantum fluctuation theorem. However, it becomes infeasible when the random variable determined by a single measurement trajectory is associated with the von-Neumann entropy…