Related papers: Permutation Excess Entropy and Mutual Information …
"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in…
Machine learning theory has mostly focused on generalization to samples from the same distribution as the training data. Whereas a better understanding of generalization beyond the training distribution where the observed distribution…
We consider a "length-biased" shift-dependent information measure, related to the differential entropy in which higher weight is assigned to large values of observed random variables. This allows us to introduce the notions of "weighted…
Systems out of equilibrium, in stationary as well as in nonstationary regimes, display a linear response to energy impulses simply expressed as the sum of two specific temporal correlation functions. There is a natural interpretation of…
We develop a unified, information theoretic interpretation of the number-phase complementarity that is applicable both to finite-dimensional (atomic) and infinite-dimensional (oscillator) systems. The relevant uncertainty principle is…
We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…
We give a systematic expansion of the crypticity--a recently introduced measure of the inaccessibility of a stationary process's internal state information. This leads to a hierarchy of k-cryptic processes and allows us to identify…
Quantum information scrambling refers to the loss of local recoverability of quantum information, which has found widespread attention from high energy physics to quantum computing. In the present analysis we propose a possible starting…
Gathering the most information by picking the least amount of data is a common task in experimental design or when exploring an unknown environment in reinforcement learning and robotics. A widely used measure for quantifying the…
Due to the absence of an external, classical time variable, the probabilistic predictions of covariant quantum theory are ambiguous when multiple measurements are considered. Here, we introduce an information theoretic framework to the…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
Mutual information and information entropies in momentum space are proposed as measures of the non-local aspects of information. Singlet and triplet state members of the helium isoelectronic series are employed to examine Coulomb and Fermi…
Entropy or information is a fundamental quantity contained in a system in statistical mechanics and information theory. In this paper, a definition of classical information entropy of parton distribution functions is suggested. The…
For sensory networks, we determine the rate with which they acquire information about the changing external conditions. Comparing this rate with the thermodynamic entropy production that quantifies the cost of maintaining the network, we…
Subshifts of deterministic substitutions are ubiquitous objects in dynamical systems and aperiodic order (the mathematical theory of quasicrystals). Two of their most striking features are that they have low complexity (zero topological…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
A well-known stochastic model for intermittent fluctuations in physical systems is investigated. The model is given by a super-position of uncorrelated exponential pulses, and the degree of pulse overlap is interpreted as an intermittency…
Whereas classical invariance principles for ergodic Markov chains address the situation in which the time horizon of observations is much larger than the mixing time, the quality of approximation is questionable when this is not the case…
We derive a geometrical expression of the excess entropy production for quasi-static transitions between nonequilibrium steady states of Markovian jump processes, which can be exactly applied to nonlinear and nonequilibrium situations. The…
In a genetic algorithm, fluctuations of the entropy of a genome over time are interpreted as fluctuations of the information that the genome's organism is storing about its environment, being this reflected in more complex organisms. The…