Related papers: On a comparative study between dependence scales d…
We consider dynamic versions of the mutual information of lifetime distributions, with focus on past lifetimes, residual lifetimes and mixed lifetimes evaluated at different instants. This allows to study multicomponent systems, by…
Renormalization is an essential technique in field-theoretic descriptions of natural phenomena, where the absence of a UV-complete description yields an abundance of divergent quantities. While the renormalization prescription has been…
This paper presents expression of mutual information that defines the information gain in planning of sensing resources, when the goal is to reduce the forecast uncertainty of some quantities of interest and the system dynamics is described…
Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number…
We report on the computer study of a lattice system that relaxes from a metastable state. Under appropriate nonequilibrium randomness, relaxation occurs by avalanches, i.e., the model evolution is discontinuous and displays many scales in a…
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…
Measuring the dependence of data plays a central role in statistics and machine learning. In this work, we summarize and generalize the main idea of existing information-theoretic dependence measures into a higher-level perspective by the…
Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and…
Through computer simulations, we research several different measures of dependence, including Pearson's and Spearman's correlation coefficients, the maximal correlation, the distance correlation, a function of the mutual information called…
The performance of a number of different measures of nonlinearity in a time series is compared numerically. Their power to distinguish noisy chaotic data from linear stochastic surrogates is determined by Monte Carlo simulation for a number…
Scaling behavior is studied of several dominant eigenvalues of spectra of Markov matrices and the associated correlation times governing critical slowing down in models in the universality class of the two-dimensional Ising model. A scheme…
We investigate the multipartite mutual information between $N$ discrete-state stochastic units interacting in a network that is invariant under unit permutations. We show that when the system relaxes to fixed point attractors, multipartite…
We introduce an independence criterion based on entropy regularized optimal transport. Our criterion can be used to test for independence between two samples. We establish non-asymptotic bounds for our test statistic and study its…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
We apply a new technique, the mutual information (MI) from information theory, to time-distance helioseismology, and demonstrate that it can successfully reproduce several classic results based on the widely used cross-covariance method. MI…
Extensions to various information theoretic quantities used for nonlinear time series analysis are discussed, as well as their relationship to the generalized correlation integral. It is shown that calculating redundancies from the…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
Exponential models of distributions are widely used in machine learning for classiffication and modelling. It is well known that they can be interpreted as maximum entropy models under empirical expectation constraints. In this work, we…
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…