Related papers: Mutual Information as a Two-Point Correlation Func…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the pair-wise sum of probabilities of system conditions that have to be distinguished during the diagnosing process. The proposed measure describes the…
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precisely the mutual…
We consider discrete stochastic processes, modeled by classical master equations, on networks. The temporal growth of the lack of information about the system is captured by its non-equilibrium entropy, defined via the transition…
The participation coefficient is a widely used metric of the diversity of a node's connections with respect to a modular partition of a network. An information-theoretic formulation of this concept of connection diversity, referred to here…
We review of the interface between (theoretical) physics and information for non-experts. The origin of information as related to the notion of entropy is described, first in the context of thermodynamics then in the context of statistical…
We consider some formulations of the entropy bounds at the semiclassical level. The entropy S(V) localized in a region V is divergent in quantum field theory (QFT). Instead of it we focus on the mutual information I(V,W)=S(V)+S(W)-S(V\cup…
The similarity of local atomic environments is an important concept in many machine-learning techniques which find applications in computational chemistry and material science. Here, we present and discuss a connection between the…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
A full-rank lattice in the Euclidean space is a discrete set formed by all integer linear combinations of a basis. Given a probability distribution on $\mathbb{R}^n$, two operations can be induced by considering the quotient of the space by…
Because of the kinematic reversibility of the Stokes equation, fluid mixing at the microscale requires an interplay between advection and diffusion. Here we introduce mutual information between particle positions before and after mixing as…
Whether or not there is growing static order accompanying the dynamical heterogeneity and increasing relaxation times seen in glassy systems is a matter of dispute. An obstacle to resolving this issue is that the order is expected to be…
The entropy production of an open system coupled to a reservoir initialized in a canonical state can be expressed as a sum of two microscopic information-theoretic contributions: the system-bath mutual information and the relative entropy…
Entropic dynamics is a framework for defining dynamical systems that is aligned with the principles of information theory. In an entropic dynamics model for motion on a statistical manifold, we find that the rate of changes for expected…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
Relating thermodynamic and kinetic properties is a conceptual challenge with many practical benefits. Here, based on first principles, we derive a rigorous inequality relating the entropy and the dynamic propagator of particle…
Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number…
The fundamental question of how information spreads in closed quantum many-body systems is often addressed through the lens of the bipartite entanglement entropy, a quantity that describes correlations in a comprehensive (nonlocal) way.…
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter…