Related papers: Groups, Information Theory and Einstein's Likeliho…
Within its range of applicability, the Boltzmann equation seems unique in its capacity to accurately describe the transition from almost any initial state to a self-equilibrated thermal state. Using information-theoretic methods to rephrase…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
We propose a generalisation of Gibbs' statistical mechanics into the domain of non-negligible phase space correlations. Derived are the probability distribution and entropy as a generalised ensemble average, replacing…
To characterize strongly interacting statistical systems within a thermodynamical framework - complex systems in particular - it might be necessary to introduce generalized entropies, $S_g$. A series of such entropies have been proposed in…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
The Heisenberg uncertainty principle is known to be connected to the entropic uncertainty principle. This correspondence is obtained employing a Gaussian probability distribution for wave functions associated to the Shannon entropy.…
The Boltzmann--Gibbs entropy is a functional on the space of probability measures. When a state space is countable, one characterization of the Boltzmann--Gibbs entropy is given by the Shannon--Khinchin axioms, which consist of continuity,…
Using the generalized entropies which depend on two parameters we propose a set of quantitative characteristics derived from the Information Geometry based on these entropies. Our aim, at this stage, is modest, as we are first constructing…
We generalize, improve and unify theorems of Rumin, and Maassen--Uffink about classical entropies associated to quantum density matrices. These theorems refer to the classical entropies of the diagonals of a density matrix in two different…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
Statistical mechanics is generalized on the basis of an additive information theory for incomplete probability distributions. The incomplete normalization $\sum_{i=1}^wp_i^q=1$ is used to obtain generalized entropy $S=-k\sum_{i=1}^wp_i^q\ln…
To account for the origin of information accumulation in nature despite the entropy-increase law, we advocate a universal mechanism due to competition/selection of general composite entities, from simple to complex. To confirm its…
We present a general holistic theory for the organization of complex networks, both human-engineered and naturally-evolved. Introducing concepts of value of interactions and satisfaction as generic network performance measures, we show that…
A novel measure, quantumness of correlations is introduced here for bipartite states, by incorporating the required measurement scheme crucial in defining any such quantity. Quantumness coincides with the previously proposed measures in…
Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
In this work, we propose a new similarity index for images considering the entropy function and group theory. This index considers an algebraic group of images, it is defined by an inner law that provides a novel approach for the…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…
The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the…