Related papers: A generalization of majorization that characterize…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
Generalised degrees provide a natural bridge between local and global topological properties of networks. We define the generalised degree to be the number of neighbours of a node within one and two steps respectively. Tailored random graph…
This paper is focused on derivations of data-processing and majorization inequalities for $f$-divergences, and their applications in information theory and statistics. For the accessibility of the material, the main results are first…
Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…
Nature is full of random networks of complex topology describing such apparently disparate systems as biological, economical or informatical ones. Their most characteristic feature is the apparent scale-free character of interconnections…
A unified combinatorial definition of the information content and entropy of different types of patterns, compatible with the traditional concepts of information and entropy, going beyond the limitations of Shannon information interpretable…
The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we…
Due to the absence of an external, classical time variable, the probabilistic predictions of covariant quantum theory are ambiguous when multiple measurements are considered. Here, we introduce an information theoretic framework to the…
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
We introduce a hierarchical classification of theories that describe systems with fundamentally limited information content. This property is introduced in an operational way and gives rise to the existence of mutually complementary…
We study the behavior of a nonlinear semiclassical system using Shannon entropy and two approaches to statistical complexity. These systems involve the interaction between classical variables (representing the environment) and quantum ones.…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
The entropic uncertainty measures of the multidimensional hydrogenic states quantify the multiple facets of the spatial delocalization of the electronic probability density of the system. The Shannon entropy is the most adequate uncertainty…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…