Related papers: Power Law and Entropy
The relationship between excess entropy and diffusion is revisited by means of large-scale computer simulation combined to supervised learning approach to determine the excess entropy for the Lennard-Jones potential. Results reveal that it…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the…
This article serves as a brief introduction to the Shannon information theory. Concepts of information, Shannon entropy and channel capacity are mainly covered. All these concepts are developed in a totally combinatorial flavor. Some issues…
We study the Shannon entropy of the cluster size distribution in classical as well as explosive percolation, in order to estimate the uncertainty in the sizes of randomly chosen clusters. At the critical point the cluster size distribution…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
Uncertainty relations have become the trademark of quantum theory since they were formulated by Bohr and Heisenberg. This review covers various generalizations and extensions of the uncertainty relations in quantum theory that involve the…
The science of thermodynamics was put together in the Nineteenth Century to describe large systems in equilibrium. One part of thermodynamics defines entropy for equilibrium systems and demands an ever-increasing entropy for non-equilibrium…
The container methods are powerful tools to bound the number of independent sets of graphs and hypergraphs, and they have been extremely influential in the area of extremal and probabilistic combinatorics. We will focus on more specialized…
A new definition of entropy is introduced using a model that simulates an expanding space-time compatible with the fundamental principle of cosmology. The entropy is obtained by mean of a state function that measures the variation of the…
These expository notes propose to follow, across fields, some aspects of the concept of entropy. Starting from the work of Boltzmann in the kinetic theory of gases, various universes are visited, including Markov processes and their…
We introduce a new measure of interdependence among the components of a random vector along the main diagonal of the vector copula, i.e. along the line $u_{1}=\ldots=u_{J}$, for $\left(u_{1},\ldots,u_{J}\right)\in\left[0,1\right]^{J}$. Our…
Similar to the generalized extreme value (GEV) family, the generalized extreme value distributions under power normalization are introduced by Roudsari (1999) and Barakat et al. (2013). In this article, we study the asymptotic behavior of…
The paper examines relationships between the Shannon entropy and the $\ell_{\alpha}$-norm for $n$-ary probability vectors, $n \ge 2$. More precisely, we investigate the tight bounds of the $\ell_{\alpha}$-norm with a fixed Shannon entropy,…
This article addresses the issue of the proof of the entropy power inequality (EPI), an important tool in the analysis of Gaussian channels of information transmission, proposed by Shannon. We analyse continuity properties of the mutual…
A hypothesis proposed in the paper (Entropy 2017, 19, 345) on the deductive formulation of a physical theory based on explicitly- and universally-introduced basic concepts is further developed. An entropic measure of time with a number of…
This paper is an introduction to the von Neumann entropy in a historic approach. Von Neumann's gedanken experiment is repeated, which led him to the formula of thermodynamic entropy of a statistical operator. In the analysis of his ideas we…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the…
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…