Related papers: Variability as a better characterization of Shanno…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
We review of the interface between (theoretical) physics and information for non-experts. The origin of information as related to the notion of entropy is described, first in the context of thermodynamics then in the context of statistical…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
The entropic uncertainty measures of the multidimensional hydrogenic states quantify the multiple facets of the spatial delocalization of the electronic probability density of the system. The Shannon entropy is the most adequate uncertainty…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
MaxEnt's variational principle, in conjunction with Shannon's logarithmic information measure, yields only exponential functional forms in straightforward fashion. In this communication we show how to overcome this limitation via the…
The basin entropy is a simple idea that aims to measure the the final state unpredictability of multistable systems. Since 2016, the basin entropy has been widely used in different contexts of physics, from cold atoms to galactic dynamics.…
The Shannon information-entropy uncertainty (in brief as "information uncertainty") is used to analyze the fragments in the measured 140$A$ MeV $^{40, 48}$Ca + $^{9}$Be and $^{58, 64}$Ni + $^{9}$Be reactions. A scaling phenomenon is found…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
In this paper we discuss about the validity of the Shannon entropy functional in connection with the correct Gibbs-Hertz probability distribution function. We show that there is no contradiction in using the Shannon-Gibbs functional and…
The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the…
We show that the way in which the Shannon entropy of sequences produced by an information source converges to the source's entropy rate can be used to monitor how an intelligent agent builds and effectively uses a predictive model of its…
Statistical properties of coupled dynamic-stochastic systems are studied within a combination of the maximum information principle and the superstatistical approach. The conditions at which the Shannon entropy functional leads to a…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…
The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates…
A well-interpretable measure of information has been recently proposed based on a partition obtained by intersecting a random sequence with its moving average. The partition yields disjoint sets of the sequence, which are then ranked…
Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a…