Related papers: A note on Shannon entropy
It is well known that to estimate the Shannon entropy for symbolic sequences accurately requires a large number of samples. When some aspects of the data are known it is plausible to attempt to use this to more efficiently compute entropy.…
We develop a theory of \emph{Katetov functors} which provide a uniform way of constructing Fraisse limits. Among applications, we present short proofs and improvements of several recent results on the structure of the group of automorphisms…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
The knotted solitons introduced by Faddeev and Niemi is presently a subject of great interest in particle and mathematical physics. In this paper we give a condensed matter interpretation of the recent results of Faddeev and Niemi.
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
In this paper we discuss about the validity of the Shannon entropy functional in connection with the correct Gibbs-Hertz probability distribution function. We show that there is no contradiction in using the Shannon-Gibbs functional and…
The diversity of a community that cannot be fully counted must be inferred. The two preeminent inference methods are the MaxEnt method, which uses information in the form of constraints and Bayes' rule which uses information in the form of…
Similarity between quantum mechanics and thermodynamics is discussed. It is found that if the Clausius equality is imposed on the Shannon entropy and the analogue of the heat quantity, then the value of the Shannon entropy comes to formally…
We investigate the memory properties of discrete sequences built upon a finite number of states. We find that the block entropy can reliably determine the memory for systems modeled as Markov chains of arbitrary finite order. Further, we…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
We present a universal relation between the flow of a Renyi entropy and the full counting statistics of energy transfers. We prove the exact relation for a flow to a system in thermal equilibrium that is weakly coupled to an arbitrary…
In the paper we present the new approach to Kolmogorov-Sinai entropy and its quantization. Our presentation stems from an application of the Choquet theory to the theory of decompositions of states and therefore, it resembles our rigorous…
We present a problem relating measurements and information theory in spin foam models. In the three dimensional case of quantum gravity we can compute probabilities of spin network graphs and study the behaviour of the Shannon entropy…
Tsallis relative operator entropy was defined as a parametric extension of relative operator entropy and the generalized Shannon inequalities were shown in the previous paper. After the review of some fundamental properties of Tsallis…
We show that the principle of entropy increase may be exactly founded on a few axioms valid not only for quantum and classical statistics, but also for a wide range of statistical processes.
We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon's entropy. As a consequence, an asymptotic 100% confidence interval of entropy is…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
I show that non-decreasing entropy provides a necessary and sufficient condition to convert the state of a physical system into a different state by a reversible transformation that acts on the system of interest and a further "catalyst"…