相关论文: Empirical entropy in context

Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…

It is well-known that the Shannon entropies of some parameterized probability distributions are concave functions with respect to the parameter. In this paper we consider a family of such distributions (including the binomial, Poisson, and…

An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…

A recently proposed history formalism is used to define temporal entanglement in quantum systems, and compute its entropy. The procedure is based on the time-reduction of the history density operator, and allows a symmetrical treatment of…

A general framework to describe a vast majority of biology-inspired systems is to model them as stochastic processes in which multiple couplings are in play at the same time. Molecular motors, chemical reaction networks, catalytic enzymes,…

The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which detailed balance and time-reversal symmetry are broken. Despite…

Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…

The use of coarse graining to connect physical and information theoretic entropies has recently been given a precise formulation in terms of ``observational entropy'', describing entropy for observers with respect to a measurement. Here we…

We compute the Hamiltonian and Lagrangian associated to the large deviations of the trajectory of the empirical distribution for independent Markov processes, and of the empirical measure for translation invariant interacting Markov…

The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…

We comment on some conceptual and and technical problems related to computational mechanics, point out some errors in several papers, and straighten out some wrong priority claims. We present explicitly the correct algorithm for…

The present paper studies continuity of generalized entropy functions and relative entropies defined using the notion of a deformed logarithmic function. In particular, two distinct definitions of relative entropy are discussed. As an…

Kolmogorov complexity of a finite binary word reflects both algorithmic structure and the empirical distribution of symbols appearing in the word. Words with symbol frequencies far from one half have smaller combinatorial richness and…

Topological entropy or spatial entropy is a way to measure the complexity of shift spaces. This study investigates the relationships between the spatial entropy and the various periodic entropies which are computed by skew-coordinated…

Intrinsic computation refers to how dynamical systems store, structure, and transform historical and spatial information. By graphing a measure of structural complexity against a measure of randomness, complexity-entropy diagrams display…

Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…

We develop a method, based on a Bochner-type identity, to obtain estimates on the exponential rate of decay of the relative entropy from equilibrium of Markov processes in discrete settings. When this method applies the relative entropy…

Information entropy is applied to the analysis of time series generated by dynamical systems. Complexity of a temporal or spatio-temporal signal is defined as the difference between the sum of entropies of the local linear regions of the…

We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…

We discuss how maximum entropy methods may be applied to the reconstruction of Markov processes underlying empirical time series and compare this approach to usual frequency sampling. It is shown that, at least in low dimension, there…