Related papers: Empirical entropy in context
We consider the differential entropy of probability measures absolutely continuous with respect to a given $\sigma$-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general…
Consider a uniformly sampled random $d$-regular graph on $n$ vertices. If $d$ is fixed and $n$ goes to $\infty$ then we can relate typical (large probability) properties of such random graph to a family of invariant random processes (called…
Lower bounds for the R\'enyi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
Interacting random field of probabilities links Kolmogorov law 0-1 and Bayesian probabilities observing Markov diffusion process under Yes-No actions of random impulse. These objective probabilities measure virtual probing impulses…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
It is well known that to estimate the Shannon entropy for symbolic sequences accurately requires a large number of samples. When some aspects of the data are known it is plausible to attempt to use this to more efficiently compute entropy.…
The problem of positive Kolmogorov-Sinai entropy of the Chirikov-Standard map with respect to the invariant Lebesgue measure on the two-dimensional is open. In 1999, we believed to have a proof that the entropy can be bounded below. This…
Starting from the pioneering works of Shannon and Weiner in 1948, a plethora of works have been reported on entropy in different directions. Entropy-related review work in the direction of statistics, reliability and information science, to…
We propose an operational definition of the entropy of cosmological perturbations based on a truncation of the hierarchy of Green functions. The value of the entropy is unambiguous despite gauge invariance and the renormalization procedure.…
We consider a Markov chain obtained by random iterations of Lipschitz maps $T_i$ chosen with a probability $p_i(x)$ depending on the current position $x$. We assume this system has a property of "contraction on average", that is $\sum_i…
Using known entropic and information inequalities new inequalities for some classical polynomials are obtained. Examples of Jacobi and Legendre polynomials are considered.
In the case of ergodicity much of the structure of a one-dimensional time-discrete dynamical system is already determined by its ordinal structure. We generally discuss this phenomenon by considering the distribution of ordinal patterns,…
A powerful tool is developed for the characterization of chaotic signals. The approach is based on the symbolic encoding of time series (according to their ordinal patterns) combined with the ensuing characterization of the corresponding…
Entropic uncertainty relations for the position and momentum within the generalized uncertainty principle are examined. Studies of this principle are motivated by the existence of a minimal observable length. Then the position and momentum…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
We discuss basic statistical properties of systems with multifractal structure. This is possible by extending the notion of the usual Gibbs--Shannon entropy into more general framework - Renyi's information entropy. We address the…
A mathematical interpretation of the usual definition of entropy (for a discrete probability distribution or a trace 1 positive operator) is given. This formulation makes some properties of entropy immediate.
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…
We consider the dynamical behavior of Martin-L\"of random points in dynamical systems over metric spaces with a computable dynamics and a computable invariant measure. We use computable partitions to define a sort of effective symbolic…