Related papers: Entropy under disintegrations
Every symbolic system supports a Borel measure that is invariant under the shift, but it is not known if every such systems supports a measure that is invariant under all of its automorphisms; known as a characteristic measure. We give…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
In this paper, we introduce two measure theoretical notions of conditional entropy for finite measurable covers conditioned to a finite measurable partition and prove that they are equal. Using this we state a local variational principle…
In this paper, we introduce and study a notion of asymptotic expansion in measure for measurable actions. This generalises expansion in measure and provides a new perspective on the classical notion of strong ergodicity. Moreover, we obtain…
We consider a set $\mbK = \bigcup_{n \in \mbbN}\mbK_n$ of {\em finite} structures such that all members of $\mbK_n$ have the same universe, the cardinality of which approaches $\infty$ as $n\to\infty$. Each structure in $\mbK$ may have a…
In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
We propose a "decomposition method" to prove non-asymptotic bound for the convergence of empirical measures in various dual norms. The main point is to show that if one measures convergence in duality with sufficiently regular observables,…
A {\it uniformly $p$-to-one endomorphism} is a measure-preserving map with entropy log $p$ which is almost everywhere $p$-to-one and for which the conditional expectation of each preimage is precisely $1/p$. The {\it standard} example of…
The entropy of a finite probability space $X$ measures the observable cardinality of large independent products $X^{\otimes n}$ of the probability space. If two probability spaces $X$ and $Y$ have the same entropy, there is an almost…
In the present paper, we introduce a natural extension of AKM-topological entropy for noncompact spaces and prove a variational principle which states that the topological entropy, the supremum of the measure theoretical entropies and the…
Information entropy and its extension, which are important generalization of entropy, have been applied in many research domains today. In this paper, a novel generalized relative entropy is constructed to avoid some defects of traditional…
In this paper we continue our earlier investigations into the asymptotic behaviour of infinite systems of coupled differential equations. Under the mild assumption that the so-called characteristic function of our system is completely…
The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
Assume that a sequence $x=x_0x_1\ldots$ is frequency-typical for a finite-valued stationary stochastic process $\textbf X$. We prove that the function associating to $x$ the entropy-rate $\bar H(\textbf X)$ of $\textbf X$ is uniformly…
We develop a notion of entropy, using hyperbolic time, for laminations by hyperbolic Riemann surfaces. When the lamination is compact and transversally smooth, we show that the entropy is finite and the Poincare metric on leaves is…
We derive explicit integrability conditions for stochastic integrals taken over time and space driven by a random measure. Our main tool is a canonical decomposition of a random measure which extends the results from the purely temporal…
Random walks on a group $G$ model many natural phenomena. A random walk is defined by a probability measure $p$ on $G$. We are interested in asymptotic properties of the random walks and in particular in the linear drift and the asymptotic…