Related papers: On analytic properties of entropy rate
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
In connection with the Entropy Conjecture it is known that the topological entropy of a continuous graph map is bounded from below by the spectral radius of the induced map on the first homology group. We show that in the case of a…
We consider actions of a tileable amenable group $\Gamma$ on a topological space $X$. For a continuous function on $X$, we define the entropy of the number of homologically detectable critical point of the average of that function over…
Statistical mechanics relies on the complete though probabilistic description of a system in terms of all the microscopic variables. Its object is to derive therefrom static and dynamic properties involving some reduced set of variables.…
Characterising the capacity region for a network can be extremely difficult, especially when the sources are dependent. Most existing computable outer bounds are relaxations of the Linear Programming bound. One main challenge to extend…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural…
We consider the differential entropy of probability measures absolutely continuous with respect to a given $\sigma$-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general…
In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…
The entropy of the random walk on the discrete contable group could be used for comparison of the system of the generators. Fundamental inequality between growth, entropy and escape gives the possibility to define "the best" system of the…
Entropy in thermodynamics is an extensive quantity, whereas standard methods in statistical mechanics give rise to a non-extensive expression for the entropy. This discrepancy is often seen as a sign that basic formulas of statistical…
Under certain conditions, the rate of increase of the statistical entropy of a simple, fully chaotic, conservative system is known to be given by a single number, characteristic of this system, the Kolmogorov-Sinai entropy rate. This…
Consider a uniformly sampled random $d$-regular graph on $n$ vertices. If $d$ is fixed and $n$ goes to $\infty$ then we can relate typical (large probability) properties of such random graph to a family of invariant random processes (called…
We produce a probabilistic space from logic, both classical and quantum, which is in addition partially ordered in such a way that entropy is monotone. In particular do we establish the following equation: Quantitative Probability = Logic +…
Entropy production (EP) is known as a fundamental quantity for measuring the irreversibility of processes in thermal equilibrium and states far from equilibrium. In stochastic thermodynamics, the EP becomes more visible in terms of the…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
A task is randomly drawn from a finite set of tasks and is described using a fixed number of bits. All the tasks that share its description must be performed. Upper and lower bounds on the minimum $\rho$-th moment of the number of performed…
I formulate an entropy-rate maximization problem at the observable level for stochastic processes observed through an information-reducing observation map. For a visible stationary law, the map determines an observational fiber of hidden…