Related papers: On analytic properties of entropy rate
The relative entropy of two n-party quantum states is an important quantity exhibiting, for example, the extent to which the two states are different. The relative entropy of the states formed by reducing two n-party to a smaller number $m$…
We consider a simple transformation (coding) of an iid source called a bit-shift channel. This simple transformation occurs naturally in magnetic or optical data storage. The resulting process is not Markov of any order. We discuss methods…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
We introduce the concept of inflation word entropy for random substitutions with a constant and primitive substitution matrix. Previous calculations of the topological entropy of such systems implicitly used this concept and established…
Random walks on a group $G$ model many natural phenomena. A random walk is defined by a probability measure $p$ on $G$. We are interested in asymptotic properties of the random walks and in particular in the linear drift and the asymptotic…
The microscopic explanation of entropy has been challenged from both experimental and theoretical point of view. The expression of entropy is derived from the first law of thermodynamics indicating that entropy or the second law of…
A convenient measure of a map or flow's chaotic action is the topological entropy. In many cases, the entropy has a homological origin: it is forced by the topology of the space. For example, in simple toral maps, the topological entropy is…
We consider some classes of piecewise expanding maps in finite dimensional spaces having invariant probability measures which are absolutely continuous with respect to Lebesgue measure. We derive an entropy formula for such measures and,…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
We consider a random walk in a random potential on a square lattice of arbitrary dimension. The potential is a function of an ergodic environment and some steps of the walk. The potential can be unbounded, but it is subject to a moment…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
For any discrete time dynamical system with a rational evolution, we define an entropy, which is a global index of complexity for the evolution map. We analyze its basic properties and its relations to the singularities and the…
Since its origin in the thermodynamics of the 19th century, the concept of entropy has also permeated other fields of physics and mathematics, such as Classical and Quantum Statistical Mechanics, Information Theory, Probability Theory,…
We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…