Related papers: Entropy rate of product of independent processes
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
Dependence among marginally constrained observations can break a finite-sample barrier. To formalize this phenomenon, we introduce the \emph{minimum list entropy coupling} $H(P\|Q_1,\dots,Q_m)$, the minimum conditional entropy…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
We present a unifying approach to the study of entropies in Mathematics, such as measure entropy, topological entropy, algebraic entropy, set-theoretic entropy. We take into account discrete dynamical systems, that is, pairs $(X,T)$, where…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
To illustrate Boltzmann's construction of an entropy function that is defined for a microstate of a macroscopic system, we present here the simple example of the free expansion of a one dimensional gas of non-interacting point particles.…
In this paper, a polynomial version of Furstenberg joining is introduced and its structure is investigated. Particularly, it is shown that if all polynomials are non-linear, then almost every ergodic component of the joining is a direct…
On a fine grained scale the Gibbs entropy of an isolated system remains constant throughout its dynamical evolution. This is a consequence of Liouville's theorem for Hamiltonian systems and appears to contradict the second law of…
The expressions for entropy production, free energy, and entropy extraction rates are derived for a Brownian particle that walks in an underdamped medium. Our analysis indicates that as long as the system is driven out of equilibrium, it…
In this paper we show that the existence of a primarily discrete space-time may be a fruitful assumption from which we may develop a new approach of statistical thermodynamics in pre-relativistic conditions. The discreetness of space-time…
The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which detailed balance and time-reversal symmetry are broken. Despite…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the…
Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…
The quantum relative entropy is known to play a key role in determining the asymptotic convertibility of quantum states in general resource-theoretic settings, often constituting the unique monotone that is relevant in the asymptotic…
The problem of stability of the optimal filter is revisited. The optimal filter (or filtering process) is the conditional probability of the current state of some stochastic process (the signal process), given both present and past values…
The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates…
Let $\Theta$ be a finite alphabet. We consider a bundle of measure preserving transformations $(T_{\theta})_{\theta \in \Theta}$ acting on a probability space $(X,\mu)$, which are chosen randomly according to an ergodic stochastic process…
We construct multiperiodic processes -- a simple example of stationary ergodic (but not mixing) processes over natural numbers that enjoy the vanishing entropy rate under a mild condition. Multiperiodic processes are supported on randomly…
The selection of an equilibrium state by maximising the entropy of a system, subject to certain constraints, is often powerfully motivated as an exercise in logical inference, a procedure where conclusions are reached on the basis of…