Related papers: Entropy Rate Bounds via Second-Order Statistics
A quantity of interest to characterise continuous-valued stochastic processes is the differential entropy rate. The rate of convergence of many properties of LRD processes is slower than might be expected, based on the intuition for…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
The estimation of entropy rates for stationary discrete-valued stochastic processes is a well studied problem in information theory. However, estimating the entropy rate for stationary continuous-valued stochastic processes has not received…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary…
We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…
Extropy, a complementary dual of entropy, (proposed by Lad et al. \cite{lad2015extropy} in 2015) has attracted considerable interest from the research community. In this study, we focus on discrete random variables and define conditional…
Some microscopic dynamics are also macroscopically irreversible, dissipating energy and producing entropy. For many-particle systems interacting with deterministic thermostats, the rate of thermodynamic entropy dissipated to the environment…
In this paper we explore how non trivial boundary conditions could influence the entanglement entropy in a topological order in 2+1 dimensions. Specifically we consider the special class of topological orders describable by the quantum…
(abbreviated) The statistical mechanics of self-gravitating systems is a long-held puzzle. In this work, we employ a phenomenological entropy form of ideal gas, first proposed by White & Narayan, to revisit this issue. By calculating the…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
Long memory or long range dependency is an important phenomenon that may arise in the analysis of time series or spatial data. Most of the definitions of long memory of a stationary process $X=\{X_1, X_2,\cdots,\}$ are based on the…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
The second law of thermodynamics states that entropy production cannot be negative. Recent developments concerning uncertainty relations in stochastic thermodynamics, such as thermodynamic uncertainty relations and speed limits, have…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bounds for upper bounding…
This paper studies the second-order asymptotics of coding rates for the discrete memoryless multiple-access channel with a fixed target error probability. Using constant-composition random coding, coded time-sharing, and a variant of…
We have shown how the intrinsic properties of a noise process can set an upper bound for the time derivative of entropy in a nonequilibrium system. The interplay of dissipation and the properties of noise processes driving the dynamical…