Related papers: Entropy Rate Bounds via Second-Order Statistics

Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…

In the paper, the approximate sequence for entropy of some binary hidden Markov models has been found to have two bound sequences, the low bound sequence and the upper bound sequence. The error bias of the approximate sequence is bound by a…

We study the entropy production of a system with a finite number of states connected by random transition rates. The stationary entropy production, driven out of equilibrium both by asymmetric transition rates and by an external probability…

Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here,…

We consider boundary value problems for stochastic differential equations of second order with a small parameter. For this case we prove a special existence and unicity theorem for strong solutions. The asymptotic behavior of these…

In 1959, R\'enyi proposed the information dimension and the $d$-dimensional entropy to measure the information content of general random variables. This paper proposes a generalization of information dimension to stochastic processes by…

Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…

The stochastic entropy generated during the evolution of a system interacting with an environment may be separated into three components, but only two of these have a non-negative mean. The third component of entropy production is…

Using fluorescence spectroscopy we directly measure entropy production of a single two-level system realized experimentally as an optically driven defect center in diamond. We exploit a recent suggestion to define entropy on the level of a…

We show that under local detailed balance the expected entropy production rate is always bounded in terms of the dynamical activity. The activity refers to the time-symmetric contribution in the action functional for path-space…

The coefficients in a second order parabolic linear stochastic partial differential equation (SPDE) are estimated from multiple spatially localised measurements. Assuming that the spatial resolution tends to zero and the number of…

We observe an infinite sequence of independent identically distributed random variables $X_1,X_2,\ldots$ drawn from an unknown distribution $p$ over $[n]$, and our goal is to estimate the entropy $H(p)=-\mathbb{E}[\log p(X)]$ within an…

Computation of entropy in liquids and liquid crystal phases is a big challenge in statistical physics. In this work, we extend the two-phase thermodynamic model (2PT) to shape anisotropic soft repulsive spherocylinders (SRSs) and report the…

It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…

I consider multipartite processes in which there are constraints on each subsystem's rate matrix, restricting which other subsystems can directly affect its dynamics. I derive a strictly nonzero lower bound on the minimal achievable entropy…

This thesis concerns sequential-access data compression, i.e., by algorithms that read the input one or more times from beginning to end. In one chapter we consider adaptive prefix coding, for which we must read the input character by…

The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…

We propose a stochastic dynamics to be associated to a deterministic motion defined by a set of first order differential equation. The transitions that defined the stochastic dynamics are unidirectional and the rates are equal to the…

We establish upper and lower bounds for the metric entropy and bracketing entropy of the class of $d$-dimensional bounded monotonic functions under $L^p$ norms. It is interesting to see that both the metric entropy and bracketing entropy…

Processes controlled by stochastic synthesis and degradation (SSD) are widespread in biology but their reaction kinetics are not well understood. Using methods borrowed from the theory of resetting processes, we determine the first-passage…