Related papers: The Shannon Lower Bound is Asymptotically Tight
This paper proves sharp lower bounds on a resonance counting function for obstacle scattering in even-dimensional Euclidean space without a need for trapping assumptions. Similar lower bounds are proved for some other compactly supported…
In this work, we introduce a notion of reachability entropy to characterize the smallest data rate which is sufficient enough to enforce reach-while-stay specification. We also define data rates of coder-controllers that can enforce this…
Recovery of the sparsity pattern (or support) of an unknown sparse vector from a limited number of noisy linear measurements is an important problem in compressed sensing. In the high-dimensional setting, it is known that recovery with a…
We address a recent conjecture stated by Z. Van Herstraeten and N.J. Cerf. They claim that the Shannon entropy for positive Wigner functions is bounded below by a positive constant, which can be attained only by Gaussian pure states. We…
On the standard microscopic model of friction we confirm the common belief that the irreversible entropy production originates from the increase of Shannon information. We reveal that the reversible microscopic dynamics would continuously…
The Shannon upper bound places a limit on the error-free information transmission rate (capacity) of a noisy channel. It has stood for over sixty years, and underlies both theoretical and practical work in the telecommunications industry.…
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…
Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions…
A lossy source code $\mathcal{C}$ with rate $R$ for a discrete memoryless source $S$ is called subset-universal if for every $0<R'< R$, almost every subset of $2^{nR'}$ of its codewords achieves average distortion close to the source's…
An analog source is to be transmitted across a Gaussian channel in more than one channel use per source symbol. This paper derives a lower bound on the asymptotic mean squared error for a strategy that consists of repeatedly quantizing the…
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…
Expressions for (EPI Shannon type) Divergence-Power Inequalities (DPI) in two cases (time-discrete and band-limited time-continuous) of stationary random processes are given. The new expressions connect the divergence rate of the sum of…
Recent advances in machine learning-aided lossy compression are incorporating perceptual fidelity into the rate-distortion theory. In this paper, we study the rate-distortion-perception trade-off when the perceptual quality is measured by…
The Shannon cipher system is studied in the context of general sources using a notion of computational secrecy introduced by Merhav & Arikan. Bounds are derived on limiting exponents of guessing moments for general sources. The bounds are…
Restriction is a natural quasi-order on $d$-way tensors. We establish a remarkable aspect of this quasi-order in the case of tensors over a fixed finite field -- namely, that it is a well-quasi-order: it admits no infinite antichains and no…
In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
The distributed remote source coding (so-called CEO) problem is studied in the case where the underlying source, not necessarily Gaussian, has finite differential entropy and the observation noise is Gaussian. The main result is a new lower…
We prove a variety of new and refined uniform continuity bounds for entropies of both classical random variables on an infinite state space and of quantum states of infinite-dimensional systems. We obtain the first tight continuity estimate…
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality has been brought to light, where the perception is measured by the closeness of the output…