Related papers: Minimax R\'enyi Redundancy

We study universal compression of sequences generated by monotonic distributions. We show that for a monotonic distribution over an alphabet of size $k$, each probability parameter costs essentially $0.5 \log (n/k^3)$ bits, where $n$ is the…

Channel simulation is to simulate a noisy channel using noiseless channels with unlimited shared randomness. This can be interpreted as the reverse problem to Shannon's noisy coding theorem. In contrast to previous works, our approach…

We show the existence of variable-rate rate-distortion codes that meet the disortion constraint almost surely and are minimax, i.e., strongly, universal with respect to an unknown source distribution and a distortion measure that is…

We extend the Rate-Distortion-Perception (RDP) framework to the R\'enyi information-theoretic regime, utilizing Sibson's $\alpha$-mutual information to characterize the fundamental limits under distortion and perception constraints. For…

Universal compression of patterns of sequences generated by independently identically distributed (i.i.d.) sources with unknown, possibly large, alphabets is investigated. A pattern is a sequence of indices that contains all consecutive…

We present a theoretical framework that extends classical information theory to finite and structured systems by redefining redundancy as a fundamental property of information organization rather than inefficiency. In this framework,…

This work explores properties of Strong Data-Processing constants for R\'enyi Divergences. Parallels are made with the well-studied $\varphi$-Divergences, and it is shown that the order $\alpha$ of R\'enyi Divergences dictates whether…

We consider the problem of decomposing the total mutual information conveyed by a pair of predictor random variables about a target random variable into redundant, unique and synergistic contributions. We focus on the relationship between…

This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of…

Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general…

We investigate the question of whether or not there exists a noncommutative/ quantum extension of a recent (commutative probabilistic) result of Clarke and Barron. They demonstrated that the Jeffreys' invariant prior of Bayesian theory…

For any Markov source, there exist universal codes whose normalized codelength approaches the Shannon limit asymptotically as the number of samples goes to infinity. This paper investigates how fast the gap between the normalized codelength…

We consider optimal scalar quantization with $r$th power distortion and constrained R\'enyi entropy of order $\alpha$. For sources with an absolutely continuous distribution the high rate asymptotics of the quantizer distortion has long…

We consider the problem of distilling uniform random bits from an unknown source with a given $p$-entropy using linear hashing. As our main result, we estimate the expected $p$-divergence from the uniform distribution over the ensemble of…

We introduce a new quantum R\'enyi divergence $D^{\#}_{\alpha}$ for $\alpha \in (1,\infty)$ defined in terms of a convex optimization program. This divergence has several desirable computational and operational properties such as an…

Clarke and Barron have recently shown that the Jeffreys' invariant prior of Bayesian theory yields the common asymptotic (minimax and maximin) redundancy of universal data compression in a parametric setting. We seek a possible analogue of…

This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…

In this paper, we examine the existence of the R\'enyi divergence between two time invariant general hidden Markov models with arbitrary positive initial distributions. By making use of a Markov chain representation of the probability…

In this paper, we investigate the optimum way to allocate redundancy of finite-length nested codes for modern nonvolatile memories suffering from both permanent defects and transient errors (erasures or random errors). A nested coding…

For variable-length coding with an almost-sure distortion constraint, Zhang et al. show that for discrete sources the redundancy is upper bounded by $\log n/n$ and lower bounded (in most cases) by $\log n/(2n)$, ignoring lower order terms.…