Related papers: Codes for Tasks and R\'enyi Entropy Rate
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
The work performed on or extracted from a non-autonomous quantum system described by means of a two-point projective-measurement approach takes the form of a stochastic variable. We show that the cumulant generating function of work can be…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
We study entropy-bounded computational geometry, that is, geometric algorithms whose running times depend on a given measure of the input entropy. Specifically, we introduce a measure that we call range-partition entropy, which unifies and…
A realistic Quantum Key Distribution (QKD) protocol necessarily deals with finite resources, such as the number of signals exchanged by the two parties. We derive a bound on the secret key rate which is expressed as an optimization problem…
In this work, binary block-to-block distribution matching is considered. m independent and uniformly distributed bits are mapped to n output bits resembling a target product distribution. A rate R is called achieved by a sequence of…
The entropy rates of the Wright-Fisher process, the Moran process, and generalizations are computed and used to compare these processes and their dependence on standard evolutionary parameters. Entropy rates are measures of the variation…
The exact range of the joined values of several R\'{e}nyi entropies is determined. The method is based on topology with special emphasis on the orientation of the objects studied. Like in the case when only two orders of R\'{e}nyi entropies…
We define and investigate a notion of entropy for quantum error correcting codes. The entropy of a code for a given quantum channel has a number of equivalent realisations, such as through the coefficients associated with the Knill-Laflamme…
We prove that the quantum relative entropy is a rate function in large deviation principle. Next, we define information criteria for quantum states and estimate the accuracy of the use of them. Most of the results in this paper are…
We prove a general lower bound of quantum decision tree complexity in terms of some entropy notion. We regard the computation as a communication process in which the oracle and the computer exchange several rounds of messages, each round…
We study convexity properties of R\'{e}nyi entropy as function of $\alpha>0$ on finite alphabets. We also describe robustness of the R\'{e}nyi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
In Tokenization and the Noiseless Channel (Zouhar et al., 2023a), R\'enyi efficiency is suggested as an intrinsic mechanism for evaluating a tokenizer: for NLP tasks, the tokenizer which leads to the highest R\'enyi efficiency of the…
Conventional information-theoretic quantities assume access to probability distributions. Estimating such distributions is not trivial. Here, we consider function-based formulations of cross entropy that sidesteps this a priori estimation…
PAE cannot be made a basis for either a generalized statistical mechanics or a generalized information theory. Either statistical independence must be waived, or the expression of the averaged conditional probability as the difference…
A relative entropy code for a source $X \sim P_X$ is a stochastic code that encodes random samples from a prescribed $P_{Y \mid X}$ using as few bits as possible. A generalisation of entropy coding, it is a standard result that the minimum…
R\'enyi entropy of order \alpha is a general measure of entropy. In this paper we derive estimations for the R\'enyi entropy of the mixture of sources in terms of the entropy of the single sources. These relations allow to compute the…
We establish the first information-theoretic limits for multimodal retrieval. Casting ranking as lossy source coding, we derive a single-letter rate-distortion function $R(D)$ for reciprocal-rank distortion and prove a converse bound that…