Related papers: Encoding Tasks and R\'enyi Entropy
Noise is conventionally viewed as a severe problem in diverse fields, e.g., engineering, learning systems. However, this paper aims to investigate whether the conventional proposition always holds. It begins with the definition of task…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
Spectral properties of an arbitrary matrix can be characterized by the entropy of its rescaled singular values. Any quantum operation can be described by the associated dynamical matrix or by the corresponding superoperator. The entropy of…
Two maximization problems of R\'enyi entropy rate are investigated: the maximization over all stochastic processes whose marginals satisfy a linear constraint, and the Burg-like maximization over all stochastic processes whose…
In this work, we introduce a notion of reachability entropy to characterize the smallest data rate which is sufficient enough to enforce reach-while-stay specification. We also define data rates of coder-controllers that can enforce this…
We show that the conditional min-entropy Hmin(A|B) of a bipartite state rho_AB is directly related to the maximum achievable overlap with a maximally entangled state if only local actions on the B-part of rho_AB are allowed. In the special…
The performance of an error correcting code is evaluated by its error probability, rate, and en/decoding complexity. The performance of a series of codes is evaluated by, as the block lengths approach infinity, whether their error…
We compute the entanglement entropy in a 2+1 dimensional topological order in the presence of gapped boundaries. Specifically, we consider entanglement cuts that cut through the boundaries. We argue that based on general considerations of…
Shannon entropy is the shortest average codeword length a lossless compressor can achieve by encoding i.i.d. symbols. However, there are cases in which the objective is to minimize the \textit{exponential} average codeword length, i.e. when…
Classification is a ubiquitous and fundamental problem in artificial intelligence and machine learning, with extensive efforts dedicated to developing more powerful classifiers and larger datasets. However, the classification task is…
In Tokenization and the Noiseless Channel (Zouhar et al., 2023a), R\'enyi efficiency is suggested as an intrinsic mechanism for evaluating a tokenizer: for NLP tasks, the tokenizer which leads to the highest R\'enyi efficiency of the…
In the index coding problem a sender holds a message $x \in \{0,1\}^n$ and wishes to broadcast information to $n$ receivers in a way that enables the $i$th receiver to retrieve the $i$th bit $x_i$. Every receiver has prior side information…
Distributions of abundances or frequencies play an important role in many fields of science, from biology to sociology, as does the R\'enyi entropy, which measures the diversity of a statistical ensemble. We derive a mathematical relation…
This paper investigates the problem of guessing subject to distortion, which was introduced by Arikan and Merhav. While the primary concern of the previous study was asymptotic analysis, our primary concern is non-asymptotic analysis. We…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
A natural measure for the amount of quantum information that a physical system E holds about another system A = A_1,...,A_n is given by the min-entropy Hmin(A|E). Specifically, the min-entropy measures the amount of entanglement between E…
We compute R\'enyi entropies for the statistics of a noisy simultaneous observation of two complementary observables in two-dimensional quantum systems. The relative amount of uncertainty between two states depends on the uncertainty…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
The entropy of probability distribution defined by Shannon has several extensions. R\'enyi entropy is one of the general extensions of Shannon entropy and is widely used in engineering, physics, and so on. On the other hand, the quantum…
In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the…