Related papers: Arimoto-R\'enyi Conditional Entropy and Bayesian $…
This paper presents an achievability bound that evaluates the exact probability of error of an ensemble of random codes that are decoded by a minimum distance decoder. Compared to the state-of-the-art which demands exponential computation…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
Approximate entropy (ApEn) has been widely used as an estimator of regularity in many scientific fields. It has proved to be a useful tool because of its ability to distinguish different system's dynamics when there is only available…
We derive a novel chain rule for a family of channel conditional entropies, covering von Neumann and sandwiched R\'{e}nyi entropies. In the process, we show that these channel conditional entropies are equal to their regularized version,…
We introduce an alternative closed form lower bound on the Gaussian process ($\mathcal{GP}$) likelihood based on the R\'enyi $\alpha$-divergence. This new lower bound can be viewed as a convex combination of the Nystr\"om approximation and…
For the sandwiched R\'enyi entropy the conditional entropy can be defined two ways: $\widetilde{H}^\downarrow_\alpha(A|B)_\rho, \widetilde{H}^\uparrow_\alpha(A|B)_\rho$. In the limiting case, $\alpha=1$, both definitions consolidate into…
The minimum error entropy (MEE) criterion has been verified as a powerful approach for non-Gaussian signal processing and robust machine learning. However, the implementation of MEE on robust classification is rather a vacancy in the…
Given an arbitrary quantum state ($\sigma$), we obtain an explicit construction of a state $\rho^*_\varepsilon(\sigma)$ (resp. $\rho_{*,\varepsilon}(\sigma)$) which has the maximum (resp. minimum) entropy among all states which lie in a…
In this paper we consider the problem of binary hypothesis testing with finite memory systems. Let $X_1,X_2,\ldots$ be a sequence of independent identically distributed Bernoulli random variables, with expectation $p$ under $\mathcal{H}_0$…
The asymptotic restriction problem for tensors can be reduced to finding all parameters that are normalized, monotone under restrictions, additive under direct sums and multiplicative under tensor products, the simplest of which are the…
Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic…
R\'enyi entropy is an important measure in the context of information theory as a generalization of Shannon entropy. This information measure was often used for uncertainty quantification of dynamical behaviour of stochastic processes. In…
The entropy accumulation theorem states that the smooth min-entropy of an $n$-partite system $A = (A_1, \ldots, A_n)$ is lower-bounded by the sum of the von Neumann entropies of suitably chosen conditional states up to corrections that are…
The fidelity-based smooth min-relative entropy is a distinguishability measure that has appeared in a variety of contexts in prior work on quantum information, including resource theories like thermodynamics and coherence. Here we provide a…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
We consider the optimal quantization problem with R\'enyi-$\alpha$-entropy constraints for centered Gaussian measures on a separable Banach space. For $\alpha = \infty$ we can compute the optimal quantization error by a moment on a ball.…
This paper provides a general technique for lower bounding the Bayes risk of statistical estimation, applicable to arbitrary loss functions and arbitrary prior distributions. A lower bound on the Bayes risk not only serves as a lower bound…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
Conventional information-theoretic quantities assume access to probability distributions. Estimating such distributions is not trivial. Here, we consider function-based formulations of cross entropy that sidesteps this a priori estimation…
Configurational entropy, or complexity, plays a critical role in characterizing disordered systems such as glasses, yet its measurement often requires significant computational resources. Recently, R\'enyi entropy, a one-parameter…