相关论文: Sharp Bounds on Arimoto's Conditional R\'{e}nyi En…

Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…

We provide an operational interpretation of the multivariate R\'enyi divergence in terms of economic-theoretic tasks based on betting, risk aversion, and multiple lotteries. We show that the multivariate R\'enyi divergence…

We recognise that an entropy inequality akin to the main intermediate goal of recent works (Gowers, Green, Manners, Tao [3],[2]) regarding a conjecture of Marton provides a black box from which we can also through a short deduction recover…

Will further scaling up of machine learning models continue to bring success? A significant challenge in answering this question lies in understanding generalization gap, which is the impact of overfitting. Understanding generalization gap…

Configurational entropy, or complexity, plays a critical role in characterizing disordered systems such as glasses, yet its measurement often requires significant computational resources. Recently, R\'enyi entropy, a one-parameter…

In this paper we examine the average R\'{e}nyi entropy $S_{\alpha}$ of a subsystem $A$ when the whole composite system $AB$ is a random pure state. We assume that the Hilbert space dimensions of $A$ and $AB$ are $m$ and $m n$ respectively.…

We show that the R\'enyi entropies of single particle, extended wave functions for disordered systems contain information about the multifractal spectrum. It is shown for moments of the R\'enyi entropy, $S_{n}$, where $|n|<1$, it is…

Entropic uncertainty relations in a finite dimensional Hilbert space are investigated. Making use of the majorization technique we derive explicit lower bounds for the sum of R\'enyi entropies describing probability distributions associated…

For the sandwiched R\'enyi entropy the conditional entropy can be defined two ways: $\widetilde{H}^\downarrow_\alpha(A|B)_\rho, \widetilde{H}^\uparrow_\alpha(A|B)_\rho$. In the limiting case, $\alpha=1$, both definitions consolidate into…

We show that in the continuum limit, the average spectrum method (ASM) is equivalent to maximizing R\'enyi entropies of order $\eta$, of which Shannon entropy is the special case $\eta=1$. The order of R\'enyi entropy is determined by the…

Ordinal classification problems, where labels exhibit a natural order, are prevalent in high-stakes fields such as medicine and finance. Accurate uncertainty quantification, including the decomposition into aleatoric (inherent variability)…

The specific relative entropy, introduced by N. Gantert, allows to quantify the discrepancy between the laws of potentially mutually singular measures. It appears naturally as the large deviations rate function in a randomized version of…

Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…

The aim of the work is to give the explicit proofs of the Renyi-entropy uncertainty relations presented in the previous work [A. Rastegin, arXiv:0805.1777]. The relations with both the state-dependent and state-independent entropic bounds…

Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…

We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we…

We present simple and computationally efficient nonparametric estimators of R\'enyi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over $\R^d$. The estimators are…

Concepts of information theory are increasingly used to characterize collective phenomena in condensed matter systems, such as the use of entanglement entropies to identify emergent topological order in interacting quantum many-body…

It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…

The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately…