Related papers: On the R\'{e}nyi Cross-Entropy
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of)…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
Understanding the fundamental limits of robust supervised learning has emerged as a problem of immense interest, from both practical and theoretical standpoints. In particular, it is critical to determine classifier-agnostic bounds on the…
We study the continuity property of the generalized entropy as a function of the underlying probability distribution, defined with an action space and a loss function, and use this property to answer the basic questions in statistical…
Astrophysical transient phenomena are traditionally classified spectroscopically in a hierarchical taxonomy; however, this graph structure is currently not utilized in neural net-based photometric classifiers for time-domain astrophysics.…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
This paper deals with the existence of global weak solutions for a wide class of (multiple species) cross-diffusions systems. The existence is based on two different ingredients: an entropy estimate giving some gradient control and a…
A large class of strongly correlated quantum systems can be described in certain large-N limits by quadratic in field actions along with self-consistency equations that determine the two-point functions. We use the replica approach and the…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…
Soft targets combined with the cross-entropy loss have shown to improve generalization performance of deep neural networks on supervised classification tasks. The standard cross-entropy loss however assumes data to be categorically…
The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
In this paper, we propose to quantitatively compare loss functions based on parameterized Tsallis-Havrda-Charvat entropy and classical Shannon entropy for the training of a deep network in the case of small datasets which are usually…
The min-entropy is a widely used metric to quantify the randomness of generated random numbers in cryptographic applications; it measures the difficulty of guessing the most likely output. An important min-entropy estimator is the…
We consider a probability distribution depending on a real parameter $x$. As functions of $x$, the R\'enyi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$. We establish recurrence…
The entropy accumulation theorem, and its subsequent generalized version, is a powerful tool in the security analysis of many device-dependent and device-independent cryptography protocols. However, it has the drawback that the finite-size…
We show how the Shannon entropy function can be used as a basis to set up complexity measures weighting the economic efficiency of countries and the specialization of products beyond bare diversification. This entropy function guarantees…
The multi-entropy and dihedral measures are a class of tractable measures for multi-partite entanglement, which are labeled by the R\'enyi index (or replica number) $n$ as in the R\'enyi entanglement entropy. The purpose of this article is…
Escort distributions have been shown to be very useful in a great variety of fields ranging from information theory, nonextensive statistical mechanics till coding theory, chaos and multifractals. In this work we give the notion and the…
Extropy was introduced as a dual complement of the Shannon entropy. In this investigation, we consider failure extropy and its dynamic version. Various basic properties of these measures are presented. It is shown that the dynamic failure…