R\'{e}nyi Cross-Entropy Measures for Common Distributions and Processes with Memory
Information Theory
2022-10-06 v2 math.IT
Abstract
Two R\'{e}nyi-type generalizations of the Shannon cross-entropy, the R\'{e}nyi cross-entropy and the Natural R\'{e}nyi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we build upon our results in [1] by deriving the R\'{e}nyi and Natural R\'{e}nyi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family and tabulating the results for ease of reference. We also summarise the R\'{e}nyi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.
Keywords
Cite
@article{arxiv.2208.06983,
title = {R\'{e}nyi Cross-Entropy Measures for Common Distributions and Processes with Memory},
author = {Ferenc Cole Thierrin and Fady Alajaji and Tamás Linder},
journal= {arXiv preprint arXiv:2208.06983},
year = {2022}
}