On the R\'{e}nyi Cross-Entropy
Information Theory
2022-08-09 v3 Machine Learning
math.IT
Abstract
The R\'{e}nyi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we examine the properties of this measure and derive closed-form expressions for it when one of the distributions is fixed and when both distributions belong to the exponential family. We also analytically determine a formula for the cross-entropy rate for stationary Gaussian processes and for finite-alphabet Markov sources.
Keywords
Cite
@article{arxiv.2206.14329,
title = {On the R\'{e}nyi Cross-Entropy},
author = {Ferenc Cole Thierrin and Fady Alajaji and Tamás Linder},
journal= {arXiv preprint arXiv:2206.14329},
year = {2022}
}
Comments
Appeared in the Proceedings of CWIT'22 (updated version)