Nonextensive Generalizations of the Jensen-Shannon Divergence
Abstract
Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-difference is built, which is a nonextensive generalization of the JSD, based on Tsallis entropies. Finally, the Jensen-Tsallis q-difference is charaterized in terms of convexity and extrema.
Keywords
Cite
@article{arxiv.0804.1653,
title = {Nonextensive Generalizations of the Jensen-Shannon Divergence},
author = {Andre Martins and Pedro Aguiar and Mario Figueiredo},
journal= {arXiv preprint arXiv:0804.1653},
year = {2008}
}
Comments
Submitted to the IEEE Transactions on Information Theory