English

Two Measures of Dependence

Information Theory 2019-08-22 v4 math.IT

Abstract

Two families of dependence measures between random variables are introduced. They are based on the R\'enyi divergence of order α\alpha and the relative α\alpha-entropy, respectively, and both dependence measures reduce to Shannon's mutual information when their order α\alpha is one. The first measure shares many properties with the mutual information, including the data-processing inequality, and can be related to the optimal error exponents in composite hypothesis testing. The second measure does not satisfy the data-processing inequality, but appears naturally in the context of distributed task encoding.

Keywords

Cite

@article{arxiv.1607.02330,
  title  = {Two Measures of Dependence},
  author = {Amos Lapidoth and Christoph Pfister},
  journal= {arXiv preprint arXiv:1607.02330},
  year   = {2019}
}

Comments

40 pages; 1 figure; published in Entropy

R2 v1 2026-06-22T14:49:10.270Z