A remark on conditional entropy
Information Theory
2024-04-04 v1 math.IT
Abstract
The following note proves that conditional entropy of a sequence is almost time-reversal invariant, specifically they only differ by a small constant factor dependent only upon the forward and backward models that the entropies are being calculated with respect to. This gives rise to a numerical value that quantifies learnability, as well as a methodology to control for distributional shift between datasets. Rough guidelines are given for practitioners.
Keywords
Cite
@article{arxiv.2404.02167,
title = {A remark on conditional entropy},
author = {Adam Wang},
journal= {arXiv preprint arXiv:2404.02167},
year = {2024}
}
Comments
Preprint, 5 pages