English

Expressing entropy and cross-entropy in expansions of common meadows

Information Theory 2025-02-12 v1 math.IT

Abstract

A common meadow is an enrichment of a field with a partial division operation that is made total by assuming that division by zero takes the a default value, a special element \bot adjoined to the field. To a common meadow of real numbers we add a binary logarithm log2()\log_2(-), which we also assume to be total with log2(p)=\log_2(p) = \bot for p0p \leq 0. With these and other auxiliary operations, such as a sign function, we form algebras over which entropy and cross entropy can be defined for probability mass functions on a finite sample space by algebraic formulae that are simple terms built from the operations of the algebras and without case distinctions or conventions to avoid partiality. The discuss the advantages of algebras based on common meadows, whose theory is established, and alternate methods to define entropy and other information measures completely for all arguments using single terms.

Keywords

Cite

@article{arxiv.2502.07148,
  title  = {Expressing entropy and cross-entropy in expansions of common meadows},
  author = {Jan A Bergstra and John V Tucker},
  journal= {arXiv preprint arXiv:2502.07148},
  year   = {2025}
}