English

A Global Characterization of $f$-Divergences Yielding PSD Mutual-Information Matrices

Information Theory 2026-05-15 v3 math.IT

Abstract

Given nn random variables, when does the matrix of pairwise ff-mutual informations define a PSD kernel over variables? For convex finite generators f:(0,)Rf:(0,\infty)\to\mathbb{R} with f(1)=0f(1)=0 and finite boundary value f(0)f(0), we give a closed characterization up to linear transformation ff+c(t1)f\sim f+c(t-1), which leaves every ff-divergence and every ff-mutual-information matrix unchanged. The matrix Mij(f):=If(Xi;Xj)M^{(f)}_{ij}:=I_f(X_i;X_j) is PSD for every finite-alphabet family if and only if the normalized representative has a globally convergent expansion fˉ(t)=m2am(t1)m\bar f(t)=\sum_{m\ge2}a_m(t-1)^m, with am0a_m\ge0, on all of (0,)(0,\infty). Sufficiency follows from a replica embedding for monomial generators plus closure under nonnegative mixtures. Necessity first extracts the local Taylor cone at 11 using biased three-point kernels HaH_a, the Belton--Guillot--Khare--Putinar (BGKP) low-rank Hankel positivity-preserver theorem, and then bootstraps analyticity to the divergence. This is a kernel characterization problem, not a metric one: PSD of the variable-indexed matrix is distinct from Hilbertian properties of divergences between distributions. The result explains why Shannon MI and Jensen--Shannon fail, why χ2\chi^2 succeeds, and why non-analytic divergences such as total variation and ReLU are excluded.

Keywords

Cite

@article{arxiv.2601.08929,
  title  = {A Global Characterization of $f$-Divergences Yielding PSD Mutual-Information Matrices},
  author = {Zachary Robertson},
  journal= {arXiv preprint arXiv:2601.08929},
  year   = {2026}
}

Comments

Revised main theorem and proof; fixes the local-to-global step and proves that the local analytic expansion extends to positive real line