English

A generalization for the expected value of the earth mover's distance

Combinatorics 2021-12-15 v2

Abstract

The earth mover's distance (EMD), also called the first Wasserstein distance, can be naturally extended to compare arbitrarily many probability distributions, rather than only two, on the set [n]={1,,n}[n]=\{1,\dots,n\}. We present the details for this generalization, along with a highly efficient algorithm inspired by combinatorics; it turns out that in the special case of three distributions, the EMD is half the sum of the pairwise EMD's. Extending the methods of Bourn and Willenbring (arXiv:1903.03673), we compute the expected value of this generalized EMD on random dd-tuples of distributions, using a generating function which coincides with the Hilbert series of the Segre embedding. We then use the EMD to analyze a real-world data set of grade distributions.

Keywords

Cite

@article{arxiv.2009.12723,
  title  = {A generalization for the expected value of the earth mover's distance},
  author = {William Q. Erickson},
  journal= {arXiv preprint arXiv:2009.12723},
  year   = {2021}
}

Comments

23 pages, 2 figures; corrected typos, simplified notation, added proof of Proposition 6, added discussion of even vs. odd d-values, updated real-world example to compare 7 distributions instead of 3, rewrote Section 7 for clarity