BB plot: A Tool for Accurate Model Selection Using Bayes factors

A common task in physics and astronomy is studying which of the competing hypotheses the data prefer. This is usually done by computing the Bayes factor between the two hypotheses, and either interpreting it in terms of the posterior odds or as a ranking statistic for a frequentist p-value test. Here we describe a relationship between the Bayes factor and its distributions under the two competing hypotheses, called the Bayes factor-Bayes factor (BB) relationship, expressed as a diagnostic plot. Using examples from gravitational wave (GW) astronomy, we demonstrate how the BB plot can validate the accuracy of Bayes factor calculations. The BB relationship may also be useful for estimating background distributions of the Bayes factor at low computational cost, even analytically in some cases. We apply this technique in the context of wave-optics lensing of GWs, extrapolating the background distribution from GWTC4 to put a rough bound of $\lesssim 4.1 \sigma$ on the statistical significance of GW231123.