Bayesian Posteriors For Arbitrarily Rare Events

We study how much data a Bayesian observer needs to correctly infer the relative likelihoods of two events when both events are arbitrarily rare. Each period, either a blue die or a red die is tossed. The two dice land on side $1$ with unknown probabilities $p_1$ and $q_1$, which can be arbitrarily low. Given a data-generating process where $p_1\ge c q_1$, we are interested in how much data is required to guarantee that with high probability the observer's Bayesian posterior mean for $p_1$ exceeds $(1-\delta)c$ times that for $q_1$. If the prior densities for the two dice are positive on the interior of the parameter space and behave like power functions at the boundary, then for every $\epsilon>0,$ there exists a finite $N$ so that the observer obtains such an inference after $n$ periods with probability at least $1-\epsilon$ whenever $np_1\ge N$. The condition on $n$ and $p_1$ is the best possible. The result can fail if one of the prior densities converges to zero exponentially fast at the boundary.