Stein's method of exchangeable pairs for the Beta distribution and generalizations

We propose a new version of Stein's method of exchangeable pairs, which, given a suitable exchangeable pair $(W,W')$ of real-valued random variables, suggests the approximation of the law of $W$ by a suitable absolutely continuous distribution. This distribution is characterized by a first order linear differential Stein operator, whose coefficients $\gamma$ and $\eta$ are motivated by two regression properties satisfied by the pair $(W,W')$. Furthermore, the general theory of Stein's method for such an absolutely continuous distribution is developed and a general characterization result as well as general bounds on the solution to the Stein equation are given. This abstract approach is a certain extension of the theory developed in the papers \cite{ChSh} and \cite{EiLo10}, which only consider the framework of the density approach, i.e. $\eta\equiv1$. As an illustration of our technique we prove a general plug-in result, which bounds a certain distance of the distribution of a given random variable $W$ to a Beta distribution in terms of a given exchangeable pair $(W,W')$ and provide new bounds on the solution to the Stein equation for the Beta distribution, which complement the existing bounds from \cite{GolRei13}. The abstract plug-in result is then applied to derive bounds of order $n^{-1}$ for the distance between the distribution of the relative number of drawn red balls after $n$ drawings in a P\'olya urn model and the limiting Beta distribution measured by a certain class of smooth test functions.