Clustering with general photo-$z$ uncertainties: Application to Baryon Acoustic Oscillations

Photometric data can be analyzed using the three-dimensional correlation function $\xi_{\rm p}$ to extract cosmological information via e.g., measurement of the Baryon Acoustic Oscillations (BAO). Previous studies modeled $\xi_{\rm p}$ assuming a Gaussian photo-$z$ approximation. In this work we improve the modeling by incorporating realistic photo-$z$ distribution. We show that the position of the BAO scale in $\xi_{\rm p}$ is determined by the photo-$z$ distribution and the Jacobian of the transformation. The latter diverges at the transverse scale of the separation $s_\perp$, and it explains why $\xi_{\rm p }$ traces the underlying correlation function at $s_\perp$, rather than $s$, when the photo-$z$ uncertainty $\sigma_z / (1+ z) \gtrsim 0.02$. We also obtain the Gaussian covariance for $\xi_{\rm p}$. Due to photo-$z$ mixing, the covariance of $\xi_{\mathrm{p}}$ shows strong off-diagonal elements. The high correlation of the data causes some issues to the data fitting. Nonetheless, we find that either it can be solved by suppressing the largest eigenvalues of the covariance or it is not directly related to the BAO. We test our BAO fitting pipeline using a set of mock catalogs. The data set is dedicated for Dark Energy Survey Year 3 (DES Y3) BAO analyses and includes realistic photo-$z$ distributions. The theory template is in good agreement with mock measurement. Based on the DES Y3 mocks, $\xi_{\rm p}$ statistic is forecast to constrain the BAO shift parameter $\alpha$ to be $1.001 \pm 0.023$, which is well consistent with the corresponding constraint derived from the angular correlation function measurements. Thus $\xi_{\rm p}$ offers a competitive alternative for the photometric data analyses.