On the Systematic Errors of Cosmological-Scale Gravity Tests using Redshift Space Distortion: Non-linear Effects and the Halo Bias

Redshift space distortion (RSD) observed in galaxy redshift surveys is a powerful tool to test gravity theories on cosmological scales, but the systematic uncertainties must carefully be examined for future surveys with large statistics. Here we employ various analytic models of RSD and estimate the systematic errors on measurements of the structure growth-rate parameter, $f\sigma_8$, induced by non-linear effects and the halo bias with respect to the dark matter distribution, by using halo catalogues from 40 realisations of $3.4 \times 10^8$ comoving $h^{-3}$Mpc$^3$ cosmological N-body simulations. We consider hypothetical redshift surveys at redshifts z=0.5, 1.35 and 2, and different minimum halo mass thresholds in the range of $5.0 \times 10^{11}$ -- $2.0 \times 10^{13} h^{-1} M_\odot$. We find that the systematic error of $f\sigma_8$ is greatly reduced to ~5 per cent level, when a recently proposed analytical formula of RSD that takes into account the higher-order coupling between the density and velocity fields is adopted, with a scale-dependent parametric bias model. Dependence of the systematic error on the halo mass, the redshift, and the maximum wavenumber used in the analysis is discussed. We also find that the Wilson-Hilferty transformation is useful to improve the accuracy of likelihood analysis when only a small number of modes are available in power spectrum measurements.