Denoising clustering covariance matrices with Rotational Invariant Estimators

Cosmological parameter inference from galaxy clustering relies critically on accurate estimates of the covariance and precision matrices. These are often obtained from a limited number of mock catalogs, introducing noise and bias in the precision matrix when the data-vector dimension becomes comparable to the number of available realizations. We present the first application of the Rotational Invariant Estimator (RIE) to the large-scale clustering of galaxies, benchmarking it against the standard sample covariance and the non-linear shrinkage estimator NERCOME for both the two-point correlation function (2PCF) and power spectrum. Using controlled synthetic data sets with analytically known covariance matrices, we estimate the covariance with all three methods across a range of mock-to-dimension ratios $q = N/D$ and data-vector sizes $D$. We then perform Bayesian inference with an EFT-based model and quantify each estimator through the Figure of Bias (FoB) and Figure of Merit (FoM). After correction for finite-$N$ effects, the sample covariance recovers unbiased average uncertainty volumes but suffers from growing best-fit scatter and bias at small $q$ due to the Dodelson--Schneider effect. Both NERCOME and RIE substantially reduce these stochastic shifts; however, the uncertainties they assign are probe-dependent. In configuration space, both estimators can yield overly tight constraints, with a bias that grows with $D$. In Fourier space, RIE delivers markedly improved best-fit stability with only mild FoM bias, whereas NERCOME tends to overestimate the constraining power. Among the estimators tested, RIE emerges as the most effective at stabilizing best-fit recovery, particularly in Fourier space, where it closely reproduces the reference posteriors even when the number of mocks barely exceeds the data-vector dimension.