English

Data Compression with Noise Suppression for Inference under Noisy Covariance

Cosmology and Nongalactic Astrophysics 2025-08-20 v1 Instrumentation and Methods for Astrophysics

Abstract

In many fields including cosmology, statistical inference often relies on Gaussian likelihoods whose covariance matrices are estimated from a finite number of simulations. This finite-sample estimation introduces noise into the covariance, which propagates to parameter estimates, a phenomenon known as the Dodelson-Schneider (DS) effect, leading to inflated uncertainties. While the Massively Optimized Parameter Estimation and Data compression (MOPED) algorithm offers lossless Fisher information-preserving compression, it does not mitigate the DS effect when the compression matrix itself is derived from noisy covariances. In this paper, we propose a modified compression scheme, powered MOPED (pp-MOPED), which suppresses noise propagation by balancing information retention and covariance estimate noise reduction through a tunable power-law transformation of the sample correlation matrix. We test pp-MOPED against standard and diagonal MOPED on toy models and on cosmological data from the Subaru Hyper Suprime-Cam Year 3 weak lensing survey. Our results demonstrate that pp-MOPED consistently outperforms other approaches, especially in regimes with limited simulations, offering a robust compression strategy for high-dimensional data analyses under practical constraints.

Keywords

Cite

@article{arxiv.2508.14021,
  title  = {Data Compression with Noise Suppression for Inference under Noisy Covariance},
  author = {Sunao Sugiyama and Minsu Park},
  journal= {arXiv preprint arXiv:2508.14021},
  year   = {2025}
}

Comments

10 pages, 5 figures, comments welcomed!

R2 v1 2026-07-01T04:57:10.317Z