Related papers: A New Data Compression Method and its Application …
The disconnected part of the power spectrum covariance matrix (also known as the "Gaussian" covariance) is the dominant contribution on large scales for galaxy clustering and weak lensing datasets. The presence of a complicated sky mask…
In industrial and IoT environments, massive amounts of real-time and historical process data are continuously generated and archived. With sensors and devices capturing every operational detail, the volume of time-series data has become a…
Real-time data collection and analysis in large experimental facilities present a great challenge across multiple domains, including high energy physics, nuclear physics, and cosmology. To address this, machine learning (ML)-based methods…
Owing to their more extensive sky coverage and tighter control on systematic errors, future deep weak lensing surveys should provide a better statistical picture of the dark matter clustering beyond the level of the power spectrum. In this…
Real-world data typically contain repeated and periodic patterns. This suggests that they can be effectively represented and compressed using only a few coefficients of an appropriate basis (e.g., Fourier, Wavelets, etc.). However, distance…
The next generation of CMB experiments can measure cosmological parameters with unprecedented accuracy - in principle. To achieve this in practice when faced with such gigantic data sets, elaborate data analysis methods are needed to make…
Weak gravitational lensing surveys are rapidly becoming important tools to probe directly the mass density fluctuations in the universe and its background dynamics. Earlier studies have shown that it is possible to model the statistics of…
The potential of cosmic shear to probe cosmology is well recognized and future optical wide field surveys are currently being designed to optimize the return of cosmic shear science. High precision cosmic shear analysis requires high…
Scientific datasets present unique challenges for machine learning-driven compression methods, including more stringent requirements on accuracy and mitigation of potential invalidating artifacts. Drawing on results from compressed sensing…
We perform a blinded cosmology analysis with cosmic shear two-point correlation functions (2PCFs) measured from more than 25 million galaxies in the Hyper Suprime-Cam three-year shear catalog in four tomographic redshift bins ranging from…
Most statistical inference from cosmic large-scale structure relies on two-point statistics, i.e.\ on the galaxy-galaxy correlation function (2PCF) or the power spectrum. These statistics capture the full information encoded in the Fourier…
Current and upcoming cosmological surveys will produce unprecedented amounts of high-dimensional data, which require complex high-fidelity forward simulations to accurately model both physical processes and systematic effects which describe…
A long standing problem in weak lensing is about how to construct cosmic shear estimators from galaxy images. Conventional methods average over a single quantity per galaxy to estimate each shear component. We show that any such shear…
In this paper, we investigate the impact of compression on stochastic gradient algorithms for machine learning, a technique widely used in distributed and federated learning. We underline differences in terms of convergence rates between…
This paper studies the problem of estimating the covariance of a collection of vectors using only highly compressed measurements of each vector. An estimator based on back-projections of these compressive samples is proposed and analyzed. A…
Cosmological probes pose an inverse problem where the measurement result is obtained through observations, and the objective is to infer values of model parameters which characterize the underlying physical system -- our Universe. Modern…
We present the first cosmic shear measurements obtained from the T0001 release of the Canada-France-Hawaii Telescope Legacy Survey. The data set covers three uncorrelated patches (D1, D3 and D4) of one square degree each observed in u*, g',…
Linear models are used in online decision making, such as in machine learning, policy algorithms, and experimentation platforms. Many engineering systems that use linear models achieve computational efficiency through distributed systems…
The covariance matrix of the matter power spectrum is a key element of the statistical analysis of galaxy clustering data. Independent realisations of observational measurements can be used to sample the covariance, nevertheless statistical…
We present a new lossy compression algorithm for statistical floating-point data through a representation learning with binary variables. The algorithm finds a set of basis vectors and their binary coefficients that precisely reconstruct…