Related papers: Massive Lossless Data Compression and Multiple Par…
We propose a new prediction method for multivariate linear regression problems where the number of features is less than the sample size but the number of outcomes is extremely large. Many popular procedures, such as penalized regression…
This paper addresses classification problems with matrix-valued data, which commonly arise in applications such as neuroimaging and signal processing. Building on the assumption that the data from each class follows a matrix normal…
We present a novel method to compress galaxy clustering three-point statistics and apply it to redshift space galaxy bispectrum monopole measurements from BOSS DR12 CMASS data considering a $k$-space range of $0.03-0.12\,h/\mathrm{Mpc}$.…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…
Scientific discoveries are increasingly constrained by limited storage space and I/O capacities. For time-series simulations and experiments, their data often need to be decimated over timesteps to accommodate storage and I/O limitations.…
Today's scientific simulations, for example in the high-performance exascale sector, produce huge amounts of data. Due to limited I/O bandwidth and available storage space, there is the necessity to reduce scientific data of high…
This paper presents a novel data-driven, direct filtering approach for unknown linear time-invariant systems affected by unknown-but-bounded measurement noise. The proposed technique combines independent multistep prediction models,…
The galaxy power spectrum is now a well-known tool of precision cosmology. In addition to the overall shape, baryon oscillations and the small-scale suppression of power by massive neutrinos capture complimentary information on cosmological…
This work establishes regularity conditions for consistency and asymptotic normality of the multiple parameter maximum likelihood estimator(MLE) from censored data, where the censoring mechanism is in the form of $1$-bit measurements. The…
We explore linear and non-linear dimensionality reduction techniques for statistical inference of parameters in cosmology. Given the importance of compressing the increasingly complex data vectors used in cosmology, we address questions…
The missing data problem has been broadly studied in the last few decades and has various applications in different areas such as statistics or bioinformatics. Even though many methods have been developed to tackle this challenge, most of…
We present a procedure for efficiently compressing astronomical radio data for high performance applications. Integrated, post-correlation data are first passed through a nearly lossless rounding step which compares the precision of the…
Error-controlled lossy compression has been studied for years because of extremely large volumes of data being produced by today's scientific simulations. None of existing lossy compressors, however, allow users to fix the peak…
Accurate parameter estimation from cosmic microwave background data requires reliable likelihood modeling, particularly at large angular scales where angular power spectrum estimators exhibit non-Gaussian statistics. We present a novel…
The statistical properties of a map of the primary fluctuations in the cosmic microwave background (CMB) may be specified to high accuracy by a few thousand power spectra measurements, provided the fluctuations are gaussian, yet the number…
We present a new method to estimate redshift distributions and galaxy-dark matter bias parameters using correlation functions in a fully data driven and self-consistent manner. Unlike other machine learning, template, or correlation…
Recently, an extensive amount of research has been focused on compressing and accelerating Deep Neural Networks (DNN). So far, high compression rate algorithms require part of the training dataset for a low precision calibration, or a…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
This paper describes a new approach, based on linear programming, for computing nonnegative matrix factorizations (NMFs). The key idea is a data-driven model for the factorization where the most salient features in the data are used to…
Precision measurements of the galaxy power spectrum P(k) require a data analysis pipeline that is both fast enough to be computationally feasible and accurate enough to take full advantage of high-quality data. We present a rigorous…