Related papers: Massive Lossless Data Compression and Multiple Par…
Many multivariate data such as social and biological data exhibit complex dependencies that are best characterized by graphs. Unlike sequential data, graphs are, in general, unordered structures. This means we can no longer use classic,…
Current radio interferometers output multi-petabyte-scale volumes of data per year making the storage, transfer, and processing of this data a sizeable challenge. This challenge is expected to grow with the next-generation telescopes such…
The current and upcoming large data volume galaxy surveys require the use of machine learning techniques to maximize their scientific return. This study explores the use of Self-Organizing Maps (SOMs) to estimate galaxy parameters with a…
We developed a method to infer the calibration parameters of multichannel measurement systems, such as channel variations of sensitivity and noise amplitude, from experimental data. We regard such uncertainties of the calibration parameters…
A new approach to data compression is developed and applied to multimedia content. This method separates messages into components suitable for both lossless coding and 'lossy' or statistical coding techniques, compressing complex objects by…
This article introduces a novel nonparametric methodology for Generalized Linear Models which combines the strengths of the binary regression and latent variable formulations for categorical data, while overcoming their disadvantages.…
Complex phenomena can be better understood when broken down into a limited number of simpler "components". Linear statistical methods such as the principal component analysis and its variants are widely used across various fields of applied…
Data used for analytics and machine learning often take the form of tables with categorical entries. We introduce a family of lossless compression algorithms for such data that proceed in four steps: $(i)$ Estimate latent variables…
The data compression technology now is fully developed and widely used in many fields such as communication, multi-media, image information processing and so on. The large physical experiments, especially the ones with Micro-pattern Gas…
Bringing a high-dimensional dataset into science-ready shape is a formidable challenge that often necessitates data compression. Compression has accordingly become a key consideration for contemporary cosmology, affecting public data…
The volume of radio-astronomical data is a considerable burden in the processing and storing of radio observations with high time and frequency resolutions and large bandwidths. Lossy compression of interferometric radio-astronomical data…
Today's HPC applications are producing extremely large amounts of data, such that data storage and analysis are becoming more challenging for scientific research. In this work, we design a new error-controlled lossy compression algorithm…
We explore two methods of compressing the redshift space galaxy power spectrum and bispectrum with respect to a chosen set of cosmological parameters. Both methods involve reducing the dimension of the original data-vector ( e.g. 1000…
We extend the modal decomposition method, previously applied to compress the information in the real-space bispectrum, to the anisotropic redshift-space galaxy bispectrum. In the modal method approach, the bispectrum is expanded on a basis…
This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated…
Background: High-throughput proteomics techniques, such as mass spectrometry (MS)-based approaches, produce very high-dimensional data-sets. In a clinical setting one is often interested in how mass spectra differ between patients of…
Most statistical inference from cosmic large-scale structure relies on two-point statistics, i.e.\ on the galaxy-galaxy correlation function (2PCF) or the power spectrum. These statistics capture the full information encoded in the Fourier…
The redshift-space bispectrum (three point statistics) of galaxies depends on the expansion rate, the growth rate, and geometry of the Universe, and hence can be used to measure key cosmological parameters. In a homogeneous Universe the…
In 2009, a lossless compression algorithm based on 1D chaotic maps known as Generalized Lur\"{o}th Series (or GLS) has been proposed. This algorithm (GLS-coding) encodes the input message as a symbolic sequence on an appropriate 1D chaotic…
We describe a new method of overcoming problems inherent in peculiar velocity surveys by using data compression as a filter with which to separate large-scale, linear flows from small-scale noise that biases the results systematically. We…