相关论文: Sifting data in the real world

The particle filter is a popular Bayesian filtering algorithm for use in cases where the state-space model is nonlinear and/or the random terms (initial state or noises) are non-Gaussian distributed. We study the behavior of the error in…

Generalization beyond the training distribution is a core challenge in machine learning. The common practice of mixing and shuffling examples when training neural networks may not be optimal in this regard. We show that partitioning the…

Investigation of the underlying physics or biology from empirical data requires a quantifiable notion of similarity - when do two observed data sets indicate nearly identical generating processes, and when they do not. The discriminating…

We investigate a new sampling scheme aimed at improving the performance of particle filters whenever (a) there is a significant mismatch between the assumed model dynamics and the actual system, or (b) the posterior probability tends to…

A physical data (such as astrophysical, geophysical, meteorological etc.) may appear as an output of an experiment or it may come out as a signal from a dynamical system or it may contain some sociological, economic or biological…

Particle filtering is used to compute good nonlinear estimates of complex systems. It samples trajectories from a chosen distribution and computes the estimate as a weighted average. Easy-to-sample distributions often lead to degenerate…

Health data are often not symmetric to be adequately modeled through the usual normal distributions; most of them exhibit skewed patterns. They can indeed be modeled better through the larger family of skew-normal distributions covering…

Unfolding, in the context of high-energy particle physics, refers to the process of removing detector distortions in experimental data. The resulting unfolded measurements are straightforward to use for direct comparisons between…

The main purpose of scattering experiments is to unveil the underlying structure of the colliding particles and their interaction. Typically one measures scattering observables (cross sections and polarizations) at discrete angles and…

Thousands of experiments are analyzed and papers are published each year involving the statistical analysis of grouped data. While this area of statistics is often perceived -- somewhat naively -- as saturated, several misconceptions still…

High-throughput data analyses are becoming common in biology, communications, economics and sociology. The vast amounts of data are usually represented in the form of matrices and can be considered as knowledge networks. Spectra-based…

Safely deploying machine learning models to the real world is often a challenging process. Models trained with data obtained from a specific geographic location tend to fail when queried with data obtained elsewhere, agents trained in a…

The Gaussian function (GF) is widely used to explain the behavior or statistical distribution of many natural phenomena as well as industrial processes in different disciplines of engineering and applied science. For example, the GF can be…

Data assimilation algorithms integrate prior information from numerical model simulations with observed data. Ensemble-based filters, regarded as state-of-the-art, are widely employed for large-scale estimation tasks in disciplines such as…

We revisit the method of cumulants for analysing dynamic light scattering data in particle sizing applications. Here the data, in the form of the time correlation function of scattered light, is written as a series involving the first few…

The performance of machine learning models relies heavily on the quality of input data, yet real-world applications often face significant data-related challenges. A common issue arises when curating training data or deploying models: two…

When reading peer-reviewed scientific literature describing any analysis of empirical data, it is natural and correct to proceed with the underlying assumption that experiments have made good faith efforts to ensure that their analyses…

Matrix inversion problems are often encountered in experimental physics, and in particular in high-energy particle physics, under the name of unfolding. The true spectrum of a physical quantity is deformed by the presence of a detector,…

Consistent experiment data are crucial to adjust parameters of physics models and to determine best estimates of observables. However, often experiment data are not consistent due to unrecognized systematic errors. Standard methods of…

Particle filters are a group of algorithms to solve inverse problems through statistical Bayesian methods when the model does not comply with the linear and Gaussian hypothesis. Particle filters are used in domains like data assimilation,…