English
Related papers

Related papers: Numerical study of high-dimensional covariance est…

200 papers

This paper is a contribution in the context of variational data assimilation combined with statistical learning. The framework of data assimilation traditionally uses data collected at sensor locations in order to bring corrections to a…

Numerical Analysis · Mathematics 2023-05-09 Amina Benaceur , Barbara Verfürth

Data assimilation addresses the problem of identifying plausible state trajectories of dynamical systems given noisy or incomplete observations. In geosciences, it presents challenges due to the high-dimensionality of geophysical dynamical…

Machine Learning · Statistics 2023-11-03 François Rozet , Gilles Louppe

Modern data assimilation schemes typically use the same discrete dynamical model to evolve the state estimate in time also to approximate the evolution, or propagation, of the estimation error covariance. Ensemble-based methods, such as the…

Analysis of PDEs · Mathematics 2025-08-25 Shay Gilpin

The performance of ensemble-based data assimilation techniques that estimate the state of a dynamical system from partial observations depends crucially on the prescribed uncertainty of the model dynamics and of the observations. These are…

Computation · Statistics 2021-02-24 Tadeo Javier Cocucci , Manuel Pulido , Magdalena Lucini , Pierre Tandeo

Data assimilation techniques are widely used to predict complex dynamical systems with uncertainties, based on time-series observation data. Error covariance matrices modelling is an important element in data assimilation algorithms which…

Machine Learning · Computer Science 2021-11-15 Sibo Cheng , Mingming Qiu

Data assimilation involves estimating the state of a system by combining observations from various sources with a background estimate of the state. The weights given to the observations and background state depend on their specified error…

Numerical Analysis · Mathematics 2025-03-13 Olivier Goux , Anthony Weaver , Selime Gürol , Oliver Guillet , Andrea Piacentini

This paper addresses variational data assimilation from a learning point of view. Data assimilation aims to reconstruct the time evolution of some state given a series of observations, possibly noisy and irregularly-sampled. Using automatic…

Computational Physics · Physics 2021-11-10 Ronan Fablet , Bertrand Chapron , Lucas. Drumetz , Etienne Memin , Olivier Pannekoucke , Francois Rousseau

Data assimilation plays a key role in large-scale atmospheric weather forecasting, where the state of the physical system is estimated from model outputs and observations, and is then used as initial condition to produce accurate future…

Methodology · Statistics 2018-02-13 Azam Moosavi , Ahmed Attia , Adrian Sandu

Data assimilation (DA) is integrated with machine learning in order to perform entirely data-driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as surrogate models to replace key components of…

Accurate estimation of error covariances (both background and observation) is crucial for efficient observation compression approaches in data assimilation of large-scale dynamical problems. We propose a new combination of a covariance…

Numerical Analysis · Mathematics 2021-06-11 Sibo Cheng , Didier Lucor , Jean-Philippe Argaud

Data assimilation (DA) aims at optimally merging observational data and model outputs to create a coherent statistical and dynamical picture of the system under investigation. Indeed, DA aims at minimizing the effect of observational and…

Data Analysis, Statistics and Probability · Physics 2022-01-05 Yumeng Chen , Alberto Carrassi , Valerio Lucarini

Data Assimilation (DA) is a methodology for combining mathematical models simulating complex systems (the background knowledge) and measurements (the reality or observational data) in order to improve the estimate of the system state. This…

Numerical Analysis · Mathematics 2019-01-15 Luisa D'Amore , Rosalba Cacciapuoti

Testing the independence between random vectors is a fundamental problem in statistics. Distance correlation, a recently popular dependence measure, is universally consistent for testing independence against all distributions with finite…

Methodology · Statistics 2024-08-22 Yuwei Ke , Hok Kan Ling , Yanglei Song

We identify a strong equivalence between neural network based machine learning (ML) methods and the formulation of statistical data assimilation (DA), known to be a problem in statistical physics. DA, as used widely in physical and…

Machine Learning · Computer Science 2017-10-23 H. D. I. Abarbanel , P. J. Rozdeba , S. Shirman

This paper studies the role of sparse regularization in a properly chosen basis for variational data assimilation (VDA) problems. Specifically, it focuses on data assimilation of noisy and down-sampled observations while the state variable…

Data Analysis, Statistics and Probability · Physics 2014-06-25 A. M. Ebtehaj , M. Zupanski , G. Lerman , E. Foufoula-Georgiou

We develop an optimal experimental design framework for adapting the covariance inflation and localization in data assimilation problems. Covariance inflation and localization are ubiquitously employed to alleviate the effect of using…

Computational Engineering, Finance, and Science · Computer Science 2019-03-26 Ahmed Attia , Emil Constantinescu

A method of data assimilation (DA) is employed to estimate electrophysiological parameters of neurons simultaneously with their synaptic connectivity in a small model biological network. The DA procedure is cast as an optimization, with a…

Neurons and Cognition · Quantitative Biology 2020-02-05 Eve Armstrong

Approximate Bayesian computation (ABC) and other likelihood-free inference methods have gained popularity in the last decade, as they allow rigorous statistical inference for complex models without analytically tractable likelihood…

Computation · Statistics 2019-06-21 Jukka Sirén , Samuel Kaski

In this study, two classes of methods including statistical and variational data assimilation algorithms will be described. In statistical methods, the model state is updated sequentially based on the previous estimate. Variational methods,…

Systems and Control · Electrical Eng. & Systems 2021-10-25 Loc Luong

Quantifying forecast uncertainty is a key aspect of state-of-the-art numerical weather prediction and data assimilation systems. Ensemble-based data assimilation systems incorporate state-dependent uncertainty quantification based on…

Atmospheric and Oceanic Physics · Physics 2023-05-17 Maximiliano A. Sacco , Manuel Pulido , Juan J. Ruiz , Pierre Tandeo