English
Related papers

Related papers: Nonlinear ensemble filtering with diffusion models…

200 papers

The particle filter (PF) and the ensemble Kalman filter (EnKF) are widely used for approximate inference in state-space models. From a Bayesian perspective, these algorithms represent the prior by an ensemble of particles and update it to…

Methodology · Statistics 2025-02-11 Chengxin Gong , Wei Lin , Cheng Zhang

The sample covariance matrix of a random vector is a good estimate of the true covariance matrix if the sample size is much larger than the length of the vector. In high-dimensional problems, this condition is never met. As a result, in…

Data Analysis, Statistics and Probability · Physics 2024-11-12 Michael Tsyrulnikov , Arseniy Sotskiy

This paper presents the machine learning-based ensemble conditional mean filter (ML-EnCMF) -- a filtering method based on the conditional mean filter (CMF) previously introduced in the literature. The updated mean of the CMF matches that of…

Machine Learning · Computer Science 2022-08-02 Truong-Vinh Hoang , Sebastian Krumscheid , Hermann G. Matthies , Raúl Tempone

Data assimilation has been applied to coastal hydrodynamic models to better estimate system states or parameters by incorporating observed data into the model. Kalman Filter (KF) is one of the most studied data assimilation methods whose…

Atmospheric and Oceanic Physics · Physics 2016-07-05 Milad Hooshyar , Stephen C. Medeiros , Dingbao Wang , Scott C. Hagen

Ensemble Kalman filtering (EnKF) is an efficient approach to addressing uncertainties in subsurface groundwater models. The EnKF sequentially integrates field data into simulation models to obtain a better characterization of the model's…

Data Analysis, Statistics and Probability · Physics 2015-11-09 Boujemaa Ait-El-Fquih , Mohamad El Gharamti , Ibrahim Hoteit

This paper develops efficient ensemble Kalman filter (EnKF) implementations based on shrinkage covariance estimation. The forecast ensemble members at each step are used to estimate the background error covariance matrix via the…

Statistics Theory · Mathematics 2015-02-03 Elias D. Nino-Ruiz , Adrian Sandu

The Ensemble Kalman Filter (EnKF) is a widely used method for data assimilation in high-dimensional systems, with an ensemble update step equivalent to an empirical version of the Matheron update popular in Gaussian process regression -- a…

Machine Learning · Computer Science 2025-09-19 Dan MacKinlay

This paper investigates an approximation scheme of the optimal nonlinear Bayesian filter based on the Gaussian mixture representation of the state probability distribution function. The resulting filter is similar to the particle filter,…

Data Analysis, Statistics and Probability · Physics 2015-05-30 Ibrahim Hoteit , Xiaodong Luo , Dinh-Tuan Pham

Contemporary data assimilation often involves more than a million prediction variables. Ensemble Kalman filters (EnKF) have been developed by geoscientists. They are successful indispensable tools in science and engineering, because they…

Probability · Mathematics 2017-05-26 Andrew J. Majda , Xin T. Tong

The Ensemble Kalman filter assumes the observations to be Gaussian random variables with a pre-specified mean and variance. In practice, observations may also have detection limits, for instance when a gauge has a minimum or maximum value.…

Optimization and Control · Mathematics 2018-11-14 Abhishek Shah , Mohamad El Gharamti , Laurent Bertino

We explore the potential of Data-Assimilation (DA) within the multi-scale framework of a shell model of turbulence, with a focus on the Ensemble Kalman Filter (EnKF). The central objective is to understand how measuring mesoscales (i.e.,…

Fluid Dynamics · Physics 2026-01-15 Francesco Fossella , Luca Biferale , Alberto Carrassi , Massimo Cencini , Vikrant Gupta

Data assimilation techniques are crucial for accurately tracking complex dynamical systems by integrating observational data with numerical forecasts. Recently, score-based data assimilation methods emerged as powerful tools for…

Machine Learning · Statistics 2026-03-03 Pengpeng Xiao , Phillip Si , Peng Chen

In this paper, we propose and develop a methodology for nonlinear systems health monitoring by modeling the damage and degradation mechanism dynamics as "slow" states that are augmented with the system "fast" dynamical states. This…

Systems and Control · Computer Science 2017-10-17 Najmeh Daroogheh , Nader Meskin , Khashayar Khorasani

We design and analyse the performance of a multilevel ensemble Kalman filter method (MLEnKF) for filtering settings where the underlying state-space model is an infinite-dimensional spatio-temporal process. We consider underlying models…

Numerical Analysis · Mathematics 2020-03-11 Alexey Chernov , Håkon Hoel , Kody J. H. Law , Fabio Nobile , Raul Tempone

Filtering - the task of estimating the conditional distribution for states of a dynamical system given partial and noisy observations - is important in many areas of science and engineering, including weather and climate prediction.…

Machine Learning · Computer Science 2025-03-25 Eviatar Bach , Ricardo Baptista , Enoch Luk , Andrew Stuart

This work embeds a multilevel Monte Carlo (MLMC) sampling strategy into the Monte Carlo step of the ensemble Kalman filter (EnKF), thereby yielding a multilevel ensemble Kalman filter (MLEnKF) which has provably superior asymptotic cost to…

Numerical Analysis · Mathematics 2016-08-31 Alexey Chernov , Haakon Hoel , Kody Law , Fabio Nobile , Raul Tempone

We propose a Neural-Enhanced Distributed Kalman Filter (NDKF) for multi-sensor state estimation in nonlinear systems. Unlike traditional Kalman filters that rely on explicit analytical models and assume centralized fusion, NDKF leverages…

Systems and Control · Electrical Eng. & Systems 2026-03-17 Siavash Farzan , Bennett Parisi

In this work, we present the ensemble-marginalized Kalman filter (EnMKF), a sequential algorithm analogous to our previously proposed approach [1,2], for estimating the state and parameters of linear parabolic partial differential equations…

Computation · Statistics 2018-05-15 Marco Iglesias , Zaid Sawlan , Marco Scavino , Raul Tempone , Christopher Wood

The FFT EnKF data assimilation method is proposed and applied to a stochastic cell simulation of an epidemic, based on the S-I-R spread model. The FFT EnKF combines spatial statistics and ensemble filtering methodologies into a localized…

Computation · Statistics 2010-03-10 Jan Mandel , Jonathan D. Beezley , Loren Cobb , Ashok Krishnamurthy

The ensemble Gaussian mixture filter (EnGMF) is a non-linear filter suited to data assimilation of highly non-Gaussian and non-linear models that has practical utility in the case of a small number of samples, and theoretical convergence to…

Optimization and Control · Mathematics 2024-06-03 Andrey A. Popov , Enrico M. Zucchelli , Renato Zanetti