English
Related papers

Related papers: Efficient methods for computing observation impact…

200 papers

State-space mixed-frequency vector autoregressions are now widely used for nowcasting. Despite their popularity, estimating such models can be computationally intensive, especially for large systems with stochastic volatility. To tackle the…

Econometrics · Economics 2021-12-22 Joshua C. C. Chan , Aubrey Poon , Dan Zhu

In variational assimilation, the most probable state of a dynamical system under Gaussian assumptions for the prior and likelihood can be found by solving a least-squares minimization problem . In recent years, we have seen the popularity…

Numerical Analysis · Mathematics 2023-06-22 Shaerdan Shataer , Amos S. Lawless , Nancy K. Nichols

In numerical weather prediction (NWP), a large number of observations are used to create initial conditions for weather forecasting through a process known as data assimilation. An assessment of the value of these observations for NWP can…

We present two analytical formulae for estimating the sensitivity -- namely, the gradient or Jacobian -- at given realizations of an arbitrary-dimensional random vector with respect to its distributional parameters. The first formula…

Machine Learning · Statistics 2025-08-14 Pi-Yueh Chuang , Ahmed Attia , Emil Constantinescu

Today's ocean numerical prediction skills depend on the availability of in-situ and remote ocean observations at the time of the predictions only. Because observations are scarce and discontinuous in time and space, numerical models are…

Signal Processing · Electrical Eng. & Systems 2022-06-06 Ali Muhamed Ali , Hanqi Zhuang , Yu Huang , Ali K. Ibrahim , Ali Salem Altaher , Laurent Chérubin

Estimation of the precision matrix (or inverse covariance matrix) is of great importance in statistical data analysis and machine learning. However, as the number of parameters scales quadratically with the dimension $p$, computation…

Computation · Statistics 2022-11-02 Qian LI , Binyan Jiang , Defeng Sun

Errors in the representation of clouds in convection-permitting numerical weather prediction models can be introduced by different sources. These can be the forcing and boundary conditions, the representation of orography, the accuracy of…

Atmospheric and Oceanic Physics · Physics 2022-03-14 Stefanie Legler , Tijana Janjic

Shapley effects are attracting increasing attention as sensitivity measures. When the value function is the conditional variance, they account for the individual and higher order effects of a model input. They are also well defined under…

Computation · Statistics 2021-10-13 Elmar Plischke , Giovanni Rabitti , Emanuele Borgonovo

The four-dimensional variational data assimilation (4D-Var) has emerged as an important methodology, widely used in numerical weather prediction, oceanographic modeling, and climate forecasting. Classical unconstrained gradient-based…

Numerical Analysis · Mathematics 2024-10-08 Bowen Li , Bin Shi

Utilizing the information in observations of a complex system to make accurate predictions through a quantitative model when observations are completed at time $T$, requires an accurate estimate of the full state of the model at time $T$.…

Chaotic Dynamics · Physics 2014-05-13 Zhe An , Daniel Rey , Henry D. I. Abarbanel

Variational data assimilation optimizes for an initial state of a dynamical system such that its evolution fits observational data. The physical model can subsequently be evolved into the future to make predictions. This principle is a…

Machine Learning · Computer Science 2021-05-21 Thomas Frerix , Dmitrii Kochkov , Jamie A. Smith , Daniel Cremers , Michael P. Brenner , Stephan Hoyer

Variational data assimilation is a technique for combining measured data with dynamical models. It is a key component of Earth system state estimation and is commonly used in weather and ocean forecasting. The approach involves a…

Numerical Analysis · Mathematics 2026-04-30 I. Daužickaitė , M. A. Freitag , S. Gürol , A. S. Lawless , A. Ramage , J. A. Scott , J. M. Tabeart

Poyiadjis et al. (2011) show how particle methods can be used to estimate both the score and the observed information matrix for state space models. These methods either suffer from a computational cost that is quadratic in the number of…

Computation · Statistics 2015-09-07 Christopher Nemeth , Paul Fearnhead , Lyudmila Mihaylova

Machine learning for time-series forecasting remains a key area of research. Despite successful application of many machine learning techniques, relating computational efficiency to forecast error remains an under-explored domain. This…

Machine Learning · Computer Science 2023-09-28 Elin Törnquist , Wagner Costa Santos , Timothy Pogue , Nicholas Wingle , Robert A. Caulk

Data assimilation combines forecasts from a numerical model with observations. Most of the current data assimilation algorithms consider the model and observation error terms as additive Gaussian noise, specified by their covariance…

We consider the large-sparse symmetric linear systems of equations that arise in the solution of weak constraint four-dimensional variational data assimilation, a method of high interest for numerical weather prediction. These systems can…

Numerical Analysis · Mathematics 2020-05-15 Ieva Daužickaitė , Amos S. Lawless , Jennifer A. Scott , Peter Jan van Leeuwen

Data assimilation techniques are widely used to predict complex dynamical systems with uncertainties, based on time-series observation data. Error covariance matrices modelling is an important element in data assimilation algorithms which…

Machine Learning · Computer Science 2021-11-15 Sibo Cheng , Mingming Qiu

Data assimilation provides algorithms for widespread applications in various fields. It is of practical use to deal with a large amount of information in the complex system that is hard to estimate. Weather forecasting is one of the…

Optimization and Control · Mathematics 2023-03-23 Yihua Yang

Extreme precipitation causes severe societal and economic damage, and weather control has long been discussed as a potential mitigation strategy. However, to the best of our knowledge, perturbation-based interventions for weather control…

Machine Learning · Computer Science 2026-05-15 Ayumu Ueyama , Kazuhiko Kawamoto , Hiroshi Kera

This paper proposes two practical implementations of Four-Dimensional Variational (4D-Var) Ensemble Kalman Filter (4D-EnKF) methods for non-linear data assimilation. Our formulations' main idea is to avoid the intrinsic need for adjoint…

Applications · Statistics 2023-05-05 Elias Nin-Ruiz , Jairo Diaz-Rodriguez