Related papers: Transformers as Implicit State Estimators: In-Cont…
Latent variable models have become instrumental in computational neuroscience for reasoning about neural computation. This has fostered the development of powerful offline algorithms for extracting latent neural trajectories from neural…
State estimation of a dynamical system refers to estimating the state of a system given an imperfect model, noisy measurements and some or no information about the initial state. While Kalman filtering is optimal for estimation of linear…
This paper introduces a novel proprioceptive state estimator for legged robots that combines model-based filters and deep neural networks. Recent studies have shown that neural networks such as multi-layer perceptron or recurrent neural…
Using Kalman techniques, it is possible to perform optimal estimation in linear Gaussian state-space models. We address here the case where the noise probability density functions are of unknown functional form. A flexible Bayesian…
Accuracy and generalization capabilities are key objectives when learning dynamical system models. To obtain such models from limited data, current works exploit prior knowledge and assumptions about the system. However, the fusion of…
The Kalman filter is a fundamental filtering algorithm that fuses noisy sensory data, a previous state estimate, and a dynamics model to produce a principled estimate of the current state. It assumes, and is optimal for, linear models and…
Kalman filter is a best linear unbiased state estimator. It is also comprehensible from the point view of the Bayesian estimation. However, this note gives a detailed derivation of Kalman filter from the mutual information perspective for…
In-context learning (ICL) refers to the ability of a model to learn new tasks from examples in its input without any parameter updates. In contrast to previous theories of ICL relying on toy models and data settings, recently it has been…
Large language models (LLMs) exhibit a strong capacity for in-context learning: Given labeled examples, they can generate good predictions without parameter updates. However, many interactive settings go beyond static prediction to online…
Providing a metric of uncertainty alongside a state estimate is often crucial when tracking a dynamical system. Classic state estimators, such as the Kalman filter (KF), provide a time-dependent uncertainty measure from knowledge of the…
The majority of data assimilation (DA) methods in the geosciences are based on Gaussian assumptions. While these assumptions facilitate efficient algorithms, they cause analysis biases and subsequent forecast degradations. Non-parametric,…
Ensemble transform Kalman filtering (ETKF) data assimilation is often used to combine available observations with numerical simulations to obtain statistically accurate and reliable state representations in dynamical systems. However, it is…
Switching Kalman Filters (SKF) are well known for their ability to solve the piecewise linear dynamic system estimation problem using the standard Kalman Filter (KF). Practical SKFs are heuristic, approximate filters that are not guaranteed…
Sequential fine-tuning of transformers is useful when new data arrive sequentially, especially with shifting distributions. Unlike batch learning, sequential learning demands that training be stabilized despite a small amount of data by…
Recent research has demonstrated that transformers, particularly linear attention models, implicitly execute gradient-descent-like algorithms on data provided in-context during their forward inference step. However, their capability in…
A Bayesian knowledge transfer mechanism that leverages external information to improve the performance of the Kalman fixed-lag interval smoother (FLIS) is proposed. Exact knowledge of the external observation model is assumed to be missing,…
This paper considers the simultaneous state and unknown input estimation for continuous-discrete stochastic systems. Two types of approaches (with and without modeling of unknown inputs) which can address this issue are investigated. A…
The ensemble Kalman filter (EnKF) is widely used for data assimilation in high-dimensional systems, but its performance often deteriorates for strongly nonlinear dynamics due to the structural mismatch between the Kalman update and the…
This paper studies the distributed state estimation in sensor network, where $m$ sensors are deployed to infer the $n$-dimensional state of a linear time-invariant (LTI) Gaussian system. By a lossless decomposition of optimal steady-state…
Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples $(x, f(x))$ presented in the input without further parameter…