Related papers: RLEKF: An Optimizer for Deep Potential with Ab Ini…
One hidden yet important issue for developing neural network potentials (NNPs) is the choice of training algorithm. Here we compare the performance of two popular training algorithms, the adaptive moment estimation algorithm (Adam) and the…
Deep Learners (DLs) are the state-of-art predictive mechanism with applications in many fields requiring complex high dimensional data processing. Although conventional DLs get trained via gradient descent with back-propagation, Kalman…
Driven by the filtering challenges in linear systems disturbed by non-Gaussian heavy-tailed noise, the robust Kalman filters (RKFs) leveraging diverse heavy-tailed distributions have been introduced. However, the RKFs rely on precise noise…
In this article, we propose a new filtering algorithm based in the Koopman operator, showing that a nonlinear filtering problem can be seen as an equivalent problem where the dynamics is infinite dimensional, but linear. Using Extended…
In this work, we present a new derivative-free optimization method and investigate its use for training neural networks. Our method is motivated by the Ensemble Kalman Filter (EnKF), which has been used successfully for solving optimization…
We study adaptive (or online) nonlinear regression with Long-Short-Term-Memory (LSTM) based networks, i.e., LSTM-based adaptive learning. In this context, we introduce an efficient Extended Kalman filter (EKF) based second-order training…
In order to integrate uncertainty estimates into deep time-series modelling, Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models, however, such approaches typically rely on approximate inference…
Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN computation and storage by decreasing the bitwidth of…
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA), address this challenge by…
Simultaneous localization and mapping (SLAM) is a method that constructs a map of an unknown environment and localizes the position of a moving agent on the map simultaneously. Extended Kalman filter (EKF) has been widely adopted as a low…
In the presence of renewable resources, distribution networks have become extremely complex to monitor, operate and control. Furthermore, for the real time applications, active distribution networks require fast real time distribution state…
The Kalman filter is a fundamental tool for state estimation in dynamical systems. While originally developed for linear Gaussian settings, it has been extended to nonlinear problems through approaches such as the extended and unscented…
Statistical signal processing based speech enhancement methods adopt expert knowledge to design the statistical models and linear filters, which is complementary to the deep neural network (DNN) based methods which are data-driven. In this…
Reinforcement learning (RL) tackles sequential decision-making problems by creating agents that interacts with their environment. However, existing algorithms often view these problem as static, focusing on point estimates for model…
Cellular Simultaneous Recurrent Neural Network (SRN) has been shown to be a function approximator more powerful than the MLP. This means that the complexity of MLP would be prohibitively large for some problems while SRN could realize the…
High fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge…
In recent years, deep learning has made remarkable progress in a wide range of domains, with a particularly notable impact on natural language processing tasks. One of the challenges associated with training deep neural networks in the…
Extended Kalman Filter (EKF) has been a popular approach to localization a mobile robot. However, the performance of the EKF and the quality of the estimation depends on the correct a priori knowledge of process and measurement noise…
We propose a Dynamical Low-Rank Ensemble Kalman Filter (DLR-ENKF) for efficient joint state-parameter estimation in high-dimensional dynamical systems. The method extends the DLR-ENKF formulation of arXiv:2509.11210 to the augmented…
The optimal fusion of estimates in a Distributed Kalman Filter (DKF) requires tracking of the complete network error covariance, problematic in terms of memory and communication. A scalable alternative is to fuse estimates under unknown…