Related papers: Kalman Filter Auto-tuning through Enforcing Chi-Sq…

Accurate estimation of the dynamic states of a synchronous machine (e.g., rotor s angle and speed) is essential in monitoring and controlling transient stability of a power system. It is well known that the covariance matrixes of process…

Minimizing the Mean Squared Error (MSE) is a key objective in machine learning and is commonly used for imputing missing values. While this approach provides accurate point estimates, it introduces systematic biases in downstream analyses.…

The Kalman filter (KF) and the extended Kalman filter (EKF) are well established techniques for state estimation. However, the choice of the filter tuning parameters still poses a major challenge for the engineers [1]. In the present work,…

This article presents an up-to-date tutorial review of nonlinear Bayesian estimation. State estimation for nonlinear systems has been a challenge encountered in a wide range of engineering fields, attracting decades of research effort. To…

We consider ECNoise, a practical tool for estimating the magnitude of noise in evaluations of a black-box function. Recent developments in numerical optimization algorithms have seen increased usage of ECNoise as a subroutine to provide a…

We consider the problem of optimizing a grey-box objective function, i.e., nested function composed of both black-box and white-box functions. A general formulation for such grey-box problems is given, which covers the existing grey-box…

Likelihood functions evaluated using particle filters are typically noisy, computationally expensive, and non-differentiable due to Monte Carlo variability. These characteristics make conventional optimization methods difficult to apply…

Filtering is a widely used methodology for the incorporation of observed data into time-evolving systems. It provides an online approach to state estimation inverse problems when data is acquired sequentially. The Kalman filter plays a…

Current quantum computers suffer from non-stationary noise channels with high error rates, which undermines their reliability and reproducibility. We propose a Bayesian inference-based adaptive algorithm that can learn and mitigate quantum…

Deep learning models are trained to minimize the error between the model's output and the actual values. The typical cost function, the Mean Squared Error (MSE), arises from maximizing the log-likelihood of additive independent, identically…

Black box discrete optimization (BBDO) appears in wide range of engineering tasks. Evolutionary or other BBDO approaches have been applied, aiming at automating necessary tuning of system parameters, such as hyper parameter tuning of…

Optimizing expensive black-box objectives over mixed search spaces is a common challenge across the natural sciences. Bayesian optimization (BO) offers sample-efficient strategies through probabilistic surrogate models and acquisition…

Accurate estimation of noise parameters is critical for optimal filter performance, especially in systems where true noise parameter values are unknown or time-varying. This article presents a quaternion left-invariant extended Kalman…

This work introduces the StoMADS-PB algorithm for constrained stochastic blackbox optimization, which is an extension of the mesh adaptive direct-search (MADS) method originally developed for deterministic blackbox optimization under…

Real-world problems often involve the optimization of several objectives under multiple constraints. An example is the hyper-parameter tuning problem of machine learning algorithms. In particular, the minimization of the estimation of the…

One of the most common misconceptions made about the Kalman filter when applied to linear systems is that it requires an assumption that all error and noise processes are Gaussian. This misconception has frequently led to the Kalman filter…

We propose a new recursive estimator for linear dynamical systems under Gaussian process noise and non-Gaussian measurement noise. Specifically, we develop an approximate maximum a posteriori (MAP) estimator using dynamic programming and…

This paper proposes a novel global optimization algorithm, Particle Filter-Based Optimization (PFO), designed for a class of stochastic optimization problems in which the objective function lacks an analytical form and is subject to noisy…

This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum…

Reinforcement learning for large language models suffers from high-variance token-level importance sampling (IS) ratios, which would destabilize policy optimization at scale. To improve stability, recent methods typically use a fixed…