Related papers: EnKSGD: A Class Of Preconditioned Black Box Optimi…

This research enhances linear regression models by integrating a Kalman filter and analysing curve areas to minimize loss. The goal is to develop an optimal linear regression equation using stochastic gradient descent (SGD) for weight…

We propose a new stochastic optimization framework for empirical risk minimization problems such as those that arise in machine learning. The traditional approaches, such as (mini-batch) stochastic gradient descent (SGD), utilize an…

As 6G networks are developed and defined, offloading of XR applications is emerging as one of the strong new use cases. The reduced 6G latency coupled with edge processing infrastructure will for the first time provide a realistic…

The Ensemble Kalman filter assumes the observations to be Gaussian random variables with a pre-specified mean and variance. In practice, observations may also have detection limits, for instance when a gauge has a minimum or maximum value.…

We investigate the convergence properties of a class of iterative algorithms designed to minimize a potentially non-smooth and noisy objective function, which may be algebraically intractable and whose values may be obtained as the output…

Many real-world problems require one to estimate parameters of interest, in a Bayesian framework, from data that are collected sequentially in time. Conventional methods for sampling from posterior distributions, such as {Markov Chain Monte…

Accurate estimation and forecasting of energy consumption are important for power-system operation, planning, and demand-side management. In practice, however, complete and timely measurements may not always be available, and the observed…

Heavy tails is a common feature of filtering distributions that results from the nonlinear dynamical and observation processes as well as the uncertainty from physical sensors. In these settings, the Kalman filter and its ensemble version -…

In recent years, the prosperity of deep learning has revolutionized the Artificial Neural Networks. However, the dependence of gradients and the offline training mechanism in the learning algorithms prevents the ANN for further improvement.…

This paper studies an output feedback stabilization control framework for discrete-time linear systems with stochastic dynamics determined by an independent and identically distributed (i.i.d.) process. The controller is constructed with an…

This paper investigates ensemble Kalman inversion (EKI) for variational inverse problems with convex, potentially non-smooth regularization. While deterministic EKI and its Tikhonov-regularized variants have primarily been analyzed for…

The ensemble Kalman filter (EnKF) is widely used for nonlinear and high-dimensional state estimation because it replaces complex covariance propagation with simple ensemble statistics. However, conventional EnKF implementations can become…

An Ensemble Kalman Filter (EnKF, the predictor) is used make a large change in the state, followed by a Particle Filer (PF, the corrector) which assigns importance weights to describe non-Gaussian distribution. The weights are obtained by…

This paper derives the extended Kalman filter (EKF) for continuous-time systems on matrix Lie groups observed through discrete-time measurements. By modeling the system noise on the Lie algebra and adopting a Stratonovich interpretation for…

This paper introduces Tempered Fractional Gradient Descent (TFGD), a novel optimization framework that synergizes fractional calculus with exponential tempering to enhance gradient-based learning. Traditional gradient descent methods often…

In this work we combine ideas from multi-index Monte Carlo and ensemble Kalman filtering (EnKF) to produce a highly efficient filtering method called multi-index EnKF (MIEnKF). MIEnKF is based on independent samples of four-coupled EnKF…

This paper extends the ensemble Kalman filter (EnKF) for inverse problems to identify trending model coefficients. This is done by repeatedly inflating the ensemble while maintaining the mean of the particles. As a benchmark serves a…

In this paper, we develop a new algorithm, Annealed Skewed SGD - AskewSGD - for training deep neural networks (DNNs) with quantized weights. First, we formulate the training of quantized neural networks (QNNs) as a smoothed sequence of…

The Bayesian approach to inverse problems is widely used in practice to infer unknown parameters from noisy observations. In this framework, the ensemble Kalman inversion has been successfully applied for the quantification of uncertainties…

We provide the first generalization error analysis for black-box learning through derivative-free optimization. Under the assumption of a Lipschitz and smooth unknown loss, we consider the Zeroth-order Stochastic Search (ZoSS) algorithm,…