Related papers: Covariance Square Root Second-Order Mapping
The Kalman filter operates by storing a Gaussian description of the state estimate in the form of a mean and covariance. Instead of storing and manipulating the covariance matrix directly, a square-root Kalman filter only forms and updates…
Using the array form of numerically stable square-root implementation methods for Kalman filtering formulas, we construct a new square-root algorithm for the log-likelihood gradient (score) evaluation. This avoids the use of the…
Square-root Kalman filters propagate state covariances in Cholesky-factor form for numerical stability, and are a natural target for gradient-based parameter learning in state-space models. Their core operation, triangularization of a…
The Fast Reciprocal Square Root Algorithm is a well-established approximation technique consisting of two stages: first, a coarse approximation is obtained by manipulating the bit pattern of the floating point argument using integer…
Researchers collecting intensive longitudinal data (ILD) are increasingly looking to model psychological processes, such as emotional dynamics, that organize and adapt across time in complex and meaningful ways. This is also the case for…
This paper suggests a few novel Cholesky-based square-root algorithms for the maximum correntropy criterion Kalman filtering. In contrast to the previously obtained results, new algorithms are developed in the so-called {\it condensed} form…
Covariance steering (CS) synthesizes a control policy which drives the state's mean and covariance matrix towards desired values. Offering tractable computation of a closed-loop policy which can obey chance constraints in uncertain…
Successive quadratic approximations, or second-order proximal methods, are useful for minimizing functions that are a sum of a smooth part and a convex, possibly nonsmooth part that promotes regularization. Most analyses of iteration…
Relying on recent advances in statistical estimation of covariance distances based on random matrix theory, this article proposes an improved covariance and precision matrix estimation for a wide family of metrics. The method is shown to…
Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$.…
We present improved algorithms for fast calculation of the inverse square root for single-precision floating-point numbers. The algorithms are much more accurate than the famous fast inverse square root algorithm and have the same or…
We propose KOALA++, a scalable Kalman-based optimization algorithm that explicitly models structured gradient uncertainty in neural network training. Unlike second-order methods, which rely on expensive second order gradient calculation,…
First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity. Second-order methods, while able to provide faster convergence, have been much less explored…
The state-of-the-art tensor network Kalman filter lifts the curse of dimensionality for high-dimensional recursive estimation problems. However, the required rounding operation can cause filter divergence due to the loss of positive…
Contemporary data assimilation often involves millions of prediction variables. The classical Kalman filter is no longer computationally feasible in such a high dimensional context. This problem can often be resolved by exploiting the…
Estimation of the precision matrix (or inverse covariance matrix) is of great importance in statistical data analysis and machine learning. However, as the number of parameters scales quadratically with the dimension $p$, computation…
Due to the limited number of bits in floating-point or fixed-point arithmetic, rounding is a necessary step in many computations. Although rounding methods can be tailored for different applications, round-off errors are generally…
Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates. At their heart, such algorithms need to…
Models in which the covariance matrix has the structure of a sparse matrix plus a low rank perturbation are ubiquitous in data science applications. It is often desirable for algorithms to take advantage of such structures, avoiding costly…
We revisit a formulation technique for inequality constrained optimization problems that has been known for decades: the substitution of squared variables for nonnegative variables. Using this technique, inequality constraints are converted…