Related papers: A Recursive Newton Method for Smoothing in Nonline…
In this paper, we present a Newton-like method based on model reduction techniques, which can be used in implicit numerical methods for approximating the solution to ordinary differential equations. In each iteration, the Newton-like method…
We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton's method (SN), and the…
The Newton-Schulz (NS) iteration has become a key technique for orthogonalization in optimizers such as Muon and for optimization on the Stiefel manifold. Despite its effectiveness, the conventional NS iteration incurs significant…
The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of…
This paper focuses on discussing Newton's method and its hybrid with machine learning for the steady state Navier-Stokes Darcy model discretized by mixed element methods. First, a Newton iterative method is introduced for solving the…
We present iterative solvers to approximate the solution of numerical schemes for stochastic Stefan problems. After briefly talking about the convergence results, we tackle the question of efficient strategies for solving the nonlinear…
State estimation in heavy-tailed process and measurement noise is an important challenge that must be addressed in, e.g., tracking scenarios with agile targets and outlier-corrupted measurements. The performance of the Kalman filter (KF)…
This work is on constrained large-scale non-convex optimization where the constraint set implies a manifold structure. Solving such problems is important in a multitude of fundamental machine learning tasks. Recent advances on Riemannian…
This paper details a novel indirect method for solving constrained optimal control problems (OCPs) directly in continuous-time function space. The KKT conditions are embedded in a non-smooth complementarity function, which enables their…
We introduce a class of quadratic support (QS) functions, many of which play a crucial role in a variety of applications, including machine learning, robust statistical inference, sparsity promotion, and Kalman smoothing. Well known…
Large-scale dynamic inverse problems are often ill-posed due to model complexity and the high dimensionality of the unknown parameters. Regularization is commonly employed to mitigate ill-posedness by incorporating prior information and…
Iterative refinement -- start with a random guess, then iteratively improve the guess -- is a useful paradigm for representation learning because it offers a way to break symmetries among equally plausible explanations for the data. This…
In this paper, we propose a new model reduction technique for linear stochastic systems that builds upon knowledge filtering and utilizes optimal Kalman filtering techniques. This new technique will reduce the dimension of the noise…
A stochastic iterative algorithm approximating second-order information using von Neumann series is discussed. We present convergence guarantees for strongly-convex and smooth functions. Our analysis is much simpler in contrast to a similar…
In this work we develop and analyze a semi-smooth Newton method for the general nonlinear conic programming problem. In particular, we study the problem with a generalized simplicial cone, i.e., the image of a symmetric cone under a linear…
In this paper, we study the affine phase retrieval problem, which aims to recover signals from the magnitudes of affine measurements. We develop second-order optimization methods based on Newton and Gauss-Newton iterations and establish…
For solving linear ill-posed problems regularization methods are required when the right hand side is with some noise. In the present paper regularized solutions are obtained by implicit iteration methods in Hilbert scales. % By exploiting…
Non-smooth regularization is widely used in image reconstruction to eliminate the noise while preserving subtle image structures. In this work, we investigate the use of proximal Newton (PN) method to solve an optimization problem with a…
Multi-scale problems, where variables of interest evolve in different time-scales and live in different state-spaces, can be found in many fields of science. Here, we introduce a new recursive methodology for Bayesian inference that aims at…
This work introduces an algorithm for state estimation on manifolds within the framework of the Kalman filter. Its primary objective is to provide a methodology enabling the evaluation of the precision of existing Kalman filter variants…