Related papers: A Framework for Generalising the Newton Method and…
In this paper we study the Newton's method for finding a singularity of a differentiable vector field defined on a Riemannian manifold. Under the assumption of invertibility of covariant derivative of the vector field at its singularity, we…
Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset…
Many machine learning models depend on solving a large scale optimization problem. Recently, sub-sampled Newton methods have emerged to attract much attention for optimization due to their efficiency at each iteration, rectified a weakness…
Manifold learning has been proven to be an effective method for capturing the implicitly intrinsic structure of non-Euclidean data, in which one of the primary challenges is how to maintain the distortion-free (isometry) of the data…
Approximating a manifold-valued function from samples of input-output pairs consists of modeling the relationship between an input from a vector space and an output on a Riemannian manifold. We propose a function approximation method that…
A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems. The proposed algorithm is fully adaptive with at most ${\cal O} (\epsilon_g^{-3/2})$ iterations to achieve a…
We consider Newton methods for common types of single commodity and multi-commodity network flow problems. Despite the potentially very large dimension of the problem, they can be implemented using the conjugate gradient method and…
The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…
Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how…
This work is concerned with an inverse elastic scattering problem of identifying the unknown rigid obstacle embedded in an open space filled with a homogeneous and isotropic elastic medium. A Newton-type iteration method relying on the…
Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors in a connected network. We study decentralized second-order…
The present author recently proposed and proved a relationship theorem between nonlinear polynomial equations and the corresponding Jacobian matrix. By using this theorem, this paper derives a Newton iterative formula without requiring the…
In this paper, we use the optimization formulation of nonlinear Kalman filtering and smoothing problems to develop second-order variants of iterated Kalman smoother (IKS) methods. We show that Newton's method corresponds to a recursion over…
Fixed-point or Newton-methods are typically employed for the numerical solution of nonlinear systems arising from discretization of nonlinear magnetic field problems. We here discuss an alternative strategy which uses local Quasi-Newton…
Nonlinear inverse problems have complicated landscapes. Hence the calculation with naive iterative schemes (e.g., Gauss-Newton or conjugate gradients) is trapped in local minima. The (first) Born approximation can avoid this trapping but…
With the widespread applications of neural networks (NNs) trained on personal data, machine unlearning has become increasingly important for enabling individuals to exercise their personal data ownership, particularly the "right to be…
We study Newton type methods for inverse problems described by nonlinear operator equations $F(u)=g$ in Banach spaces where the Newton equations $F'(u_n;u_{n+1}-u_n) = g-F(u_n)$ are regularized variationally using a general data misfit…
Normalizing Flows (NFs) are universal density estimators based on Neural Networks. However, this universality is limited: the density's support needs to be diffeomorphic to a Euclidean space. In this paper, we propose a novel method to…
Motivated by the problem of learning a linear regression model whose parameter is a large fixed-rank non-symmetric matrix, we consider the optimization of a smooth cost function defined on the set of fixed-rank matrices. We adopt the…
The problem of minimizing a continuously differentiable convex function over an intersection of closed convex sets is ubiquitous in applied mathematics. It is particularly interesting when it is easy to project onto each separate set, but…