相关论文: IRKA is a Riemannian Gradient Descent Method

We develop a structure-preserving system-theoretic model reduction framework for nonlinear power grid networks. First, via a lifting transformation, we convert the original nonlinear system with trigonometric nonlinearities to an equivalent…

In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model…

Gradient descent is a widely used iterative algorithm for finding local minima in multivariate functions. However, the final iterations often either overshoot the minima or make minimal progress, making it challenging to determine an…

The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate…

Projected gradient descent and its Riemannian variant belong to a typical class of methods for low-rank matrix estimation. This paper proposes a new Nesterov's Accelerated Riemannian Gradient algorithm by efficient orthographic retraction…

The iterated Crank-Nicolson (ICN) method is a successful numerical algorithm in numerical relativity for solving partial differential equations. The $\theta$-ICN method is the extension of the original ICN method where $\theta$ is the…

Bilinear dynamical systems are commonly used in science and engineering because they form a bridge between linear and non-linear systems. However, simulating them is still a challenge because of their large size. Hence, a lot of research is…

The purpose of this paper is to introduce a new Kirk type iterative algorithm called Kirk multistep iteration and to study its convergence. We also prove some theorems related with the stability results for the Kirk-multistep and Kirk-SP…

Depending on the frequency range of interest, finite element-based modeling of acoustic problems leads to dynamical systems with very high dimensional state spaces. As these models can mostly be described with second order linear dynamical…

Efficient and accurate low-rank approximation (LRA) methods are of great significance for large-scale data analysis. Randomized tensor decompositions have emerged as powerful tools to meet this need, but most existing methods perform poorly…

Natural Gradient Descent, a second-degree optimization method motivated by the information geometry, makes use of the Fisher Information Matrix instead of the Hessian which is typically used. However, in many cases, the Fisher Information…

Information geometry applies concepts in differential geometry to probability and statistics and is especially useful for parameter estimation in exponential families where parameters are known to lie on a Riemannian manifold. Connections…

Iterative gradient-based optimization algorithms are widely used to solve difficult or large-scale optimization problems. There are many algorithms to choose from, such as gradient descent and its accelerated variants such as Polyak's Heavy…

In this paper, we investigate interpolatory projection framework for model reduction of descriptor systems. With a simple numerical example, we first illustrate that employing subspace conditions from the standard state space settings to…

An arc-search interior-point method is a type of interior-point methods that approximates the central path by an ellipsoidal arc, and it can often reduce the number of iterations. In this work, to further reduce the number of iterations and…

We are concerned with the efficient implementation of symplectic implicit Runge-Kutta (IRK) methods applied to systems of (non-necessarily Hamiltonian) ordinary differential equations by means of Newton-like iterations. We pay particular…

Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient descent implicitly regularize models by penalizing gradient…

Gradient descent methods are fundamental first-order optimization algorithms in both Euclidean spaces and Riemannian manifolds. However, the exact gradient is not readily available in many scenarios. This paper proposes a novel inexact…

This paper presents the first optimal-rate $p$-th order methods with $p\geq 1$ for finding first and second-order stationary points of non-convex smooth objective functions over Riemannian manifolds. In contrast to the geodesically convex…

We investigate the regularizing behavior of an iterative Krylov subspace method for the solution of linear inverse problems in precisions lower than double. Recent works have considered the projection of iterated Tikhonov methods using…