Related papers: IRKA is a Riemannian Gradient Descent Method

We propose an inexact optimization algorithm on Riemannian manifolds, motivated by quadratic discrimination tasks in high-dimensional, low-sample-size (HDLSS) imaging settings. In such applications, gradient evaluations are often biased due…

Derivative-free Riemannian optimization (DFRO) aims to minimize an objective function using only function evaluations, under the constraint that the decision variables lie on a Riemannian manifold. The rapid increase in problem dimensions…

In this paper we present deflation and augmentation techniques that have been designed to accelerate the convergence of Krylov subspace methods for the solution of linear systems of equations. We review numerical approaches both for linear…

This work introduces a novel algorithm to solve large-scale eigenvalue problems and seek a small set of eigenpairs. The method, called randomized Krylov-Schur (rKS), has a simple implementation and benefits from fast and efficient…

Stacking, a heuristic technique for training deep residual networks by progressively increasing the number of layers and initializing new layers by copying parameters from older layers, has proven quite successful in improving the…

Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in…

Inverse reinforcement learning (IRL) is the problem of finding a reward function that generates a given optimal policy for a given Markov Decision Process. This paper looks at an algorithmic-independent geometric analysis of the IRL problem…

The iterated Arnoldi-Tikhonov (iAT) method is a regularization technique particularly suited for solving large-scale ill-posed linear inverse problems. Indeed, it reduces the computational complexity through the projection of the…

Existing implementations of gradient-based optimisation methods typically assume that the problem is posed in Euclidean space. When solving optimality problems on function spaces, the functional derivative is then inaccurately represented…

The Iterative Filtering method is a technique developed recently for the decomposition and analysis of non-stationary and non-linear signals. In this work we propose two alternative formulations of the original algorithm which allows to…

We propose a practical implementation of high-order fully implicit Runge-Kutta(IRK) methods in a multiple precision floating-point environment. Although implementations based on IRK methods in an IEEE754 double precision environment have…

The incremental gradient method is a prominent algorithm for minimizing a finite sum of smooth convex functions, used in many contexts including large-scale data processing applications and distributed optimization over networks. It is a…

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical…

The Randomized Kaczmarz method (RK) is a stochastic iterative method for solving linear systems that has recently grown in popularity due to its speed and low memory requirement. Selectable Set Randomized Kaczmarz (SSRK) is an variant of RK…

In this paper, we propose an implicit gradient descent algorithm for the classic $k$-means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch…

A discrete rotation algorithm can be apprehended as a parametric application $f\_\alpha$ from $\ZZ[i]$ to $\ZZ[i]$, whose resulting permutation ``looks like'' the map induced by an Euclidean rotation. For this kind of algorithm, to be…

The gradient descent method aims at finding local minima of a given multivariate function by moving along the direction of its gradient, and hence, the algorithm typically involves computing all partial derivatives of a given function,…

The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates…

Minimizing the empirical risk is a popular training strategy, but for learning tasks where the data may be noisy or heavy-tailed, one may require many observations in order to generalize well. To achieve better performance under less…

This paper describes a new MATLAB software package of iterative regularization methods and test problems for large-scale linear inverse problems. The software package, called IR Tools, serves two related purposes: we provide implementations…