Related papers: Optimal error bounds for nonexpansive fixed-point …
Fixed point iterations play a central role in the design and the analysis of a large number of optimization algorithms. We study a new iterative scheme in which the update is obtained by applying a composition of quasinonexpansive operators…
In this paper, we introduce a new iteration method and show that this iteration method can be used to approximate fixed point of almost contraction mappings. Furthermore, we prove that the new iteration method is equivalent to both Mann…
In this article, we present the mathematical analysis of the convergence of the linearized Crank-Nicolson Galerkin method for a nonlinear Schrodinger problem related to a domain with a moving boundary. The convergence analysis of the…
Two optimization algorithms are proposed for solving a stochastic programming problem for which the objective function is given in the form of the expectation of convex functions and the constraint set is defined by the intersection of…
We study a classical iterative algorithm for balancing matrices in the $L_\infty$ norm via a scaling transformation. This algorithm, which goes back to Osborne and Parlett \& Reinsch in the 1960s, is implemented as a standard preconditioner…
Despite its important applications in Machine Learning, min-max optimization of nonconvex-nonconcave objectives remains elusive. Not only are there no known first-order methods converging even to approximate local min-max points, but the…
The proximal gradient algorithm has been popularly used for convex optimization. Recently, it has also been extended for nonconvex problems, and the current state-of-the-art is the nonmonotone accelerated proximal gradient algorithm.…
The maximum-marginal-a-posteriori success rate of statistical decision under multivariate Gaussian error distribution on an integer lattice is almost rigorously calculated by using union-bound approximation and Monte Carlo integration.…
In this paper, we develop a novel accelerated fixed-point-based framework using delayed inexact oracles to approximate a fixed point of a nonexpansive operator (or equivalently, a root of a co-coercive operator), a central problem in…
Regularization by the Shannon entropy enables us to efficiently and approximately solve optimal transport problems on a finite set. This paper is concerned with regularized optimal transport problems via Bregman divergence. We introduce the…
The question of fast convergence in the classical problem of high dimensional linear regression has been extensively studied. Arguably, one of the fastest procedures in practice is Iterative Hard Thresholding (IHT). Still, IHT relies…
The aim of this paper is to prove that, in an appropriate setting, every iterative sequence generated by the hybrid steepest descent method is convergent whenever so is every iterative sequence generated by the Halpern type iterative…
From optimal transport to robust dimensionality reduction, a plethora of machine learning applications can be cast into the min-max optimization problems over Riemannian manifolds. Though many min-max algorithms have been analyzed in the…
We propose a Riemannian version of Nesterov's Accelerated Gradient algorithm (RAGD), and show that for geodesically smooth and strongly convex problems, within a neighborhood of the minimizer whose radius depends on the condition number as…
Fast Incremental Expectation Maximization (FIEM) is a version of the EM framework for large datasets. In this paper, we first recast FIEM and other incremental EM type algorithms in the {\em Stochastic Approximation within EM} framework.…
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic…
Piecewise linearization is a key technique for solving nonlinear problems in transportation network design and other optimization fields, in which generating breakpoints is a fundamental task. This paper proposes an optimal breakpoint…
In unconstrained optimisation on an Euclidean space, to prove convergence in Gradient Descent processes (GD) $x_{n+1}=x_n-\delta _n \nabla f(x_n)$ it usually is required that the learning rates $\delta _n$'s are bounded: $\delta _n\leq…
We give a rate of metastability for Halpern's iteration relative to a rate of metastability for the resolvent for nonexpansive mappings in uniformly smooth Banach spaces, extracted from a proof due to Xu. In Hilbert space, the latter is…
Recent applications that arise in machine learning have surged significant interest in solving min-max saddle point games. This problem has been extensively studied in the convex-concave regime for which a global equilibrium solution can be…