Related papers: Iterative Descent Method for Generalized Leontief …
The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…
When an inverse problem is solved by a gradient-based optimization algorithm, the corresponding forward and adjoint problems, which are introduced to compute the gradient, can be also solved iteratively. The idea of iterating at the same…
We study the finite convergence of iterative methods for solving convex feasibility problems. Our key assumptions are that the interior of the solution set is nonempty and that certain overrelaxation parameters converge to zero, but with a…
In this paper, we will present a generalization for a minimization problem from I. Daubechies, M. Defrise, and C. Demol [3]. This generalization is useful for solving many practical problems in which more than one constraint are involved.…
A generalized conditional gradient method for minimizing the sum of two convex functions, one of them differentiable, is presented. This iterative method relies on two main ingredients: First, the minimization of a partially linearized…
Consider the consensus problem of minimizing $f(x)=\sum_{i=1}^n f_i(x)$ where each $f_i$ is only known to one individual agent $i$ out of a connected network of $n$ agents. All the agents shall collaboratively solve this problem and obtain…
In this paper, we propose a generalized conditional gradient method for multiobjective optimization, which can be viewed as an improved extension of the classical Frank-Wolfe (conditional gradient) method for single-objective optimization.…
In this work we are interested in general linear inverse problems where the corresponding forward problem is solved iteratively using fixed point methods. Then one-shot methods, which iterate at the same time on the forward problem solution…
Solutions to differential equations, which are used to model physical systems, are computed numerically by solving a set of discretized equations. This set of discretized equations is reduced to a large linear system, whose solution is…
We consider the convergence of iterative solvers for problems of nonlinear magnetostatics. Using the equivalence to an underlying minimization problem, we can establish global linear convergence of a large class of methods, including the…
In this paper, we develop a global descent method for non-convex multi-objective optimization problems. The proposed approach builds upon foundational concepts from single-objective global descent techniques while removing the need for…
Gradient descent is one of the most widely used iterative algorithms in modern statistical learning. However, its precise algorithmic dynamics in high-dimensional settings remain only partially understood, which has limited its broader…
We prove the familiar Lazy Online Gradient Descent algorithm is universal on polytope domains. That means it gets $O(1)$ pseudo-regret against i.i.d opponents, while simultaneously achieving the well-known $O(\sqrt N)$ worst-case regret…
In this article, we extend our previous work (Applicable Analysis, 2024, pp. 1-25) on the steepest descent method for uncertain multiobjective optimization problems. While that study established local convergence, it did not address global…
We propose a new approach to solving bilevel optimization problems, intermediate between solving full-system optimality conditions with a Newton-type approach, and treating the inner problem as an implicit function. The overall idea is to…
In distributed training of machine learning models, gradient descent with local iterative steps, commonly known as Local (Stochastic) Gradient Descent (Local-(S)GD) or Federated averaging (FedAvg), is a very popular method to mitigate…
In this article we propose a descent method for equality and inequality constrained multiobjective optimization problems (MOPs) which generalizes the steepest descent method for unconstrained MOPs by Fliege and Svaiter to constrained…
We propose a novel iterative numerical method to solve the three-dimensional inverse obstacle scattering problem of recovering the shape of the obstacle from far-field measurements. To address the inherent ill-posed nature of the inverse…
Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set. For such challenging instances, we develop a new…
We propose a general modeling and algorithmic framework for discrete structure recovery that can be applied to a wide range of problems. Under this framework, we are able to study the recovery of clustering labels, ranks of players, signs…