Related papers: First-Order Averaging Principles for Maps with App…
A set of accelerated first order algorithms with memory are proposed for minimising strongly convex functions. The algorithms are differentiated by their use of the iterate history for the gradient step. The increased convergence rate of…
The usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant $L$. However, in many settings the…
The time discrete scheme of characteristics type is especially effective for convection-dominated diffusion problems. The scheme has been used in various engineering areas with different approximations in spatial direction. The lowest-order…
These notes focus on the minimization of convex functionals using first-order optimization methods, which are fundamental in many areas of applied mathematics and engineering. The primary goal of this document is to introduce and analyze…
Consider $\Xi$ a homogeneous Poisson point process on $\mathbb{R}^d$ ($d\geq 2$) with unit intensity with respect to the Lebesgue measure. For $\varepsilon\geq 0$, we define the Boolean model $\Sigma_{p, \varepsilon}$ as the union of the…
Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained bilevel problems, the constrained setting remains relatively…
A framework is developed for applying accelerated methods to general hyperbolic programming, including linear, second-order cone, and semidefinite programming as special cases. The approach replaces a hyperbolic program with a convex…
We study a random dynamical system such that one transformation is randomly selected from a family of transformations and then applied on each iteration. For such random dynamical systems, we consider estimates of absolutely continuous…
We consider a class of popular distributed non-convex optimization problems, in which agents connected by a network $\mathcal{G}$ collectively optimize a sum of smooth (possibly non-convex) local objective functions. We address the…
The stochastic Auxiliary Problem Principle (APP) algorithm is a general Stochastic Approximation (SA) scheme that turns the resolution of an original optimization problem into the iterative resolution of a sequence of auxiliary problems.…
The work [8] established memory loss in the time-dependent (non-random) case of uniformly expanding maps of the interval. Here we find conditions under which we have convergence to the normal distribution of the appropriately scaled…
Using an optimization algorithm to solve a machine learning problem is one of mainstreams in the field of science. In this work, we demonstrate a comprehensive comparison of some state-of-the-art first-order optimization algorithms for…
We present a new class of preconditioned iterative methods for solving linear systems of the form $Ax = b$. Our methods are based on constructing a low-rank Nystr\"om approximation to $A$ using sparse random matrix sketching. This…
We present a simple scheme for restarting first-order methods for convex optimization problems. Restarts are made based only on achieving specified decreases in objective values, the specified amounts being the same for all optimization…
This work is devoted to averaging principle of a two-time-scale stochastic partial differential equation on a bounded interval $[0, l]$, where both the fast and slow components are directly perturbed by additive noises. Under some regular…
In this paper, we give explicit estimates that insure the existence of solutions for first order partial differential operators on compact manifolds, using a viscosity method. In the linear case, an explicit integral formula can be found,…
In this paper, we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems, and variational inequalities. This framework allows obtaining many…
Diffusion models, which iteratively denoise data samples to synthesize high-quality outputs, have achieved empirical success across domains. However, optimizing these models for downstream tasks often involves nested bilevel structures,…
Understanding the optimization dynamics of neural networks is necessary for closing the gap between theory and practice. Stochastic first-order optimization algorithms are known to efficiently locate favorable minima in deep neural…
This paper presents a framework for designing a class of distributed, asynchronous optimization algorithms, realized as signal processing architectures utilizing various conservation principles. The architectures are specifically based on…