Related papers: A link between the steepest descent method and fix…
In this work, we analyze the regularizing property of the stochastic gradient descent for the efficient numerical solution of a class of nonlinear ill-posed inverse problems in Hilbert spaces. At each step of the iteration, the method…
We propose a new approach to solving bilevel optimization problems, intermediate between solving full-system optimality conditions with a Newton-type approach, and treating the inner problem as an implicit function. The overall idea is to…
Randomness is ubiquitous in modern engineering. The uncertainty is often modeled as random coefficients in the differential equations that describe the underlying physics. In this work, we describe a two-step framework for numerically…
This paper considers a class of distributed resource allocation problems where each agent privately holds a smooth, potentially non-convex local objective, subject to a globally coupled equality constraint. Built upon the existing method,…
Discrete gradient methods are geometric integration techniques that can preserve the dissipative structure of gradient flows. Due to the monotonic decay of the function values, they are well suited for general convex and nonconvex…
We establish sharp energy decay rates for a large class of nonlinearly first-order damped systems, and we design discretization schemes that inherit of the same energy decay rates, uniformly with respect to the space and/or time…
Even for the gradient descent (GD) method applied to neural network training, understanding its optimization dynamics, including convergence rate, iterate trajectories, function value oscillations, and especially its implicit acceleration,…
In this work, we study a novel class of projection-based algorithms for linearly constrained problems (LCPs) which have a lot of applications in statistics, optimization, and machine learning. Conventional primal gradient-based methods for…
We investigate relaxations for a class of discrete bilevel programs where the interaction constraints linking the leader and the follower are linear. Our approach reformulates the upper-level optimality constraints by projecting the…
The purpose of this paper is to introduce two new classes of accelerated distributed proximal conjugate gradient algorithms for multi-agent constrained optimization problems; given as minimization of a function decomposed as a sum of M…
We consider regression problems with binary weights. Such optimization problems are ubiquitous in quantized learning models and digital communication systems. A natural approach is to optimize the corresponding Lagrangian using variants of…
A generalized conditional gradient method for minimizing the sum of two convex functions, one of them differentiable, is presented. This iterative method relies on two main ingredients: First, the minimization of a partially linearized…
This paper is concerned with the rigorous analysis of a recently proposed model of Zheng et. al. for describing nematic liquid crystals within the dense regime, with the orientation distribution function as the variable. A key feature of…
The $\ell_p$ regularization problem with $0< p< 1$ has been widely studied for finding sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. The proximal gradient…
This paper investigates the comparative performance of two fundamental approaches to solving linear regression problems: the closed-form Moore-Penrose pseudoinverse and the iterative gradient descent method. Linear regression is a…
In this work, we have discretized a system of time-dependent nonlinear convection-diffusion-reaction equations with the virtual element method over the spatial domain and the Euler method for the temporal interval. For the nonlinear…
We explore probability modelling of discretization uncertainty for system states defined implicitly by ordinary or partial differential equations. Accounting for this uncertainty can avoid posterior under-coverage when likelihoods are…
The paper deals with the optimal control problem described by second order evolution differential inclusions; to this end first we use an auxiliary problem with second order discrete and discrete-approximate inclusions. Then applying…
This paper considers sufficient descent Riemannian conjugate gradient methods with line search algorithms. We propose two kinds of sufficient descent nonlinear conjugate gradient methods and prove these methods satisfy the sufficient…
In this paper, we introduce a new iterative method which we call one step back approach: the main idea is to anticipate the consequence of the iterative computation per coordinate and to optimize on the choice of the sequence of the…