Related papers: A Hybrid Algorithm for Monotone Variational Inequa…

In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification,…

We address composite optimization problems, which consist in minimizing the sum of a smooth and a merely lower semicontinuous function, without any convexity assumptions. Numerical solutions of these problems can be obtained by proximal…

We introduce a novel algorithm for gradient-based optimization of stochastic objective functions. The method may be seen as a variant of SGD with momentum equipped with an adaptive learning rate automatically adjusted by an 'energy'…

Stochastic variance reduced optimization methods are known to be globally convergent while they suffer from slow local convergence, especially when moderate or high accuracy is needed. To alleviate this problem, we propose an optimization…

Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training neural networks is a challenging task in many applications. The prevalent…

Through theoretical and experimental validation, unlike all existing adaptive methods like Adam which penalize frequently-changing parameters and are only applicable to sparse gradients, we propose the simplest SGD enhanced method,…

An adaptive proximal method for a special class of variational inequalities and related problems is proposed. For example, the so-called mixed variational inequalities and composite saddle problems are considered. Some estimates of the…

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many…

Variance reduction is a crucial tool for improving the slow convergence of stochastic gradient descent. Only a few variance-reduced methods, however, have yet been shown to directly benefit from Nesterov's acceleration techniques to match…

In this paper an explicit algorithm is proposed for solving an equilibrium problem whose associated bifunction is pseudomonotone and satisfies a Lipschitz-type condition. Contrary to many algorithms, our algorithm is done without using…

In this paper, we design a new iterative algorithm for solving pseudomonotone equilibrium problems in real Hilbert spaces. The advantage of our algorithm is that it requires only one strongly convex programming problem at each iteration.…

We introduce and study the convergence properties of a projection-type algorithm for solving the variational inequality problem for point-to-set operators. No monotoni\-city assumption is used in our analysis. The operator defining the…

This paper is devoted to the variational inequality problems. We consider two classes of problems, the first is classical constrained variational inequality and the second is the same problem with functional (inequality type) constraints.…

Tracking the solution of time-varying variational inequalities is an important problem with applications in game theory, optimization, and machine learning. Existing work considers time-varying games or time-varying optimization problems.…

This paper considers stochastic monotone variational inequalities whose feasible region is the intersection of a (possibly infinite) number of convex functional level sets. A projection-based approach or direct Lagrangian-based techniques…

Variational inequalities play a key role in machine learning research, such as generative adversarial networks, reinforcement learning, adversarial training, and generative models. This paper is devoted to the constrained variational…

In this paper, we discuss variational inequality (VI) problems without monotonicity from the perspective of convergence of projection-type algorithms. In particular, we identify existing conditions as well as present new conditions that are…

Variational inequalities are a formalism that includes games, minimization, saddle point, and equilibrium problems as special cases. Methods for variational inequalities are therefore universal approaches for many applied tasks, including…

Based on SGD, previous works have proposed many algorithms that have improved convergence speed and generalization in stochastic optimization, such as SGDm, AdaGrad, Adam, etc. However, their convergence analysis under non-convex conditions…

In this paper, we introduce and study a class of resolvent dynamical systems to investigate some inertial proximal methods for solving mixed variational inequalities. These proposed methods along with their discretizations and derived rates…