Related papers: Fej\'er* monotonicity in optimization algorithms
Submodularity is a fundamental phenomenon in combinatorial optimization. Submodular functions occur in a variety of combinatorial settings such as coverage problems, cut problems, welfare maximization, and many more. Therefore, a lot of…
In many naturally occurring optimization problems one needs to ensure that the definition of the optimization problem lends itself to solutions that are tractable to compute. In cases where exact solutions cannot be computed tractably, it…
This Note is inspired by the recent paper by Djafary Rouhani and Moradi [J. Optim. Theory Appl. 172 (2017) 222-235], where a proximal point algorithm proposed by Boikanyo and Moro\c{s}anu [Optim. Lett. 7 (2013) 415-420] is discussed. We…
Learned optimizers -- neural networks that are trained to act as optimizers -- have the potential to dramatically accelerate training of machine learning models. However, even when meta-trained across thousands of tasks at huge…
We propose a new class of primal-dual Fejer monotone algorithms for solving systems of com- posite monotone inclusions. Our construction is inspired by a framework used by Eckstein and Svaiter for the basic problem of finding a zero of the…
In many practical applications, heuristic or approximation algorithms are used to efficiently solve the task at hand. However their solutions frequently do not satisfy natural monotonicity properties of optimal solutions. In this work we…
This paper is devoted to the theoretical study of the efficiency, namely, stability of some greedy algorithms. In the greedy approximation theory researchers are mostly interested in the following two important properties of an algorithm --…
We deal with monotone inclusion problems of the form $0\in Ax+Dx+N_C(x)$ in real Hilbert spaces, where $A$ is a maximally monotone operator, $D$ a cocoercive operator and $C$ the nonempty set of zeros of another cocoercive operator. We…
In this paper, we focus on applications in machine learning, optimization, and control that call for the resilient selection of a few elements, e.g. features, sensors, or leaders, against a number of adversarial denial-of-service attacks or…
We provide a convergence result for sequences of random variables taking values in a metric space that satisfy a stochastic quasi-Fej\'er monotonicity condition, in the context of a (local) compactness assumption. Our result is quantitative…
We propose a linesearch projection algorithm for solving non-monotone and non-Lipschitzian equilibrium problems in Hilbert spaces. It is proved that the sequence generated by the proposed algorithm converges strongly to a solution of the…
This paper considers scheduling on identical machines. The scheduling objective considered in this paper generalizes most scheduling minimization problems. In the problem, there are $n$ jobs and each job $j$ is associated with a…
We propose in this paper a unifying scheme for several algorithms from the literature dedicated to the solving of monotone inclusion problems involving compositions with linear continuous operators in infinite dimensional Hilbert spaces. We…
We prove that the iterates produced by, either the scalar step size variant, or the coordinatewise variant of AdaGrad algorithm, are convergent sequences when applied to convex objective functions with Lipschitz gradient. The key insight is…
Theoretically exploring the advantages of neural networks might be one of the most challenging problems in the AI era. An adaptive feature program has recently been proposed to analyze feature learning, the characteristic property of neural…
In this work, we propose and study a framework of generalized proximal point algorithms associated with a maximally monotone operator. We indicate sufficient conditions on the regularization and relaxation parameters of generalized proximal…
In the last several years, the intimate connection between convex optimization and learning problems, in both statistical and sequential frameworks, has shifted the focus of algorithmic machine learning to examine this interplay. In…
This paper concerns models and convergence principles for dealing with stochasticity in a wide range of algorithms arising in nonlinear analysis and optimization in Hilbert spaces. It proposes a flexible geometric framework within which…
In this paper, we discuss about monotone vector fields, which is a typical extension to the theory of convex functions, by exploiting the tangent space structure. This new approach to monotonicity in CAT(0) spaces stands in opposed to the…
We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent with a variance reduction property that enables linear…