Related papers: Variance-Reduced Fast Operator Splitting Methods f…
Despite the rise to fame of incremental variance-reduced methods in recent years, their use in nonsmooth optimization is still limited to few simple cases. This is due to the fact that existing methods require to evaluate the proximity…
We introduce a hybrid stochastic estimator to design stochastic gradient algorithms for solving stochastic optimization problems. Such a hybrid estimator is a convex combination of two existing biased and unbiased estimators and leads to…
We introduce new planning and reinforcement learning algorithms for discounted MDPs that utilize an approximate model of the environment to accelerate the convergence of the value function. Inspired by the splitting approach in numerical…
Classical optimization is a cornerstone of the success of variational quantum algorithms, which often require determining the derivatives of the cost function relative to variational parameters. The computation of the cost function and its…
This paper investigates a category of constrained fractional optimization problems that emerge in various practical applications. The objective function for this category is characterized by the ratio of a numerator and denominator, both…
We propose a forward-backward splitting dynamical system for solving inclusion problems of the form $0\in A(x)+B(x)$ in Hilbert spaces, where $A$ is a maximal operator and $B$ is a single-valued operator. Involved operators are assumed to…
This paper provides a comprehensive study of the nonmonotone forward-backward splitting (FBS) method for solving a class of nonsmooth composite problems in Hilbert spaces. The objective function is the sum of a Fr\'echet differentiable (not…
Minimizing PDE-residual losses is a common strategy to promote physical consistency in neural operators. However, standard formulations often lack variational correctness, meaning that small residuals do not guarantee small solution errors…
Alternating Direction Method of Multipliers (ADMM) is a popular method for solving large-scale Machine Learning problems. Stochastic ADMM was proposed to reduce the per iteration computational complexity, which is more suitable for big data…
We present adaptive gradient methods (both basic and accelerated) for solving convex composite optimization problems in which the main part is approximately smooth (a.k.a. $(\delta, L)$-smooth) and can be accessed only via a (potentially…
We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate the communication bottleneck, recent work proposed various schemes…
We develop two "Nesterov's accelerated" variants of the well-known extragradient method to approximate a solution of a co-hypomonotone inclusion constituted by the sum of two operators, where one is Lipschitz continuous and the other is…
In this paper, we consider solving a class of convex optimization problem which minimizes the sum of three convex functions $f(x)+g(x)+h(Bx)$, where $f(x)$ is differentiable with a Lipschitz continuous gradient, $g(x)$ and $h(x)$ have a…
Machine learning approaches relying on such criteria as adversarial robustness or multi-agent settings have raised the need for solving game-theoretic equilibrium problems. Of particular relevance to these applications are methods targeting…
This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It introduces an algorithm designed to optimize differentiable objective functions $f$ whose values can…
In this work, we propose low-complexity adaptive biased estimation algorithms, called group-based shrinkage estimators (GSEs), for parameter estimation and interference suppression scenarios with mechanisms to automatically adjust the…
A variety of complex biological, natural and man-made systems exhibit non-Markovian dynamics that can be modeled through fractional order differential equations, yet, we lack sample comlexity aware system identification strategies. Towards…
This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic…
Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, $\ell_1$ norm regularized optimization, and $\ell_0$ norm regularized…
We propose a quasi-random operator splitting method for evolution equations driven by multiple mechanisms. The method uses a low-discrepancy sequence to generate the ordering of the subflows, while requiring only one application of each…