Related papers: Toward Designing Convergent Deep Operator Splittin…
The key task of machine learning is to minimize the loss function that measures the model fit to the training data. The numerical methods to do this efficiently depend on the properties of the loss function. The most decisive among these…
A sequential piecewise linear programming method is presented where bounded domains of non-convex functions are successively contracted about the solution of a piecewise linear program at each iteration of the algorithm. Although…
This paper investigates the problems large-scale distributed composite convex optimization, with motivations from a broad range of applications, including multi-agent systems, federated learning, smart grids, wireless sensor networks,…
Multi-task learning (MTL) algorithms typically rely on schemes that combine different task losses or their gradients through weighted averaging. These methods aim to find Pareto stationary points by using heuristics that require access to…
We propose a unifying algorithm for non-smooth non-convex optimization. The algorithm approximates the objective function by a convex model function and finds an approximate (Bregman) proximal point of the convex model. This approximate…
In this work, we explore the use of operator splitting algorithms for solving regularized structural topology optimization problems. The context is the classical structural design problems (e.g., compliance minimization and compliant…
We present a method for constructing numerical schemes with up to 3rd strong convergence order for solution of a class of stochastic differential equations, including equations of the Langevin type. The construction proceeds in two stages.…
Split learning recently emerged as a solution for distributed machine learning with heterogeneous IoT devices, where clients can offload part of their training to computationally-powerful helpers. The core challenge in split learning is to…
The recently developed Distributed Block Proximal Method, for solving stochastic big-data convex optimization problems, is studied in this paper under the assumption of constant stepsizes and strongly convex (possibly non-smooth) local…
As the complexity of learning tasks surges, modern machine learning encounters a new constrained learning paradigm characterized by more intricate and data-driven function constraints. Prominent applications include Neyman-Pearson…
The linearized Bregman method is a method to calculate sparse solutions to systems of linear equations. We formulate this problem as a split feasibility problem, propose an algorithmic framework based on Bregman projections and prove a…
The Douglas-Rachford splitting method is a classical and widely used algorithm for solving monotone inclusions involving the sum of two maximally monotone operators. It was recently shown to be the unique frugal, no-lifting…
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level Strong Convexity (LLSC) condition or on solving a series…
We consider the problem of minimizing a sum of several convex non-smooth functions. We introduce a new algorithm called the selective linearization method, which iteratively linearizes all but one of the functions and employs simple…
In this paper we introduce a numerical method for nonlinear parabolic PDEs that combines operator splitting with deep learning. It divides the PDE approximation problem into a sequence of separate learning problems. Since the computational…
In this paper, we focus on the nonconvex-strongly-convex bilevel optimization problem (BLO). In this BLO, the objective function of the upper-level problem is nonconvex and possibly nonsmooth, and the lower-level problem is smooth and…
Operator splitting methods allow to split the operator describing a complex dynamical system into a sequence of simpler subsystems and treat each part independently. In the modeling of dynamical problems, systems of (possibly coupled)…
We address the problem of multiple local optima arising due to non-convex objective functions in cooperative multi-agent optimization problems. To escape such local optima, we propose a systematic approach based on the concept of boosting…
Split computing ($\neq$ split learning) is a promising approach to deep learning models for resource-constrained edge computing systems, where weak sensor (mobile) devices are wirelessly connected to stronger edge servers through channels…
A class of monotone operator equations, which can be decomposed into sum of the gradient of a strongly convex function and a linear and skew-symmetric operator, is considered in this work. Based on discretization of the generalized gradient…