Related papers: Online and stochastic Douglas-Rachford splitting m…
We study the problem of online learning and online regret minimization when samples are drawn from a general unknown non-stationary process. We introduce the concept of a dynamic changing process with cost $K$, where the conditional…
We consider finite Markov decision processes (MDPs) with convex constraints and known dynamics. In principle, this problem is amenable to off-the-shelf convex optimization solvers, but typically this approach suffers from poor scalability.…
Online learning methods yield sequential regret bounds under minimal assumptions and provide in-expectation risk bounds for statistical learning. However, despite the apparent advantage of online guarantees over their statistical…
We propose an online data compression approach for efficiently solving distributionally robust optimization (DRO) problems with streaming data while maintaining out-of-sample performance guarantees. Our method dynamically constructs…
In this paper, we develop a unified framework for analyzing the tracking error and dynamic regret of inexact online optimization methods under a variety of settings. Specifically, we leverage the quadratic constraint approach from control…
Maximizing a monotone submodular function is a fundamental task in machine learning, economics, and statistics. In this paper, we present two communication-efficient decentralized online algorithms for the monotone continuous DR-submodular…
We introduce a general framework of stochastic online convex optimization to obtain fast-rate stochastic regret bounds. We prove that algorithms such as online newton steps and a scale-free 10 version of Bernstein online aggregation achieve…
We propose a methodology for studying the performance of common splitting methods through semidefinite programming. We prove tightness of the methodology and demonstrate its value by presenting two applications of it. First, we use the…
The Douglas-Rachford splitting method is a classical and widely used algorithm for solving monotone inclusions involving the sum of two maximally monotone operators. It was recently shown to be the unique frugal, no-lifting…
Splitting schemes are a class of powerful algorithms that solve complicated monotone inclusion and convex optimization problems that are built from many simpler pieces. They give rise to algorithms in which the simple pieces of the…
Douglas-Rachford Splitting (DRS) methods based on the proximal point algorithms for the Poisson and Gaussian log-likelihood functions are proposed for ptychography and phase retrieval. Fixed point analysis shows that the DRS iterated…
In this paper, we consider an online distributed composite optimization problem over a time-varying multi-agent network that consists of multiple interacting nodes, where the objective function of each node consists of two parts: a loss…
Randomized iterative methods have gained recent interest in machine learning and signal processing for solving large-scale linear systems. One such example is the randomized Douglas-Rachford (RDR) method, which updates the iterate by…
Non-stationary online learning has drawn much attention in recent years. Despite considerable progress, dynamic regret minimization has primarily focused on convex functions, leaving the functions with stronger curvature (e.g., squared or…
Decentralized Online Learning (online learning in decentralized networks) attracts more and more attention, since it is believed that Decentralized Online Learning can help the data providers cooperatively better solve their online problems…
We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting…
Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online…
In this work, we aim to establish the exact worst-case convergence rates of Douglas--Rachford splitting (DRS) and Davis--Yin splitting (DYS) when applied to convex optimization problems. Both DRS and DYS have two variants as swapping the…
To deal with non-stationary online problems with complex constraints, we investigate the dynamic regret of online Frank-Wolfe (OFW), which is an efficient projection-free algorithm for online convex optimization. It is well-known that in…
We address the solution of time-varying optimization problems characterized by the sum of a time-varying strongly convex function and a time-invariant nonsmooth convex function. We design an online algorithmic framework based on…