Related papers: A Field Guide to Forward-Backward Splitting with a…
We consider the problem of reconstructing an infinite set of sparse, finite-dimensional vectors, that share a common sparsity pattern, from incomplete measurements. This is in contrast to the work [17], where the single vector signal can be…
Features based on sparse representation, especially using the synthesis dictionary model, have been heavily exploited in signal processing and computer vision. However, synthesis dictionary learning typically involves NP-hard sparse coding…
The ability to learn new concepts continually is necessary in this ever-changing world. However, deep neural networks suffer from catastrophic forgetting when learning new categories. Many works have been proposed to alleviate this…
Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks. However, due to its nested structure, evaluating exact gradients for high-dimensional problems is…
A Forster transform is an operation that turns a distribution into one with good anti-concentration properties. While a Forster transform does not always exist, we show that any distribution can be efficiently decomposed as a disjoint…
Advanced feedforward control methods enable mechatronic systems to perform varying motion tasks with extreme accuracy and throughput. The aim of this paper is to develop a data-driven feedforward controller that addresses input…
The "Inertial Forward-Backward algorithm" (IFB) is a powerful tool for convex nonsmooth minimization problems, it gives the well known "fast iterative shrinkage-thresholding algorithm " (FISTA), which enjoys $O\left( {\frac{1}{{{k^2}}}}…
This work explores a novel approach for adaptive, differentiable parametrization of large-scale non-stationary random fields. Coupled with any gradient-based algorithm, the method can be applied to variety of optimization problems,…
The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an…
Clustering is an essential data mining tool for analyzing and grouping similar objects. In big data applications, however, many clustering algorithms are infeasible due to their high memory requirements and/or unfavorable runtime…
The aim of this paper is to present a new fast-convergent numerically stable space-time adaptive processing (STAP) algorithm derived using a novel technique of feedback orthogonalization. The main advantages of this approach lie in its…
Convex nonsmooth optimization problems, whose solutions live in very high dimensional spaces, have become ubiquitous. To solve them, the class of first-order algorithms known as proximal splitting algorithms is particularly adequate: they…
We develop two variance-reduced fast operator splitting methods to approximate solutions of a class of generalized equations, covering fundamental problems such as \rvs{minimization}, minimax problems, and variational inequalities as…
Nowadays, Large Language Models (LLMs) have been gradually employed to solve complex tasks. To face the challenge, task decomposition has become an effective way, which proposes to divide a complex task into multiple simpler subtasks and…
Self-supervised learning has brought about a revolutionary paradigm shift in various computing domains, including NLP, vision, and biology. Recent approaches involve pre-training transformer models on vast amounts of unlabeled data, serving…
We consider Lie and Strang splitting for the time integration of constrained partial differential equations with a nonlinear reaction term. Since such systems are known to be sensitive with respect to perturbations, the splitting procedure…
In this work, we propose some new Douglas-Rashford splitting algorithms for solving a class of generalized DC (difference of convex functions) in real Hilbert spaces. The proposed methods leverage the proximal properties of the nonsmooth…
Auto-regressive Large Language Models (LLMs) demonstrate remarkable performance across different domains such as vision and language processing. However, due to sequential processing through a stack of transformer layers, autoregressive…
The goal of this paper is to present two algorithms for solving systems of inclusion problems, with all component of the systems being a sum of two maximal monotone operators. The algorithms are variants of the forward-backward splitting…
We propose and study a weakly convergent variant of the forward--backward algorithm for solving structured monotone inclusion problems. Our algorithm features a per-iteration deviation vector which provides additional degrees of freedom.…