Related papers: Metastability of the proximal point algorithm with…

Ample empirical evidence in deep neural network training suggests that a variety of optimizers tend to find nearly global optima. In this article, we adopt the reversed perspective that convergence to an arbitrary point is assumed rather…

In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practical impact is clear, as well as their theoretical properties.…

Focus of this work is solving a non-smooth constraint minimization problem by a primal-dual splitting algorithm involving proximity operators. The problem is penalized by the Bregman divergence associated with the non-smooth total variation…

We study a conical extension of averaged nonexpansive operators and the role it plays in convergence analysis of fixed point algorithms. Various properties of conically averaged operators are systematically investigated, in particular, the…

Two distributed algorithms are described that enable all users connected over a network to cooperatively solve the problem of minimizing the sum of all users' objective functions over the intersection of all users' constraint sets, where…

We study a generalized framework for structured sparsity. It extends the well-known methods of Lasso and Group Lasso by incorporating additional constraints on the variables as part of a convex optimization problem. This framework provides…

In [C.W. Gear, T.J. Kaper, I.G. Kevrekidis, and A. Zagaris, Projecting to a Slow Manifold: Singularly Perturbed Systems and Legacy Codes, SIAM J. Appl. Dyn. Syst. 4 (2005) 711-732], we developed a class of iterative algorithms within the…

(Stochastic) bilevel optimization is a frequently encountered problem in machine learning with a wide range of applications such as meta-learning, hyper-parameter optimization, and reinforcement learning. Most of the existing studies on…

Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case. The limiting performance of such estimators depends on the…

Primal-dual splitting involving proximity operators in order to be able to find some approximation to the minimizer for a general form of Tikhonov type functional is in the focus of this work. This approximation is produced by a pair of…

In this paper, we propose a systematic approach for extending first-order optimization algorithms, originally designed for unconstrained strongly convex problems, to handle closed and convex set constraints. We show that the resulting…

Iterative algorithms based on thresholding, feedback and null space tuning (NST+HT+FB) for sparse signal recovery are exceedingly effective and fast, particularly for large scale problems. The core algorithm is shown to converge in finitely…

Research on bias in machine learning algorithms has generally been concerned with the impact of bias on predictive accuracy. We believe that there are other factors that should also play a role in the evaluation of bias. One such factor is…

We consider the model introduced by Bilu and Linial (2010), who study problems for which the optimal clustering does not change when distances are perturbed. They show that even when a problem is NP-hard, it is sometimes possible to obtain…

We investigate the asymptotic behavior of sequences generated by the proximal point algorithm for convex functions in complete geodesic spaces with curvature bounded above. Using the notion of resolvents of such functions, which was…

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a…

In this paper, we consider a network of processors aiming at cooperatively solving mixed-integer convex programs subject to uncertainty. Each node only knows a common cost function and its local uncertain constraint set. We propose a…

In this work, we analyze an efficient sampling-based algorithm for general-purpose reachability analysis, which remains a notoriously challenging problem with applications ranging from neural network verification to safety analysis of…

Stochastic optimization has found wide applications in minimizing objective functions in machine learning, which motivates a lot of theoretical studies to understand its practical success. Most of existing studies focus on the convergence…

We prove a general quantitative theorem on the asymptotic behavior of stochastic quasi-Fej\'er monotone sequences in a broad metric context. Concretely, our result explicitly constructs a rate of convergence for such process, both in mean…