Related papers: Parameter Estimation with the Ordered $\ell_{2}$ R…
In this paper, we review the parallel and distributed optimization algorithms based on the alternating direction method of multipliers (ADMM) for solving "big data" optimization problems in modern communication networks. We first introduce…
The mean-variance (MV) model is the core of modern portfolio theory. Nevertheless, it suffers from the over-fitting problem due to the estimation errors of model parameters. We consider the $\ell_{1}$ regularized MV model, which adds an…
Adversarial robustness has become an important research topic given empirical demonstrations on the lack of robustness of deep neural networks. Unfortunately, recent theoretical results suggest that adversarial training induces a strict…
We introduce a new algorithm to solve a regularized spatial-spectral image estimation problem. Our approach is based on the linearized alternating directions method of multipliers (LADMM), which is a variation of the popular ADMM algorithm.…
In this paper, we propose and analyze an inexact version of the symmetric proximal alternating direction method of multipliers (ADMM) for solving linearly constrained optimization problems. Basically, the method allows its first subproblem…
High-dimensional predictive models, those with more measurements than observations, require regularization to be well defined, perform well empirically, and possess theoretical guarantees. The amount of regularization, often determined by…
Binary segmentation, which is sequential in nature is thus far the most widely used method for identifying multiple change points in statistical models. Here we propose a top down methodology called arbitrary segmentation that proceeds in a…
Parameter identification problems typically consist of a model equation, e.g. a (system of) ordinary or partial differential equation(s), and the observation equation. In the conventional reduced setting, the model equation is eliminated…
Ordinary Differential Equations are a simple but powerful framework for modeling complex systems. Parameter estimation from times series can be done by Nonlinear Least Squares (or other classical approaches), but this can give…
In this paper, we review the parallel and distributed optimization algorithms based on alternating direction method of multipliers (ADMM) for solving "big data" optimization problem in smart grid communication networks. We first introduce…
This work proposes a novel adaptive linearized alternating direction multiplier method (LADMM) to convex optimization, which improves the convergence rate of the LADMM-based algorithm by adjusting step-size iteratively.The innovation of…
Overparametrized neural networks trained by gradient descent (GD) can provably overfit any training data. However, the generalization guarantee may not hold for noisy data. From a nonparametric perspective, this paper studies how well…
Regularization is a popular technique to solve the overfitting problem of machine learning algorithms. Most regularization technique relies on parameter selection of the regularization coefficient. Plug-in method and cross-validation…
This article reports an algorithm for multi-agent distributed optimization problems with a common decision variable, local linear equality and inequality constraints and set constraints with convergence rate guarantees.…
Optimization problems over discrete or quantized variables are very challenging in general due to the combinatorial nature of their search space. Piecewise-affine regularization (PAR) provides a flexible modeling and computational framework…
Many statistical estimators for high-dimensional linear regression are M-estimators, formed through minimizing a data-dependent square loss function plus a regularizer. This work considers a new class of estimators implicitly defined…
By explaining how humans would solve a given task, human rationales can provide strong learning signal for neural language models (LMs). Explanation regularization (ER) aims to improve LM generalization by pushing the LM's machine…
We present a powerful and easy-to-implement iterative algorithm for solving large-scale optimization problems that involve $L_1$/total-variation (TV) regularization. The method is based on combining the Alternating Directions Method of…
Primal-Dual Hybrid Gradient (PDHG) and Alternating Direction Method of Multipliers (ADMM) are two widely-used first-order optimization methods. They reduce a difficult problem to simple subproblems, so they are easy to implement and have…
State-of-the-art neural networks can be trained to become remarkable solutions to many problems. But while these architectures can express symbolic, perfect solutions, trained models often arrive at approximations instead. We show that the…