Related papers: L$_1$ Regularization for Reconstruction of a non-e…

Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least…

Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A well-known example is…

The idea of exploiting sparseness in under-determined damage characterization problems is not new, and regularizations techniques that tend to promote sparseness, such as L1-norm minimization, have been investigated in the last ten years or…

$\ell_1$ regularization has been used for logistic regression to circumvent the overfitting and use the estimated sparse coefficient for feature selection. However, the challenge of such a regularization is that the $\ell_1$ norm is not…

We analyze an Iteratively Re-weighted Least Squares (IRLS) algorithm for promoting l1-minimization in sparse and compressible vector recovery. We prove its convergence and we estimate its local rate. We show how the algorithm can be…

$L_p$-norm regularization schemes such as $L_0$, $L_1$, and $L_2$-norm regularization and $L_p$-norm-based regularization techniques such as weight decay, LASSO, and elastic net compute a quantity which depends on model weights considered…

Numerical experiments in literature on compressed sensing have indicated that the reweighted $l_1$ minimization performs exceptionally well in recovering sparse signal. In this paper, we develop exact recovery conditions and algorithm for…

We study Ising models for describing data and show that autoregressive methods may be used to learn their connections, also in the case of asymmetric connections and for multi-spin interactions. For each link the linear Granger causality is…

A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space…

Regularization techniques such as $\mathcal{L}_1$ and $\mathcal{L}_2$ regularizers are effective in sparsifying neural networks (NNs). However, to remove a certain neuron or channel in NNs, all weight elements related to that neuron or…

Consider reconstructing a signal $x$ by minimizing a weighted sum of a convex differentiable negative log-likelihood (NLL) (data-fidelity) term and a convex regularization term that imposes a convex-set constraint on $x$ and enforces its…

In this work, we propose an optimization framework for estimating a sparse robust one-dimensional subspace. Our objective is to minimize both the representation error and the penalty, in terms of the l1-norm criterion. Given that the…

We present a renormalization group (RG) procedure which works naturally on a wide class of interacting one-dimension models based on perturbed (possibly strongly) continuum conformal and integrable models. This procedure integrates Kenneth…

We propose a practical method for $L_0$ norm regularization for neural networks: pruning the network during training by encouraging weights to become exactly zero. Such regularization is interesting since (1) it can greatly speed up…

Training deep neural networks with an $L_0$ regularization is one of the prominent approaches for network pruning or sparsification. The method prunes the network during training by encouraging weights to become exactly zero. However,…

The analysis of complex nonlinear systems is often carried out using simpler piecewise linear representations of them. A principled and practical technique is proposed to linearize and evaluate arbitrary continuous nonlinear functions using…

We consider the problem of recovering an unknown effectively $(s_1,s_2)$-sparse low-rank-$R$ matrix $X$ with possibly non-orthogonal rank-$1$ decomposition from incomplete and inaccurate linear measurements of the form $y = \mathcal A (X) +…

Iterative regularization is a classic idea in regularization theory, that has recently become popular in machine learning. On the one hand, it allows to design efficient algorithms controlling at the same time numerical and statistical…

In this paper, we carry out a unified study for $L_1$ over $L_2$ sparsity promoting models, which are widely used in the regime of coherent dictionaries for recovering sparse nonnegative/arbitrary signals. First, we provide a unified…

In this paper, we propose a novel algorithm for analysis-based sparsity reconstruction. It can solve the generalized problem by structured sparsity regularization with an orthogonal basis and total variation regularization. The proposed…