Related papers: Generalized Singular Value Thresholding
We present new convolution based smooth approximations to the absolute value function and apply them to construct gradient based algorithms such as the nonlinear conjugate gradient scheme to obtain sparse, regularized solutions of linear…
In a real Hilbert space $\mathcal{H}$. Given any function $f$ convex differentiable whose solution set $\argmin_{\mathcal{H}}\,f$ is nonempty, by considering the Proximal Algorithm $x_{k+1}=\text{prox}_{\b_k f}(d x_k)$, where $0<d<1$ and…
We develop a novel and single-loop variance-reduced algorithm to solve a class of stochastic nonconvex-convex minimax problems involving a nonconvex-linear objective function, which has various applications in different fields such as…
Stochastic gradient descent (SGD) is one of the most widely used optimization methods for parallel and distributed processing of large datasets. One of the key limitations of distributed SGD is the need to regularly communicate the…
This paper considers constrained stochastic nonsmooth minimax optimization problem of the form…
Stochastic Proximal Gradient (SPG) methods have been widely used for solving optimization problems with a simple (possibly non-smooth) regularizer in machine learning and statistics. However, to the best of our knowledge no non-asymptotic…
Over the past decade, various matrix completion algorithms have been developed. Thresholded singular value decomposition (SVD) is a popular technique in implementing many of them. A sizable number of studies have shown its theoretical and…
Random Reshuffling (RR), also known as Stochastic Gradient Descent (SGD) without replacement, is a popular and theoretically grounded method for finite-sum minimization. We propose two new algorithms: Proximal and Federated Random…
This paper concerns the minimization of the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function $g$ without closed-form proximal mapping. For this class of nonconvex and nonsmooth problems, we propose a…
Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given an $m \times n$ matrix $\widehat{{\mathbf M}}$, the prototypical RSVD algorithm…
Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for…
This paper deals with the minimization of large sum of convex functions by Inexact Newton (IN) methods employing subsampled functions, gradients and Hessian approximations. The Conjugate Gradient method is used to compute the inexact Newton…
In the present work we show that the local generalized monotonicity of a lower semicontinuous set-valued operator on some certain type of dense sets ensures the global generalized monotonicity of that operator. We achieve this goal…
The generalized singular value decomposition (GSVD) is a powerful tool for solving discrete ill-posed problems. In this paper, we propose a two-sided uniformly randomized GSVD algorithm for solving the large-scale discrete ill-posed problem…
In a recent work [2] with Datta, we introduced the mu vector (with respect to a given field) of simplicial complexes and used it to study tightness and lower bounds. In this paper, we modify the definition of mu vectors. With the new…
In this paper, we obtain subdifferential representation of a proper $w^*$-lower semicontinous convex function on $X^*$ as follows: Let $g$ be a proper convex $w^*$-lower semicontinuous function on $X^*$. Assume that int dom $g$…
Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem.…
In this paper, we present the definition of generalized tensor function according to the tensor singular value decomposition (T-SVD) via the tensor T-product. Also, we introduce the compact singular value decomposition (T-CSVD) of tensors…
We propose a CJ-FEAST GSVDsolver to compute a partial generalized singular value decomposition (GSVD) of a large matrix pair $(A,B)$ with the generalized singular values in a given interval. The solver is a highly nontrivial extension of…
We analyze the sample complexity of full-batch Gradient Descent (GD) in the setup of non-smooth Stochastic Convex Optimization. We show that the generalization error of GD, with common choice of hyper-parameters, can be $\tilde \Theta(d/m +…