相关论文: Quadratic Convergence of Smoothing Newton's Method…

We study a variant of Newton's algorithm applied to under-determined systems of non-smooth equations. The notion of regularity employed in our work is based on Newton differentiability, which generalizes semi-smoothness. The classic notion…

An optimal control problem for the linear wave equation with control cost chosen as the BV semi-norm in time is analyzed. This formulation enhances piecewise constant optimal controls and penalizes the number of jumps. Existence of optimal…

Optimization models with non-convex constraints arise in many tasks in machine learning, e.g., learning with fairness constraints or Neyman-Pearson classification with non-convex loss. Although many efficient methods have been developed…

This paper proposes a novel technique called "successive stochastic smoothing" that optimizes nonsmooth and discontinuous functions while considering various constraints. Our methodology enables local and global optimization, making it a…

Zero-One Composite Optimization (0/1-COP) is a prototype of nonsmooth, nonconvex optimization problems and it has attracted much attention recently. The augmented Lagrangian Method (ALM) has stood out as a leading methodology for such…

Optimizing neural networks for quantized objectives is fundamentally challenging because the quantizer is piece-wise constant, yielding zero gradients everywhere except at quantization thresholds where the derivative is undefined. Most…

We study the problem of minimizing a sum of convex objective functions where the components of the objective are available at different nodes of a network and nodes are allowed to only communicate with their neighbors. The use of…

We consider the unconstrained optimization problem whose objective function is composed of a smooth and a non-smooth conponents where the smooth component is the expectation a random function. This type of problem arises in some interesting…

Sparse optimization has seen its advances in recent decades. For scenarios where the true sparsity is unknown, regularization turns out to be a promising solution. Two popular non-convex regularizations are the so-called $L_0$ norm and…

Zeroth-order optimization methods are developed to overcome the practical hurdle of having knowledge of explicit derivatives. Instead, these schemes work with merely access to noisy functions evaluations. One of the predominant approaches…

Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most popular assumptions of this type is $(L_0,L_1)$-smoothness…

We examine a new form of smooth approximation to the zero one loss in which learning is performed using a reformulation of the widely used logistic function. Our approach is based on using the posterior mean of a novel generalized…

In this paper, we present new second-order algorithms for composite convex optimization, called Contracting-domain Newton methods. These algorithms are affine-invariant and based on global second-order lower approximation for the smooth…

Optimization on Riemannian manifolds widely arises in eigenvalue computation, density functional theory, Bose-Einstein condensates, low rank nearest correlation, image registration, and signal processing, etc. We propose an adaptive…

This paper addresses stochastic optimization of Lipschitz-continuous, nonsmooth and nonconvex objectives over compact convex sets, where only noisy function evaluations are available. While gradient-free methods have been developed for…

The paper considers functional linear regression, where scalar responses $Y_1,...,Y_n$ are modeled in dependence of random functions $X_1,...,X_n$. We propose a smoothing splines estimator for the functional slope parameter based on a…

Optimization problems in disciplines such as machine learning are commonly solved with iterative methods. Gradient descent algorithms find local minima by moving along the direction of steepest descent while Newton's method takes into…

We propose a novel algorithm, termed soft quasi-Newton (soft QN), for optimization in the presence of bounded noise. Traditional quasi-Newton algorithms are vulnerable to such perturbations. To develop a more robust quasi-Newton method, we…

Under the hypothesis that an initial point is a quasi-regular point, we use a majorant condition to present a new semi-local convergence analysis of an extension of the Gauss-Newton method for solving convex composite optimization problems.…

In this paper, we devise a $\operatorname{prox}$-based semi-smooth Newton method for the non-differentiable TV-minimization problem. To this end, the primal-dual optimality conditions are reformulated as a nonlinear operator equation with…