Related papers: Revisiting Linearized Bregman Iterations under Lip…
We consider deep neural networks with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias…
The Levenberg-Marquardt algorithm is one of the most popular algorithms for finding the solution of nonlinear least squares problems. Across different modified variations of the basic procedure, the algorithm enjoys global convergence, a…
A new approach for robust Hinfty filtering for a class of Lipschitz nonlinear systems with time-varying uncertainties both in the linear and nonlinear parts of the system is proposed in an LMI framework. The admissible Lipschitz constant of…
We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant---for multiple…
We consider regularization of non-convex optimization problems involving a non-linear least-squares objective. By adding an auxiliary set of variables, we introduce a novel regularization framework whose corresponding objective function is…
This manuscript bridges nonparametric smoothness-based and shape-restricted estimation, which may appear as two disjoint paradigms in the field. The proposed approach is motivated by a conceptually simple observation: every Lipschitz…
The paper is devoted to investigating a Cauchy problem for nonlinear elliptic PDEs in the abstract Hilbert space. The problem is hardly solved by computation since it is severely ill-posed in the sense of Hadamard. We shall use a modified…
We propose a unifying algorithm for non-smooth non-convex optimization. The algorithm approximates the objective function by a convex model function and finds an approximate (Bregman) proximal point of the convex model. This approximate…
This work is concerned with an abstract inf-sup problem generated by a bilinear Lagrangian and convex constraints. We study the conditions that guarantee no gap between the inf-sup and related sup-inf problems. The key assumption introduced…
A typical assumption for the analysis of first order optimization methods is the Lipschitz continuity of the gradient of the objective function. However, for many practical applications this assumption is violated, including loss functions…
Discrete inverse problems correspond to solving a system of equations in a stable way with respect to noise in the data. A typical approach to enforce uniqueness and select a meaningful solution is to introduce a regularizer. While for most…
Contraction theory formulates the analysis of nonlinear systems in terms of Jacobian matrices. Although this provides the potential to develop a linear matrix inequality (LMI) framework for nonlinear control design, conditions are imposed…
High-dimensional data with intrinsic low-dimensional structure is ubiquitous in machine learning and data science. While various approaches allow one to learn a data manifold with a Riemannian structure from finite samples, performing…
Bundle methods have been intensively studied for solving both convex and nonconvex optimization problems. In most of the bundle methods developed thus far, at least one quadratic programming (QP) subproblem needs to be solved in each…
We propose a generic framework based on a new stochastic variance-reduced gradient descent algorithm for accelerating nonconvex low-rank matrix recovery. Starting from an appropriate initial estimator, our proposed algorithm performs…
In this work, we propose two derivative-free methods to address the problem of large-scale nonlinear equations with convex constraints. These algorithms satisfy the sufficient descent condition. The search directions can be considered…
In this work, we develop a level-set subdifferential error bound condition aiming towards convergence rate analysis of a variable Bregman proximal gradient (VBPG) method for a broad class of nonsmooth and nonconvex optimization problems. It…
We consider the minimization problem with the truncated quadratic regularization with gradient operator, which is a nonsmooth and nonconvex problem. We cooperated the classical preconditioned iterations for linear equations into the…
Probabilistic learning is increasingly being tackled as an optimization problem, with gradient-based approaches as predominant methods. When modelling multivariate likelihoods, a usual but undesirable outcome is that the learned model fits…
We introduce a derivative-free computational framework for approximating solutions to nonlinear PDE-constrained inverse problems. The aim is to merge ideas from iterative regularization with ensemble Kalman methods from Bayesian inference…