Related papers: nlstac: Non-Gradient Separable Nonlinear Least Squ…
We introduce a method for the nonlinear dimension reduction of a high-dimensional function $u:\mathbb{R}^d\rightarrow\mathbb{R}$, $d\gg1$. Our objective is to identify a nonlinear feature map $g:\mathbb{R}^d\rightarrow\mathbb{R}^m$, with a…
Conventional research attributes the improvements of generalization ability of deep neural networks either to powerful optimizers or the new network design. Different from them, in this paper, we aim to link the generalization ability of a…
In this paper we address the convergence of stochastic approximation when the functions to be minimized are not convex and nonsmooth. We show that the "mean-limit" approach to the convergence which leads, for smooth problems, to the ODE…
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with…
In this paper, we present a new ellipsoid-type algorithm for solving nonsmooth problems with convex structure. Examples of such problems include nonsmooth convex minimization problems, convex-concave saddle-point problems and variational…
Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection…
In this paper, we present a novel nonlinear programming-based approach to fine-tune pre-trained neural networks to improve robustness against adversarial attacks while maintaining high accuracy on clean data. Our method introduces…
We propose a penalized least-squares method to fit the linear regression model with fitted values that are invariant to invertible linear transformations of the design matrix. This invariance is important, for example, when practitioners…
Real-time identification of electrical equivalent circuit models is a critical requirement in many practical systems, such as batteries and electric motors. Significant work has been done in the past developing different types of algorithms…
We investigate the randomized Kaczmarz method that adaptively updates the stepsize using readily available information for solving inconsistent linear systems. A novel geometric interpretation is provided which shows that the proposed…
Numerous fields of nonlinear physics, very different in nature, produce signals and images, that share the common feature of being essentially constituted of piecewise homogeneous phases. Analyzing signals and images from corresponding…
This document is an introduction to the Matlab package SDLS (Semi-Definite Least-Squares) for solving least-squares problems over convex symmetric cones. The package is shortly presented through the addressed problem, a sketch of the…
We propose a novel stochastic gradient descent method for solving linear least squares problems with partially observed data. Our method uses submatrices indexed by a randomly selected pair of row and column index sets to update the iterate…
We propose a least-squares penalization as a means to extend the discontinuous Petrov-Galerkin (DPG) method with optimal test functions to a class of semilinear elliptic problems. The nonlinear contributions are replaced with independent…
We introduce a new technique for solving uni-parametric versions of linear programs, convex quadratic programs, and linear complementarity problems in which a single parameter is permitted to be present in any of the input data. We…
The non-stationary evolution of observable quantities in complex systems can frequently be described as a juxtaposition of quasi-stationary spells. Given that standard theoretical and data analysis approaches usually rely on the assumption…
We investigate implicit regularization schemes for gradient descent methods applied to unpenalized least squares regression to solve the problem of reconstructing a sparse signal from an underdetermined system of linear measurements under…
This paper puts forth a new formulation and algorithm for the elastic matching problem on unparametrized curves and surfaces. Our approach combines the frameworks of square root normal fields and varifold fidelity metrics into a novel…
A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results.…
Partial least squares regression (PLSR) has been a popular technique to explore the linear relationship between two datasets. However, most of algorithm implementations of PLSR may only achieve a suboptimal solution through an optimization…