Related papers: An adaptive damped Newton method for strongly mono…
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be…
Superlinear convergence has been an elusive goal for black-box nonsmooth optimization. Even in the convex case, the subgradient method is very slow, and while some cutting plane algorithms, including traditional bundle methods, are popular…
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and…
We introduce a new approach to deriving approximate analytical solutions of a harmonic oscillator damped by purely nonlinear, or combinations of linear and nonlinear damping forces. Our approach is based on choosing a suitable trial…
In this paper, we devise a $\operatorname{prox}$-based semi-smooth Newton method for the non-differentiable TV-minimization problem. To this end, the primal-dual optimality conditions are reformulated as a nonlinear operator equation with…
We propose and analyze an adaptive step-size variant of the Davis-Yin three operator splitting. This method can solve optimization problems composed by a sum of a smooth term for which we have access to its gradient and an arbitrary number…
We investigate the dynamics of a lattice soliton on a monatomic chain in the presence of damping and external forces. We consider Stokes and hydrodynamical damping. In the quasi-continuum limit the discrete system leads to a damped and…
Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the…
We are concerned with the tensor equations whose coefficient tensor is an M-tensor. We first propose a Newton method for solving the equation with a positive constant term and establish its global and quadratic convergence. Then we extend…
We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods…
This paper aims to develop a Newton-type method to solve a class of nonconvex composite programs. In particular, the nonsmooth part is possibly nonconvex. To tackle the nonconvexity, we develop a notion of strong prox-regularity which is…
Variational Inequality (VI) problems have attracted great interest in the machine learning (ML) community due to their application in adversarial and multi-agent training. Despite its relevance in ML, the oft-used strong-monotonicity and…
This paper considers the generalized continuation Newton method and thetrust-region updating strategy for the underdetermined system of nonlinear equations. Moreover, in order to improve its computational efficiency, the new method will not…
We study stochastic second-order methods for solving general non-convex optimization problems. We propose using a special version of momentum to stabilize the stochastic gradient and Hessian estimates in Newton's method. We show that…
An adaptive analogue of the Yu. E. Nesterov method for variational inequalities with a strongly monotone operator is proposed. Some estimates are obtained for the parameters determining the quality of the solution of the variational…
This paper answers the question if a qualitatively heterogeneous passive networked system containing damped and undamped nodes shows consensus in the output of the nodes in the long run. While a standard Lyapunov analysis shows that the…
We analyze the performance of a variant of Newton method with quadratic regularization for solving composite convex minimization problems. At each step of our method, we choose regularization parameter proportional to a certain power of the…
Stiff and chaotic differential equations are challenging for time-stepping numerical methods. For explicit methods, the required time step resolution significantly exceeds the resolution associated with the smoothness of the exact solution…
Adam has achieved strong empirical success, but its theory remains incomplete even in the deterministic full-batch setting, largely because adaptive preconditioning and momentum are tightly coupled. In this work, a convergent reformulation…
This paper addresses a class of nonsmooth and nonconvex optimization problems defined on complete Riemannian manifolds. The objective function has a composite structure, combining convex, differentiable, and lower semicontinuous terms,…