Related papers: Structured two-point stepsize gradient methods for…
A new method for solving numerically stochastic partial differential equations (SPDEs) with multiple scales is presented. The method combines a spectral method with the heterogeneous multiscale method (HMM) presented in [W. E, D. Liu, and…
This work proposes a hyper-reduction method for nonlinear parametric dynamical systems characterized by gradient fields such as Hamiltonian systems and gradient flows. The gradient structure is associated with conservation of invariants or…
In this paper, we propose algorithms that exploit negative curvature for solving noisy nonlinear nonconvex unconstrained optimization problems. We consider both deterministic and stochastic inexact settings, and develop two-step algorithms…
A scaled conjugate gradient method that accelerates existing adaptive methods utilizing stochastic gradients is proposed for solving nonconvex optimization problems with deep neural networks. It is shown theoretically that, whether with…
We show how to adapt the quasi-Newton method to the electronic-structure calculations using systematic basis sets. Our implementation requires less iterations than the conjugate gradient method, while the computational cost per iteration is…
In this paper, we consider a modified projected Gauss-Newton method for solving constrained nonlinear least-squares problems. We assume that the functional constraints are smooth and the the other constraints are represented by a simple…
We study the convergence of the gradient descent method for solving ill-posed problems where the solution is characterized as a global minimum of a differentiable functional in a Hilbert space. The classical least-squares functional for…
In this paper, we propose a new stochastic column-block gradient descent method for solving nonlinear systems of equations. It has a descent direction and holds an approximately optimal step size obtained through an optimization problem. We…
We propose a numerical method for solving high dimensional fully nonlinear partial differential equations (PDEs). Our algorithm estimates simultaneously by backward time induction the solution and its gradient by multi-layer neural…
In this paper, we first propose a new Levenberg-Marquardt method for solving constrained (and not necessarily square) nonlinear systems. Basically, the method combines the unconstrained Levenberg-Marquardt method with a type of feasible…
This chapter offers a comprehensive introduction to the least-squares neural network (LSNN) method introduced in [14,16], for solving scalar first-order hyperbolic partial differential equations, specifically linear advection-reaction…
This paper presents a unified Least-Squares framework for solving nonlinear partial differential equations by recasting the governing system as a residual minimisation problem. A Least-Squares functional is formulated and the corresponding…
When the available information is noisy zeroth-order (ZO) oracle, stochastic approximation methods are popular for estimating the root of the multivariate gradient equation. Inspired by the Stein's identity, this work establishes a novel…
In recent years, various subspace algorithms have been developed to handle large-scale optimization problems. Although existing subspace Newton methods require fewer iterations to converge in practice, the matrix operations and full…
This paper deals with nonsmooth convex optimization problems in Euclidean spaces. We identify special elements of the subdifferential of a convex function, called specular gradients. Based on this observation, we propose three numerical…
The quadratic termination property is important to the efficiency of gradient methods. We consider equipping a family of gradient methods, where the stepsize is given by the ratio of two norms, with two dimensional quadratic termination.…
A new stepsize for gradient method is proposed. Combining it with the exact line search stepsizes, the gradient method achieves the optimal solution in 5 steps for 3 dimensional quadratic function minimization problem. The new stepsize is…
We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints. Instances of this template naturally arise from SDP-relaxations of…
We introduce a new approach to spectral sparsification that approximates the quadratic form of the pseudoinverse of a graph Laplacian restricted to a subspace. We show that sparsifiers with a near-linear number of edges in the dimension of…
The following document presents a possible solution and a brief stability analysis for a nonlinear system, which is obtained by studying the possibility of building a hybrid solar receiver; It is necessary to mention that the solution of…