Related papers: Parameter Estimation in the Hermitian and Skew-Her…

Hermite polynomials and functions have extensive applications in scientific and engineering problems. Although it is recognized that employing the scaled Hermite functions rather than the standard ones can remarkably enhance the…

The spectral gradient method is known to be a powerful low-cost tool for solving large-scale optimization problems. In this paper, our goal is to exploit its advantages in the stochastic optimization framework, especially in the case of…

We propose an improved version of the Hermitian/skew-Hermitian splitting (HSS) iterative method, which we call HSS(0), to solve non-Hermitian linear systems with a positive definite Hermitian part. The improvement is based on solving the…

This article introduces an iterative method for solving nonsingular non-Hermitian positive semidefinite systems of linear equations. To construct the iteration process, the coefficient matrix is split into two non-Hermitian positive…

In this paper we propose a new fast splitting algorithm to solve the Weighted Split Bregman minimization problem in the backward step of an accelerated Forward-Backward algorithm. Beside proving the convergence of the method, numerical…

This paper explores variants of the subspace iteration algorithm for computing approximate invariant subspaces. The standard subspace iteration approach is revisited and new variants that exploit gradient-type techniques combined with a…

Gradient methods are widely used in optimization problems. In practice, while the smoothness parameter can be estimated utilizing techniques such as backtracking, estimating the strong convexity parameter remains a challenge; moreover, even…

This paper is devoted to first-order algorithms for smooth convex optimization with inexact gradients. Unlike the majority of the literature on this topic, we consider the setting of relative rather than absolute inexactness. More…

This paper proposes a novel parameter selection strategy for kernel-based gradient descent (KGD) algorithms, integrating bias-variance analysis with the splitting method. We introduce the concept of empirical effective dimension to quantify…

We present some extensions to the limited memory steepest descent method based on spectral properties and cyclic iterations. Our aim is to show that it is possible to combine sweep and delayed strategies for improving the performance of…

We show that the herding procedure of Welling (2009) takes exactly the form of a standard convex optimization algorithm--namely a conditional gradient algorithm minimizing a quadratic moment discrepancy. This link enables us to invoke…

Recently, Krukier et al. [Generalized skew-Hermitian triangular splitting iteration methods for saddle-point linear systems, Numer. Linear Algebra Appl. 21 (2014) 152-170] proposed an efficient generalized skew-Hermitian triangular…

We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent. This setup is…

Hamiltonian splitting methods are an established technique to derive stable and accurate integration schemes in molecular dynamics, in which additional accuracy can be gained using force gradients. For rigid bodies, a tradition exists in…

We consider a class of nonsmooth fractional programming problems with fixed-point constraints, where the numerator is convex and the denominator is concave. To solve this problem, we propose splitting algorithms that compute subgradient…

An efficient proximal-gradient-based method, called proximal extrapolated gradient method, is designed for solving monotone variational inequality in Hilbert space. The proposed method extends the acceptable range of parameters to obtain…

We propose new local error estimators for splitting and composition methods. They are based on the construction of lower order schemes obtained at each step as a linear combination of the intermediate stages of the integrator, so that the…

In the recent years, various gradient descent algorithms including the methods of gradient descent, gradient descent with momentum, adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp) and adaptive moment estimation (Adam)…

High dimensional and/or nonconvex optimization remains a challenging and important problem across a wide range of fields, such as machine learning, data assimilation, and partial differential equation (PDE) constrained optimization. Here we…

Iterative procedures for parameter estimation based on stochastic gradient descent allow the estimation to scale to massive data sets. However, in both theory and practice, they suffer from numerical instability. Moreover, they are…