相关论文: Averaged Heavy-Ball Method

This paper studies accelerated gradient methods for nonconvex optimization with Lipschitz continuous gradient and Hessian. We propose two simple accelerated gradient methods, restarted accelerated gradient descent (AGD) and restarted heavy…

When analysing extreme values, two alternative statistical approaches have historically been held in contention: the block maxima method (or annual maxima method, spurred by hydrological applications) and the peaks-over-threshold. Clamoured…

In this paper, we show that the rate of convergence in periodic homogenization of convex Hamilton-Jacobi equations is always $O(\varepsilon)$, which is optimal. This is a natural extension of a result concerning stable norms in metric…

Proximal bundle methods (PBM) are a powerful class of algorithms for convex optimization. Compared to gradient descent, PBM constructs more accurate surrogate models that incorporate gradients and function values from multiple past…

The proximal bundle method (PBM) is a powerful and widely used approach for minimizing nonsmooth convex functions. However, for smooth objectives, its best-known convergence rate remains suboptimal, and whether PBM can be accelerated…

In this paper, we present a novel penalty approach for the numerical solution of continuously controlled HJB equations and HJB obstacle problems. Our results include estimates of the penalisation error for a class of penalty terms, and we…

Stochastic gradient descent (SGD) has been widely studied in the literature from different angles, and is commonly employed for solving many big data machine learning problems. However, the averaging technique, which combines all iterative…

In this paper, we consider an unconstrained optimization model where the objective is a sum of a large number of possibly nonconvex functions, though overall the objective is assumed to be smooth and convex. Our bid to solving such model…

Distributed optimization advances centralized machine learning methods by enabling parallel and decentralized learning processes over a network of computing nodes. This work provides an accelerated consensus-based distributed algorithm for…

In this work, we first present an adaptive deterministic block coordinate descent method with momentum (mADBCD) to solve the linear least-squares problem, which is based on Polyak's heavy ball method and a new column selection criterion for…

In this paper, we examine the convergence properties of heavy-ball dynamics with Hessian-driven damping in smooth non-convex optimization problems satisfying a {\L}ojasiewicz condition. In this general setting, we provide a series of tight,…

We propose a new approach to the numerical solution of ergodic problems arising in the homogenization of Hamilton-Jacobi (HJ) equations. It is based on a Newton-like method for solving inconsistent systems of nonlinear equations, coming…

We study convergence rates of the classic proximal bundle method for a variety of nonsmooth convex optimization problems. We show that, without any modification, this algorithm adapts to converge faster in the presence of smoothness or a…

A control theoretic approach is presented in this paper for both batch and instantaneous updates of weights in feed-forward neural networks. The popular Hamilton-Jacobi-Bellman (HJB) equation has been used to generate an optimal weight…

Periodic dynamical systems ubiquitously exist in science and engineering. The harmonic balance (HB) method and its variants have been the most widely-used approaches for such systems, but are either confined to low-order approximations or…

In this article, two methods for solving mean-field type optimal control problems are proposed and investigated. The two methods are iterative methods: at each iteration, a Hamilton-Jacobi-Bellman equation is solved, for a terminal…

In this work, we consider smooth unconstrained optimization problems and we deal with the class of gradient methods with momentum, i.e., descent algorithms where the search direction is defined as a linear combination of the current…

We solve the Hartree-Fock-Bogoliubov (HFB) equations for a spherical mean field and a pairing potential with the inverse Hamiltonian method, which we have developed for the solution of the Dirac equation. This method is based on the…

The homotopy analysis method is studied in the present paper. The question of convergence of the homotopy analysis method is resolved. It is proven that under a special constraint the homotopy analysis method does converge to the exact…

Batting average is one of the principle performance measures for an individual baseball player. It is natural to statistically model this as a binomial-variable proportion, with a given (observed) number of qualifying attempts (called…