相关论文: An Enhancement Algorithm of Cyclic Adaptive Fourie…
This article introduces adaptive Fourier decomposition (AFD) type methods, emphasizing on those that can be applied to stochastic processes and random fields, mainly including stochastic adaptive Fourier decomposition and stochastic…
Nesterov's accelerated gradient (AG) method for minimizing a smooth strongly convex function $f$ is known to reduce $f({\bf x}_k)-f({\bf x}^*)$ by a factor of $\epsilon\in(0,1)$ after $k=O(\sqrt{L/\ell}\log(1/\epsilon))$ iterations, where…
A coarse grid correction (CGC) approach is proposed to enhance the efficiency of the matrix exponential and $\varphi$ matrix function evaluations. The approach is intended for iterative methods computing the matrix-vector products with…
It is known that adaptive Fourier decomposition (AFD) offers efficient rational approxima- tions to functions in the classical Hardy H2 spaces with significant applications. This study aims at rational approximation in Bergman, and more…
We study the performance of adaptive Fourier-Galerkin methods in a periodic box in $\mathbb{R}^d$ with dimension $d\ge 1$. These methods offer unlimited approximation power only restricted by solution and data regularity. They are of…
The performance of gradient-based optimization methods, such as standard gradient descent (GD), greatly depends on the choice of learning rate. However, it can require a non-trivial amount of user tuning effort to select an appropriate…
We present a novel randomized block coordinate descent method for the minimization of a convex composite objective function. The method uses (approximate) partial second-order (curvature) information, so that the algorithm performance is…
This paper introduces a new method for minimizing matrix-smooth non-convex objectives through the use of novel Compressed Gradient Descent (CGD) algorithms enhanced with a matrix-valued stepsize. The proposed algorithms are theoretically…
The cyclic block coordinate descent-type (CBCD-type) methods, which performs iterative updates for a few coordinates (a block) simultaneously throughout the procedure, have shown remarkable computational performance for solving strongly…
We present a first-order method for solving constrained optimization problems. The method is derived from our previous work, a modified search direction method inspired by singular value decomposition. In this work, we simplify its…
Second-order optimizers hold intriguing potential for deep learning, but suffer from increased cost and sensitivity to the non-convexity of the loss surface as compared to gradient-based approaches. We introduce a coordinate descent method…
In this article, we propose a two-grid based adaptive proper orthogonal decomposition (POD) method to solve the time dependent partial differential equations. Based on the error obtained in the coarse grid, we propose an error indicator for…
This article introduces the multi-objective adaptive order Caputo fractional gradient descent (MOAOCFGD) algorithm for solving unconstrained multi-objective problems. The proposed method performs equally well for both smooth and non-smooth…
Motivated by broad applications in machine learning, we study the popular accelerated stochastic gradient descent (ASGD) algorithm for solving (possibly nonconvex) optimization problems. We characterize the finite-time performance of this…
In this paper, we present an efficient adaptive multigrid strategy for the geometry optimization of large-scale three dimensional molecular mechanics. The resulting method can achieve significantly reduced complexity by exploiting the…
In this paper, we propose a gradient-based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of…
The iteration complexity of the block-coordinate descent (BCD) type algorithm has been under extensive investigation. It was recently shown that for convex problems the classical cyclic BCGD (block coordinate gradient descent) achieves an…
Asynchronous stochastic gradient descent (ASGD) is a popular parallel optimization algorithm in machine learning. Most theoretical analysis on ASGD take a discrete view and prove upper bounds for their convergence rates. However, the…
Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linear model with $d$ parameters, previous work has found that…
Active Appearance Models (AAMs) are one of the most popular and well-established techniques for modeling deformable objects in computer vision. In this paper, we study the problem of fitting AAMs using Compositional Gradient Descent (CGD)…