Related papers: Accelerating Extremum Seeking Convergence by Richa…
Parallel-in-time methods are developed to accelerate the direct-adjoint looping procedure. Particularly, we utilize the Paraexp algorithm, previously developed to integrate equations forward in time, to accelerate the direct-adjoint looping…
We give sufficient conditions under which the convergence of finite difference approximations in the space variable of possibly degenerate second order parabolic and elliptic equations can be accelerated to any given order of convergence by…
The tracking method based on the extreme learning machine (ELM) is efficient and effective. ELM randomly generates input weights and biases in the hidden layer, and then calculates and computes the output weights by reducing the iterative…
We generalize the Safe Extremum Seeking algorithm to address the minimization of an unknown objective function subject to multiple unknown inequality and equality constraints, relying on recent results of gradient flow systems. These…
This paper presents a novel extended dynamic programming approach for energy minimization (EDP) to solve the correspondence problem for stereo and motion. A significant speedup is achieved using a recursive minimum search strategy (RMS).…
In this paper, we present the discrete-time unbiased extremum seeking (ES) algorithm for n-dimensional (nD) static quadratic maps in the presence of unknown time-varying measurement delays bounded by known constants which can be large. The…
Based on SGD, previous works have proposed many algorithms that have improved convergence speed and generalization in stochastic optimization, such as SGDm, AdaGrad, Adam, etc. However, their convergence analysis under non-convex conditions…
In this paper, we formulate a new \emph{multiple-correction method}. The goal is to accelerate the rate of convergence. In particular, we construct some sequences to approximate the Euler-Mascheroni and Landau constants, which are faster…
This review is focused on the borderline region of theoretical physics and mathematics. First, we describe numerical methods for the acceleration of the convergence of series. These provide a useful toolbox for theoretical physics which has…
This paper examines a number of extrapolation and acceleration methods, and introduces a few modifications of the standard Shanks transformation that deal with general sequences. One of the goals of the paper is to lay out a general…
Extreme learning machine (ELM) as a neural network algorithm has shown its good performance, such as fast speed, simple structure etc, but also, weak robustness is an unavoidable defect in original ELM for blended data. We present a new…
This paper introduces a simple variant of the power method. It is shown analytically and numerically to accelerate convergence to the dominant eigenvalue/eigenvector pair; and, it is particularly effective for problems featuring a small…
Looped Transformers provide advantages in parameter efficiency, computational capabilities, and generalization for reasoning tasks. However, their expressive power regarding function approximation remains underexplored. In this paper, we…
A pervasive approach in scientific computing is to express the solution to a given problem as the limit of a sequence of vectors or other mathematical objects. In many situations these sequences are generated by slowly converging iterative…
This paper investigates multivariable extremum seeking using unit-vector control. By employing the gradient algorithm and a polytopic embedding of the unknown Hessian matrix, we establish sufficient conditions, expressed as linear matrix…
In recent years, accelerated extra-gradient methods have attracted much attention by researchers, for solving monotone inclusion problems. A limitation of most current accelerated extra-gradient methods lies in their direct utilization of…
Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed…
This article introduces new acceleration methods for fixed-point iterations. Extrapolations are computed using two or three mappings alternately and a new type of step length is proposed with good properties for nonlinear applications. The…
Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recent works have empirically shown its success in some machine…
Some continuous optimization methods can be connected to ordinary differential equations (ODEs) by taking continuous limits, and their convergence rates can be explained by the ODEs. However, since such ODEs can achieve any convergence rate…