相关论文: Asymptotic behaviour of a family of gradient algor…

This paper presents a comprehensive study on the convergence rates of the stochastic gradient descent (SGD) algorithm when applied to overparameterized two-layer neural networks. Our approach combines the Neural Tangent Kernel (NTK)…

We introduce a novel adaptive damping technique for an inertial gradient system which finds application as a gradient descent algorithm for unconstrained optimisation. In an example using the non-convex Rosenbrock's function, we show an…

Many recent applications in machine learning and data fitting call for the algorithmic solution of structured smooth convex optimization problems. Although the gradient descent method is a natural choice for this task, it requires exact…

Motivated by energy based analyses for descent methods in the Euclidean setting, we investigate a generalisation of such analyses for descent methods over Riemannian manifolds. In doing so, we find that it is possible to derive…

In this article, we deal with some problems involving a class of singularly perturbed elliptic operator. We prove the asymptotic preserving of a general Galerkin method associated to a semilinear problem. We use a particular Galerkin…

Estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with (very) large samples of high dimensional data. The collected data may be contaminated by noise so that robust…

This paper studies the convergence properties of a family of Relaxed $\ell$-Minimal Gradient Descent methods for quadratic optimization; the family includes the omnipresent Steepest Descent method, as well as the Minimal Gradient method.…

We provide the detailed asymptotic behavior for first-order aggregation models of heterogeneous oscillators. Due to the dissimilarity of natural frequencies, one could expect that all relative distances converge to definite positive value…

In a Hilbert setting $H$, we study the asymptotic behavior of the trajectories of nonautonomous evolution equations $\dot x(t)+A_t(x(t))\ni 0$, where for each $t\geq 0$, $A_t:H\tto H$ denotes a maximal monotone operator. We provide general…

In this paper, we propose and analyze a fast two-point gradient algorithm for solving nonlinear ill-posed problems, which is based on the sequential subspace optimization method. A complete convergence analysis is provided under the…

Gradient algorithms are classical in adaptive control and parameter estimation. For instantaneous quadratic cost functions they lead to a linear time-varying dynamic system that converges exponentially under persistence of excitation…

In this paper, we establish the almost sure convergence of two-timescale stochastic gradient descent algorithms in continuous time under general noise and stability conditions, extending well known results in discrete time. We analyse…

In stochastic optimization, a common tool to deal sequentially with large sample is to consider the well-known stochastic gradient algorithm. Nevertheless, since the stepsequence is the same for each direction, this can lead to bad results…

We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i)…

In this paper, we study the gradient descent-ascent method for convex-concave saddle-point problems. We derive a new non-asymptotic global convergence rate in terms of distance to the solution set by using the semidefinite programming…

This paper investigates the asymptotic behaviour of solutions to certain infinite systems of coupled recurrence relations. In particular, we obtain a characterisation of those initial values which lead to a convergent solution, and for…

The effective and efficient numerical solution of Riemann-Hilbert problems has been demonstrated in recent work. With the aid of ideas from the method of nonlinear steepest descent for Riemann-Hilbert problems, the resulting numerical…

In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way.…

In the framework of real Hilbert spaces we study continuous in time dynamics as well as numerical algorithms for the problem of approaching the set of zeros of a single-valued monotone and continuous operator $V$. The starting poin is a…

The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper (2007) for estimation of unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk. It…