Related papers: Comments on "Generalization of the gradient method…

Recent studies have shown that fractional calculus is an effective alternative mathematical tool in various scientific fields. However, some investigations indicate that results established in differential and integral calculus do not…

This paper focuses on the convergence problem of the emerging fractional order gradient descent method, and proposes three solutions to overcome the problem. In fact, the general fractional gradient method cannot converge to the real…

We consider a fractional generalization of gradient systems. We use differential forms and exterior derivatives of fractional orders. Examples of fractional gradient systems are considered. We describe the stationary states of these…

In this article, we first establish derivative formulae for fractional Gruschin type process, which generalize the result of Wang (J Theor Probab 27:80--95, Theorem 1.1, 2012). Since we work on a non-Markovian context, some technical…

Fractional derivatives are a well-studied generalization of integer order derivatives. Naturally, for optimization, it is of interest to understand the convergence properties of gradient descent using fractional derivatives. Convergence…

In this paper, we show that the stability analysis in the paper A note on stability of fractional logistic maps, Appl. Math. Lett. 125 (2022) 107787 is incorrect and repeat a proof of a theorem on the convergence of a convolution of the…

In this paper, we give some new thoughts about the classical gradient method (GM) and recall the proposed fractional order gradient method (FOGM). It is proven that the proposed FOGM holds a super convergence capacity and a faster…

In the present article, a new method for the evaluation of fractional derivatives of arbitrary real order is proposed. Numerous but inequivalent formulations have been given in the past. Some of them exhibit unsatisfactory properties such…

Fractional gradient descent has been studied extensively, with a focus on its ability to extend traditional gradient descent methods by incorporating fractional-order derivatives. This approach allows for more flexibility in navigating…

This note discusses proofs for convergence of first-order methods based on simple potential-function arguments. We cover methods like gradient descent (for both smooth and non-smooth settings), mirror descent, and some accelerated variants.

The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…

We consider some possible approaches to the fractional-order generalization of definition of variation (functional) derivative. Some problems of formulation of a fractional-order variational derivative are discussed. To give a consistent…

The policy gradient theorem describes the gradient of the expected discounted return with respect to an agent's policy parameters. However, most policy gradient methods drop the discount factor from the state distribution and therefore do…

A few families of counterexamples are provided to "A proof that the discrete singular convolution (DSC)/Lagrange-distributed approximation function (LDAF) method is inferior to high order finite differences", Journal of Computational…

In this comment, we raise serious concerns over the derivation of the rate of convergence of fractional steepest descent algorithm in Fractional Adaptive Learning (FAL) approach presented in `Fractional Extreme Value Adaptive Training…

This note provides a counterexample to a theorem announced in the last part of the paper ''Analysis of direct searches for discontinuous functions'', Mathematical Programming Vol. 133, pp.~299--325, 2012. The counterexample involves an…

In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization…

Given a two-variable function $f$ without critical points and a compact region $R$ bounded by two level curves of $f$, this note proves that the integral over $R$ of the second-order directional derivative of $f$ in the tangential…

In this article, we demonstrate that the claim made by Xiaoyan Li and Ni Sun \cite{bib 1} regarding the incorrectness of Theorem 7 in the paper \cite{bib 2} is wrong, and show that this Theorem is based on the integral with respect to…

Considering the large number of fractional operators that exist, and since it does not seem that their number will stop increasing soon at the time of writing this paper, it is presented for the first time, as far as the authors know, a…