相关论文: Design of generalized fractional order gradient de…

We consider a class of nonsmooth fractional programming problems with fixed-point constraints, where the numerator is convex and the denominator is concave. To solve this problem, we propose splitting algorithms that compute subgradient…

Based on the method of FGD, we apply the method of adaptive gradient descent which uses different step length at different epoch. Adaptive gradient descent performs much better than FGD in the tests and keeps the guarantee of convergence…

The constrained gradient method (CGM) has recently been proposed to solve convex optimization and monotone variational inequality (VI) problems with general functional constraints. While existing literature has established convergence…

We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective…

Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow.…

Using kicked differential equations of motion with derivatives of noninteger orders, we obtain generalizations of the dissipative standard map. The main property of these generalized maps, which are called fractional maps, is long-term…

A unified fast time-stepping method for both fractional integral and derivative operators is proposed. The fractional operator is decomposed into a local part with memory length $\Delta T$ and a history part, where the local part is…

Variable order structures model situations in which the comparison between two points depends on a point-to-cone map. In this paper, an inexact projected gradient method for solving smooth constrained vector optimization problems on…

There has recently been considerable interest in using a nonstandard piecewise approximation to formulate fractional order differential equations as difference equations that describe the same dynamical behaviour and are more amenable to a…

In this paper, we consider the dual formulation of minimizing $\sum_{i\in I}f_i(x_i)+\sum_{j\in J} g_j(\mathcal{A}_jx)$ with the index sets $I$ and $J$ being large. To address the difficulties from the high dimension of the variable $x$…

First-order optimization methods tend to inherently favor certain solutions over others when minimizing an underdetermined training objective that has multiple global optima. This phenomenon, known as implicit bias, plays a critical role in…

In this comment, we raise serious concerns over the derivation of the rate of convergence of fractional steepest descent algorithm in Fractional Adaptive Learning (FAL) approach presented in `Fractional Extreme Value Adaptive Training…

In this article, a high-order time-stepping scheme based on the cubic interpolation formula is considered to approximate the generalized Caputo fractional derivative (GCFD). Convergence order for this scheme is $(4-\alpha)$, where $\alpha…

A finite difference numerical method is investigated for fractional order diffusion problems in one space dimension. For this, a mathematical model is developed to incorporate homogeneous Dirichlet and Neumann type boundary conditions. The…

We present a new feasible proximal gradient method for constrained optimization where both the objective and constraint functions are given by the summation of a smooth, possibly nonconvex function and a convex simple function. The…

Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gradient flow is amenable to theoretical analysis, but is…

In this paper we consider a class of optimization problems with a strongly convex objective function and the feasible set given by an intersection of a simple convex set with a set given by a number of linear equality and inequality…

In this paper, we explore two fundamental first-order algorithms in convex optimization, namely, gradient descent (GD) and proximal gradient method (ProxGD). Our focus is on making these algorithms entirely adaptive by leveraging local…

We propose an Adagrad-like algorithm for multi-objective unconstrained optimization that relies on the computation of a common descent direction only. Unlike classical local algorithms for multi-objective optimization, our approach does not…

In this paper we consider large-scale composite optimization problems having the objective function formed as a sum of two terms (possibly nonconvex), one has (block) coordinate-wise Lipschitz continuous gradient and the other is…