相关论文: Friction-adaptive descent: a family of dynamics-ba…

Fractional-order stochastic gradient descent (FOSGD) leverages fractional exponents to capture long-memory effects in optimization. However, its utility is often limited by the difficulty of tuning and stabilizing these exponents. We…

This paper provides a rigorous derivation and analysis of accelerated optimization algorithms through the lens of High-Resolution Ordinary Differential Equations (ODEs). While classical Nesterov acceleration is well-understood via…

The certified descent algorithm (CDA) is a gradient-based method for shape optimization which certifies that the direction computed using the shape gradient is a genuine descent direction for the objective functional under analysis. It…

For complex nonlinear systems, it is challenging to design algorithms that are fast, scalable, and give an accurate approximation of the stability region. This paper proposes a sampling-based approach to address these challenges. By…

In this paper, we develop and analyze an integral fixed-time sliding mode control method for a scenario in which the system model is only partially known, utilizing Gaussian processes. We present two theorems on fixed-time convergence. The…

A large class of non-smooth practical optimization problems can be written as minimization of a sum of smooth and partly smooth functions. We examine such structured problems which also depend on a parameter vector and study the problem of…

This paper studies the continuous-time dynamics of primal-dual algorithms for linearly constrained convex optimization problems and provides a quantitative convergence analysis using the Lyapunov functions. With the growing prevalence of…

We offer a measure-theoretic extension of the concept and theory of $k$-contraction, including their generalization on fractional dimensions $d$. The respective contraction property is defined through the exponential decay of the…

This paper introduces the Fej\'er-monotone hybrid steepest descent method (FM-HSDM), a new member to the HSDM family of algorithms, for solving affinely constrained minimization tasks in real Hilbert spaces, where convex smooth and…

One of the most widely used methods for solving large-scale stochastic optimization problems is distributed asynchronous stochastic gradient descent (DASGD), a family of algorithms that result from parallelizing stochastic gradient descent…

A novel probabilistic approach for the design of mechanical structures with friction interfaces is proposed. The objective function is defined as the probability that a specified performance measure of the forced vibration response is…

We present a first-order method for solving constrained optimization problems. The method is derived from our previous work, a modified search direction method inspired by singular value decomposition. In this work, we simplify its…

Fixed-time stable dynamical systems are capable of achieving exact convergence to an equilibrium point within a fixed time that is independent of the initial conditions of the system. This property makes them highly appealing for designing…

Gradient descent-based optimization methods underpin the parameter training of neural networks, and hence comprise a significant component in the impressive test results found in a number of applications. Introducing stochasticity is key to…

In order to achieve faster and more robust convergence (especially under noisy working environments), a sliding mode theory-based learning algorithm has been proposed to tune both the premise and consequent parts of type-2 fuzzy neural…

In the paper, we propose a class of faster adaptive Gradient Descent Ascent (GDA) methods for solving the nonconvex-strongly-concave minimax problems by using the unified adaptive matrices, which include almost all existing coordinate-wise…

We introduce new adaptive artificial anti-diffusion (AAAD) methods for one- and two-dimensional hyperbolic systems of conservation laws. The key idea is to reduce the amount of numerical dissipation present in a given numerical method by…

We propose AdaNAG, an adaptive accelerated gradient method based on Nesterov's accelerated gradient method. AdaNAG is line-search-free, parameter-free, and achieves the accelerated convergence rates $f(x_k) - f_\star =…

In this paper, we present a comprehensive study on the convergence properties of Adam-family methods for nonsmooth optimization, especially in the training of nonsmooth neural networks. We introduce a novel two-timescale framework that…

Classical worst-case optimization theory neither explains the success of optimization in machine learning, nor does it help with step size selection. In this paper we demonstrate the viability and advantages of replacing the classical…