Related papers: On the zero-stability of multistep methods on smoo…
In this paper, nonstandard multistep methods are considered. It is shown that under some (sufficient and necessary) conditions, these methods attain the same order as their standard counterparts - to prove this statement, a nonstandard…
The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization algorithms. For example, stable solutions of gradient descent (GD) correspond to flat minima, which have been…
We consider split-step Milstein methods for the solution of stiff stochastic differential equations with an emphasis on systems driven by multi-channel noise. We show their strong order of convergence and investigate mean-square stability…
This work has the goal of briefly surveying some key stabilization techniques for general nonlinear systems, for which, as it is well known, a smooth control Lyapunov function may fail to exist. A general overview of the situation with…
The increased integration of intermittent and decentralised forms of power production has eroded the stability margins of power grids and made it more challenging to ensure reliable and secure power transmission. Reliable grid operation…
This work deals with the stability analysis of nonlinear sampled-data systems under nonuniform sampling. It establishes novel relationships between the stability property of the exact discrete-time model for a given sequence of (aperiodic)…
In a series of papers \cite{LSJR16, PP17, LPP}, it was established that some of the most commonly used first order methods almost surely (under random initializations) and with step-size being small enough, avoid strict saddle points, as…
The analysis of strong-stability-preserving (SSP) linear multistep methods is extended to semi-discretized problems for which different terms on the right-hand side satisfy different forward Euler (or circle) conditions. Optimal additive…
In nonsmooth optimization, a negative subgradient is not necessarily a descent direction, making the design of convergent descent methods based on zeroth-order and first-order information a challenging task. The well-studied bundle methods…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem…
We present a linear stability analysis of stationary states (or fixed points) in large dynamical systems defined on random directed graphs with a prescribed distribution of indegrees and outdegrees. We obtain two remarkable results for such…
Most existing analyses of (stochastic) gradient descent rely on the condition that for $L$-smooth costs, the step size is less than $2/L$. However, many works have observed that in machine learning applications step sizes often do not…
We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our…
The heterogeneous multi-scale method (HMM) is a general strategy for dealing with problems involving multi-scales, with multi-physics, using multi-grids. It not only unifies several existing multi-scale methods, but also provide a…
Linear stability of braneworld models constructed with multi-scalar fields is very different from that of single-scalar field models. It is well known that both the tensor and scalar perturbation equations of the later can always be written…
Stability of the BDF methods of order up to five for parabolic equations can be established by the energy technique via Nevanlinna--Odeh multipliers. The nonexistence of Nevanlinna--Odeh multipliers makes the six-step BDF method special;…
Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…
Stability properties of the well-known Fourier split-step method used to simulate a soliton and similar solutions of the nonlinear Dirac equations, known as the Gross--Neveu model, are studied numerically and analytically. Three distinct…
This paper introduces and studies the notion of output-input stability, which represents a variant of the minimum-phase property for general smooth nonlinear control systems. The definition of output-input stability does not rely on a…
We investigate the stochastic gradient descent (SGD) method where the step size lies within a banded region instead of being given by a fixed formula. The optimal convergence rate under mild conditions and large initial step size is proved.…