Related papers: From Stationary Phase to Steepest Descent
This paper investigates a new class of equations called measure functional differential equations with state-dependent delays. We establish the existence and uniqueness of solutions and present a discussion concerning the appropriate phase…
This article concerns the nonlinear Korteweg-de Vries equation with boundary time-delay feedback. Under appropriate assumption on the coefficients of the feedbacks (delayed or not), we first prove that this nonlinear infinite dimensional…
Nonequilibrium statistical mechanics exhibit a variety of complex phenomena far from equilibrium. It inherits challenges of equilibrium, including accurately describing the joint distribution of a large number of configurations, and also…
We propose a fixed-point-based numerical framework for computing stationary states of nonlocal Fokker-Planck-type equations. Instead of discretising the differential operators directly, we reformulate the stationary problem as a nonlinear…
The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models. As the main workhorse for deep learning, stochastic gradient descent…
We develop a new continuous-time stochastic gradient descent method for optimizing over the stationary distribution of stochastic differential equation (SDE) models. The algorithm continuously updates the SDE model's parameters using an…
Stochastic gradient descent (SGD) has been a go-to algorithm for nonconvex stochastic optimization problems arising in machine learning. Its theory however often requires a strong framework to guarantee convergence properties. We hereby…
A nonlinear inequality is formulated in the paper. An estimate of the rate of growth/decay of solutions to this inequality is obtained. This inequality is of interest in a study of dynamical systems and nonlinear evolution equations. It can…
Inspired by the ubiquitous use of differential equations to model continuous dynamics across diverse scientific and engineering domains, we propose a novel and intuitive approach to continuous sequence modeling. Our method interprets…
A one dimensional stochastic exclusion process with two species of particles, $+$ and $-$, is studied where density of each species can fluctuate but the total particle density is conserved. From the exact stationary state weights we show…
A nonlinear model of the scalar field with a coupling between the field and its gradient is developed. It is shown, that such model is suitable for the description of phase transition accompanied by formation of spatial inhomogeneous…
Optimization algorithms for solving nonconvex inverse problem have attracted significant interests recently. However, existing methods require the nonconvex regularization to be smooth or simple to ensure convergence. In this paper, we…
Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are…
This paper studies the problem of stability of a parameterized delay differential equations (DDE see equation (0.1)). After discretizing the DDE (0.1), we show that the problem can be equivalently casted into a semi-definite programming…
Change-point detection and locally stationary time series modeling are two major approaches for the analysis of non-stationary data. The former aims to identify stationary phases by detecting abrupt changes in the dynamics of a time series…
Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is well understood in the convex learning setting, the existing…
We construct stable periodic solutions for a simple form nonlinear delay differential equation (DDE) with a periodic coefficient. The equation involves one underlying nonlinearity with the multiplicative periodic coefficient. The well-known…
We present here the details of a backstepping transformation aiming at reformulating the dynamics of a nonlinear systems subject to unknown long input delay in a form which is suitable for Lyapunov stability analysis. The control law…
How does the choice of optimization algorithm shape a model's ability to learn features? To address this question for steepest descent methods --including sign descent, which is closely related to Adam --we introduce steepest mirror flows…
For solving pseudo-convex global optimization problems, we present a novel fully adaptive steepest descent method (or ASDM) without any hard-to-estimate parameters. For the step-size regulation in an $\varepsilon$-normalized direction, we…