Related papers: From Stationary Phase to Steepest Descent

Steepest descent is central in variational mathematics. We present a new transparent existence proof for curves of near-maximal slope --- an influential notion of steepest descent in a nonsmooth setting. We moreover show that for…

Many iterative procedures in stochastic optimization exhibit a transient phase followed by a stationary phase. During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure…

We apply the method of nonlinear steepest descent to compute the long-time asymptotics of the Korteweg-de Vries equation with steplike initial data leading to a rarefaction wave. In addition to the leading asymptotic we also compute the…

This work is divided into two parts. First, we analyze the existence of positive bound and ground states for a second order stationary system coming from a coupled system of nonlinear Schr\"odinger--Korteweg-de Vries equations. Second, we…

Slow concept drift is a ubiquitous, yet under-studied problem in practical machine learning systems. In such settings, although recent data is more indicative of future data, naively prioritizing recent instances runs the risk of losing…

This work presents the design of nonlinear stabilization techniques for the finite element discretization of Euler equations in both steady and transient form. Implicit time integration is used in the case of the transient form. A…

We develop a novel approach towards causal inference. Rather than structural equations over a causal graph, we learn stochastic differential equations (SDEs) whose stationary densities model a system's behavior under interventions. These…

A nonlinear partial differential equation is a nonlinear relationship between an unknown function and how it changes due to two or more input variables. A numerical method reduces such an equation to arithmetic for quick visualization, but…

In this paper, we propose a successive pseudo-convex approximation algorithm to efficiently compute stationary points for a large class of possibly nonconvex optimization problems. The stationary points are obtained by solving a sequence of…

This paper presents a minimum flow approach applicable to a wide range of doubly nonlinear diffusion problems. We introduce a minimum flow steepest descent algorithm that seeks an optimal traffic flow by minimizing an internal energy…

Convergence detection of iterative stochastic optimization methods is of great practical interest. This paper considers stochastic gradient descent (SGD) with a constant learning rate and momentum. We show that there exists a transient…

We prove a nonlinear steepest descent theorem for Riemann-Hilbert problems with Carleson jump contours and jump matrices of low regularity and slow decay. We illustrate the theorem by deriving the long-time asymptotics for the mKdV equation…

Large-scale nonconvex optimization problems are ubiquitous in modern machine learning, and among practitioners interested in solving them, Stochastic Gradient Descent (SGD) reigns supreme. We revisit the analysis of SGD in the nonconvex…

We propose an informal test for stationarity in a time series which checks for the compatibility of nonlinear approximations to the dynamics made in different segments of the sequence. The segments are compared directly, rather than via…

A wide body of work has applied the concept of critical slowing down to estimate the stability of different Earth system components. Most of them -- such as global vegetation -- are inherently non-stationary, for example due to strong…

In this paper, we propose a new stochastic column-block gradient descent method for solving nonlinear systems of equations. It has a descent direction and holds an approximately optimal step size obtained through an optimization problem. We…

Under many circumstances many soft and hard materials are present in a puzzling wealth of non-equilibrium amorphous states, whose properties are not stationary and depend on preparation. They are often summarized in unconventional "phase…

We present a data-driven approach to efficiently approximate nonlinear transient dynamics in solid-state systems. Our proposed machine-learning model combines a dimensionality reduction stage with a nonlinear vector autoregression scheme.…

Assessment of the degree of boundedness/stability of multidimensional nonlinear systems with time-dependent and nonperiodic coefficients is an important problem in various applied areas which has no adequate resolution yet. Most of the…

In this paper, we propose a definition of phase for a class of stable nonlinear systems called semi-sectorial systems, from an input-output perspective. The definition involves the Hilbert transform as a critical instrument to complexify…