Related papers: Step Sizes for Strong Stability Preservation with …
In this paper, we suggest a new framework for analyzing primal subgradient methods for nonsmooth convex optimization problems. We show that the classical step-size rules, based on normalization of subgradient, or on the knowledge of optimal…
The class of stochastic Runge-Kutta methods for stochastic differential equations due to R\"o{\ss}ler is considered. Coefficient families of diagonally drift-implicit stochastic Runge-Kutta (DDISRK) methods of weak order one and two are…
Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is…
We present a new class of efficient and robust discontinuous spectral-element methods of arbitrary order for nonlinear hyperbolic systems of conservation laws on curved triangular and tetrahedral unstructured grids. Such discretizations…
This paper studies whether numerically preserving monotonic properties can offer modelling advantages in data assimilation, particularly when the signal or data is a realization of a stochastic partial differential equation (SPDE) or…
In this paper, we present a novel strategy to systematically construct linearly implicit energy-preserving schemes with arbitrary order of accuracy for Hamiltonian PDEs. Such novel strategy is based on the newly developed exponential scalar…
Under interpolation-type assumptions such as the strong growth condition, stochastic optimization methods can attain convergence rates comparable to full-batch methods, but their performance, particularly for SGD, remains highly sensitive…
This paper studies the complexity of finding an $\epsilon$-stationary point for stochastic bilevel optimization when the upper-level problem is nonconvex and the lower-level problem is strongly convex. Recent work proposed the first-order…
The estimation of phase transitions in random boolean Constraint Satisfaction Problems (CSP) is based on two fundamental tools: the first and second moment methods. While the first moment method on the number of solutions permits to compute…
Stochastic Gradient Descent (SGD) is widely used in machine learning research. Previous convergence analyses of SGD under the vanishing step-size setting typically require Robbins-Monro conditions. However, in practice, a wider variety of…
In stochastic gradient descent, especially for neural network training, there are currently dominating first order methods: not modeling local distance to minimum. This information required for optimal step size is provided by second order…
Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are batch methods designed mainly based on the convex optimization, say, the…
We introduce a model for limit order book of a certain security with two main features: First, both the limit orders and market orders for the given asset are allowed to appear and interact with each other. Second, the high frequency…
We study the Allen-Cahn equation with a cubic-quintic nonlinear term and a stochastic $Q$-trace-class stochastic forcing in two spatial dimensions. This stochastic partial differential equation (SPDE) is used as a test case to understand,…
In practical computation with Runge--Kutta methods, the stage equations are not satisfied exactly, due to roundoff errors, algebraic solver errors, and so forth. We show by example that propagation of such errors within a single step can…
This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…
Two-step predictor/corrector methods are provided to solve three classes of problems that present themselves as systems of ordinary differential equations (ODEs). In the first class, velocities are given from which displacements are to be…
This paper studies fully discrete finite element approximations to the Navier-Stokes equations using inf-sup stable elements and grad-div stabilization. For the time integration two implicit-explicit second order backward differentiation…
High order upwind summation-by-parts finite difference operators have recently been developed. When combined with the simultaneous-approximation-term method to impose boundary conditions, the method converges faster than using traditional…
Temporal difference (TD) learning is a foundational algorithm in reinforcement learning (RL). For nearly forty years, TD learning has served as a workhorse for applied RL as well as a building block for more complex and specialized…