Related papers: Revisionist Integral Deferred Correction with Adap…

We study Stochastic Gradient Descent with AdaGrad stepsizes: a popular adaptive (self-tuning) method for first-order stochastic optimization. Despite being well studied, existing analyses of this method suffer from various shortcomings:…

In this article, we present a simple technique for boosting the order of accuracy of finite difference schemes for time dependent partial differential equations by optimally selecting the time step used to advance the numerical solution and…

This paper proposes a novel approach to adaptive step sizes in stochastic gradient descent (SGD) by utilizing quantities that we have identified as numerically traceable -- the Lipschitz constant for gradients and a concept of the local…

We present a continuous-time equivalent to the well-known iterative linear-quadratic algorithm including an implementation of a backtracking line-search policy and a novel regularization approach based on the necessary conditions in the…

We consider systems of ordinary differential equations with multiple scales in time. In general, we are interested in the long time horizon of a slow variable that is coupled to solution components that act on a fast scale. Although the…

Mirror Descent (MD) is a well-known method of solving non-smooth convex optimization problems. This paper analyzes the stochastic variant of MD with adaptive stepsizes. Its convergence on average is shown to be faster than with the fixed…

Scalable algorithms of posterior approximation allow Bayesian nonparametrics such as Dirichlet process mixture to scale up to larger dataset at fractional cost. Recent algorithms, notably the stochastic variational inference performs local…

Learning adversarial examples can be formulated as an optimization problem of maximizing the loss function with some box-constraints. However, for solving this induced optimization problem, the state-of-the-art gradient-based methods such…

This paper is concerned with a linear quadratic optimal control problem of delayed backward stochastic differential equations. An explicit representation is derived for the optimal control, which is a linear feedback of the entire past…

Model Predictive Control (MPC) is a well-established approach to solve infinite horizon optimal control problems. Since optimization over an infinite time horizon is generally infeasible, MPC determines a suboptimal feedback control by…

We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This provides not only an elegant and flexible framework to parametrize…

This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be considerably improved by scaling the update with a vector of…

In this work, we propose a numerical approach for simulations of large deformations of interfaces in a level set framework. To obtain a fast and viable numerical solution in both time and space, temporal discretization is based on the…

Adaptive (or deformable) mirrors are widely used as wavefront correctors in adaptive optics systems. The optical calibration of an adaptive mirror is a fundamental step during its life-cycle: the process is in facts required to compute a…

We revisit adaptive time stepping, one of the classical topics of numerical analysis and computational engineering. While widely used in application and subject of many theoretical works, a complete understanding is still missing. Apart…

Discontinuous Galerkin (DG) methods for hyperbolic partial differential equations (PDEs) with explicit time-stepping schemes, such as strong stability-preserving Runge-Kutta (SSP-RK), suffer from time-step restrictions that are…

In this paper we deal with global approximation of solutions of stochastic differential equations (SDEs) driven by countably dimensional Wiener process. Under certain regularity conditions imposed on the coefficients, we show lower bounds…

We propose a parallel adaptive constraint-tightening approach to solve a linear model predictive control problem for discrete-time systems, based on inexact numerical optimization algorithms and operator splitting methods. The underlying…

Primal-dual algorithms for the resolution of convex-concave saddle point problems usually come with one or several step size parameters. Within the range where convergence is guaranteed, choosing well the step size can make the difference…

Error control by means of a posteriori error estimators or indica-tors and adaptive discretizations, such as adaptive mesh refinement, have emerged in the late seventies. Since then, numerous theoretical developments and improvements have…