Related papers: Efficient adaptive step size control for exponenti…

Optimizers like Adam and AdaGrad have been very successful in training large-scale neural networks. Yet, the performance of these methods is heavily dependent on a carefully tuned learning rate schedule. We show that in many large-scale…

Stochastic Gradient Descent (SGD) is a popular tool in training large-scale machine learning models. Its performance, however, is highly variable, depending crucially on the choice of the step sizes. Accordingly, a variety of strategies for…

Explicit step-truncation tensor methods have recently proven successful in integrating initial value problems for high-dimensional partial differential equations (PDEs). However, the combination of non-linearity and stiffness may introduce…

The machine learning explosion has created a prominent trend in modern computer hardware towards low precision floating-point operations. In response, there have been growing efforts to use low and mixed precision in general scientific…

A Monte Carlo method for computing the action of a matrix exponential for a certain class of matrices on a vector is proposed. The method is based on generating random paths, which evolve through the indices of the matrix, governed by a…

We use one-step conditional risk mappings to formulate a risk averse version of a total cost problem on a controlled Markov process in discrete time infinite horizon. The nonnegative one step costs are assumed to be lower semi-continuous…

Selecting an effective step-size is a fundamental challenge in first-order optimization, especially for problems with non-Euclidean geometries. This paper presents a novel adaptive step-size strategy for optimization algorithms that rely on…

We propose new local error estimators for splitting and composition methods. They are based on the construction of lower order schemes obtained at each step as a linear combination of the intermediate stages of the integrator, so that the…

In this research we consider linear time-invariant plants and assume that the regressor finite excitation requirement is met. In such case, a new law to adjust the controller parameters, which ensures the exponential stability of the…

The problem of designing adaptive stepsize sequences for the gradient descent method applied to convex and locally smooth functions is studied. We take an adaptive control perspective and design update rules for the stepsize that make use…

A technique is described in this paper to avoid order reduction when integrating reaction-diffusion initial boundary value problems with explicit exponential Rosenbrock methods. The technique is valid for any Rosenbrock method, without…

In this paper, we consider the application of exponential integrators to problems that are advection dominated, either on the entire or on a subset of the domain. In this context, we compare Leja and Krylov based methods to compute the…

Model Predictive Control (MPC) is often tuned by trial and error. When a baseline linear controller exists that is already well tuned in the absence of constraints and MPC is introduced to enforce them, one would like to avoid altering the…

We introduce a class of explicit exponential Rosenbrock methods for the time integration of large systems of stiff differential equations. Their application with respect to simulation tasks in the field of visual computing is discussed…

We change a previous time-stepping algorithm for solving a multi-scale Vlasov-Poisson system within a Particle-In-Cell method, in order to do accurate long time simulations. As an exponential integrator, the new scheme allows to use large…

Several classical adaptive optimization algorithms, such as line search and trust region methods, have been recently extended to stochastic settings where function values, gradients, and Hessians in some cases, are estimated via stochastic…

Many time-dependent differential equations are equipped with invariants. Preserving such invariants under discretization can be important, e.g., to improve the qualitative and quantitative properties of numerical solutions. Recently,…

In this paper we consider mean-field optimal control problems with selective action of the control, where the constraint is a continuity equation involving a non-local term and diffusion. First order optimality conditions are formally…

We investigate a local incremental stationary scheme for the numerical solution of rate-independent systems. Such systems are characterized by a (possibly) non-convex energy and a dissipation potential, which is positively homogeneous of…

Extracting maximum power available from photovoltaic arrays requires the system operating at the maximum power point (MPP). Therefore, finding the MPP is necessary for efficient operation of PV arrays. The MPP changes with multiple…