Related papers: Adaptive gradient-augmented level set method with …
Time integration of ODEs or time-dependent PDEs with required resolution of the fastest time scales of the system, can be very costly if the system exhibits multiple time scales of different magnitudes. If the different time scales are…
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks. Meanwhile, given the need for distributed computing, distributed optimization…
Multilevel optimization has gained renewed interest in machine learning due to its promise in applications such as hyperparameter tuning and continual learning. However, existing methods struggle with the inherent difficulty of efficiently…
We introduce a novel algorithm for gradient-based optimization of stochastic objective functions. The method may be seen as a variant of SGD with momentum equipped with an adaptive learning rate automatically adjusted by an 'energy'…
Simulation of complex dynamical systems arising in many applications is computationally challenging due to their size and complexity. Model order reduction, machine learning, and other types of surrogate modeling techniques offer cheaper…
This paper reviews the adaptive sparse grid discontinuous Galerkin (aSG-DG) method for computing high dimensional partial differential equations (PDEs) and its software implementation. The C\texttt{++} software package called AdaM-DG,…
We develop an adaptive method of time layers with a linearly implicit Rosenbrock method as time integrator and symmetric interior penalty Galerkin method for space discretization for the advective Allen-Cahn equation with…
In this paper, we propose an augmented subspace based adaptive proper orthogonal decomposition (POD) method for solving the time dependent partial differential equations. By augmenting the POD subspace with some auxiliary modes, we obtain…
We consider a Runge--Kutta method for the numerical time integration of the nonstationary incompressible Navier--Stokes equations. This yields a sequence of nonlinear problems to be solved for the stages of the Runge--Kutta method. The…
We compute time-dependent solutions of the sharp-interface model of dendritic solidification in two dimensions by using a level set method. The steady-state results are in agreement with solvability theory. Solutions obtained from the level…
In this paper, we propose a novel adaptive-rank method for simulating multi-scale BGK equations, based on a greedy sampling strategy. The method adaptively selects important rows and columns of the solution matrix and updates them using a…
This paper presents enhancement strategies for the Hermitian and skew-Hermitian splitting method based on gradient iterations. The spectral properties are exploited for the parameter estimation, often resulting in a better convergence. In…
Parallel-in-time algorithms have been successfully employed for reducing time-to-solution of a variety of partial differential equations, especially for diffusive (parabolic-type) equations. A major failing of parallel-in-time approaches to…
Benders decomposition with adaptive oracles was proposed to solve large-scale optimisation problems with a column bounded block-diagonal structure, where subproblems differ on the right-hand side and cost coefficients. Adaptive Benders…
Bilevel optimization provides a powerful framework for modelling hierarchical decision-making systems. This work presents a sensitivity-based algorithm that addresses the bilevel structure directly by treating the lower-level optimal…
In this paper, we incorporate the Barzilai-Borwein step size into gradient descent methods used to train deep networks. This allows us to adapt the learning rate using a two-point approximation to the secant equation which quasi-Newton…
We propose an adaptive Hermite spectral method for the three-dimensional velocity space of the Boltzmann equation guided by a newly developed frequency indicator. For the homogeneous problem, the indicator is defined by the contribution of…
We address the challenge of estimating the learning rate for adaptive gradient methods used in training deep neural networks. While several learning-rate-free approaches have been proposed, they are typically tailored for steepest descent.…
Boundary integral methods are attractive for solving homogeneous linear constant coefficient elliptic partial differential equations on complex geometries, since they can offer accurate solutions with a computational cost that is linear or…
We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate…