Related papers: Discrete gradient algorithms of high order for one…
The concept of effective order is a popular methodology in the deterministic literature for the construction of efficient and accurate integrators for differential equations over long times. The idea is to enhance the accuracy of a…
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the existing convergence…
We explore the utilization of higher-order discretization techniques in optimizing the gate count needed for quantum computer based solutions of partial differential equations. To accomplish this, we present an efficient approach for…
In this paper, we propose high order numerical methods to solve a 2D advection diffusion equation, in the highly oscillatory regime. We use an integrator strategy that allows the construction of arbitrary high-order schemes {leading} to an…
In this work, we develop a class of high-order multiderivative time integration methods that is able to preserve certain functionals discretely. Important ingredients are the recently developed Hermite-Birkhoff-Predictor-Corrector methods…
Any gradient descent optimization requires to choose a learning rate. With deeper and deeper models, tuning that learning rate can easily become tedious and does not necessarily lead to an ideal convergence. We propose a variation of the…
The proximal gradient algorithm has been popularly used for convex optimization. Recently, it has also been extended for nonconvex problems, and the current state-of-the-art is the nonmonotone accelerated proximal gradient algorithm.…
Discrete distributions, particularly in high-dimensional deep models, are often highly multimodal due to inherent discontinuities. While gradient-based discrete sampling has proven effective, it is susceptible to becoming trapped in local…
Uncertainty is unavoidable in modeling dynamical systems and it may be represented mathematically by differential inclusions. In the past, we proposed an algorithm to compute validated solutions of differential inclusions; here we provide…
We propose new sequential simulation-optimization algorithms for general convex optimization via simulation problems with high-dimensional discrete decision space. The performance of each choice of discrete decision variables is evaluated…
We propose randomized subspace gradient methods for high-dimensional constrained optimization. While there have been similarly purposed studies on unconstrained optimization problems, there have been few on constrained optimization problems…
First order optimization algorithms play a major role in large scale machine learning. A new class of methods, called adaptive algorithms, were recently introduced to adjust iteratively the learning rate for each coordinate. Despite great…
Discrete variational methods show excellent performance in numerical simulations of mechanical systems. In this paper, we adapt discrete variational integrators for the case of mechanical systems with double-bracket dissipation. In…
Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of…
In the last two decades, significant effort has been put in understanding and designing so-called structure-preserving numerical methods for the simulation of mechanical systems. Geometric integrators attempt to preserve the geometry…
We consider a class of stochastic gradient optimization schemes. Assuming that the objective function is strongly convex, we prove weak error estimates which are uniform in time for the error between the solution of the numerical scheme,…
This paper presents a sequence of deferred correction (DC) schemes built recursively from the implicit midpoint scheme for the numerical solution of general first order ordinary differential equations (ODEs). It is proven that each scheme…
In this paper we design discrete port-Hamiltonian systems systematically in two different ways, by applying discrete gradient methods and splitting methods respectively. The discrete port-Hamiltonian systems we get satisfy a discrete notion…
Differential equations in general and neural ODEs in particular are an essential technique in continuous-time system identification. While many deterministic learning algorithms have been designed based on numerical integration via the…
Since the expense of the numerical integration of large scale dynamical systems is often computationally prohibitive, model reduction methods, which approximate such systems by simpler and much lower order ones, are often employed to reduce…