Related papers: Arbitrary-order functionally fitted energy-diminis…
Gradient descent, or negative gradient flow, is a standard technique in optimization to find minima of functions. Many implementations of gradient descent rely on discretized versions, i.e., moving in the gradient direction for a set step…
Computing gradients of a cost function is central to design-based optimization and machine learning algorithms. Equilibrium propagation provides an exact method to compute gradients in hardware by exploiting the inherent physical laws. The…
We consider adaptive finite element methods for second-order elliptic PDEs, where the arising discrete systems are not solved exactly. For contractive iterative solvers, we formulate an adaptive algorithm which monitors and steers the…
Discrete gradient methods are geometric integration techniques that can preserve the dissipative structure of gradient flows. Due to the monotonic decay of the function values, they are well suited for general convex and nonconvex…
Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution. In particular Stein variational gradient descent has gained attention…
In this paper, we introduce a Lagrange multiplier approach to construct linearly implicit energy-preserving schemes of arbitrary order for general Hamiltonian PDEs. Unlike the widely used auxiliary variable methods, this novel approach does…
A new exponentially fitted version of the Discrete Variational Derivative method for the efficient solution of oscillatory complex Hamiltonian Partial Differential Equations is proposed. When applied to the nonlinear Schroedinger equation,…
The recently introduced Gradient Methods with Memory use a subset of the past oracle information to create an accurate model of the objective function that enables them to surpass the Gradient Method in practical performance. The model…
Minimization of energy in gradient systems leads to formation of oscillatory and Turing patterns in reaction-diffusion systems. These patterns should be accurately computed using fine space and time meshes over long time horizons to reach…
In this paper, gradient-based optimization methods are combined with finite-element modeling for improving electric devices. Geometric design parameters are considered by affine decomposition of the geometry or by the design element…
The discrete gradient structure and the positive definiteness of discrete fractional integrals or derivatives are fundamental to the numerical stability in long-time simulation of nonlinear integro-differential models. We build up a…
This paper studies large-scale optimization problems on Riemannian manifolds whose objective function is a finite sum of negative log-probability losses. Such problems arise in various machine learning and signal processing applications. By…
Critical points of energy functionals, which are of broad interest, for instance, in physics and chemistry, in solid and quantum mechanics, in material science, or in general diffusion-reaction models arise as solutions to the associated…
In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way.…
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In this article, we study these types of methods through the…
Deep learning is widely used in tasks including image recognition and generation, in learning dynamical systems from data and many more. It is important to construct learning architectures with theoretical guarantees to permit safety in the…
In this paper, we analyze the optimization landscape of gradient descent methods for static output feedback (SOF) control of discrete-time linear time-invariant systems with quadratic cost. The SOF setting can be quite common, for example,…
The existing discrete variational derivative method is only second-order accurate and fully implicit. In this paper, we propose a framework to construct an arbitrary high-order implicit (original) energy stable scheme and a second-order…
We propose a new paradigm for designing efficient p-adaptive arbitrary high order methods. We consider arbitrary high order iterative schemes that gain one order of accuracy at each iteration and we modify them in order to match the…
A consequent approach is proposed to construct symplectic force-gradient algorithms of arbitrarily high orders in the time step for precise integration of motion in classical and quantum mechanics simulations. Within this approach the basic…