Related papers: An efficient and robust SAV based algorithm for di…
For the past few years, scalar auxiliary variable (SAV) and SAV-type approaches became very hot and efficient methods to simulate various gradient flows. Inspired by the new SAV approach in \cite{huang2020highly}, we propose a novel…
Two primary scalar auxiliary variable (SAV) approaches are widely applied for simulating gradient flow systems, i.e., the nonlinear energy-based approach and the Lagrange multiplier approach. The former guarantees unconditional energy…
We propose a new numerical technique to deal with nonlinear terms in gradient flows. By introducing a scalar auxiliary variable (SAV), we construct efficient and robust energy stable schemes for a large class of gradient flows. The SAV…
The scalar auxiliary variable (SAV) approach is a highly efficient method widely used for solving gradient flow systems. This approach offers several advantages, including linearity, unconditional energy stability, and ease of…
In recent years, the scalar auxiliary variable (SAV) approach has become very popular and hot in the design of linear, high-order and unconditional energy stable schemes of gradient flow models. However, the nature of SAV-based numerical…
Optimizing the learning rate remains a critical challenge in machine learning, essential for achieving model stability and efficient convergence. The Vector Auxiliary Variable (VAV) algorithm introduces a novel energy-based self-adjustable…
We present a novel optimization algorithm, element-wise relaxed scalar auxiliary variable (E-RSAV), that satisfies an unconditional energy dissipation law and exhibits improved alignment between the modified and the original energy. Our…
We introduce a novel algorithm for gradient-based optimization of stochastic objective functions. The method may be seen as a variant of SGD with momentum equipped with an adaptive learning rate automatically adjusted by an 'energy'…
The scalar auxiliary variable (SAV) approach \cite{shen2018scalar} and its generalized version GSAV proposed in \cite{huang2020highly} are very popular methods to construct efficient and accurate energy stable schemes for nonlinear…
In this paper, we present a novel investigation of the so-called SAV approach, which is a framework to construct linearly implicit geometric numerical integrators for partial differential equations with variational structure. SAV approach…
For a class of fourth order gradient flow problems, integration of the scalar auxiliary variable (SAV) time discretization with the penalty-free discontinuous Galerkin (DG) spatial discretization leads to SAV-DG schemes. These schemes are…
Scalar auxiliary variable (SAV) methods are a class of linear schemes for solving gradient flows that are known for the stability of a `modified' energy. In this paper, we propose an improved SAV (iSAV) scheme that not only retains the…
The scalar auxiliary variable (SAV) approach is a very popular and efficient method to simulate various phase field models. To save the computational cost, a new SAV approach is given by introducing a new variable $\theta$. The new SAV…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
The scalar auxiliary variable (SAV) approach of Shen et al. (2018), which presents a novel way to discretize a large class of gradient flows, has been extended and improved by many authors for general dissipative systems. In this work we…
The scalar auxiliary variable (SAV)-type methods are very popular techniques for solving various nonlinear dissipative systems. Compared to the semi-implicit method, the baseline SAV method can keep a modified energy dissipation law but…
We introduce a novel and efficient algorithm called the stochastic approximate gradient descent (SAGD), as an alternative to the stochastic gradient descent for cases where unbiased stochastic gradients cannot be trivially obtained.…
We propose AEGD, a new algorithm for first-order gradient-based optimization of non-convex objective functions, based on a dynamically updated energy variable. The method is shown to be unconditionally energy stable, irrespective of the…
The scalar auxiliary variable (SAV) method was introduced by Shen et al. and has been broadly used to solve thermodynamically consistent PDE problems. By utilizing scalar auxiliary variables, the original PDE problems are reformulated into…
This paper introduces a novel optimization algorithm designed for nonlinear least-squares problems. The method is derived by preconditioning the gradient descent direction using the Singular Value Decomposition (SVD) of the Jacobian. This…