Related papers: An Energy Stable Approach for Learning Derivative …
In this paper, a stochastic alternating direction method of multipliers (ADMM) is proposed for a class of nonsmooth composite and stochastic convex optimization problems in Hilbert space, motivated by optimization problems constrained by…
In this study, we investigated the stability of dynamic mode decomposition (DMD) algorithms to noisy data. To achieve a stable DMD algorithm, we applied the truncated total least squares (T-TLS) regression and optimal truncation level…
The stochastic sandpile model (SSM) is a generalisation of the standard Abelian sandpile model (ASM), in which topplings of unstable vertices are made random. When unstable, a vertex sends one grain to each of its neighbours independently…
Distributed cooperative localization in wireless networks is a challenging problem since it typically requires solving a large-scale nonconvex and nonsmooth optimization problem. In this paper, we reformulate the classic cooperative…
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artificial neural networks. However very little is known on to what extent SGD is crucial for to the success of this technology and, in…
We present a structure-preserving scheme based on a recently-proposed mixed formulation for incompressible hyperelasticity formulated in principal stretches. Although there exist Hamiltonians introduced for quasi-incompressible…
We develop a numerical method for solving the acoustic wave equation in covariant form on staggered curvilinear grids in an energy conserving manner. The use of a covariant basis decomposition leads to a rotationally invariant scheme that…
Dense Associative Memory (DAM) models have been attracting renewed attention since they were shown to be robust to adversarial examples and closely related to cutting edge machine learning paradigms, such as the attention mechanism and…
This paper studies whether numerically preserving monotonic properties can offer modelling advantages in data assimilation, particularly when the signal or data is a realization of a stochastic partial differential equation (SPDE) or…
We propose a stochastic modified equations (SME) for modeling the asynchronous stochastic gradient descent (ASGD) algorithms. The resulting SME of Langevin type extracts more information about the ASGD dynamics and elucidates the…
While data-driven model reduction techniques are well-established for linearizable mechanical systems, general approaches to reducing non-linearizable systems with multiple coexisting steady states have been unavailable. In this paper, we…
We propose a continuous-time scheme for large-scale optimization that introduces individual, adaptive momentum coefficients regulated by the kinetic energy of each model parameter. This approach automatically adjusts to local landscape…
The self-attention mechanism (SAM) is widely used in various fields of artificial intelligence and has successfully boosted the performance of different models. However, current explanations of this mechanism are mainly based on intuitions…
We propose a stochastic optimization method for minimizing loss functions, expressed as an expected value, that adaptively controls the batch size used in the computation of gradient approximations and the step size used to move along such…
Stochastic gradient methods (SGMs) are predominant approaches for solving stochastic optimization. On smooth nonconvex problems, a few acceleration techniques have been applied to improve the convergence rate of SGMs. However, little…
Spiking neural networks (SNNs) have gained significant attention for their potential to enable energy-efficient artificial intelligence. However, effective and efficient training of SNNs remains an unresolved challenge. While…
We present \textsc{dm-PhiSNet}, a physically constrained \textsc{PhiSNet}-based equivariant model that predicts one-electron reduced density matrices (1-RDMs) directly from molecular geometries in an atomic-orbital (AO) basis for…
The feasibility of deep neural networks (DNNs) to address data stream problems still requires intensive study because of the static and offline nature of conventional deep learning approaches. A deep continual learning algorithm, namely…
This paper studies how a stochastic gradient algorithm (SG) can be controlled to hide the estimate of the local stationary point from an eavesdropper. Such problems are of significant interest in distributed optimization settings like…
Traditional energy-based learning models associate a single energy metric to each configuration of variables involved in the underlying optimization process. Such models associate the lowest energy state to the optimal configuration of…