Related papers: Nonlinear Schwarz preconditioning for Quasi-Newton…
This paper examines a number of extrapolation and acceleration methods, and introduces a few modifications of the standard Shanks transformation that deal with general sequences. One of the goals of the paper is to lay out a general…
We propose an alternative implementation of preconditioning techniques for the solution of non-linear problems. Within the framework of Newton-Krylov methods, preconditioning techniques are needed to improve the performance of the solvers.…
A thermo-elastoplastic finite element approach is used to perform the simulation of a laser beam welding (LBW) process. This results in a nonlinear, nonsymmetric saddle point multiphysics system, for which the nonlinearity is handled via…
Numerical techniques for solving partial differential equations (PDEs) are integral for many fields across science and engineering. Such techniques usually involve solving large, sparse linear systems, where preconditioning methods are…
Although first-order stochastic algorithms, such as stochastic gradient descent, have been the main force to scale up machine learning models, such as deep neural nets, the second-order quasi-Newton methods start to draw attention due to…
We analyze the convergence of the one-level overlapping domain decomposition preconditioner SORAS (Symmetrized Optimized Restricted Additive Schwarz) applied to a generic linear system whose matrix is not necessarily symmetric/self-adjoint…
Fast computation of demagnetization curves is essential for the computational design of soft magnetic sensors or permanent magnet materials. We show that a sparse preconditioner for a nonlinear conjugate gradient energy minimizer can lead…
In this paper, we propose a globally convergent method for solving constrained nonlinear systems. The method combines an efficient Newton conditional gradient method with a derivative-free and nonmonotone linesearch strategy. The global…
In this work, we propose new variants of Anderson acceleration and nonlinear GMRES for general fixed-point iterations, based on modified least-squares problems associated with the methods. To solve the underlying linear systems, we apply…
Using quasi-Newton methods in stochastic optimization is not a trivial task given the difficulty of extracting curvature information from the noisy gradients. Moreover, pre-conditioning noisy gradient observations tend to amplify the noise.…
A linearly implicit conservative difference scheme is applied to discretize the attractive coupled nonlinear Schr\"odinger equations with fractional Laplacian. Complex symmetric linear systems can be obtained, and the system matrices are…
Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks. Despite its success, BN is not theoretically well…
Non-asymptotic convergence analysis of quasi-Newton methods has gained attention with a landmark result establishing an explicit local superlinear rate of O$((1/\sqrt{t})^t)$. The methods that obtain this rate, however, exhibit a well-known…
Domain decomposition (DD) methods are a natural way to take advantage of parallel computers when solving large scale linear systems. Their scalability depends on the design of the coarse space used in the two-level method. The analysis of…
Owing to the ability of nonlinear domain decomposition methods to improve the nonlinear convergence behavior of Newton's method, they have experienced a rise in popularity recently in the context of problems for which Newton's method…
Nonlinear acceleration methods are powerful techniques to speed up fixed-point iterations. However, many acceleration methods require storing a large number of previous iterates and this can become impractical if computational resources are…
Adaptive protocols enable the construction of more efficient state preparation circuits in variational quantum algorithms (VQAs) by utilizing data obtained from the quantum processor during the execution of the algorithm. This idea…
Training reinforcement learning (RL) agents often requires significant computational resources and prolonged training durations. To address this challenge, we build upon prior work that introduced a neural architecture with…
Non-linear least squares solvers are used across a broad range of offline and real-time model fitting problems. Most improvements of the basic Gauss-Newton algorithm tackle convergence guarantees or leverage the sparsity of the underlying…
Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This raises a key question for large-scale training: how much performance is forfeited by these…