Related papers: A short report on preconditioned Anderson accelera…
When Perturbation Analysis (PA) yields unbiased sensitivity estimators for expected-value performance functions in discrete event dynamic systems, it can be used for performance optimization of those functions. However, when PA is known to…
The convergence rates of iterative methods for solving a linear system $\mathbf{A} x = b$ typically depend on the condition number of the matrix $\mathbf{A}$. Preconditioning is a common way of speeding up these methods by reducing that…
Despite the broad use of fixed-point iterations throughout applied mathematics, the optimal convergence rate of general fixed-point problems with nonexpansive nonlinear operators has not been established. This work presents an acceleration…
The analysis of the acceleration behavior of gradient-based eigensolvers with preconditioning presents a substantial theoretical challenge. In this work, we present a novel framework for preconditioning on Riemannian manifolds and introduce…
Sparse principal component analysis (PCA), an important variant of PCA, attempts to find sparse loading vectors when conducting dimension reduction. This paper considers the nonsmooth Riemannian optimization problem associated with the…
The Anderson Mixing (AM) method is a popular approach for accelerating fixed-point iterations by leveraging historical information from previous steps. In this paper, we introduce the Riemannian Anderson Mixing (RAM) method, an extension of…
This work investigates the local convergence behavior of Anderson acceleration in solving nonlinear systems. We establish local R-linear convergence results for Anderson acceleration with general depth $m$ under the assumptions that the…
This paper describes the design of a safeguarding scheme for Anderson acceleration to improve its practical performance and stability when used for first-order optimisation methods. We show how the combination of a non-expansiveness…
Despite recent progress in video generation, inference speed remains a major bottleneck. A common acceleration strategy involves reusing model outputs via caching mechanisms at fixed intervals. However, we find that such fixed-frequency…
Despite their frequent slow convergence, proximal gradient schemes are widely used in large-scale optimization tasks due to their tremendous stability, scalability, and ease of computation. In this paper, we develop and investigate a…
Attention-based Transformers have revolutionized natural language processing (NLP) and shown strong performance in computer vision (CV) tasks. However, as the input sequence varies, the computational bottlenecks in Transformer models…
A mapping of the process on a continuous configuration space to the symbolic representation of the motion on a discrete state space will be combined with an iterative aggregation and disaggregation (IAD) procedure to obtain steady state…
In this paper, we introduce a new iteration method and show that this iteration method can be used to approximate fixed point of almost contraction mappings. Furthermore, we prove that the new iteration method is equivalent to both Mann…
The method of alternation projections (MAP) is an iterative procedure for finding the projection of a point on the intersection of closed subspaces of an Hilbert space. The convergence of this method is usually slow, and several methods for…
BoostConv has been introduced in earlier works as an effective acceleration technique for nonlinear iterative processes and has been successfully employed in a variety of applications to enhance convergence rates or to compute unstable…
We study acceleration and preconditioning strategies for a class of Douglas-Rachford methods aiming at the solution of convex-concave saddle-point problems associated with Fenchel-Rockafellar duality. While the basic iteration converges…
For some typical and widely used non-convex half-quadratic regularization models and the Ambrosio-Tortorelli approximate Mumford-Shah model, based on the Kurdyka-\L ojasiewicz analysis and the recent nonconvex proximal algorithms, we…
We develop an Accelerated Back Pressure (ABP) algorithm using Accelerated Dual Descent (ADD), a distributed approximate Newton-like algorithm that only uses local information. Our construction is based on writing the backpressure algorithm…
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data. Both problems can be formulated as…
We consider parameterized variational inverse problems that are constrained by partial differential equations (PDEs). We seek to efficiently compute the solution of the inverse problem when auxiliary model parameters, which appear in the…