Related papers: Preconditioning the Restarted and Shifted Block FO…
We investigate the application of windowed Fourier frames (WFFs) to the numerical solution of partial differential equations, focussing on elliptic equations. The action of a partial differential operator (PDO) on a windowed plane wave is…
This paper develops the preconditioning technique as a method to address the accuracy issue caused by ill-conditioning. Given a preconditioner $M$ for an ill-conditioned linear system $Ax=b$, we show that, if the inverse of the…
Radial basis functions provide highly useful and flexible interpolants to multivariate functions. Further, they are beginning to be used in the numerical solution of partial differential equations. Unfortunately, their construction requires…
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods, that involve second derivatives and/or second…
The matrix exponential is a fundamental operator in scientific computing and system simulation, with applications ranging from control theory and quantum mechanics to modern generative machine learning. While Pad\'e approximants combined…
Under appropriate technical assumptions, the simple-loop theory allows to deduce various types of asymptotic expansions for the eigenvalues of Toeplitz matrices $T_{n}(f)$ generated by a function $f$, unfortunately, such a theory is not…
In this paper, we begin the exploration of vertex-ordering problems through the lens of exponential-time approximation algorithms. In particular, we ask the following question: Can we simultaneously beat the running times of the fastest…
In recent years, there has been a renewed interest in preconditioning for multilevel Toeplitz systems, a research field that has been extensively explored over the past several decades. This work introduces novel preconditioning strategies…
Preconditioned stochastic optimization algorithms, exemplified by Shampoo, outperform first-order optimizers by offering theoretical convergence benefits and practical gains in large-scale neural network training. However, they incur…
In this work, we propose a new splitting algorithm for solving structured monotone inclusion problems composed of a maximally monotone operator, a maximally monotone and Lipschitz continuous operator and a cocoercive operator. Our method…
We present a fast algorithm for modular exponentiation when the factorization of the modulus is known. Let $a,n,m$ be positive integers and suppose $m$ factors canonically as $\prod_{i=1}^k p_i^{e_i}$. Choose integer parameters $t_i\in [1,…
We derive sharp approximation error bounds for inverse block Toeplitz matrices associated with multivariate long-memory stationary processes. The error bounds are evaluated for both column and row sums. These results are used to prove the…
A block lower triangular Toeplitz system arising from time-space fractional diffusion equation is discussed. For efficient solutions of such the linear system, the preconditioned biconjugate gradient stabilized method and flexible general…
The efficient solution of moderately large-scale linear systems arising from the KKT conditions in optimal control problems (OCPs) is a critical challenge in robotics. With the stagnation of Moore's law, there is growing interest in…
This paper presents a new parameter estimation algorithm for the adaptive control of a class of time-varying plants. The main feature of this algorithm is a matrix of time-varying learning rates, which enables parameter estimation error…
Large language model (LLM) inference systems face a fundamental tension between minimizing Time-to-First-Token (TTFT) latency for new requests and maintaining a high, steady token generation rate (low Time-Per-Output-Token, or TPOT) for…
In this paper we describe a systematic procedure to analyze the convergence of degenerate preconditioned proximal point algorithms. We establish weak convergence results under mild assumptions that can be easily employed in the context of…
For smooth finite fields $F_q$ (i.e., when $q-1$ factors into small primes) the Fast Fourier Transform (FFT) leads to the fastest known algebraic algorithms for many basic polynomial operations, such as multiplication, division,…
Submodular maximization subject to matroid constraints is a central problem with many applications in machine learning. As algorithms are increasingly used in decision-making over datapoints with sensitive attributes such as gender or race,…
In this paper, we consider an unconstrained stochastic optimization problem where the objective function exhibits high-order smoothness. Specifically, we propose a new stochastic first-order method (SFOM) with multi-extrapolated momentum,…