Related papers: The simplified topological $\varepsilon$--algorith…
Slowly convergent series and sequences as well as divergent series occur quite frequently in the mathematical treatment of scientific problems. In this report, a large number of mainly nonlinear sequence transformations for the acceleration…
Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed…
Sequence transformations are important tools for the convergence acceleration of slowly convergent scalar sequences or series and for the summation of divergent series. Transformations that depend not only on the sequence elements or…
Convergence problems occur abundantly in all branches of mathematics or in the mathematical treatment of the sciences. Sequence transformations are principal tools to overcome convergence problems of the kind. They accomplish this by…
We construct new sequence transformations based on Wynn's epsilon and rho algorithms. The recursions of the new algorithms include the recursions of Wynn's epsilon and rho algorithm and of Osada's generalized rho algorithm as special cases.…
This paper introduces a new mathematical framework for analysis and optimization of tensor expressions within an enclosing loop. Tensors are multi-dimensional arrays of values. They are common in high performance computing (HPC) and machine…
This paper examines a number of extrapolation and acceleration methods, and introduces a few modifications of the standard Shanks transformation that deal with general sequences. One of the goals of the paper is to lay out a general…
Deep learning became the method of choice in recent year for solving a wide variety of predictive analytics tasks. For sequence prediction, recurrent neural networks (RNN) are often the go-to architecture for exploiting sequential…
We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above…
A method is presented to exploit adaptive integration algorithms using importance sampling, like VEGAS, for the task of scanning theoretical predictions depending on a multi-dimensional parameter space. Usually, a parameter scan is…
In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and…
The weighted ensemble (WE) method, an enhanced sampling approach based on periodically replicating and pruning trajectories in a set of parallel simulations, has grown increasingly popular for computational biochemistry problems, due in…
Stochastic variance reduction has proven effective at accelerating first-order algorithms for solving convex finite-sum optimization tasks such as empirical risk minimization. Incorporating second-order information has proven helpful in…
In this paper, we give a multistep extension of the epsilon-algorithm of Wynn, and we show that it implements a multistep extension of the Shanks' sequence transformation which is defined by ratios of determinants. Reciprocally, the…
Variational Physics-Informed Neural Networks (VPINNs) utilize a variational loss function to solve partial differential equations, mirroring Finite Element Analysis techniques. Traditional hp-VPINNs, while effective for high-frequency…
We present Epsilon, a system for general convex programming using fast linear and proximal operators. As with existing convex programming frameworks, users specify convex optimization problems using a natural grammar for mathematical…
Despite their simple intuition, convolutions are more tedious to analyze than dense layers, which complicates the transfer of theoretical and algorithmic ideas to convolutions. We simplify convolutions by viewing them as tensor networks…
Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences. To address this limitation, we express the…
Despite recent advances in subquadratic attention mechanisms or state-space models, processing long token sequences still imposes significant computational requirements. Token merging has emerged as a solution to increase computational…
We present an acceleration method for sequences of large-scale linear systems, such as the ones arising from the numerical solution of time-dependent partial differential equations coupled with algebraic constraints. We discuss different…