Related papers: Faster arithmetic for number-theoretic transforms
The performance of neural networks improves when more parameters are used. However, the model sizes are constrained by the available on-device memory during training and inference. Although applying techniques like quantization can…
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an…
We study the effect of a splitting operator S_t on the L^p norm of the Fourier transform of a function f and on the operator norm of a Fourier multiplier m. Most of our results assume p is an even integer, and are often stronger when f or m…
Fast Fourier transform was included in the Top 10 Algorithms of 20th Century by Computing in Science & Engineering. In this paper, we provide a new simple derivation of both the discrete Fourier transform and fast Fourier transform by means…
Sparsity is regarded as a desirable property of representations, especially in terms of explanation. However, its usage has been limited due to the gap with dense representations. Most NLP research progresses in recent years are based on…
Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However,…
Discrete Fourier transforms~(DFTs) over finite fields have widespread applications in error correction coding. Hence, reducing the computational complexities of DFTs is of great significance, especially for long DFTs as increasingly longer…
We define the notion of the Fourier transform for the rook monoid (also called the symmetric inverse semigroup) and provide two efficient divide-and-conquer algorithms (fast Fourier transforms, or FFTs) for computing it. This paper marks…
In order to avoid unnecessary applications of Miller-Rabin algorithm to the number in question, we resort to trial division by a few initial prime numbers, since such a division take less time. How far we should go with such a division is…
We describe a methodology for designing efficient parallel and distributed scientific software. This methodology utilizes sequences of mechanizable algebra--based optimizing transformations. In this study, we apply our methodology to the…
By exploiting the correlation between the structure and the solution of Mixed-Integer Linear Programming (MILP), Machine Learning (ML) has become a promising method for solving large-scale MILP problems. Existing ML-based MILP solvers…
This work aims to introduce the framework of polynomial optimization theory to solve fractional polynomial problems (FPPs). Unlike other widely used optimization frameworks, the proposed one applies to a larger class of FPPs, not…
A large share of today's HPC workloads is used for Ab-Initio Molecular Dynamics (AIMD) simulations, where the interatomic forces are computed on-the-fly by means of accurate electronic structure calculations. They are computationally…
The discrete Fourier transform has proven to be an essential tool in many geometric and combinatorial problems in vector spaces over finite fields. In general, sets with good uniform bounds for the Fourier transform appear more `random' and…
The $N$th power of a polynomial matrix of fixed size and degree can be computed by binary powering as fast as multiplying two polynomials of linear degree in~$N$. When Fast Fourier Transform (FFT) is available, the resulting complexity is…
An exact, one-to-one transform is presented that not only allows digital circular convolutions, but is free from multiplications and quantisation errors for transform lengths of arbitrary powers of two. The transform is analogous to the…
We study compute efficiency of LLM training when using different parameterizations, i.e., rules for adjusting model and optimizer hyperparameters (HPs) as model size changes. Some parameterizations fail to transfer optimal base HPs (such as…
Neural document ranking approaches, specifically transformer models, have achieved impressive gains in ranking performance. However, query processing using such over-parameterized models is both resource and time intensive. In this paper,…
Pre-training and then fine-tuning large language models is commonly used to achieve state-of-the-art performance in natural language processing (NLP) tasks. However, most pre-trained models suffer from low inference speed. Deploying such…
In the paper it is shown that there exist infinite classes of fast DFT algorithms having multiplicative complexity lower than O(NlogN), i.e. smaller than their arithmetical complexity. The derivation starts with nesting of Discrete Fourier…