Related papers: Faster arithmetic for number-theoretic transforms

Recently, prompt tuning (PT) has gained increasing attention as a parameter-efficient way of tuning pre-trained language models (PLMs). Despite extensively reducing the number of tunable parameters and achieving satisfying performance, PT…

Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences. We introduce two techniques to improve the efficiency of…

Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements. Recent approaches tackle these shortcomings by training smaller models, dynamically reducing the…

Let $p$ be a odd prime such that 2 is a primitive element of finite field $F_p*$. In this short note we propose a new algorithm for the computation of discrete logarithm in $F_p*$. This algorithm is based on elementary properties of finite…

Symmetric Nonnegative Matrix Factorization (SNMF) models arise naturally as simple reformulations of many standard clustering algorithms including the popular spectral clustering method. Recent work has demonstrated that an elementary…

Transformer-based large language models have achieved remarkable performance across various natural language processing tasks. However, they often struggle with seemingly easy tasks like arithmetic despite their vast capabilities. This…

Despite their tremendous success and versatility, Deep Neural Networks (DNNs) such as Large Language Models (LLMs) suffer from inference inefficiency and rely on advanced computational infrastructure. To address these challenges and make…

Scientific computing programs often undergo aggressive compiler optimization to achieve high performance and efficient resource utilization. While performance is critical, we also need to ensure that these optimizations are correct. In this…

Two methods for fast Fourier transforms are used in a quantum context. The first method is for systems with dimension of the Hilbert space $D=d^n$ with $d$ an odd integer, and is inspired by the Cooley-Tukey formalism. The `large Fourier…

Adaptive Fourier decomposition (AFD, precisely 1-D AFD or Core-AFD) was originated for the goal of positive frequency representations of signals. It achieved the goal and at the same time offered fast decompositions of signals. There then…

Fine-tuning a pretrained transformer for a downstream task has become a standard method in NLP in the last few years. While the results from these models are impressive, applying them can be extremely computationally expensive, as is…

We present a new general method for performing basic arithmetic in the finite field~$\mathbb{F}_p$ for any prime $p>2$ by using traditional binary operations over~$\mathbb{F}_2$. Our new approach is efficient and competitive with current…

We consider the problem of the computation of $\inf_p \theta p$ over the set of exponent pairs $P \ni p$ under linear constraints for a certain class of objective functions $\theta$. An effective algorithm is presented. The output of the…

In this paper we present a new efficient variant to compute strong Gr\"obner basis over quotients of principal ideal domains. We show an easy lifting process which allows us to reduce one computation over the quotient $R/nR$ to two…

In this work, we study the problem of monotone non-submodular maximization with partition matroid constraint. Although a generalization of this problem has been studied in literature, our work focuses on leveraging properties of partition…

Cryptographic primitives have been used for various non-cryptographic objectives, such as eliminating or reducing randomness and interaction. We show how to use cryptography to improve the time complexity of solving computational problems.…

Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to specific positions in a sequence, optimising the…

This review article was first published in 2008 as chapter 11 in the book "Fast Fourier Transforms," edited by C. S. Burrus, for the Connexions project at Rice University, which is sadly no longer online. It gives a high-level overview of…

In ISSAC 2017, van der Hoeven and Larrieu showed that evaluating a polynomial P in GF(q)[x] of degree <n at all n-th roots of unity in GF($q^d$) can essentially be computed d-time faster than evaluating Q in GF($q^d$)[x] at all these roots,…

When comparing the fastest algorithm for computing the largest simulation preorder over Kripke structures with the one for labeled transition systems (LTS), there is a noticeable time and space complexity blow-up proportional to the size of…