Related papers: Parameter-Efficient Fine-Tuning with Circulant and…

There is growing interest in learning Fourier domain sampling strategies (particularly for magnetic resonance imaging, MRI) using optimization approaches. For non-Cartesian sampling patterns, the system models typically involve non-uniform…

Continuous neural representations have recently emerged as a powerful and flexible alternative to classical discretized representations of signals. However, training them to capture fine details in multi-scale signals is difficult and…

Discrete trigonometric transformations, such as the discrete Fourier and cosine/sine transforms, are important in a variety of applications due to their useful properties. For example, one well-known property is the convolution theorem for…

Numeric modeling of electromagnetics and acoustics frequently entails matrix-vector multiplication with block Toeplitz structure. When the corresponding block Toeplitz matrix is not highly sparse, e.g. when considering the electromagnetic…

Channel estimation is one of the most important parts in current mobile communication systems. Among the huge contributions in channel estimation studies, the discrete Fourier transform (DFT)-based channel estimation has attracted lots of…

Foundation models have revolutionized artificial intelligence by providing robust, versatile architectures pre-trained on large-scale datasets. However, adapting these massive models to specific downstream tasks requires fine-tuning, which…

The number of phase wraps in 2D wrapped phase map can be completely eliminated, or greatly reduced by frequency shifting. But the wraps usually cannot be optimally reduced using the conventional fast Fourier transform (FFT) because the…

We propose a simple architecture for deep reinforcement learning by embedding inputs into a learned Fourier basis and show that it improves the sample efficiency of both state-based and image-based RL. We perform infinite-width analysis of…

Discrete Fourier transforms~(DFTs) over finite fields have widespread applications in error correction coding. Hence, reducing the computational complexities of DFTs is of great significance, especially for long DFTs as increasingly longer…

Adapting vision transformer foundation models through parameter-efficient fine-tuning (PEFT) methods has become increasingly popular. These methods optimize a limited subset of parameters, enabling efficient adaptation without the need to…

Performance of neural networks can be significantly improved by encoding known invariance for particular tasks. Many image classification tasks, such as those related to cellular imaging, exhibit invariance to rotation. We present a novel…

Low-bit floating-point (FP) formats, such as FP8, provide significant acceleration and memory savings in model training thanks to native hardware support on modern GPUs and NPUs. However, we analyze that FP8 quantization offers speedup…

We present a method for training a deep neural network containing sinusoidal activation functions to fit to time-series data. Weights are initialized using a fast Fourier transform, then trained with regularization to improve…

Large Language Model training with 8-bit floating point (FP8) formats promises significant efficiency improvements, but reduced numerical precision makes training challenging. It is currently possible to train in FP8 only if one is willing…

Image Representation learning via input reconstruction is a common technique in machine learning for generating representations that can be effectively utilized by arbitrary downstream tasks. A well-established approach is using…

We present a differentiation framework for plane-wave density-functional theory (DFT) that combines the strengths of forward-mode algorithmic differentiation (AD) and density-functional perturbation theory (DFPT). In the resulting AD-DFPT…

This work presents a method of computing Voigt functions and their derivatives, to high accuracy, on a uniform grid. It is based on an adaptation of Fourier-transform based convolution. The relative error of the result decreases as the…

Pruning is an effective way to reduce the huge inference cost of Transformer models. However, prior work on pruning Transformers requires retraining the models. This can add high training cost and high complexity to model deployment, making…

As a well-established approach, factorization machine (FM) is capable of automatically learning high-order interactions among features to make predictions without the need for manual feature engineering. With the prominent development of…

Computational micromechanics and homogenization require the solution of the mechanical equilibrium of a periodic cell that comprises a (generally complex) microstructure. Techniques that apply the Fast Fourier Transform have attracted much…