Related papers: Deep convolutional neural networks and data approx…
Transformers are a type of neural network that have demonstrated remarkable performance across various domains, particularly in natural language processing tasks. Motivated by this success, research on the theoretical understanding of…
We propose a novel formulation of deep networks that do not use dot-product neurons and rely on a hierarchy of voting tables instead, denoted as Convolutional Tables (CT), to enable accelerated CPU-based inference. Convolutional layers are…
The Fractional Fourier Transform (FRT) corresponds to an arbitrary-angle rotation in the phase space, e.g. the time-frequency (TF) space, and generalizes the fundamentally important Fourier Transform. FRT applications range from classical…
Graph fractional Fourier transform (GFRFT) is an extension of graph Fourier transform (GFT) that provides an additional fractional analysis tool for graph signal processing (GSP) by generalizing temporal-vertex domain Fourier analysis to…
Frequency estimation is a fundamental problem in many areas. The well-known A&M and its variant estimators have established an estimation framework by iteratively interpolating the discrete Fourier transform (DFT) coefficients. In general,…
In this paper, we present a frequency domain neural network for image super-resolution. The network employs the convolution theorem so as to cast convolutions in the spatial domain as products in the frequency domain. Moreover, the…
The nonlinear Fourier transform (NFT) has recently gained significant attention in fiber optic communications and other engineering fields. Although several numerical algorithms for computing the NFT have been published, the design of…
Progress towards the energy breakthroughs needed to combat climate change can be significantly accelerated through the efficient simulation of atomic systems. Simulation techniques based on first principles, such as Density Functional…
We present approximation theories and efficient training methods for derivative-informed Fourier neural operators (DIFNOs) with applications to PDE-constrained optimization. A DIFNO is an FNO trained by minimizing its prediction error…
Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed…
We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual fully convolutional networks and their continuous-layer…
We derive fundamental lower bounds on the connectivity and the memory requirements of deep neural networks guaranteeing uniform approximation rates for arbitrary function classes in $L^2(\mathbb R^d)$. In other words, we establish a…
We consider functions $f$ of two real variables, given as trigonometric functions over a finite set $F$ of frequencies. This set is assumed to be closed under rotations in the frequency plane of angle $\frac{2k\pi}{M}$ for some integer $M$.…
\citet{farrell2021deep} establish non-asymptotic high-probability bounds for general deep feedforward neural network (with rectified linear unit activation function) estimators, with \citet[Theorem 1]{farrell2021deep} achieving a suboptimal…
Spectral analysis provides one of the most effective paradigms for information-preserving dimensionality reduction, as simple descriptions of naturally occurring signals are often obtained via few terms of periodic basis functions. In this…
We unify the discrete Fourier transform (DFT), discrete cosine transform (DCT), Walsh-Hadamard, Haar wavelet, Karhunen-Lo\`eve transform, and several others along with their continuous counterparts (Fourier transform, Fourier series,…
Filtering of digital signals is accomplished on an Excel spreadsheet using fast Fourier transform (FFT) convolution in which the kernel is either a Gaussian or a cosine modulated Gaussian. Pedagogical examples of low-pass and band-pass…
Estimates of the generalization error are proved for a residual neural network with $L$ random Fourier features layers $\bar z_{\ell+1}=\bar z_\ell + \mathrm{Re}\sum_{k=1}^K\bar b_{\ell k}e^{\mathrm{i}\omega_{\ell k}\bar z_\ell}+…
Function regression/approximation is a fundamental application of machine learning. Neural networks (NNs) can be easily trained for function regression using a sufficient number of neurons and epochs. The forward-forward learning algorithm…
Discrete trigonometric transformations, such as the discrete Fourier and cosine/sine transforms, are important in a variety of applications due to their useful properties. For example, one well-known property is the convolution theorem for…