Related papers: Asymptotic Singular Value Distribution of Linear C…

Convolution, a cornerstone of signal processing and optical neural networks, has traditionally been implemented by mapping mathematical operations onto complex hardware. Here, we overcome this challenge by revealing that wave dynamics in…

A block decomposition method is proposed for minimizing a (possibly non-convex) continuously differentiable function subject to one linear equality constraint and simple bounds on the variables. The proposed method iteratively selects a…

One of the hallmarks of active galactic nuclei are that they are highly variable with time. In watching the spectra vary it has been observed that the emission-lines often appear to "reverberate" -- that is they vary in response to…

We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently…

In the proposed SEHybridSN model, a dense block was used to reuse shallow feature and aimed at better exploiting hierarchical spatial spectral feature. Subsequent depth separable convolutional layers were used to discriminate the spatial…

We study a class of generalized linear programs (GLP) in a large-scale setting, which includes simple, possibly nonsmooth convex regularizer and simple convex set constraints. By reformulating (GLP) as an equivalent convex-concave min-max…

Structured pruning compresses neural networks by reducing channels (filters) for fast inference and low footprint at run-time. To restore accuracy after pruning, fine-tuning is usually applied to pruned networks. However, too few remaining…

Standard convolutions are prevalent in image processing and deep learning, but their fixed kernels limits adaptability. Several deformation strategies of the reference kernel grid have been proposed. Yet, they lack a unified theoretical…

Lipschitz Bound Estimation is an effective method of regularizing deep neural networks to make them robust against adversarial attacks. This is useful in a variety of applications ranging from reinforcement learning to autonomous systems.…

In two earlier papers, we designed a distributed deterministic asynchronous algorithm for minimizing the sum of subdifferentiable and proximable functions and a regularizing quadratic on time-varying graphs based on Dykstra's algorithm, or…

It is an established fact that a finite difference operator approximates a derivative with a fixed algebraic rate of convergence. Nevertheless, we exhibit a new finite difference operator and prove it has spectral accuracy. Its rate of…

It is known that the generating function $f$ of a sequence of Toeplitz matrices $\{T_n(f)\}_n$ may not describe the asymptotic distribution of the eigenvalues of $T_n(f)$ if $f$ is not real. In a recent paper, we assume as a working…

Continuing our recent work we study polynomial masks of multivariate tight wavelet frames from two additional and complementary points of view: convexity and system theory. We consider such polynomial masks that are derived by means of the…

We provide exact asymptotic expressions for the performance of regression by an $L-$layer deep random feature (RF) model, where the input is mapped through multiple random embedding and non-linear activation functions. For this purpose, we…

We study norm-based uniform convergence bounds for neural networks, aiming at a tight understanding of how these are affected by the architecture and type of norm constraint, for the simple class of scalar-valued one-hidden-layer networks,…

Learned image compression research has achieved state-of-the-art compression performance with auto-encoder based neural network architectures, where the image is mapped via convolutional neural networks (CNN) into a latent representation…

Discrete circular convolution over $\mathbb{Z}/N\mathbb{Z}$ is a linear operator and can be implemented on quantum hardware within the linear-combination-of-unitaries (LCU) framework. In this work, we make this connection explicit through…

This paper reveals a novel linear characteristic exclusive to transformer decoders, including models such as GPT, LLaMA, OPT, BLOOM and others. We analyze embedding transformations between sequential layers, uncovering a near-perfect linear…

Entropic regularization provides a simple way to approximate linear programs whose constraints split into two or more tractable blocks. The resulting objectives are amenable to cyclic Kullback-Leibler (KL) Bregman projections, with…

Linear regression is a fundamental and primitive problem in supervised machine learning, with applications ranging from epidemiology to finance. In this work, we propose methods for speeding up distributed linear regression. We do so by…