Related papers: Asymptotic Singular Value Distribution of Linear C…
Matrix completion algorithms recover a low rank matrix from a small fraction of the entries, each entry contaminated with additive errors. In practice, the singular vectors and singular values of the low rank matrix play a pivotal role for…
A fruitful approach for solving signal deconvolution problems consists of resorting to a frame-based convex variational formulation. In this context, parallel proximal algorithms and related alternating direction methods of multipliers have…
Although deep convolutional neural networks (CNNs) have obtained outstanding performance in image superresolution (SR), their computational cost increases geometrically as CNN models get deeper and wider. Meanwhile, the features of…
The classical random matrix theory is mostly focused on asymptotic spectral properties of random matrices as their dimensions grow to infinity. At the same time many recent applications from convex geometry to functional analysis to…
We study the sample complexity of learning one-hidden-layer convolutional neural networks (CNNs) with non-overlapping filters. We propose a novel algorithm called approximate gradient descent for training CNNs, and show that, with high…
The scattering transform is a wavelet-based model of Convolutional Neural Networks originally introduced by S. Mallat. Mallat's analysis shows that this network has desirable stability and invariance guarantees and therefore helps explain…
A considerable amount of research in harmonic analysis has been devoted to non-linear estimators of signals contaminated by additive Gaussian noise. They are implemented by thresholding coefficients in a frame, which provide a sparse signal…
Regularization in convolutional neural networks (CNNs) is usually addressed with dropout layers. However, dropout is sometimes detrimental in the convolutional part of a CNN as it simply sets to zero a percentage of pixels in the feature…
We consider a product of an arbitrary number of independent rectangular Gaussian random matrices. We derive the mean densities of its eigenvalues and singular values in the thermodynamic limit, eventually verified numerically. These…
This paper is devoted to the asymptotic behavior of all eigenvalues of Symmetric (in general non Hermitian) Toeplitz matrices with moderately smooth symbols which trace out a simple loop on the complex plane line as the dimension of the…
Convolution is an efficient technique to obtain abstract feature representations using hierarchical layers in deep networks. Although performing convolution in Euclidean geometries is fairly straightforward, its extension to other…
We present a neighborhood similarity layer (NSL) which induces appearance invariance in a network when used in conjunction with convolutional layers. We are motivated by the observation that, even though convolutional networks have low…
We study two specific symmetric random block Toeplitz (of dimension $k \times k$) matrices: where the blocks (of size $n \times n$) are (i) matrices with i.i.d. entries, and (ii) asymmetric Toeplitz matrices. Under suitable assumptions on…
We establish universal Gaussian fluctuations for the mesoscopic linear eigenvalue statistics in the vicinity of the cusp-like singularities of the limiting spectral density for Wigner-type random matrices. Prior to this work, the linear…
Singular source terms expressed as weighted summations of Dirac-delta functions are regularized through approximation theory with convolution operators. We consider the numerical solution of scalar and one-dimensional hyperbolic…
We show the effectiveness of automatic differentiation in efficiently and correctly computing and controlling the spectrum of implicitly linear operators, a rich family of layer types including all standard convolutional and dense layers.…
The successful training of deep neural networks requires addressing challenges such as overfitting, numerical instabilities leading to divergence, and increasing variance in the residual stream. A common solution is to apply regularization…
Convolutional neural networks are very popular nowadays. Training neural networks is not an easy task. Each convolution corresponds to a structured transformation matrix. In order to help avoid the exploding/vanishing gradient problem, it…
We study the spectrum of an asymmetric random matrix with block structured variances. The rows and columns of the random square matrix are divided into $D$ partitions with arbitrary size (linear in $N$). The parameters of the model are the…
Deep neural networks have enormous representational power which leads them to overfit on most datasets. Thus, regularizing them is important in order to reduce overfitting and enhance their generalization capabilities. Recently, channel…