相关论文: Asymptotic Singular Value Distribution of Linear C…

We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation. This characterization also leads to an algorithm for projecting a…

Since the control of the Lipschitz constant has a great impact on the training stability, generalization, and robustness of neural networks, the estimation of this value is nowadays a real scientific challenge. In this paper we introduce a…

This paper leverages the use of \emph{Gram iteration} an efficient, deterministic, and differentiable method for computing spectral norm with an upper bound guarantee. Designed for circular convolutional layers, we generalize the use of the…

We consider block-structured matrices $A_n$, where the blocks are of (block) unilevel Toeplitz type with $s\times t$ matrix-valued generating functions. Under mild assumptions on the size of the (rectangular) blocks, the asymptotic…

In general, convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of…

Pseudospectral analysis serves as a powerful tool in matrix computation and the study of both linear and nonlinear dynamical systems. Among various numerical strategies, random sampling, especially in the form of rank-$1$ perturbations,…

Consider the ensembles of real symmetric Toeplitz matrices and real symmetric Hankel matrices whose entries are i.i.d. random variables chosen from a fixed probability distribution p of mean 0, variance 1, and finite higher moments.…

This paper tackles the problem of Lipschitz regularization of Convolutional Neural Networks. Lipschitz regularity is now established as a key property of modern deep learning with implications in training stability, generalization,…

Token uniformity is commonly observed in transformer-based models, in which different tokens share a large proportion of similar information after going through stacked multiple self-attention layers in a transformer. In this paper, we…

We study the spectral norm of large rectangular random Toeplitz and circulant matrices with independent entries. For Toeplitz matrices, we show that the scaled norm converges to the norm of a bilinear operator defined via the pointwise…

Consider the ensemble of real symmetric Toeplitz matrices, each independent entry an i.i.d. random variable chosen from a fixed probability distribution p of mean 0, variance 1, and finite higher moments. Previous investigations showed that…

In recent years more and more involved block structures appeared in the literature in the context of numerical approximations of complex infinite dimensional operators modeling real-world applications. In various settings, thanks the theory…

In deep neural networks, the spectral norm of the Jacobian of a layer bounds the factor by which the norm of a signal changes during forward/backward propagation. Spectral norm regularizations have been shown to improve generalization,…

Although Convolution Neural Networks (CNNs) has made substantial progress in the low-light image enhancement task, one critical problem of CNNs is the paradox of model complexity and performance. This paper presents a novel SurroundNet…

Recent studies have shown that aggregating convolutional features of a pre-trained Convolutional Neural Network (CNN) can obtain impressive performance for a variety of visual tasks. The symmetric Positive Definite (SPD) matrix becomes a…

We study random matrices acting on tensor product spaces which have been transformed by a linear block operation. Using operator-valued free probability theory, under some mild assumptions on the linear map acting on the blocks, we compute…

In the study of condensed matter physics, spectral information plays an important role for understand the mechanism of materials. However, it is difficult to obtain the spectrum directly through experiments or simulation. For example, the…

A positive semidefinite Toeplitz matrix, which often arises as the finite covariance matrix of a stationary random process, can be decomposed as the sum of a nonnegative multiple of the identity corresponding to a white noise, and a…

The spectral symbols are useful tools to analyse the eigenvalue distribution when dealing with high dimensional linear systems. Given a matrix sequence with an asymptotic symbol, the last one depends only on the spectra of the individual…

We explore the block nature of the matrix representation of multiplex networks, introducing a new formalism to deal with its spectral properties as a function of the inter-layer coupling parameter. This approach allows us to derive…