相关论文: Fantastic Four: Differentiable Bounds on Singular …

Regularization techniques such as L2 regularization (Weight Decay) and Dropout are fundamental to training deep neural networks, yet their underlying physical mechanisms regarding feature frequency selection remain poorly understood. In…

Previous work of the second author and Wolf showed that given a set $A\subseteq \mathbb{F}_p^n$ of bounded $\textrm{VC}_2$-dimension, there is a high rank quadratic factor $\mathcal{B}$ of bounded complexity such that $A$ is approximately…

We consider a second order, two-point, singularly perturbed boundary value problem, of reaction-convection-diffusion type with two small parameters, and we obtain regularity results for its solution. First we establish classical…

We develop exact representations of training two-layer neural networks with rectified linear units (ReLUs) in terms of a single convex program with number of variables polynomial in the number of training samples and the number of hidden…

Uniform bounds on sketched inner products of vectors or matrices underpin several important computational and statistical results in machine learning and randomized algorithms, including the Johnson-Lindenstrauss (J-L) lemma, the Restricted…

For any given neural network architecture a permutation of weights and biases results in the same functional network. This implies that optimization algorithms used to `train' or `learn' the network are faced with a very large number (in…

It is commonly recognized that the expressiveness of deep neural networks is contingent upon a range of factors, encompassing their depth, width, and other relevant considerations. Currently, the practical performance of the majority of…

The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting…

Regularizing the input gradient has shown to be effective in promoting the robustness of neural networks. The regularization of the input's Hessian is therefore a natural next step. A key challenge here is the computational complexity.…

Deep learning has transformed computer vision, natural language processing, and speech recognition\cite{badrinarayanan2017segnet, dong2016image, ren2017faster, ji20133d}. However, two critical questions remain obscure: (1) why do deep…

The information recoverable from galaxy spectra depends fundamentally on spectral resolution, yet assembling large samples at high resolution remains observationally expensive. We present a deep-learning framework for spectral…

Performance of optimization on quadratic problems sensitively depends on the low-lying part of the spectrum. For large (effectively infinite-dimensional) problems, this part of the spectrum can often be naturally represented or approximated…

We propose a new point of view for regularizing deep neural networks by using the norm of a reproducing kernel Hilbert space (RKHS). Even though this norm cannot be computed, it admits upper and lower approximations leading to various…

The stability of spike deconvolution, which aims at recovering point sources from their convolution with a point spread function (PSF), is known to be related to the separation between those sources. When the observations are noisy, it is…

Boundary value problems for integrable nonlinear evolution PDEs formulated on the finite interval can be analyzed by the unified method introduced by one of the authors and used extensively in the literature. The implementation of this…

Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a…

Geometric variations like rotation, scaling, and viewpoint changes pose a significant challenge to visual understanding. One common solution is to directly model certain intrinsic structures, e.g., using landmarks. However, it then becomes…

Current self-supervised denoising methods for paired noisy images typically involve mapping one noisy image through the network to the other noisy image. However, after measuring the spectral bias of such methods using our proposed Image…

We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently…

Bolton and Schlegel presented a promising deconvolution method to extract 1D spectra from a 2D optical fiber spectral CCD image. The method could eliminate the PSF difference between fibers, extract spectra to the photo noise level, as well…