Related papers: Adaptive Convolutions
Behavior of neural networks is irremediably determined by the specific loss and data used during training. However it is often desirable to tune the model at inference time based on external factors such as preferences of the user or…
The Steklov function $\mu_f(\cdot,t)$ is defined to average a continuous function $f$ at each point of its domain by using a window of size given by $t>0$. It has traditionally been used to approximate $f$ smoothly with small values of $t$.…
Adjusting for an unmeasured confounder is generally an intractable problem, but in the spatial setting it may be possible under certain conditions. In this paper, we derive necessary conditions on the coherence between the treatment…
Adaptive cubic regularization methods have emerged as a credible alternative to linesearch and trust-region for smooth nonconvex optimization, with optimal complexity amongst second-order methods. Here we consider a general/new class of…
Convolutional neural network is an important model in deep learning. To avoid exploding/vanishing gradient problems and to improve the generalizability of a neural network, it is desirable to have a convolution operation that nearly…
The free metaplectic transformation (FMT) is widely used in many fields such as filter design, pattern recognition, image processing and optics. In order to obtain a more concise and intuitive convolution form, this paper studies two kinds…
Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias and the weight parameters. The bias catches the center of the…
We focus on establishing the foundational paradigm of a novel optimization theory based on convolution with convex kernels. Our goal is to devise a morally deterministic model of locating the global optima of an arbitrary function, which is…
It is well-known that the high computational complexity and the insufficient samples in large-scale array signal processing restrict the real-world applications of the conventional full-dimensional adaptive beamforming (sample matrix…
Smooth activation functions are ubiquitous in modern deep learning, yet their theoretical advantages over non-smooth counterparts remain poorly understood. In this work, we study both approximation and statistical properties of neural…
We tensorize the Faber spline system from [14] to prove sequence space isomorphisms for multivariate function spaces with higher mixed regularity. The respective basis coefficients are local linear combinations of discrete function values…
The approximation properties of the finite element method can often be substantially improved by choosing smooth high-order basis functions. It is extremely difficult to devise such basis functions for partitions consisting of arbitrarily…
Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in target tasks. Fine-tuning is able to alleviate the challenge…
We study the problem of optimizing a function under a \emph{budgeted number of evaluations}. We only assume that the function is \emph{locally} smooth around one of its global optima. The difficulty of optimization is measured in terms of…
The mutation strength adaptation properties of a multi-recombinative $(\mu/\mu_I, \lambda)$-ES are studied for isotropic mutations. To this end, standard implementations of cumulative step-size adaptation (CSA) and mutative self-adaptation…
In the present paper we consider the problem of estimating a periodic $(r+1)$-dimensional function $f$ based on observations from its noisy convolution. We construct a wavelet estimator of $f$, derive minimax lower bounds for the $L^2$-risk…
We consider the estimation of the slope function in functional linear regression, where scalar responses are modeled in dependence of random functions. Cardot and Johannes [J. Multivariate Anal. 101 (2010) 395-408] have shown that a…
We study the convolution function $$ C[f(x)] := \int_1^x f(y)f({x\over y}) {{\rm d} y\over y} $$ when $f(x)$ is a suitable number-theoretic error term. Asymptotics and upper bounds for $C[f(x)]$ are derived from mean square bounds for…
We present here a new method for approximating functions defined on superreflexive Banach spaces by differentiable functions with $\alpha$-H\"older derivatives (for some $0<\alpha\leq 1$). The smooth approximation is given by means of an…
This paper deals with estimation with functional covariates. More precisely, we aim at estimating the regression function $m$ of a continuous outcome $Y$ against a standard Wiener coprocess $W$. Following Cadre and Truquet (2015) and Cadre,…