Related papers: LULU operators for functions of continuous argumen…

We consider neural networks with rational activation functions. The choice of the nonlinear activation function in deep learning architectures is crucial and heavily impacts the performance of a neural network. We establish optimal bounds…

Gated Linear Units (arXiv:1612.08083) consist of the component-wise product of two linear projections, one of which is first passed through a sigmoid function. Variations on GLU are possible, using different nonlinear (or even linear)…

Ranges of the real-valued parameters $\alpha$, $a$, $b$, and $m$ are identified for which the operator $$\mathcal{A}_{\alpha}(a,b)f(x):=x^\alpha\left(f''(x)+\frac{a}{x}f'(x)+\frac{b}{x^2}f(x)\right), \quad x>0,$$ generates an analytic…

We point out that (continuous or discontinuous) piecewise linear functions on a convex polytope mesh can be represented by two-hidden-layer ReLU neural networks in a weak sense. In addition, the numbers of neurons of the two hidden layers…

In the present paper, we introduce three neural network operators of convolution type activated by symmetrized, deformed and parametrized B-generalized logistic function. We deal with the approximation properties of these operators to the…

The Rectified Linear Unit is currently a state-of-the-art activation function in deep convolutional neural networks. To combat ReLU's dying neuron problem, we propose the Parametric Variational Linear Unit (PVLU), which adds a sinusoidal…

It is difficult to describe in mathematical terms what a neural network trained on data represents. On the other hand, there is a growing mathematical understanding of what neural networks are in principle capable of representing.…

Let $\mathcal{L}(X;Y)$ be the space of bounded linear operators from a Banach space $X$ to a Banach space $Y$. Given an operator-valued function $u:\mathbb{R}_{\geq 0}\rightarrow \mathcal{L}(X;Y)$, suppose that every orbit $t\mapsto u(t)x$…

We show that, under certain regularity assumptions, there exists a linear extension operator.

In this article we identify a general class of high-dimensional continuous functions that can be approximated by deep neural networks (DNNs) with the rectified linear unit (ReLU) activation without the curse of dimensionality. In other…

In this work we investigate and characterize linear functionals $L:\mathbb{R}[x_1,\dots,x_n]\to\mathbb{R}$ with absolutely continuous representing measures $\mu$, i.e., $\mathrm{d}\mu(x) = g(x)\,\mathrm{d} x$ for some density $g$. We focus…

Recently, self-normalizing neural networks (SNNs) have been proposed with the intention to avoid batch or weight normalization. The key step in SNNs is to properly scale the exponential linear unit (referred to as SELU) to inherently…

We build on the work by Davies, extending the Helffer-Sj\"ostrand Functional Calculus domain for semi-bounded operators on Banach spaces given a priori controlled growth of the resolvents. We employ Seeley's Extension Theorem to extend…

We propose the Hyperbolic Tangent Exponential Linear Unit (TeLU), a neural network hidden activation function defined as TeLU(x)=xtanh(exp(x)). TeLU's design is grounded in the core principles of key activation functions, achieving strong…

Let $X$ and $Y$ be separable Banach spaces and $T:X\to Y$ be a bounded linear operator. We characterize the non-separability of $T^*(Y^*)$ by means of fixing properties of the operator $T$.

In this paper the spectrum of composition operators on the space of real analytic functions is investigated. In some cases it is completely determined while in some other cases it is only estimated.

Multivalued linear operators, also known as linear relations, are studied on a specific class of weighted, composition transforms on Fock space. Basic properties of this class of linear relations, such as closed graph, boundedness, complex…

Associated to each set $S$ of simple roots for $SL(n,\mathbb{C})$ is an equivariant fibration $X\to X_S$ of the space $X$ of complete flags of $\mathbb{C}^n$. To each such fibration we associate an algebra $J_S$ of operators on $L^2(X)$…

Form factors of composite operators in the SL(2) sector of N=4 SYM theory are studied up to two loops via the on-shell unitarity method. The non-compactness of this subsector implies the novel feature and technical challenge of an unlimited…

A leading twist expansion in terms of bi-local operators is proposed for the structure functions of deeply inelastic scattering near the elastic limit $x \to 1$, which is also applicable to a range of other processes. Operators of…