Related papers: LULU operators for functions of continuous argumen…
Selecting the most suitable activation function is a critical factor in the effectiveness of deep learning models, as it influences their learning capacity, stability, and computational efficiency. In recent years, the Gaussian Error Linear…
Specific global symbol classes and corresponding pseudodifferential operators of infinite order that act continuously on the space of tempered ultradistributions of Beurling and Roumieu type are constructed. For these classes, symbolic…
Non-linear activation functions are crucial in Convolutional Neural Networks. However, until now they have not been well described in the frequency domain. In this work, we study the spectral behavior of ReLU, a popular activation function.…
For an elliptic, semilinear differential operator of the form $S(u) = A : D^2 u + b(x, u , Du)$, consider the functional $E_\infty(u) = \mathop{\mathrm{ess \, sup}}_\Omega |S(u)|$. We study minimisers of $E_\infty$ for prescribed boundary…
In [1], an operator was introduced which acts parallel to the Riemann-Liouville differintegral on a transformation of the space of real analytic functions and commutes with itself. This paper aims to extend the technique - and its defining…
Some concepts, such as non-compactness measure and condensing operators, defined on metric spaces are extended to uniform spaces. Such extensions allow us to locate, in the context of uniform spaces, some classical results existing in…
Artificial neural networks usually consist of successive linear multiply-accumulate operations and nonlinear activation functions. However, most optical neural networks only achieve the linear operation in the optical domain, while the…
A continuous linear operator L defined on the space of entire functions H(C) is said to be an extended $lambda$-eigenoperator of the differentiation operator D provided DL = $lambda$LD. Here we fully characterize when an extended…
We introduce the "exponential linear unit" (ELU) which speeds up learning in deep neural networks and leads to higher classification accuracies. Like rectified linear units (ReLUs), leaky ReLUs (LReLUs) and parametrized ReLUs (PReLUs), ELUs…
We study a system of partial differential equations defined by commuting family of differential operators with regular singularities. We construct ideally analytic solutions depending on a holomorphic parameter. We give some explicit…
A semiregular operator on a Hilbert C^*-module, or equivalently, on the C^*-algebra of `compact' operators on it, is a closable densely defined operator whose adjoint is also densely defined. It is shown that for operators on extensions of…
By considering generalized logarithm and exponential functions used in nonextensive statistics, the four usual algebraic operators : addition, subtraction, product and division, are generalized. The properties of the generalized operators…
A simple version for the extension of the Taylor theorem to the operator functions was found. The expansion was done with respect to a value given by a diagonal matrix for the non-commutative case, and the coefficients are given both by…
In this paper, we introduce a new class of positive linear operators that generalize the classical Bernstein operators. Specifically, we construct a sequence of operators that reproduce the logarithmic function $\ln(1+\mu+x)$, with $\mu >…
Sequential decision-making algorithms such as reinforcement learning (RL) in real-world scenarios inevitably face environments with partial observability. This paper scrutinizes the effectiveness of a popular architecture, namely…
We propose a novel activation function that implements piece-wise orthogonal non-linear mappings based on permutations. It is straightforward to implement, and very computationally efficient, also it has little memory requirements. We…
In a function approximation with a neural network, an input dataset is mapped to an output index by optimizing the parameters of each hidden-layer unit. For a unary function, we present constraints on the parameters and its second…
This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in $\mathbb{R}^d$. Under the assumption that the discontinuity interface $\Gamma$ may be…
Neural Arithmetic Logic Modules have become a growing area of interest, though remain a niche field. These modules are neural networks which aim to achieve systematic generalisation in learning arithmetic and/or logic operations such as…
We consider various systematic ways of defining unbounded operator valued integrals of complex functions with respect to (mostly) positive operator measures and positive sesquilinear form measures, and investigate their relationships to…