Related papers: LULU operators for functions of continuous argumen…
Neural networks have to capture mathematical relationships in order to learn various tasks. They approximate these relations implicitly and therefore often do not generalize well. The recently proposed Neural Arithmetic Logic Unit (NALU) is…
Reduction operators (called also nonclassical or $Q$-conditional symmetries) of variable coefficient semilinear reaction-diffusion equations with exponential source $f(x)u_t=(g(x)u_x)_x+h(x)e^{mu}$ are investigated using the algorithm…
We introduce sliced ReLU attention, a new attention mechanism that departs structurally from both softmax and its approximation alternatives. Instead of applying a nonlinearity to pairwise dot products, we operate on one-dimensional…
Amongst others, the adoption of Rectified Linear Units (ReLUs) is regarded as one of the ingredients of the success of deep learning. ReLU activation has been shown to mitigate the vanishing gradient issue, to encourage sparsity in the…
We introduce certain linear positive operators and study some approximation properties of these operators in the space of functions, continuous on a compact set, of two variables. We also find the order of this approximation by using…
Rectified linear units (ReLU) are commonly used in deep neural networks. So far ReLU and its generalizations (non-parametric or parametric) are static, performing identically for all input samples. In this paper, we propose dynamic ReLU…
Most of the special functions of mathematical physics are connected with the representation of Lie groups. The action of elements $D$ of the associated Lie algebras as linear differential operators gives relations among the functions in a…
We define strongly continuous max-additive and max-plus linear operator semigroups and study their main properties. We present some important examples of such semigroups coming from non-linear evolution equations.
We introduce multilinear operators, that generalize Hirota's bilinear $D$ operator, based on the principle of gauge invariance of the $\tau$ functions. We show that these operators can be constructed systematically using the bilinear $D$'s…
Reduction operators, i.e. the operators of nonclassical (or conditional) symmetry of a class of variable coefficient nonlinear wave equations with power nonlinearities is investigated within the framework of singular reduction operator. A…
The dominant paradigm in modern neural networks relies on simple, monotonically-increasing activation functions like ReLU. While effective, this paradigm necessitates large, massively-parameterized models to approximate complex functions.…
The activation function is at the heart of a deep neural networks nonlinearity; the choice of the function has great impact on the success of training. Currently, many practitioners prefer the Rectified Linear Unit (ReLU) due to its…
The main objective of this article is to present $\nu$-fractional derivative $\mu$-differentiable functions by considering 4-parameters extended Mittag-Leffler function (MLF). We investigate that the new $\nu$-fractional derivative…
LU-factorization of matrices is one of the fundamental algorithms of linear algebra. The widespread use of supercomputers with distributed memory requires a review of traditional algorithms, which were based on the common memory of a…
We prove that for a homogeneous linear partial differential operator $\mathcal A$ of order $k \le 2$ and an integrable map $f$ taking values in the essential range of that operator, there exists a function $u$ of special bounded variation…
In recent years, there has been growing interest in the field of functional neural networks. They have been proposed and studied with the aim of approximating continuous functionals defined on sets of functions on Euclidean domains. In this…
ReLU (rectified linear units) neural network has received significant attention since its emergence. In this paper, a univariate ReLU (UReLU) neural network is proposed to both modelling the nonlinear dynamic system and revealing insights…
In the present paper we continue the project of systematic construction of invariant differential operators on the example of the non-compact algebras $su(n,n)$. Earlier were given the main multiplets of indecomposable elementary…
We construct bounded linear operators that map $H^1$ conforming Lagrange finite element spaces to $H^2$ conforming virtual element spaces in two and three dimensions. These operators are useful for the analysis of nonstandard finite element…
In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions. We…