Related papers: A Generalization Method of Partitioned Activation …
Researchers are often perplexed when their machine learning algorithms are required to deal with complex number. Various strategies are commonly employed to project complex number into real number, although it is frequently sacrificing the…
This paper expresses the structure of artificial neural network (ANN) as a functional form, using the activation integral concept derived from the activation function. In this way, the structure of ANN can be represented by a simple…
In this paper, we propose novel quaternion activation functions where we modify either the quaternion magnitude or the phase, as an alternative to the commonly used split activation functions. We define criteria that are relevant for…
Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers. However, several analytical properties of the complex domain (e.g., holomorphicity) make…
In this work a rationalized algorithm for calculating the quotient of two complex numbers is presented which reduces the number of underlying real multiplications. The performing of a complex number division using the naive method takes 4…
We introduce an efficient algorithm, called partition of unity extension or PUX, to construct an extension of desired regularity of a function given on a complex multiply connected domain in $2D$. Function extension plays a fundamental role…
Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible…
A complex-analytic structure within the unit disk of the complex plane is presented. It can be used to represent and analyze a large class of real functions. It is shown that any integrable real function can be obtained by means of the…
We construct explicit easily implementable polynomial approximations of sufficiently high accuracy for locally constant functions on the union of disjoint segments. This problem has important applications in several areas of numerical…
Successive linear transforms followed by nonlinear "activation" functions can approximate nonlinear functions to arbitrary precision given sufficient layers. The number of necessary layers is dependent on, in part, by the nature of the…
For a real function, automatic differentiation is such a standard algorithm used to efficiently compute its gradient, that it is integrated in various neural network frameworks. However, despite the recent advances in using complex…
Consider a binary classification problem solved using a feed-forward artificial neural network (ANN). Let the ANN be composed of a ReLU layer and several linear layers (convolution, sum-pooling, or fully connected). We assume the network…
Our work proposes a novel approach to designing activation functions by focusing on their gradients and deriving the corresponding activation functions using integration. We introduce the Expanded Integral of the Exponential Linear Unit…
Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique…
Activation function is crucial to the recent successes of deep neural networks. In this paper, we first propose a new activation function, Multiple Parametric Exponential Linear Units (MPELU), aiming to generalize and unify the rectified…
Real numbers provide a sufficient description of classical physics and all measurable phenomena; however, complex numbers are occasionally utilized as a convenient mathematical tool to aid our calculations. On the other hand, the formalism…
Activation functions are essential to introduce nonlinearity into neural networks, with the Rectified Linear Unit (ReLU) often favored for its simplicity and effectiveness. Motivated by the structural similarity between a shallow…
The number partition problem is a well-known problem, which is one of 21 Karp's NP-complete problems \cite{karp}. The partition function is a boolean function that is equivalent to the number partition problem with number range restricted.…
We give a method of solution to the problem of iterating holomorphic functions to fractional or complex heights. We construct an auxiliary function from natural iterates of a holomorphic function; the auxiliary function will be…
A simple approach is proposed to obtain complexity controls for neural networks with general activation functions. The approach is motivated by approximating the general activation functions with one-dimensional ReLU networks, which reduces…