Related papers: A Generalization Method of Partitioned Activation …
We show a power 2.5 separation between bounded-error randomized and quantum query complexity for a total Boolean function, refuting the widely believed conjecture that the best such separation could only be quadratic (from Grover's…
Rectified linear activation units are important components for state-of-the-art deep convolutional networks. In this paper, we propose a novel S-shaped rectified linear activation unit (SReLU) to learn both convex and non-convex functions,…
A generalization of the factorization technique is shown to be a powerful algebraic tool to discover further properties of a class of integrable systems in Quantum Mechanics. The method is applied in the study of radial oscillator, Morse…
This paper shows that, for matrix multiplications and convolutions, it is possible to asymptotically replace each real multiplication with a single squaring operation. Similarly, a single complex multiplication can be replaced with 3…
For many biological image segmentation tasks, including topological knowledge, such as the nesting of classes, can greatly improve results. However, most `out-of-the-box' CNN models are still blind to such prior information. In this paper,…
Partition functions of probability distributions are important quantities for model evaluation and comparisons. We present a new method to compute partition functions of complex and multimodal distributions. Such distributions are often…
We propose the Moderate Adaptive Linear Unit (MoLU), a novel activation function for deep neural networks, defined analytically as: f(x)=x \times (1+tanh(x))/2. MoLU combines mathematical elegance with empirical effectiveness, exhibiting…
We describe a unified approach to calculating the partition functions of a general multi-level system with a free Hamiltonian. Particularly, we present new results for parastatistical systems of any order in the second quantized approach.…
The efficacy of deep neural networks is heavily reliant on the design of non-linear activation functions, yet existing approaches often struggle to balance optimization stability with computational efficiency. While piecewise linear…
In a distributed quantum computer scalability is accomplished by networking together many elementary nodes. Typically the network is optical and inter-node entanglement involves photon detection. In complex networks the entanglement…
We present a unified representation of the most popular neural network activation functions. Adopting Mittag-Leffler functions of fractional calculus, we propose a flexible and compact functional form that is able to interpolate between…
A system of commutative complex numbers in 5 dimensions of the form u=x_0+h_1x_1+h_2x_2+h_3x_3+h_4x_4 is described in this paper, the variables x_0, x_1, x_2, x_3, x_4 being real numbers. The operations of addition and multiplication of the…
An appropriate choice of the activation function (like ReLU, sigmoid or swish) plays an important role in the performance of (deep) multilayer perceptrons (MLP) for classification and regression learning. Prototype-based classification…
Verification of neural networks relies on activation functions being piecewise affine (pwa) -- enabling an encoding of the verification problem for theorem provers. In this paper, we present the first formalization of pwa activation…
We present a simple, effective, and general activation function we term ACON which learns to activate the neurons or not. Interestingly, we find Swish, the recent popular NAS-searched activation, can be interpreted as a smooth approximation…
An artificial neuron is modelled as a weighted summation followed by an activation function which determines its output. A wide variety of activation functions such as rectified linear units (ReLU), leaky-ReLU, Swish, MISH, etc. have been…
We show that a real analytic restricted log-exp-analytic function has a holomorphic extension which is again restricted log-exp-analytic. We also establish a parametric version of this result.
In this work, we introduce the concept of Active Representation Learning, a novel class of problems that intertwines exploration and representation learning within partially observable environments. We extend ideas from Active Simultaneous…
This work presents an adaptive activation method for neural networks that exploits the interdependency of features. Each pixel, node, and layer is assigned with a polynomial activation function, whose coefficients are provided by an…
In our previous paper an effective algorithm for inverting polynomial automorphisms was proposed. We extend its application to the case of formal power series over a field of arbitrary characteristic and illustrate the proposed approach…