Related papers: Variations on the Chebyshev-Lagrange Activation Fu…

Activation functions in neural networks are typically selected from a set of empirically validated, commonly used static functions such as ReLU, tanh, or sigmoid. However, by optimizing the shapes of a network's activation functions, we can…

This research is concerned with finding the roots of a function in an interval using Chebyshev Interpolation. Numerical results of Chebyshev Interpolation are presented to show that this is a powerful way to simultaneously calculate all the…

Image resizing is a basic tool in image processing and in literature we have many methods, based on different approaches, which are often specialized in only upscaling or downscaling. In this paper, independently of the (reduced or…

The analysis of complex nonlinear systems is often carried out using simpler piecewise linear representations of them. A principled and practical technique is proposed to linearize and evaluate arbitrary continuous nonlinear functions using…

As function approximators, deep neural networks have served as an effective tool to represent various signal types. Recent approaches utilize multi-layer perceptrons (MLPs) to learn a nonlinear mapping from a coordinate to its corresponding…

Previous works show convergence of rational Chebyshev approximants to the Pad\'e approximant as the underlying domain of approximation shrinks to the origin. In the present work, the asymptotic error and interpolation properties of rational…

In the framework of mapped pseudospectral methods, we introduce a new polynomial-type mapping function in order to describe accurately the dynamics of systems developing almost singular structures. Using error criteria related to the…

In this paper, we propose novel quaternion activation functions where we modify either the quaternion magnitude or the phase, as an alternative to the commonly used split activation functions. We define criteria that are relevant for…

In a previous study [B. Li, S. Tang and H. Yu, Commun. Comput. Phy. 27(2):379-411, 2020], it is shown that deep neural networks built with rectified power units (RePU) as activation functions can give better approximation for sufficient…

Model extraction attacks have renewed interest in the classic problem of learning neural networks from queries. In this work we give the first polynomial-time algorithm for learning arbitrary one hidden layer neural networks activations…

In this article we present new results on neural networks with linear threshold activation functions. We precisely characterize the class of functions that are representable by such neural networks and show that 2 hidden layers are…

Unions of graph Fourier multipliers are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators to the high-dimensional signals…

We consider a distributed optimization problem over a network of agents aiming to minimize a global objective function that is the sum of local convex and composite cost functions. To this end, we propose a distributed Chebyshev-accelerated…

The widespread use of Multi-layer perceptrons (MLPs) often relies on a fixed activation function (e.g., ReLU, Sigmoid, Tanh) for all nodes within the hidden layers. While effective in many scenarios, this uniformity may limit the networks…

This study focuses on constructing efficient rank-1 lattices that enable the exact integration and reconstruction of functions within Chebyshev spaces, based on finite lower index sets. We establish the equivalence of different…

We investigate neural network image reconstruction for magnetic particle imaging. The network performance depends strongly on the convolution effects of the spectrum input data. The larger convolution effect appearing at a relatively…

In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions. We…

An artificial neural network is presented based on the idea of connections between units that are only active for a specific range of input values and zero outside that range (and so are not evaluated outside the active range). The…

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning research. In this paper, we replace the output activation function of DNNs, typically the…

Object recognition is an important task for improving the ability of visual systems to perform complex scene understanding. Recently, the Exponential Linear Unit (ELU) has been proposed as a key component for managing bias shift in…