Related papers: On Polynomial Approximation of Activation Function
ReLU is widely seen as the default choice for activation functions in neural networks. However, there are cases where more complicated functions are required. In particular, recurrent neural networks (such as LSTMs) make extensive use of…
In this paper, we consider the problem of computing estimates of the domain-of-attraction for non-polynomial systems. A polynomial approximation technique, based on multivariate polynomial interpolation and error analysis for remaining…
We introduce a numerical method for generating the approximating polynomials used in fermionic calculations with smeared link actions. We investigate the stability of the algorithm and determine the optimal weight function and the optimal…
In activation network design problems we are given an undirected graph $G=(V,E)$ and a pair of activation costs $\{c_e^u,c_e^v\}$ for each $e=uv \in E$. The goal is to find an edge set $F \subseteq E$ that satisfies a prescribed property of…
Many quantum algorithms for ground-state preparation and energy estimation require the implementation of high-degree polynomials of a Hamiltonian to achieve better convergence rates. Their circuit implementation typically relies on quantum…
This paper discusses how to find the global minimum of functions that are summations of small polynomials (``small'' means involving a small number of variables). Some sparse sum of squares (SOS) techniques are proposed. We compare their…
We present an iterative algorithm for calculating approximate greatest common divisor (GCD) of univariate polynomials with the real or the complex coefficients. For a given pair of polynomials and a degree, our algorithm finds a pair of…
We consider polynomials of a few linear forms and show how exploit this type of sparsity for optimization on some particular domains like the Euclidean sphere or a polytope. Moreover, a simple procedure allows to detect this form of…
We present an extension of our GPGCD method, an iterative method for calculating approximate greatest common divisor (GCD) of univariate polynomials, to multiple polynomial inputs. For a given pair of polynomials and a degree, our algorithm…
Memory footprint is one of the main limiting factors for large neural network training. In backpropagation, one needs to store the input to each operation in the computational graph. Every modern neural network model has quite a few…
We consider the global minimization of smooth functions based solely on function evaluations. Algorithms that achieve the optimal number of function evaluations for a given precision level typically rely on explicitly constructing an…
We propose a proximal variable smoothing algorithm for a nonsmooth optimization problem whose cost function is the sum of three functions including a weakly convex composite function. The proposed algorithm has a single-loop structure…
The purpose of this note is to compare various approximation methods as applied to the inverse of the Bessel function of the first kind, in a given domain of the complex plane.
We seek to approximate a composite function h(x) = g(f(x)) with a global polynomial. The standard approach chooses points x in the domain of f and computes h(x) at each point, which requires an evaluation of f and an evaluation of g. We…
A partial least squares regression is proposed for estimating the function-on-function regression model where a functional response and multiple functional predictors consist of random curves with quadratic and interaction effects. The…
We present a new active learning algorithm based on nonparametric estimators of the regression function. Our investigation provides probabilistic bounds for the rates of convergence of the generalization error achievable by proposed method…
In this paper, we will discuss the notion of almost orthogonality in a functional sequence.Especially, we will define a few sequences of almost orthogonal polynomials which can be used successfully for modeling of electronic systems which…
Factorization of polynomials arises in numerous areas in symbolic computation. It is an important capability in many symbolic and algebraic computation. There are two type of factorization of polynomials. One is convention polynomial…
A method for moving least squares interpolation and differentiation is presented in the framework of orthogonal polynomials on discrete points. This yields a robust and efficient method which can avoid singularities and breakdowns in the…
In this paper we propose a method for the approximation of high-dimensional functions over finite intervals with respect to complete orthonormal systems of polynomials. An important tool for this is the multivariate classical analysis of…