Related papers: On Polynomial Approximation of Activation Function
We investigate a way to approximate the maximum of a polynomial over a polytopal region by using Handelman's polynomial decomposition and continuous multivariate generating functions. The maximization problem is NP-hard, but our…
Motivated by the result that an `approximate' evaluation of the Jones polynomial of a braid at a $5^{th}$ root of unity can be used to simulate the quantum part of any algorithm in the quantum complexity class BQP, and results relating BQP…
The approximate degree of a Boolean function $f \colon \{-1, 1\}^n \rightarrow \{-1, 1\}$ is the least degree of a real polynomial that approximates $f$ pointwise to error at most $1/3$. We introduce a generic method for increasing the…
We initiate the study of active learning polynomial threshold functions (PTFs). While traditional lower bounds imply that even univariate quadratics cannot be non-trivially actively learned, we show that allowing the learner basic access to…
We present a complete algorithm for finding an exact minimal polynomial from its approximate value by using an improved parameterized integer relation construction method. Our result is superior to the existence of error controlling on…
We show the existence of a deep neural network capable of approximating a wide class of high-dimensional approximations. The construction of the proposed neural network is based on a quasi-optimal polynomial approximation. We show that this…
We study the computational power of polynomial threshold functions, that is, threshold functions of real polynomials over the boolean cube. We provide two new results bounding the computational power of this model. Our first result shows…
Gradient descent method, as one of the major methods in numerical optimization, is the key ingredient in many machine learning algorithms. As one of the most fundamental way to solve the optimization problems, it promises the function value…
Universal approximation theorem suggests that a shallow neural network can approximate any function. The input to neurons at each layer is a weighted sum of previous layer neurons and then an activation is applied. These activation…
We introduce an algorithm of joint approximation of a function and its first derivative by alternative orthogonal polynomials on the interval [0,1].The algorithm exhibits properties of shape preserving approximation for the function. A weak…
The proximal gradient descent method, well-known for composite optimization, can be completely described by the concept of proximal gradient mapping. In this paper, we highlight our previous two discoveries of proximal gradient…
Structured convex optimization problems typically involve a mix of smooth and nonsmooth functions. The common practice is to activate the smooth functions via their gradient and the nonsmooth ones via their proximity operator. We show that,…
We consider least squares approximation of a function of one variable by a continuous, piecewise-linear approximand that has a small number of breakpoints. This problem was notably considered by Bellman who proposed an approximate algorithm…
This paper presents a polynomial-time $1/2$-approximation algorithm for maximizing nonnegative $k$-submodular functions. This improves upon the previous $\max\{1/3, 1/(1+a)\}$-approximation by Ward and \v{Z}ivn\'y~(SODA'14), where…
Let $\mathbb{F}_p$ be the finite field of prime order $p$. For any function $f \colon \mathbb{F}_p{}^n \to \mathbb{F}_p$, there exists a unique polynomial over $\mathbb{F}_p$ having degree at most $p-1$ with respect to each variable which…
We present a new rational approximation algorithm based on the empirical interpolation method for interpolating a family of parametrized functions to rational polynomials with invariant poles, leading to efficient numerical algorithms for…
Motivated by conforming finite element methods for elliptic problems of second order, we analyze the approximation of the gradient of a target function by continuous piecewise polynomial functions over a simplicial mesh. The main result is…
Multivariate functions encountered in high-dimensional uncertainty quantification problems often vary most strongly along a few dominant directions in the input parameter space. We propose a gradient-based method for detecting these…
In this paper, approximation by means of algebraic polynomials of classes of functions defined by a generalised modulus of smoothness of operators of differentiation of these functions is considered. We give structural characteristics of…
We propose a new optimization method for training feed-forward neural networks. By rewriting the activation function as an equivalent proximal operator, we approximate a feed-forward neural network by adding the proximal operators to the…