Related papers: A Generalization Method of Partitioned Activation …
Adaptive Partition-based Methods (APM) are numerical methods to solve two-stage stochastic linear problems (2SLP). The core idea is to iteratively construct an adapted partition of the space of alea in order to aggregate scenarios while…
Splitting methods for the numerical integration of differential equations of order greater than two involve necessarily negative coefficients. This order barrier can be overcome by considering complex coefficients with positive real part.…
We survey current term-wise techniques for quadratizing high-degree pseudo-Boolean functions and introduce a new one, which allows multiple splits of terms. We also introduce the first aggregative approach, which splits a collection of…
During my study of the iteration of functions of the form $f(z)=z^{\alpha}+c$, where $z,c \in \mathbbC$, and $\alpha$ is a rational non-integer larger than 2 (\cite{s1}), I encountered a fundamental difficulty in the exponentiation of a…
Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks. However, the benefits of novel activation functions have been inconsistent and task dependent, and…
Given a sequence of $N$ positive real numbers $\{a_1,a_2,..., a_N \}$, the number partitioning problem consists of partitioning them into two sets such that the absolute value of the difference of the sums of $a_j$ over the two sets is…
We propose a novel extension to symmetrized neural network operators by incorporating fractional and mixed activation functions. This study addresses the limitations of existing models in approximating higher-order smooth functions,…
Hypercomplex numbers are unital algebras over the real numbers. We offer a short demonstration of the practical value of hypercomplex analytic functions in the field of partial differential equations.
Diverse optimization algorithms correctly identify, in finite time, intrinsic constraints that must be active at optimality. Analogous behavior extends beyond optimization to systems involving partly smooth operators, and in particular to…
The activation function is at the heart of a deep neural networks nonlinearity; the choice of the function has great impact on the success of training. Currently, many practitioners prefer the Rectified Linear Unit (ReLU) due to its…
We consider the following "partition and sum" operation on a natural number: Treating the number as a long string of digits insert several plus signs in between some of the digits and carry out the indicated sum. This results in a smaller…
This paper explores the expressive power of deep neural networks for a diverse range of activation functions. An activation function set $\mathscr{A}$ is defined to encompass the majority of commonly used activation functions, such as…
Recent seminal work at the intersection of deep neural networks practice and random matrix theory has linked the convergence speed and robustness of these networks with the combination of random weight initialization and nonlinear…
In this work, we propose an interesting method that aims to approximate an activation function over some domain by polynomials of the presupposing low degree. The main idea behind this method can be seen as an extension of the ordinary…
In this note we consider setups in which variational objectives for Bayesian neural networks can be computed in closed form. In particular we focus on single-layer networks in which the activation function is piecewise polynomial (e.g.…
Artificial Neural Networks (ANN) have been employed for a range of modelling and prediction tasks using financial data. However, evidence on their predictive performance, especially for time-series data, has been mixed. Whereas some…
This work considers the problem of distributing matrix multiplication over the real or complex numbers to helper servers, such that the information leakage to these servers is close to being information-theoretically secure. These servers…
While it is well-known that neural networks enjoy excellent approximation capabilities, it remains a big challenge to compute such approximations from point samples. Based on tools from Information-based complexity, recent work by Grohs and…
Within recent years, considerable progress has been made regarding high-performance solvers for Partial Differential Equations (PDEs), yielding potential gains in efficiency compared to industry standard tools. However, the latter largely…
For manipulations of multipartite quantum systems, it was well known that all local operations assisted by classical communication (LOCC) constitute a proper subset of the class of separable operations. Recently, Gheorghiu and Griffiths…