Related papers: Positivity sets of hinge functions
An additive submonoid of the nonnegative cone of the real line is called a positive monoid. Positive monoids consisting of rational numbers (also known as Puiseux monoids) have been the subject of several recent papers. Moreover, those…
Activation functions are non-linearities in neural networks that allow them to learn complex mapping between inputs and outputs. Typical choices for activation functions are ReLU, Tanh, Sigmoid etc., where the choice generally depends on…
We propose ReDense as a simple and low complexity way to improve the performance of trained neural networks. We use a combination of random weights and rectified linear unit (ReLU) activation function to add a ReLU dense (ReDense) layer to…
Rectified linear unit (ReLU) activations can also be thought of as 'gates', which, either pass or stop their pre-activation input when they are 'on' (when the pre-activation input is positive) or 'off' (when the pre-activation input is…
This paper is concerned with the utilization of deterministically modeled chemical reaction networks for the implementation of (feed-forward) neural networks. We develop a general mathematical framework and prove that the ordinary…
This article concerns the expressive power of depth in neural nets with ReLU activations and bounded width. We are particularly interested in the following questions: what is the minimal width $w_{\text{min}}(d)$ so that ReLU nets of width…
This paper is concerned with configurations of points in a plane lattice which determine angles that are rational multiples of $\pi$. We shall study how many such angles may appear in a given lattice and in which positions, allowing the…
This theoretical paper is devoted to developing a rigorous theory for demystifying the global convergence phenomenon in a challenging scenario: learning over-parameterized Rectified Linear Unit (ReLU) nets for very high dimensional dataset…
Convolutional neural networks (CNNs), one of the key architectures of deep learning models, have achieved superior performance on many machine learning tasks such as image classification, video recognition, and power systems. Despite their…
Let $T$ be a positive closed current of unit mass on the complex projective space $\mathbb P^n$. For certain values $\alpha<1$, we prove geometric properties of the set of points in $\mathbb P^n$ where the Lelong number of $T$ exceeds…
We study the problem of computing the preimage of a set under a neural network with piecewise-affine activation functions. We recall an old result that the preimage of a polyhedral set is again a union of polyhedral sets and can be…
It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al.,…
A rational homogeneous (of degree one) positive real matrix-valued function is presented as the Schur complement of a block of the linear pencil with positive semidefinite matrix coefficients. The partial derivative numerators of a rational…
We introduce the class of rational plane curves parameterizable by conics as an extension of the family of curves parameterizable by lines (also known as monoid curves). We show that they are the image of monoid curves via suitable…
Encoding constraints into neural networks is attractive. This paper studies how to introduce the popular positive linear satisfiability to neural networks. We propose the first differentiable satisfiability layer based on an extension of…
Neural networks are playing a crucial role in everyday life, with the most modern generative models able to achieve impressive results. Nonetheless, their functioning is still not very clear, and several strategies have been adopted to…
Motivated by the human way of memorizing images we introduce their functional representation, where an image is represented by a neural network. For this purpose, we construct a hypernetwork which takes an image and returns weights to the…
Rectified Linear Unit (ReLU) networks are piecewise-linear (PWL), so universal linear safety properties can be reduced to reasoning about linear constraints. Modern verifiers rely on SMT(LRA) procedures or MILP encodings, but a safety claim…
We consider rational surface automorphisms with positive entropy. A Fatou component is said to be a rotation domain if the automorphism induces a torus action on it. Here we construct a rational surface automorphism with positive entropy…
The structure of cones of positive and k-positive maps acting on a finite-dimensional Hilbert space is investigated. Special emphasis is given to their duality relations to the sets of superpositive and k-superpositive maps. We characterize…