Related papers: Enumeration of max-pooling responses with generali…
The aims of this article are two-fold. First, we give a geometric characterization of the optimal basic solutions of the general linear programming problem (no compactness assumptions) and provide a simple, self-contained proof of it…
Deep Neural Networks now excel at image classification, detection and segmentation. When used to scan images by means of a sliding window, however, their high computational complexity can bring even the most powerful hardware to its knees.…
It is well-known that the convex and concave envelope of a multilinear polynomial over a box are polyhedral functions. Exponential-sized extended and projected formulations for these envelopes are also known. We consider the convexification…
We derive tight expressions for the maximum number of $k$-faces, $0\le k\le d-1$, of the Minkowski sum, $P_1+P_2+P_3$, of three $d$-dimensional convex polytopes $P_1$, $P_2$ and $P_3$, as a function of the number of vertices of the…
The information processing abilities of a multilayer neural network with a number of hidden units scaling as the input dimension are studied using statistical mechanics methods. The mapping from the input layer to the hidden units is…
The volume and the number of lattice points of the permutohedron P_n are given by certain multivariate polynomials that have remarkable combinatorial properties. We give several different formulas for these polynomials. We also study a more…
In machine learning and neuroscience, certain computational structures and algorithms are known to yield disentangled representations without us understanding why, the most striking examples being perhaps convolutional neural networks and…
Maxout polytopes are defined by feedforward neural networks with maxout activation function and non-negative weights after the first layer. We characterize the parameter spaces and extremal f-vectors of maxout polytopes for shallow…
A permutation polytope is the convex hull of a group of permutation matrices. In this paper we investigate the combinatorics of permutation polytopes and their faces. As applications we completely classify permutation polytopes in…
Signed Minkowski decomposition is an expression of a polytope as a Minkowski sum and difference of smaller polytopes. Signed Minkowski decompositions of a polytope can be interpreted as factorizations of a max-plus (tropical) function. We…
Pooling is one of the main elements in convolutional neural networks. The pooling reduces the size of the feature map, enabling training and testing with a limited amount of computation. This paper proposes a new pooling method named…
Partitionings (or segmentations) divide a given domain into disjoint connected regions whose union forms again the entire domain. Multi-dimensional partitionings occur, for example, when analyzing parameter spaces of simulation models,…
Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems. In those methods a deep neural network is used as a solution generator which is then trained by…
Feature pooling layers (e.g., max pooling) in convolutional neural networks (CNNs) serve the dual purpose of providing increasingly abstract representations as well as yielding computational savings in subsequent convolutional layers. We…
The objective of this paper is to present two types of results on Minkowski sums of convex polytopes. The first is about a special class of polytopes we call perfectly centered and the combinatorial properties of the Minkowski sum with…
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries…
Any homogeneous polynomial $P(x, y, z)$ of degree $d$, being restricted to a unit sphere $S^2$, admits essentially a unique representation of the form $\lambda_0 + \sum_{k = 1}^d \lambda_k [\prod_{j = 1}^k L_{kj}]$, where $L_{kj}$'s are…
The goals of this paper are two-fold. The first goal is to serve as an expository tutorial on the working of deep learning models which emphasizes geometrical intuition about the reasons for success of deep learning. The second goal is to…
How to utilize deep learning methods for graph classification tasks has attracted considerable research attention in the past few years. Regarding graph classification tasks, the graphs to be classified may have various graph sizes (i.e.,…
Our formal understanding of the inductive bias that drives the success of convolutional networks on computer vision tasks is limited. In particular, it is unclear what makes hypotheses spaces born from convolution and pooling operations so…