R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network
Machine Learning
2018-03-14 v1 Machine Learning
Abstract
We consider a neural network architecture with randomized features, a sign-splitter, followed by rectified linear units (ReLU). We prove that our architecture exhibits robustness to the input perturbation: the output feature of the neural network exhibits a Lipschitz continuity in terms of the input perturbation. We further show that the network output exhibits a discrimination ability that inputs that are not arbitrarily close generate output vectors which maintain distance between each other obeying a certain lower bound. This ensures that two different inputs remain discriminable while contracting the distance in the output feature space.
Cite
@article{arxiv.1803.04186,
title = {R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network},
author = {Arun Venkitaraman and Alireza M. Javid and Saikat Chatterjee},
journal= {arXiv preprint arXiv:1803.04186},
year = {2018}
}