A single gradient step finds adversarial examples on random two-layers neural networks
Machine Learning
2021-04-13 v2 Cryptography and Security
Machine Learning
Abstract
Daniely and Schacham recently showed that gradient descent finds adversarial examples on random undercomplete two-layers ReLU neural networks. The term "undercomplete" refers to the fact that their proof only holds when the number of neurons is a vanishing fraction of the ambient dimension. We extend their result to the overcomplete case, where the number of neurons is larger than the dimension (yet also subexponential in the dimension). In fact we prove that a single step of gradient descent suffices. We also show this result for any subexponential width random neural network with smooth activation function.
Keywords
Cite
@article{arxiv.2104.03863,
title = {A single gradient step finds adversarial examples on random two-layers neural networks},
author = {Sébastien Bubeck and Yeshwanth Cherapanamjeri and Gauthier Gidel and Rémi Tachet des Combes},
journal= {arXiv preprint arXiv:2104.03863},
year = {2021}
}
Comments
Added a comment about universal adversarial perturbations. 18 pages, 7 figures