This work presents a web-based interactive neural network (NN) calculator and a NN inefficiency measurement that has been investigated for the purpose of detecting trojans embedded in NN models. This NN Calculator is designed on top of TensorFlow Playground with in-memory storage of data and NN graphs plus coefficients. It is "like a scientific calculator" with analytical, visualization, and output operations performed on training datasets and NN architectures. The prototype is aaccessible at https://pages.nist.gov/nn-calculator. The analytical capabilities include a novel measurement of NN inefficiency using modified Kullback-Liebler (KL) divergence applied to histograms of NN model states, as well as a quantification of the sensitivity to variables related to data and NNs. Both NN Calculator and KL divergence are used to devise a trojan detector approach for a variety of trojan embeddings. Experimental results document desirable properties of the KL divergence measurement with respect to NN architectures and dataset perturbations, as well as inferences about embedded trojans.
@article{arxiv.2006.03707,
title = {Scientific Calculator for Designing Trojan Detectors in Neural Networks},
author = {Peter Bajcsy and Nicholas J. Schaub and Michael Majurski},
journal= {arXiv preprint arXiv:2006.03707},
year = {2020}
}
Comments
Presented at AAAI FSS-20: Artificial Intelligence in Government and Public Sector, Washington, DC, USA