Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces
Machine Learning
2019-11-21 v3 Machine Learning
Abstract
We present a method to extract a weighted finite automaton (WFA) from a recurrent neural network (RNN). Our algorithm is based on the WFA learning algorithm by Balle and Mohri, which is in turn an extension of Angluin's classic \lstar algorithm. Our technical novelty is in the use of \emph{regression} methods for the so-called equivalence queries, thus exploiting the internal state space of an RNN to prioritize counterexample candidates. This way we achieve a quantitative/weighted extension of the recent work by Weiss, Goldberg and Yahav that extracts DFAs. We experimentally evaluate the accuracy, expressivity and efficiency of the extracted WFAs.
Keywords
Cite
@article{arxiv.1904.02931,
title = {Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces},
author = {Takamasa Okudono and Masaki Waga and Taro Sekiyama and Ichiro Hasuo},
journal= {arXiv preprint arXiv:1904.02931},
year = {2019}
}
Comments
AAAI 2020. We are preparing to distribute the implementation