Variational Inference In Pachinko Allocation Machines
Abstract
The Pachinko Allocation Machine (PAM) is a deep topic model that allows representing rich correlation structures among topics by a directed acyclic graph over topics. Because of the flexibility of the model, however, approximate inference is very difficult. Perhaps for this reason, only a small number of potential PAM architectures have been explored in the literature. In this paper we present an efficient and flexible amortized variational inference method for PAM, using a deep inference network to parameterize the approximate posterior distribution in a manner similar to the variational autoencoder. Our inference method produces more coherent topics than state-of-art inference methods for PAM while being an order of magnitude faster, which allows exploration of a wider range of PAM architectures than have previously been studied.
Cite
@article{arxiv.1804.07944,
title = {Variational Inference In Pachinko Allocation Machines},
author = {Akash Srivastava and Charles Sutton},
journal= {arXiv preprint arXiv:1804.07944},
year = {2018}
}