An Automatic Approach for Document-level Topic Model Evaluation
Computation and Language
2017-06-19 v1
Abstract
Topic models jointly learn topics and document-level topic distribution. Extrinsic evaluation of topic models tends to focus exclusively on topic-level evaluation, e.g. by assessing the coherence of topics. We demonstrate that there can be large discrepancies between topic- and document-level model quality, and that basing model evaluation on topic-level analysis can be highly misleading. We propose a method for automatically predicting topic model quality based on analysis of document-level topic allocations, and provide empirical evidence for its robustness.
Cite
@article{arxiv.1706.05140,
title = {An Automatic Approach for Document-level Topic Model Evaluation},
author = {Shraey Bhatia and Jey Han Lau and Timothy Baldwin},
journal= {arXiv preprint arXiv:1706.05140},
year = {2017}
}
Comments
10 pages; accepted for the Twenty First Conference on Computational Natural Language Learning (CoNLL 2017)