English

Term-community-based topic detection with variable resolution

Computation and Language 2021-07-27 v2

Abstract

Network-based procedures for topic detection in huge text collections offer an intuitive alternative to probabilistic topic models. We present in detail a method that is especially designed with the requirements of domain experts in mind. Like similar methods, it employs community detection in term co-occurrence graphs, but it is enhanced by including a resolution parameter that can be used for changing the targeted topic granularity. We also establish a term ranking and use semantic word-embedding for presenting term communities in a way that facilitates their interpretation. We demonstrate the application of our method with a widely used corpus of general news articles and show the results of detailed social-sciences expert evaluations of detected topics at various resolutions. A comparison with topics detected by Latent Dirichlet Allocation is also included. Finally, we discuss factors that influence topic interpretation.

Keywords

Cite

@article{arxiv.2103.13550,
  title  = {Term-community-based topic detection with variable resolution},
  author = {Andreas Hamm and Simon Odrowski},
  journal= {arXiv preprint arXiv:2103.13550},
  year   = {2021}
}

Comments

31 pages, 6 figures