ToMMeR -- Efficient Entity Mention Detection from Large Language Models
Abstract
Identifying which text spans refer to entities - mention detection - is both foundational for information extraction and a known performance bottleneck. We introduce ToMMeR, a lightweight model (<300K parameters) probing mention detection capabilities from early LLM layers. Across 13 NER benchmarks, ToMMeR achieves 93% recall zero-shot, with an estimated 90% precision under a human-calibrated LLM-judge protocol, showing that ToMMeR rarely produces spurious predictions despite high recall. Cross-model analysis reveals that diverse architectures (14M-15B parameters) converge on similar mention boundaries (DICE >75%), confirming that mention detection emerges naturally from language modeling. When extended with span classification heads, ToMMeR achieves competitive NER performance (80-87% F1 on standard benchmarks). Our work provides evidence that structured entity representations exist in early transformer layers and can be efficiently recovered with minimal parameters.
Cite
@article{arxiv.2510.19410,
title = {ToMMeR -- Efficient Entity Mention Detection from Large Language Models},
author = {Victor Morand and Nadi Tomeh and Josiane Mothe and Benjamin Piwowarski},
journal= {arXiv preprint arXiv:2510.19410},
year = {2026}
}
Comments
Accepted at ACL2026 - Code: https://github.com/VictorMorand/llm2ner