English

Unlocking Crowdsourcing for Ontology Matching Validation

Information Retrieval 2026-05-28 v3

Abstract

Recent advances in large language models (LLMs) pose new challenges for ontology matching (OM). While OM systems built on LLMs have shown remarkable capabilities in discovering more matching candidates, traditional OM validation that relies on domain experts has become overwhelming. In this study, we explore the use of crowdsourcing for OM validation and introduce a novel crowdsourcing system. We propose three domain-specific mechanisms, namely differential trustworthiness, coherence pre-filling, and time-dependent opinion, to ensure the quality of crowdsourcing for OM validation. We demonstrate that our crowdsourcing system can be integrated with existing OM systems to enable human-in-the-loop validation. The evaluation of the system shows its effectiveness in handling diverse user groups and different annotation settings. We discuss two real-world use cases of the system and current limitations for improvement.

Keywords

Cite

@article{arxiv.2605.12226,
  title  = {Unlocking Crowdsourcing for Ontology Matching Validation},
  author = {Zhangcheng Qiang},
  journal= {arXiv preprint arXiv:2605.12226},
  year   = {2026}
}

Comments

6 pages, 7 figures

R2 v1 2026-07-22T07:07:52.942Z