English

KG2QA: Knowledge Graph-enhanced Retrieval-augmented Generation for Communication Standards Question Answering

Computation and Language 2025-10-16 v2

Abstract

The rapid evolution of communication technologies has led to an explosion of standards, rendering traditional expert-dependent consultation methods inefficient and slow. To address this challenge, we propose \textbf{KG2QA}, a question answering (QA) framework for communication standards that integrates fine-tuned large language models (LLMs) with a domain-specific knowledge graph (KG) via a retrieval-augmented generation (RAG) pipeline. We construct a high-quality dataset of 6,587 QA pairs from ITU-T recommendations and fine-tune Qwen2.5-7B-Instruct, achieving significant performance gains: BLEU-4 increases from 18.86 to 66.90, outperforming both the base model and Llama-3-8B-Instruct. A structured KG containing 13,906 entities and 13,524 relations is built using LLM-assisted triple extraction based on a custom ontology. In our KG-RAG pipeline, the fine-tuned LLMs first retrieves relevant knowledge from KG, enabling more accurate and factually grounded responses. Evaluated by DeepSeek-V3 as a judge, the KG-enhanced system improves performance across five dimensions, with an average score increase of 2.26\%, demonstrating superior factual accuracy and relevance. Integrated with Web platform and API, KG2QA delivers an efficient and interactive user experience. Our code and data have been open-sourced https://github.com/luozhongze/KG2QA.

Keywords

Cite

@article{arxiv.2506.07037,
  title  = {KG2QA: Knowledge Graph-enhanced Retrieval-augmented Generation for Communication Standards Question Answering},
  author = {Zhongze Luo and Weixuan Wan and Tianya Zhang and Dan Wang and Xiaoying Tang},
  journal= {arXiv preprint arXiv:2506.07037},
  year   = {2025}
}
R2 v1 2026-07-01T03:05:25.681Z