Agri-Query：评估跨语言技术问答中 RAG 与长上下文 LLM 的案例研究

我们提出了一个案例研究，评估拥有 128K token 上下文窗口的大型语言模型（LLM） on technical question answering（QA）任务。我们的 benchmark is built on one of agricultural machine 的 user manual，可用英文、法语和德语。它模拟了 cross-lingual information retrieval scenario，其中 questions are posed in English against all three language versions of the manual。evaluation focuses on realistic needle-in-a-haystack challenges and includes unanswerable questions to test for hallucinations。我们比较了 nine long-context LLMs using direct prompting against three Retrieval-Augmented Generation（RAG）strategies（keyword、semantic、hybrid），with an LLM-as-a-judge for evaluation。Our findings for this specific manual show that Hybrid RAG consistently outperforms direct long-context prompting。Models like Gemini 2.5 Flash and the smaller Qwen 2.5 7B achieve high accuracy（over 85%）across all languages with RAG。This paper contributes a detailed analysis of LLM performance in a specialized industrial domain and an open framework for similar evaluations， highlighting practical trade-offs and challenges。