English

Beyond Factual QA: Mentorship-Oriented Question Answering over Long-Form Multilingual Content

Computation and Language 2026-01-27 v1 Artificial Intelligence

Abstract

Question answering systems are typically evaluated on factual correctness, yet many real-world applications-such as education and career guidance-require mentorship: responses that provide reflection and guidance. Existing QA benchmarks rarely capture this distinction, particularly in multilingual and long-form settings. We introduce MentorQA, the first multilingual dataset and evaluation framework for mentorship-focused question answering from long-form videos, comprising nearly 9,000 QA pairs from 180 hours of content across four languages. We define mentorship-focused evaluation dimensions that go beyond factual accuracy, capturing clarity, alignment, and learning value. Using MentorQA, we compare Single-Agent, Dual-Agent, RAG, and Multi-Agent QA architectures under controlled conditions. Multi-Agent pipelines consistently produce higher-quality mentorship responses, with especially strong gains for complex topics and lower-resource languages. We further analyze the reliability of automated LLM-based evaluation, observing substantial variation in alignment with human judgments. Overall, this work establishes mentorship-focused QA as a distinct research problem and provides a multilingual benchmark for studying agentic architectures and evaluation design in educational AI. The dataset and evaluation framework are released at https://github.com/AIM-SCU/MentorQA.

Keywords

Cite

@article{arxiv.2601.17173,
  title  = {Beyond Factual QA: Mentorship-Oriented Question Answering over Long-Form Multilingual Content},
  author = {Parth Bhalerao and Diola Dsouza and Ruiwen Guan and Oana Ignat},
  journal= {arXiv preprint arXiv:2601.17173},
  year   = {2026}
}
R2 v1 2026-07-01T09:18:03.664Z