English

Objective Metrics for Evaluating Large Language Models Using External Data Sources

Computation and Language 2025-08-13 v1 Machine Learning

Abstract

Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing well-defined benchmarks, factual datasets, and structured evaluation pipelines, the approach ensures consistent, reproducible, and bias-minimized measurements. The framework emphasizes automation and transparency in scoring, reducing reliance on human interpretation while ensuring alignment with real-world applications. This method addresses the limitations of subjective evaluation methods, providing a scalable solution for performance assessment in educational, scientific, and other high-stakes domains.

Keywords

Cite

@article{arxiv.2508.08277,
  title  = {Objective Metrics for Evaluating Large Language Models Using External Data Sources},
  author = {Haoze Du and Richard Li and Edward Gehringer},
  journal= {arXiv preprint arXiv:2508.08277},
  year   = {2025}
}

Comments

This version of the paper is lightly revised from the EDM 2025 proceedings for the sake of clarity

R2 v1 2026-07-01T04:44:51.110Z