English

Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation

Computation and Language 2025-11-04 v1

Abstract

This study proposes a cognitive benchmarking framework to evaluate how large language models (LLMs) process and apply culturally specific knowledge. The framework integrates Bloom's Taxonomy with Retrieval-Augmented Generation (RAG) to assess model performance across six hierarchical cognitive domains: Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating. Using a curated Taiwanese Hakka digital cultural archive as the primary testbed, the evaluation measures LLM-generated responses' semantic accuracy and cultural relevance.

Keywords

Cite

@article{arxiv.2511.01649,
  title  = {Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation},
  author = {Hung-Shin Lee and Chen-Chi Chang and Ching-Yuan Chen and Yun-Hsiang Hsu},
  journal= {arXiv preprint arXiv:2511.01649},
  year   = {2025}
}

Comments

This paper has been accepted by The Electronic Library, and the full article is now available on Emerald Insight

R2 v1 2026-07-01T07:19:24.823Z