NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge
Abstract
The rapid progress of large language models (LLMs) raises concerns about cultural bias, fairness, and performance in diverse languages and underrepresented regions. Addressing these gaps requires large-scale resources grounded in multilingual, local, and cultural contexts. We systematize and extend the earlier NativQA framework to multimodality by adding image, audio, and video support, enabling scalable construction of culturally and regionally aligned QA datasets in native languages. Given user-defined seed queries, the framework uses search engines to collect location-specific everyday information. We evaluate it across 39 locations in 24 countries and 7 languages, spanning extremely low-resource to high-resource settings, and collect over 300K text QA pairs, 312K images, and 29K videos with associated audio. The developed resources can be used for LLMs benchmarking and further fine-tuning. The framework has been made publicly available for the community (https://gitlab.com/nativqa/nativqa-framework). Demo video is available here: \href{https://shorturl.at/DAVn9}{https://shorturl.at/DAVn9}.
Cite
@article{arxiv.2504.05995,
title = {NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge},
author = {Firoj Alam and Md Arid Hasan and Sahinur Rahman Laskar and Mucahid Kutlu and Kareem Darwish and Shammur Absar Chowdhury},
journal= {arXiv preprint arXiv:2504.05995},
year = {2026}
}
Comments
LLMs, Native, Multilingual, Language Diversity, Contextual Understanding, Minority Languages, Culturally Informed, Foundation Models, Large Language Models