Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for combining the precision of retrieval systems with the fluency of large language models. While several studies have investigated RAG pipelines for high-resource languages, the optimization of RAG components for Arabic remains underexplored. This study presents a comprehensive empirical evaluation of state-of-the-art RAG components-including chunking strategies, embedding models, rerankers, and language models-across a diverse set of Arabic datasets. Using the RAGAS framework, we systematically compare performance across four core metrics: context precision, context recall, answer faithfulness, and answer relevancy. Our experiments demonstrate that sentence-aware chunking outperforms all other segmentation methods, while BGE-M3 and Multilingual-E5-large emerge as the most effective embedding models. The inclusion of a reranker (bge-reranker-v2-m3) significantly boosts faithfulness in complex datasets, and Aya-8B surpasses StableLM in generation quality. These findings provide critical insights for building high-quality Arabic RAG pipelines and offer practical guidelines for selecting optimal components across different document types.
@article{arxiv.2506.06339,
title = {Optimizing RAG Pipelines for Arabic: A Systematic Analysis of Core Components},
author = {Jumana Alsubhi and Mohammad D. Alahmadi and Ahmed Alhusayni and Ibrahim Aldailami and Israa Hamdine and Ahmad Shabana and Yazeed Iskandar and Suhayb Khayyat},
journal= {arXiv preprint arXiv:2506.06339},
year = {2025}
}