RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
Abstract
Large Language Models (LLMs) demonstrate human-level capabilities in dialogue, reasoning, and knowledge retention. However, even the most advanced LLMs face challenges such as hallucinations and real-time updating of their knowledge. Current research addresses this bottleneck by equipping LLMs with external knowledge, a technique known as Retrieval Augmented Generation (RAG). However, two key issues constrained the development of RAG. First, there is a growing lack of comprehensive and fair comparisons between novel RAG algorithms. Second, open-source tools such as LlamaIndex and LangChain employ high-level abstractions, which results in a lack of transparency and limits the ability to develop novel algorithms and evaluation metrics. To close this gap, we introduce RAGLAB, a modular and research-oriented open-source library. RAGLAB reproduces 6 existing algorithms and provides a comprehensive ecosystem for investigating RAG algorithms. Leveraging RAGLAB, we conduct a fair comparison of 6 RAG algorithms across 10 benchmarks. With RAGLAB, researchers can efficiently compare the performance of various algorithms and develop novel algorithms.
Cite
@article{arxiv.2408.11381,
title = {RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation},
author = {Xuanwang Zhang and Yunze Song and Yidong Wang and Shuyun Tang and Xinfeng Li and Zhengran Zeng and Zhen Wu and Wei Ye and Wenyuan Xu and Yue Zhang and Xinyu Dai and Shikun Zhang and Qingsong Wen},
journal= {arXiv preprint arXiv:2408.11381},
year = {2024}
}
Comments
6 pages, 3 figures