QA-RAG: Exploring LLM Reliance on External Knowledge
Mansurova A. Mansurova A. Nugumanova A.
September 2024Multidisciplinary Digital Publishing Institute (MDPI)
Big Data and Cognitive Computing
2024#8Issue 9
Large language models (LLMs) can store factual knowledge within their parameters and have achieved superior results in question-answering tasks. However, challenges persist in providing provenance for their decisions and keeping their knowledge up to date. Some approaches aim to address these challenges by combining external knowledge with parametric memory. In contrast, our proposed QA-RAG solution relies solely on the data stored within an external knowledge base, specifically a dense vector index database. In this paper, we compare RAG configurations using two LLMs—Llama 2 7b and 13b—systematically examining their performance in three key RAG capabilities: noise robustness, knowledge gap detection, and external truth integration. The evaluation reveals that while our approach achieves an accuracy of 83.3%, showcasing its effectiveness across all baselines, the model still struggles significantly in terms of external truth integration. These findings suggest that considerable work is still required to fully leverage RAG in question-answering tasks.
external knowledge base , large language model , question-answering system , retrieval-augmented generation
Text of the article Перейти на текст статьи
Big Data and Blockchain Technologies Science and Innovation Center, Astana IT University, Astana, 020000, Kazakhstan
Big Data and Blockchain Technologies Science and Innovation Center
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026