Legal AI in Low-Resource Languages: Building and Evaluating QA Systems for the Kazakh Legislation
Rakhimova D. Turarbek A. Karyukin V. Sarsenbayeva A. Alieyev R.
September 2025Multidisciplinary Digital Publishing Institute (MDPI)
Computers
2025#14Issue 9
The research focuses on the development and evaluation of a legal question–answer system for the Kazakh language, a low-resource and morphologically complex language. Four datasets were compiled from open legal sources—Adilet, Zqai, Gov, and a manually created synthetic set—containing question–аnswer pairs extracted from official legislative documents and government portals. Seven large language models (GPT-4o mini, GEMMA, KazLLM, LLaMA, Phi, Qwen, and Mistral) were fine-tuned using structured prompt templates, quantization methods, and domain-specific training to enhance contextual understanding and efficiency. The evaluation employed both automatic metrics (ROUGE and METEOR) and expert-based manual assessment. GPT-4o mini achieved the highest overall performance, with ROUGE-1: 0.309, ROUGE-2: 0.175, ROUGE-L: 0.263, and METEOR: 0.320, and received an expert score of 3.96, indicating strong legal reasoning capabilities and adaptability to Kazakh legal contexts. The results highlight GPT-4o mini’s superiority over other tested models in both quantitative and qualitative evaluations. This work demonstrates the feasibility and importance of developing localized legal AI solutions for low-resource languages, contributing to improved legal accessibility, transparency, and digital governance in Kazakhstan.
GEMMA , government websites , GPT-4o mini , KazLLM , legislative documents , LLaMA , low-resource language , Phi , question-answer system , Qwen
Text of the article Перейти на текст статьи
Department of Information Systems, Al-Farabi Kazakh National University, Almaty, 050040, Kazakhstan
Institute of Information and Computational Technologies, Almaty, 050010, Kazakhstan
Department of Information Systems
Institute of Information and Computational Technologies
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026