Advanced Implementation of a Multilevel Model for Text Summarization in Kazakh Using Pretrained Models
Oralbekova D. Mamyrbayev O. Othman M. Zhumagulova S.
6 October 2025Dr D. Pylarinos
Engineering, Technology and Applied Science Research
2025#15Issue 526711 - 26721 pp.
This study investigates transformer models for the task of hybrid text summarization in the Kazakh language. Using mBART, mT5, and XLM-RoBERTa models, a multilevel architecture was developed that processes text at the character, subword, word, and contextual levels. The proposed system performs feature fusion across multiple linguistic layers, enabling the model to capture both fine-grained lexical variation and broader contextual dependencies. The architecture also allows flexible integration with various transformer models, supporting both encoder-decoder and hybrid configurations. This approach significantly improved the quality of generated summaries by effectively accounting for the morphological and semantic features of the Kazakh language. The experimental results showed that mBART achieved the best performance in terms of ROUGE-1, ROUGE-2, ROUGE-L, and BERTScore-F1 metrics, confirming the high effectiveness of the proposed multilevel transformer architecture. This is the first implementation of such an architecture for hybrid summarization in Kazakh, which is a low-resource and morphologically rich language.
hybrid summarization , Kazakh language , mBART , mT5 , multilevel modeling , transformer models , XLM-RoBERTa
Text of the article Перейти на текст статьи
Institute of Information and Computational Technologies, International Engineering and Technological University, Almaty, Kazakhstan
Institute of Information and Computational Technologies, Almaty, Kazakhstan
Malaysia Department of Communication Technology and Networks, Universiti Putra Malaysia, Serdang, Malaysia
Laboratory of Computational Science and Mathematical Physics, Institute for Mathematical Research, Universiti Putra Malaysia, Serdang, Malaysia
Al-Farabi Kazakh National University, Almaty, Kazakhstan
Institute of Information and Computational Technologies
Institute of Information and Computational Technologies
Malaysia Department of Communication Technology and Networks
Laboratory of Computational Science and Mathematical Physics
Al-Farabi Kazakh National University
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026