A study of transformer-based end-to-end speech recognition system for Kazakh language
Orken M. Dina O. Keylan A. Tolganay T. Mohamed O.
December 2022Nature Research
Scientific Reports
2022#12Issue 1
Today, the Transformer model, which allows parallelization and also has its own internal attention, has been widely used in the field of speech recognition. The great advantage of this architecture is the fast learning speed, and the lack of sequential operation, as with recurrent neural networks. In this work, Transformer models and an end-to-end model based on connectionist temporal classification were considered to build a system for automatic recognition of Kazakh speech. It is known that Kazakh is part of a number of agglutinative languages and has limited data for implementing speech recognition systems. Some studies have shown that the Transformer model improves system performance for low-resource languages. Based on our experiments, it was revealed that the joint use of Transformer and connectionist temporal classification models contributed to improving the performance of the Kazakh speech recognition system and with an integrated language model it showed the best character error rate 3.7% on a clean dataset.
Text of the article Перейти на текст статьи
Institute of Information and Computational Technologies CS MES RK, Almaty, Kazakhstan
Satbayev University, Almaty, Kazakhstan
L.N. Gumilyov Eurasian National University, Nur-Sultan, Kazakhstan
Universiti Putra Malaysia, Kuala Lumpur, Malaysia
Institute of Information and Computational Technologies CS MES RK
Satbayev University
L.N. Gumilyov Eurasian National University
Universiti Putra Malaysia
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026