Deep Learning-Based Continuous Sign Language Recognition
Zholshiyeva L. Zhukabayeva T. Serek A. Duisenbek R. Berdieva M. Shapay N.
2025Department of Agribusiness, Universitas Muhammadiyah Yogyakarta
Journal of Robotics and Control (JRC)
2025#6Issue 31106 - 1119 pp.
This study focuses on the development of a continuous sign language recognition system based on deep neural network models. A new Kazakh Sign Language (QazSL) dataset is created. DL models for continuous KazSL are developed, their accuracy and robustness under different environmental conditions are analyzed, and an optimized model algorithm to improve sign recognition processes are proposed. The main goal is to improve gesture recognition accuracy, account for gesture variability and environmental conditions, and promote the development of adaptive technologies for low-resource languages. This paper proposes a QazSL recognition system using an YOLOv8n and optimized 2DCNN models to improve accessibility for the hearing impaired. The optimized 2DCNN method includes optimal data preprocessing techniques and new training architecture, followed by model training and testing with precision, recall, and accuracy metrics. The proposed systems were trained using an open-course K-RSL dataset with 5 signers and a newly created QazSL dataset, recorded by 7 signers. The test accuracy of gesture recognition are 98.12% for Yolov8n and 98, 57% for 2DCNN, indicating the robustness and capability of the models for real-time application. Certain issues, such as background variation and gesture consistency, were found to affect recognition under different conditions. This research contributes to the development of AI-based assistive technology to facilitate social inclusion and access to communication for deaf and hard-of-hearing people. By addressing the challenges identified in gesture recognition, this study paves the way for more reliable interactions between users and technology. Future work will focus on optimizing the model further to enhance its performance in varied environments and to expand its applicability across different languages and sign systems.
2DCNN , Computer Vision , Deep Learning , Real-Time Recognition , Sign Language , Yolo Optimization
Text of the article Перейти на текст статьи
Department of Information Systems, L.N. Gumilyov Eurasian National University, Astana, Kazakhstan
School of Information Technology and Engineering, Kazakh-British Technical University (KBTU), Almaty, Kazakhstan
Department of Computer Engineering, Astana IT University, Astana, Kazakhstan
Department of Medical Biophysics and IT, South Kazakhstan Medical Academy, Shymkent, Kazakhstan
Department of Information Systems, SDU University, Kaskelen, Kazakhstan
Department of Information Systems
School of Information Technology and Engineering
Department of Computer Engineering
Department of Medical Biophysics and IT
Department of Information Systems
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026