Real-Time Lightweight Sign Language Recognition on Hybrid Deep CNN-BiLSTM Neural Network with Attention Mechanism

Kazbekova G. Ismagulova Z. Ibrayeva G. Sundetova A. Abdrazakh Y. Baimurzayev B.
2025 Science and Information Organization

International Journal of Advanced Computer Science and Applications
2025 #16 Issue 4 510 - 522 pp.

Sign language recognition (SLR) plays a crucial role in bridging communication gaps for individuals with hearing and speech impairments. This study proposes a hybrid deep CNN-BiLSTM neural network with an attention mechanism for real-time and lightweight sign language recognition. The CNN module extracts spatial features from individual gesture frames, while the BiLSTM module captures temporal dependencies, enhancing classification accuracy. The attention mechanism further refines feature selection by focusing on the most relevant time steps in a sign sequence. The proposed model was evaluated on the Sign Language MNIST dataset, achieving state-of-the-art performance with high accuracy, precision, recall, and F1-score. Experimental results indicate that the model converges rapidly, maintains low misclassification rates, and effectively distinguishes between visually similar signs. Confusion matrix analysis and feature map visualizations provide deeper insights into the hierarchical feature extraction process. The results demonstrate that integrating spatial, temporal, and attention-based learning significantly improves recognition performance while maintaining computational efficiency. Despite its effectiveness, challenges such as misclassification in ambiguous gestures and real-time computational constraints remain, suggesting future improvements in multi-modal fusion, transformer-based architectures, and lightweight model optimizations. The proposed approach offers a scalable and efficient solution for real-time sign language recognition, contributing to the development of assistive technologies for individuals with communication disabilities.

assistive technology , attention mechanism , CNN-BiLSTM , deep learning , gesture classification , real-time processing , Sign language recognition

Text of the article Перейти на текст статьи

Khoja Akhmet Yassawi International Kazakh-Turkish University, Turkistan, Kazakhstan
ALT University, Almaty, Kazakhstan
Military Institute of the Air Defense Forces Named After Twice Hero of the Soviet Union T.Ya. Bigeldinov, Aktobe, Kazakhstan
Baishev University, Aktobe, Kazakhstan

Khoja Akhmet Yassawi International Kazakh-Turkish University
ALT University
Military Institute of the Air Defense Forces Named After Twice Hero of the Soviet Union T.Ya. Bigeldinov
Baishev University

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026