Real-Time Sign Language Fingerspelling Recognition System Using 2D Deep CNN with Two-Stream Feature Extraction Approach

Zhidebayeva A. Nurmukhanbetova G. Aldeshov S. Zhamalova K. Mamikov S. Torebay N.
2024 Science and Information Organization

International Journal of Advanced Computer Science and Applications
2024 #15 Issue 9 1062 - 1072 pp.

This research paper introduces a novel sign language recognition system developed using advanced deep learning (DL) techniques aimed at enhancing communication capabilities between deaf and hearing individuals. The system leverages a convolutional neural network (CNN) architecture, optimized for the real-time interpretation of dynamic hand gestures that constitute sign language. A comprehensive dataset was employed to train and validate the model, encompassing a diverse range of gestures across different environmental settings. Comparative analysis revealed that the deep learning-based model significantly outperforms traditional machine learning techniques in terms of recognition accuracy, particularly with the increase in the volume of training data. This was illustrated through various performance metrics, including a detailed confusion matrix and Levenshtein distance measurements, highlighting the system’s efficacy in accurately identifying complex gestures. Real-time application tests further demonstrated the model’s robustness and adaptability to varying lighting conditions and backgrounds, essential for practical deployment. Key challenges identified include the need for broader linguistic diversity in training datasets and enhanced model sensitivity to subtle gestural distinctions. The paper concludes with suggestions for future research directions, emphasizing algorithm optimization, data diversification, and user-centric design improvements to foster wider adoption and usability. This study underscores the potential of deep learning technologies to revolutionize assistive communication tools, making them more accessible and effective for the deaf community.

accessibility technology , convolutional neural networks , Deep learning , gesture recognition , machine learning , real-time processing , sign language recognition

Text of the article Перейти на текст статьи

University of Friendship of People’s Academician, A. Kuatbekov, Shymkent, Kazakhstan
South Kazakhstan Pedagogical University named after Ozbekali Zhanibekov, Shymkent, Kazakhstan
Miras University, Shymkent, Kazakhstan

University of Friendship of People’s Academician
South Kazakhstan Pedagogical University named after Ozbekali Zhanibekov
Miras University

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026