A Real-Time Dynamic Gesture Variability Recognition Method Based on Convolutional Neural Networks

Amangeldy N. Milosz M. Kudubayeva S. Kassymova A. Kalakova G. Zhetkenbay L.
October 2023 Multidisciplinary Digital Publishing Institute (MDPI)

Applied Sciences (Switzerland)
2023 #13 Issue 19

Among the many problems in machine learning, the most critical ones involve improving the categorical response prediction rate based on extracted features. In spite of this, it is noted that most of the time from the entire cycle of multi-class machine modeling for sign language recognition tasks is spent on data preparation, including collection, filtering, analysis, and visualization of data. To find the optimal solution for the above-mentioned problem, this paper proposes a methodology for automatically collecting the spatiotemporal features of gestures by calculating the coordinates of the found area of the pose and hand, normalizing them, and constructing an optimal multilayer perceptron for multiclass classification. By extracting and analyzing spatiotemporal data, the proposed method makes it possible to identify not only static features, but also the spatial (for gestures that touch the face and head) and dynamic features of gestures, which leads to an increase in the accuracy of gesture recognition. This classification was also carried out according to the form of the gesture demonstration to optimally extract the characteristics of gestures (display ability of all connection points), which also led to an increase in the accuracy of gesture recognition for certain classes to the value of 0.96. This method was tested using the well-known Ankara University Turkish Sign Language Dataset and the Dataset for Argentinian Sign Language to validate the experiment, which proved effective with a recognition accuracy of 0.98.

CNN , multimodality , preprocessing , sign language recognition

Text of the article Перейти на текст статьи

Department of Artificial Intelligence Technologies, Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Pushkina 11, Astana, 010008, Kazakhstan
Department of Computer Science, Faculty of Electrical Engineering and Computer Science, Lublin University of Technology, Nadbystrzycka 36B, Lublin, 20-618, Poland
Higher School of Information Technologies, Faculty of Economics, Information Technology and Vocational Education, Zhangir Khan University, Zhangir Khan 51, Uralsk, 090009, Kazakhstan
Department of Physics, Mathematics and Digital Technology, A. Baitursynov, Kostanay Regional University, Kostanay, 110000, Kazakhstan

Department of Artificial Intelligence Technologies
Department of Computer Science
Higher School of Information Technologies
Department of Physics

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026