Development of a verbal robot hand gesture recognition system

Kenshimov C. Sundetov T. Kunelbayev M. Amirgaliyeva Z. Yedilkhan D. Auelbekov O.
2021 World Scientific and Engineering Academy and Society

WSEAS Transactions on Systems and Control
2021 #16 573 - 583 pp.

This article analyzes the most famous sign languages, the correlation of sign languages, and also considers the development of a verbal robot hand gesture recognition system in relation to the Kazakh language. The proposed system contains a touch sensor, in which the contact of the electrical property of the users skin is measured, which provides more accurate information for simulating and indicating the gestures of the robot hand. Within the framework of the system, the speed and accuracy of recognition of each gesture of the verbal robot are calculated. The average recognition accuracy was over 98%. The detection time was 3ms on a 1.9 GHz Jetson Nano processor, which is enough to create a robot showing natural language gestures. A complete fingerprint of the Kazakh sign language for a verbal robot is also proposed. To improve the quality of gesture recognition, a machine learning method was used. The operability of the developed technique for recognizing gestures by a verbal robot was tested, and on the basis of computational experiments, the effectiveness of algorithms and software for responding to a verbal robot to a voice command was evaluated based on automatic recognition of a multilingual human voice. Thus, we can assume that the authors have proposed an intelligent verbal complex implemented in Python with the CMUSphinx communication module and the PyOpenGL graphical command execution simulator. Robot manipulation module based on 3D modeling from ABB.

Accurate recognition , Hand gesture , Kinect , Verbal robot

Text of the article Перейти на текст статьи

Institute of Information and Computing Technologies CS MES RK, Almaty, Kazakhstan

Institute of Information and Computing Technologies CS MES RK

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026