Sign Language Detection using Action Recognit...

Sign Language Detection using Action Recognition with LSTM Deep Learning Model

Author Name : Prof. Asha P, Md Wajid Ali, Chetan Almaje, Mirza Maaz Baig, Mohd Mahroof Ali

ABSTRACT

Sign Language Detection has become crucial and effective for humans and research in this area is in progress and is one of the applications of Computer Vision. Earlier works included detection using static signs with the help of a simple deep learning-based Convolutional Neural Network. This proposal is based on continuous detection of image frames in real-time using action detection so as to detect the action performed by the user. The model uses LSTM neural network model after identifying keypoints using mediapipe holistic which includes face, pose and hand features. The proposed work is done by collecting key value points for training and testing, pre-processing the data, and creating labels and features. It saves the weights and evaluates the model using confusion matrix accuracy.

Keywords— Neural Network (Convolutional and Recurrent), Long-Short Term Memory(LSTM), Mediapipe Holistic, Computer Vision, Sign Language Detection