|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Engineering / Faculty of Engineering >
Peer-reviewed Journal Articles, etc >
American Sign Language Alphabet Recognition Using Inertial Motion Capture System with Deep Learning
| Title: | American Sign Language Alphabet Recognition Using Inertial Motion Capture System with Deep Learning |
| Authors: | Gu, Yutong Browse this author | | Sherrine, Sherrine Browse this author | | Wei, Weiyi Browse this author | | Li, Xinya Browse this author | | Yuan, Jianan Browse this author | | Todoh, Masahiro Browse this author →KAKEN DB |
| Keywords: | American sign language alphabet | | hand gesture classification | | sequence recognition |
| Issue Date: | 1-Dec-2022 |
| Publisher: | MDPI |
| Journal Title: | Inventions |
| Volume: | 7 |
| Issue: | 4 |
| Start Page: | 112 |
| Publisher DOI: | 10.3390/inventions7040112 |
| Abstract: | Sign language is designed as a natural communication method for the deaf community to convey messages and connect with society. In American sign language, twenty-six special sign gestures from the alphabet are used for the fingerspelling of proper words. The purpose of this research is to classify the hand gestures in the alphabet and recognize a sequence of gestures in the fingerspelling using an inertial hand motion capture system. In this work, time and time-frequency domain features and angle-based features are extracted from the raw data for classification with convolutional neural network-based classifiers. In fingerspelling recognition, we explore two kinds of models: connectionist temporal classification and encoder-decoder structured sequence recognition model. The study reveals that the classification model achieves an average accuracy of 74.8% for dynamic ASL gestures considering user independence. Moreover, the proposed two sequence recognition models achieve 55.1%, 93.4% accuracy in word-level evaluation, and 86.5%, 97.9% in the letter-level evaluation of fingerspelling. The proposed method has the potential to recognize more hand gestures of sign language with highly reliable inertial data from the device. |
| Type: | article |
| URI: | http://hdl.handle.net/2115/87729 |
| Appears in Collections: | 工学院・工学研究院 (Graduate School of Engineering / Faculty of Engineering) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
|
