Development Of A First-Person View-Based Korean Sign Language Education System Using VIVE Hand Tracking

Abstract

In this study, using VIVE and Tracking, we developed a ‘first-person view-based Korean sign language education system’ that guides sign language handshapes using a 3D virtual hand from the user's point of view and then evaluates the accuracy of the handshapes that learners perform. The learning effects of a Korean sign language education system that guides sign language handshapes using 3D virtual hands in a virtual space and a system that only learns with a teacher's lecture video in a virtual space were compared and analyzed. The number of times the two groups of participants tried to make the sign language correctly was measured. Then, the measurement results were compared and analyzed and interviews also conducted. As a result of the experiment, the ‘first-person view-based Korean sign language education system’ demonstrated that some sign language handshapes, which are not easy for participants to make, were implemented faster and more accurately than traditional learning methods. Therefore, if this system is used, it is expected that sign language learners will be able to learn more intuitively through the first-person perspective. Furthermore, it is expected that learning effects can be improved through the ability of being able to judge whether users are making the correct shapes with their hands. In addition, since it is possible to evaluate the accuracy of the hand shapes, it is believed that single-person learning, which was not easy in the existing sign language education paradigm, will become possible. However, for the sign language examined in the experiment, some showed significant differences while others were less meaningful. These were analyzed because the difficulty of the learning contents used in the experiment was not high. In future research, in order to more accurately verify the effectiveness of the sign language education system using hand tracking and 3D virtual hand guidance, we intend to conduct an experiment implementing learning content with more complex sentences