An evaluation of depth camera-based hand pose recognition for virtual reality systems.
Loading...
Date
2018
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Camera-based hand gesture recognition for interaction in virtual reality systems
promises to provide a more immersive and less distracting means of input than the
usual hand-held controllers. It is unknown if a camera would effectively distinguish
hand poses made in a virtual reality environment, due to lack of research in this area.
This research explores and measures the effectiveness of static hand pose input with a
depth camera, specifically the Leap Motion controller, for user interaction in virtual
reality applications. A pose set was derived by analyzing existing gesture taxonomies
and Leap Motion controller-based virtual reality applications, and a dataset of these
poses was constructed using data captured by twenty-five participants. Experiments
on the dataset utilizing three popular machine learning classifiers were not able to
classify the poses with a high enough accuracy, primarily due to occlusion issues affecting
the input data. Therefore, a significantly smaller subset was empirically derived
using a novel algorithm, which utilized a confusion matrix from the machine learning
experiments as well as a table of Hamming Distances between poses. This improved
the recognition accuracy to above 99%, making this set more suitable for real-world
use. It is concluded that while camera-based pose recognition can be reliable on a
small set of poses, finger occlusion hinders the use of larger sets. Thus, alternative
approaches, such as multiple input cameras, should be explored as a potential solution
to the occlusion problem.
Description
Masters Degree. University of KwaZulu-Natal, Durban.