Exploring Mid-Air Gestures in Human-Computer Interfaces

This thesis investigates three hand gesture detection technologies and their integration into everyday tasks for interacting with technology, including text entry, target selection, and gaming. Through three user studies, these technologies have been incorporated and evaluated within novel applications relying on mid-air hand gestures.

The first user study introduces a one-handed mid-air gesture-based text entry method in Virtual Reality with two layouts of a four-key ambiguous keyboard. Five participants took part in a five-day longitudinal study using a computer camera and an open-source gesture detection framework for gesture detection. Their entry speed (wpm) and error rate (%) were recorded and analyzed.

In the second user study, a Leap Motion Controller (LMC) was integrated with a physical keyboard, introducing a novel computer keyboard, LeapBoard, that combines mid-air hand gestures with physical keys. The evaluation compared LeapBoard’s point-and-select ability to a touch-based method (using a touchpad) and a mid-air gesture-based method in a target selection task. A user study with 12 participants measured throughput (bps) and error rates (%) across different selection methods, movement amplitudes, and target widths.

The third user study investigates the effects of two types of mid-air hand gesture-based in-put methods on children’s performance, fun, and preference and compares them to a mouse input method. The evaluation was done using a card-matching game on a laptop with 18 children between five and seven. The trial completion time (s), number of selected cards, and children’s perception regarding ease of use, likability, and willingness to play the game using each input method again were recorded and analyzed.