Moving Toward an Ecologically Valid Data Collection Protocol for 2D Gestures In Video Games
Abstract
Those who design gesture recognizers and user interfaces often use data collection applications that enable users to comfortably produce gesture training samples. In contrast, games present unique contexts that impact cognitive load and have the potential to elicit rapid gesticulations as players react to dynamic conditions, which can result in high gesture form variability. However, the extent to which these gestures differ is presently unknown. To this end, we developed two games with unique mechanics, Follow the Leader (FTL) and Sleepy Town, as well as a standard data collection application. We collected gesture samples from 18 participants across all conditions for gestures of varying complexity, and through an analysis using relative, global, and distribution coverage measures, we confirm significant differences between conditions. We discuss the implications of our findings, and show that our FTL design is closer to being an ecologically valid data collection protocol with low implementation complexity.
Short Summary
When playing video games, users input gestures that are more variable than those collected in lab conditions. Systems trained on lab data will underperform in real-world scenarios. We designed a simple-to-implement protocol which helps elicit more variability during user gesture collection.
Mykola Maslych
Computer Science PhD Candidate
My research interests include machine learning applied to 3D User interfaces and HCI in general.