Gesture Knitter: A Hand Gesture Design Tool for Head-Mounted Mixed Reality Applications


Hand gestures are a natural and expressive input method enabled by modern mixed reality headsets. However, it remains challenging for developers to create custom gestures for their applications. Conventional strategies to bespoke gesture recognition involve either hand-crafting or data-intensive deep-learning. Neither approach is well suited for rapid prototyping of new interactions. This paper introduces a flexible and efficient alternative approach for constructing hand gestures. We present Gesture Knitter: a design tool for creating custom gesture recognizers with minimal training data. Gesture Knitter allows the specification of gesture primitives that can then be combined to create more complex gestures using a visual declarative script. Designers can build custom recognizers by declaring them from scratch or by providing a demonstration that is automatically decoded into its primitive components. Our developer study shows that Gesture Knitter achieves high recognition accuracy despite minimal training data and delivers an expressive and creative design experience.

George B. Mo
University of Cambridge, Cambridge, United Kingdom
John J. Dudley
University of Cambridge, Cambridge, United Kingdom
Per Ola Kristensson
University of Cambridge, Cambridge, United Kingdom




会議: CHI 2021

The ACM CHI Conference on Human Factors in Computing Systems (

セッション: Computational Physical Interaction

[A] Paper Room 02, 2021-05-10 17:00:00~2021-05-10 19:00:00 / [B] Paper Room 02, 2021-05-11 01:00:00~2021-05-11 03:00:00 / [C] Paper Room 02, 2021-05-11 09:00:00~2021-05-11 11:00:00
Paper Room 02
12 件の発表
2021-05-10 17:00:00
2021-05-10 19:00:00