Hand Interfaces: Using Hands to Imitate Objects in AR/VR for Expressive Interactions

Abstract

Augmented reality (AR) and virtual reality (VR) technologies create exciting new opportunities for people to interact with computing resources and information. Less exciting is the need for holding hand controllers, which limits applications that demand expressive, readily available interactions. Prior research investigated freehand AR/VR input by transforming the user's body into an interaction medium. In contrast to previous work that has users' hands grasp virtual objects, we propose a new interaction technique that lets users' hands become virtual objects by imitating the objects themselves. For example, a thumbs-up hand pose is used to mimic a joystick. We created a wide array of interaction designs around this idea to demonstrate its applicability in object retrieval and interactive control tasks. Collectively, we call these interaction designs Hand Interfaces. From a series of user studies comparing Hand Interfaces against various baseline techniques, we collected quantitative and qualitative feedback, which indicates that Hand Interfaces are effective, expressive, and fun to use.

Award
Honorable Mention
Authors
Siyou Pei
University of California, Los Angeles, Los Angeles, California, United States
Alexander Chen
University of California, Los Angeles, Los Angeles, California, United States
Jaewook Lee
University of Illinois at Urbana-Champaign, Urbana, Illinois, United States
Yang Zhang
University of California, Los Angeles, Los Angeles, California, United States
Paper URL

https://dl.acm.org/doi/abs/10.1145/3491102.3501898

Video

Conference: CHI 2022

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2022.acm.org/)

Session: Hands & Fingers

286–287
5 items in this session
2022-05-03 11:00:00
2022-05-03 12:15:00