Point & Grasp: Flexible Selection of Out-of-Reach Objects Through Probabilistic Cue Integration

要旨

Selecting out-of-reach objects is a fundamental task in mixed reality (MR). Existing methods rely on a single cue or deterministically fuse multiple cues, leading to performance degradation when the dominant cue becomes unreliable. In this work, we introduce a probabilistic cue integration framework that enables flexible combination of multiple user-generated cues for intent inference. Inspired by natural grasping behavior, we instantiate the framework with pointing direction and grasp gestures as a new interaction technique, \textsc{Point\&Grasp}. To this end, we collect the \datasetfullname~(\dataset) dataset to train a robust likelihood model of the gestural cue, which captures grasping patterns not present in existing in-reach datasets. User studies demonstrate that our selection method with cue integration not only improves accuracy and speed over single-cue baselines, but also remains practically effective compared to state-of-the-art methods across various sources of ambiguity. The dataset and code are available at \url{https://github.com/drlxj/point-and-grasp}.

著者
Xuejing Luo
Aalto University, Espoo, Finland
Hee-Seung Moon
Chung-Ang University, Seoul, Korea, Republic of
Christian Holz
ETH Zürich, Zurich, Switzerland
Antti Oulasvirta
Aalto University, Helsinki, Finland

会議: CHI 2026

ACM CHI Conference on Human Factors in Computing Systems

セッション: XR Selection

P1 - Room 131
7 件の発表
2026-04-14 18:00:00
2026-04-14 19:30:00