A common problem of mid-air interaction is excessive arm fatigue, known as the "Gorilla arm" effect. To predict and prevent such problems at a low cost, we investigate user testing of mid-air interaction without real users, utilizing biomechanically simulated AI agents trained using deep Reinforcement Learning (RL). We implement this in a pointing task and four experimental conditions, demonstrating that the simulated fatigue data matches human fatigue data. We also compare two effort models: 1) instantaneous joint torques commonly used in computer animation and robotics, and 2) the recent Three Compartment Controller (3CC-) model from biomechanical literature. 3CC- yields movements that are both more efficient and relaxed, whereas with instantaneous joint torques, the RL agent can easily generate movements that are quickly tiring or only reach the targets slowly and inaccurately. Our work demonstrates that deep RL combined with the 3CC- provides a viable tool for predicting both interaction movements and user experience \textit{in silico}, without users.
https://doi.org/10.1145/3313831.3376701
Mid-air arm movements are ubiquitous in VR, AR, and gestural interfaces. While mouse movements have received some attention, the dynamics of mid-air movements are understudied in HCI. In this paper we present an exploratory analysis of the dynamics of aimed mid-air movements. We explore the 3rd order lag (3OL) and existing 2nd order lag (2OL) models for modeling these dynamics. For a majority of movements the 3OL model captures mid-air dynamics better, in particular acceleration. The models can effectively predict the complete time series of position, velocity and acceleration of aimed movements given an initial state and a target using three (2OL) or four (3OL) free parameters.
Input amplification enables easier movement in virtual reality (VR) for users with mobility issues or in confined spaces. However, current techniques either do not focus on maintaining feelings of body ownership, or are not applicable to general VR tasks. We investigate a general purpose non-linear transfer function that keeps the user's reach within reasonable bounds to maintain body ownership. The technique amplifies smaller movements from a user-definable neutral point into the expected larger movements using a configurable Hermite curve. Two experiments evaluate the approach. The first establishes that the technique has comparable performance to the state-of-the-art, increasing physical comfort while maintaining task performance and body ownership. The second explores the characteristics of the technique over a wide range of amplification levels. Using the combined results, design and implementation recommendations are provided with potential applications to related VR transfer functions.
https://doi.org/10.1145/3313831.3376687
Collaborative Virtual Environments (CVEs) offer unique opportunities for human communication. Humans can interact with each other over a distance in any environment and visual embodiment they want. Although deictic gestures are especially important as they can guide other humans' attention, humans make systematic errors when using and interpreting them. Recent work suggests that the interpretation of vertical deictic gestures can be significantly improved by warping the pointing arm. In this paper, we extend previous work by showing that models enable to also improve the interpretation of deictic gestures at targets all around the user. Through a study with 28 participants in a CVE, we analyzed the errors users make when interpreting deictic gestures. We derived a model that rotates the arm of a pointing user's avatar to improve the observing users' accuracy. A second study with 24 participants shows that we can improve observers' accuracy by 22.9%. As our approach is not noticeable for users, it improves their accuracy without requiring them to learn a new interaction technique or distracting from the experience.
https://doi.org/10.1145/3313831.3376340
Industrial design review is an iterative process which mainly relies on two steps involving many stakeholders: design discussion and CAD data adjustment. We investigate how a wall-sized display could be used to merge these two steps by allowing multidisciplinary collaborators to simultaneously generate and explore design alternatives. We designed ShapeCompare based on the feedback from a usability study. It enables multiple users to compute and distribute CAD data with touch interaction. To assess the benefit of the wall-sized display in such context, we ran a controlled experiment which aims to compare ShapeCompare with a visualization technique suitable for standard screens. The results show that pairs of participants performed a constraint solving task faster and used more deictic instructions with ShapeCompare. From these findings, we draw generic recommendations for collaborative exploration of alternatives.
https://doi.org/10.1145/3313831.3376736