Action pointing involves choosing and executing an action at a specific place in the workspace (e.g., choosing a tool and clicking to start drawing, or selecting an object and copying with a shortcut). The elements of action pointing (choosing an action, specifying a position, and triggering the action) can be carried out in many ways - and our analysis of current techniques identified limitations on performance, particularly for repeated sequences of interactions. To empirically analyse interaction alternatives for action pointing, we developed and evaluated two techniques: ModeKeys removes modifier keys from keyboard shortcuts used to choose actions; AimKeys goes further by using the shortcut (not the mouse) to trigger the action. Three studies over three tasks showed that these reconfigurations were highly effective - in all studies, either AimKeys or ModeKeys were faster, easier, and preferred overall. Our studies show that small variations in the configuration of action pointing can have a large impact, offering opportunities to improve performance with direct-manipulation systems.
This paper investigates multi-selection in XR interfaces based on eye and hand interaction. We propose enabling multi-selection using different variations of techniques that combine gaze with a semi-pinch gesture, allowing users to select multiple objects, while on the way to a full-pinch. While our exploration is based on the semi-pinch mode for activating a quasi-mode, we explore four methods for confirming subselections in multi-selection mode, varying in effort and complexity: dwell-time (SemiDwell), swipe (SemiSwipe), tilt (SemiTilt), and non-dominant hand input (SemiNDH), and compare them to a baseline technique. In the user study, we evaluate their effectiveness in reducing task completion time, errors, and effort. The results indicate the strengths and weaknesses of each technique, with SemiSwipe and SemiDwell as the most preferred methods by participants. We also demonstrate their utility in file managing and RTS gaming application scenarios. This study provides valuable insights to advance 3D input systems in XR.
Designing efficient selection techniques for graphical user interfaces (GUIs) is fundamental in HCI research. We derive selection techniques based on the multiple process model, a theory that details the motor control processes during goal-directed movements. Specifically, we deduce three theoretical assumptions on how control processes of pre-planning, impulse control, and limb-target control could influence selection movements when adjusting GUI elements, including visual feedback, cursor position, and target position. Corresponding to our assumptions, we develop three techniques that hide the cursor when a target is highlighted, snap the cursor when selection begins, and expand clustered objects during selection movements. After that, we pre-register the assumptions and research methodology and evaluate the techniques in three crowdsourcing-based pointing studies. Our results show that all techniques improved the selection efficiency compared to established baselines. We further discuss the design implications and reflect on how we derived techniques from theory.
When using indirect pointing devices in modern operating systems (OS), users' perception of the pointing transfer function is easily influenced by the device's hardware or OS-native transfer function settings. This could hinder users from finding and fully adapting to the transfer function that is optimal for them. We propose a novel hardware-embedded transfer function technique that is expected to allow users to consistently experience the desired function even when device hardware or OS settings change. The technique (1) allows users to define the desired function within the device firmware in physical units and (2) enables the firmware to cancel out the influence of OS-native functions and hardware setting perturbations, so that the uploaded function can persist regardless of the external environment. Through technical evaluation including transfer functions of various shapes, we showed that the proposed technique has comparable robustness and accuracy to the conventional approach.
Window selection is a fundamental method in desktop environments for interacting with multiple targets, typically performed by successive operations like click-drag-release (i.e., a single sequence of dragging). Although this method is common in GUI interactions, there has been limited research to understand user behavior during window selection. This study explores user behavior and performance during window selection using dragging. We empirically studied the impact of several GUI parameters — including the size, interval, number, and layout of targets — on window selection for multiple targets. Based on well-established existing motor models, we analyzed user behavior in terms of time performance and derived a more suitable model. Additionally, our new prediction model effectively predicted time performance in partially constrained scenarios. This study provides new insights into user behavior during window selection for multiple targets. We hope that our research findings will assist GUI designers, practitioners, and researchers in testing their designs.
Free-hand interactions have been widely deployed for AR/VR interfaces to promote a natural and seamless interaction experience. Among various types of hand interactions, microgestures are still limited in supporting discrete inputs and in lacking a continuous interaction theme. To this end, we propose a new pointing technique, T2IRay, which enables continuous indirect pointing through microgestures for continuous spatial input. We employ our own local coordinate system based on the thumb-to-index finger relationship to map the computed raycasting direction for indirect pointing in a virtual environment. Furthermore, we examine various mapping methodologies and collect thumb-click behaviors to formulate thumb-to-index microgesture design guidelines to foster continuous, reliable input. We evaluate the design parameters for mapping indirect pointing with acceptable speed, depth, and range. We collect and analyze the characteristics of click behaviors for future implementation. Our research demonstrates the potential and practicality of free-hand micro-finger input methods for advancing future interaction paradigms.