Acquiring tacit knowledge and practical skills often depends on direct observation and in situ training. AR offers an alternative by overlaying first-person step-by-step instructions that guide users through tasks such as assembly and repair. Previous work demonstrates the effectiveness of AR instruction for specific applications. In our experimental work, we systematically explore aspects of the broader design space. We conducted a controlled experiment (n = 40) to investigate three key factors identified in learning theory and XR embodiment research: imitation timing (parallel vs. sequential), limb visualization (hand vs. full arm), and limb visibility (opaque vs. semi-transparent). Across all conditions, participants followed AR instructions and afterward repeated the tasks from memory. We assessed performance, user experience, and retention. Our results show that parallel imitation is faster and increases embodiment, whereas sequential imitation enhances memory retention and comfort. Our findings provide guidance for the temporal and visual design of first-person AR tutorials.
We explore new interactions for wayfinding in complex, hazardous environments through the concrete case of multi-pitch rock climbing. Working with seven expert climbers, we derived design principles for navigation tools that support both pre-climb planning and on-the-wall decision making with limited physical and mental bandwidth. We then demonstrate these principles in practice through the creation of a mobile application that incorporates high-fidelity vertical terrain models, spatially-anchored multi-modal annotations, integrated human-scale avatars, and adaptable one-handed interactions. Based on our expert feedback and experiences developing the tool, as well as results from a 3-week public deployment and responses from 16 climbers, we highlight new opportunities for interactive tools for rock climbing as well as other hazardous, high-focus activities.
In virtual reality (VR) experiences, mismatches between reality and virtuality are usually undesirable, as they can disrupt immersion and induce cybersickness. However, when carefully controlled, they may expand the design space of VR. This research investigates perceptual detection thresholds for mismatches between real and virtual inclinations during cycling in VR. Using a custom simulation, N=30 participants cycled through a virtual city while physical and visual inclinations were independently manipulated. Real inclinations were implemented with a tilting indoor bike, providing vestibular and proprioceptive feedback, while virtual inclinations within the simulated environment were presented visually. A multiple staircase procedure derived estimates for perceptual thresholds that approximate which mismatches in visual and physical inclination were still perceived as congruent. These thresholds reveal a window of perceived congruence before mismatches become noticeable to users. These findings advance understanding of sensory integration in VR cycling and inform applications in immersive training, exergames, and rehabilitation systems.
Mixed Reality (MR) interfaces have traditionally relied on visual modalities, but this poses challenges in high-stakes or cognitively demanding contexts where continuous visual attention is impractical. This exploratory study investigates audio-centric interaction in MR, specifically focusing on 3D spatial navigation. Unlike prior work limited to 2D navigation, we developed a custom augmented reality (AR) navigation system to compare \textit{audio AR} (AAR) with (1) visual AR, (2) combined audio-visual AR, and (3) traditional navigation without aids. Results show that AAR enables users to maintain environmental awareness comparable to those without AR assistance, while showing no meaningful difference in navigation performance with visual AR in complex 3D environments. Qualitative reports and awareness tests suggest AAR reduces visual tunneling. These exploratory findings suggest the potential use of AAR in many surrounding fields, such as accessibility for blind or visually impaired individuals or training resources for first responders, offering insights into audio interface design for safety-critical applications.
With a stylus, users can both sweep sketches across models and pinpoint locations with precision. Building on this dual capability, we explore how teleportation can be integrated into stylus interaction without disrupting the flow of common stylus usage. We introduce two key ideas: flipping the stylus as an intuitive mode switch between drawing and teleportation, and using gaze to set orientation while the stylus handles positioning. In a user study that features a teleport-and-orient task, we evaluate six teleportation techniques, covering two mode-switching methods (Button and Flip) and three orientation approaches (StylusRoll, StylusPoint, and GazePoint). The results offer new insights into the relative merits and limitations of each technique. Our work contributes to knowledge about teleportation in VR and fills the gap in seamlessly integrating teleportation with stylus use in 3D.
Various virtual locomotion techniques and visual transition methods are used in VR-based navigation research, yet few studies have systematically examined their effects on spatial learning, cognitive map formation, and navigational performance in complex indoor environments. We conducted a between-subjects study (N=142) in two high-fidelity VR hospital contexts, including free exploration and task-based wayfinding, while treating locomotion and viewpoint transitions as experimental factors. Spatial learning was measured through pointing, distance estimation, and sketch-map accuracy; performance was measured through completion time and distance traveled; and experience was measured through cybersickness, perceived presence, and usability. Locomotion techniques affected task completion time, with teleportation associated with faster performance in the task-based context. Spatial learning effects were mixed, with patterns indicating that techniques without viewpoint transitions may better support cognitive mapping. Empirical insights and guidelines are provided to improve the reliability and real-world applicability of VR-based wayfinding research.
In mixed reality environments, virtual objects can obscure real-world obstacles, creating a risk of collision when users walk through them. Although users may choose to detour around virtual objects, this behavior also carries risks, such as colliding with obstacles or pedestrians along the detour route. To reduce collision risks, it is essential to understand the factors that determine whether users walk through or detour, as well as the walking paths associated with each behavior. In this research, we investigated users’ walking behavior toward both a static virtual obstacle and a virtual obstacle that disappeared as the user approached. Our findings suggest that individual characteristics and the width of the virtual obstacle influence the decision to walk through or detour. Furthermore, while most users initially chose paths that detoured around the virtual obstacle, once the obstacle began to disappear, they switched their walking paths toward the space where it had been.