Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both -- their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. Code is available at https://github.com/VIDA-NYU/satori-assistance.
Adaptive AR assistance can automatically trigger content to support users based on their context. Such intelligent automation offers many benefits but also alters users' degree of control, which is seldom explored in existing research. In this paper, we compare high- and low-agency control in AR-assisted construction assembly to understand the role of user agency. We designed cognitive and physical assembly scenarios and conducted a lab study (N=24), showing that low-agency control reduced mental workloads and perceived autonomy in several tasks. A follow-up domain expert study with trained carpenters (N=8) contextualised these results in an ecologically valid setting. Through semi-structured interviews, we examined the carpenters' perspectives on AR support in their daily work and the trade-offs of automating interactions. Based on these findings, we summarise key design considerations to inform future adaptive AR designs in the context of timber construction.
Reliable augmented reality (AR) cues can support the resumption of interrupted tasks. We investigated how sub-optimal AR cue reliability (100%, 86%, 64%, or no cue) affected users’ resumption performance and strategies. In a between-subjects experiment, 120
participants conducted a physical sorting task including interruptions, and we manipulated AR cue reliability (i.e., the AR cue was present or absent at the end of interruptions). In trials with AR cue, performance with 86% and 64% reliable AR cues was as well as with
100% reliable cues. In trials without AR cue, performance with suboptimal AR cue reliability declined but was still better than with no cue. Cue reliability affected task resumption strategies of the 86% (slow but no increase in errors) and the 64% (fast but increase in errors) reliability groups differently. Our results extend reliability research to interruptions and the observed efficiency-thoroughness trade-offs in resumption strategies provide insight for design
Ubiquitous computing devices like Augmented Reality (AR) glasses allow countless spontaneous interactions - all serving different goals. AR devices rely on data transfer to personalize recommendations and adapt to the user. Today's consent mechanisms, such as privacy policies, are suitable for long-lasting interactions; however, how users can consent to fast, spontaneous interactions is unclear. We first conducted two focus groups (N=17) to identify privacy-relevant scenarios in AR. We then conducted expert interviews (N=11) with co-design activities to establish effective consent mechanisms. Based on that, we contribute (1) a validated scenario taxonomy to define privacy-relevant AR interaction scenarios, (2) a flowchart to decide on the type of mechanisms considering contextual factors, (3) a design continuum and design aspects chart to create the mechanisms, and (4) a trade-off and prediction chart to evaluate the mechanism. Thus, we contribute a conceptual framework fostering a privacy-preserving future with AR.
Virtual reality (VR) applications achieve their high immersive potential by detaching the user from the real world, replacing it through a virtual environment. This detachment also blocks real-world orientation cues, which might cause fear of colliding with the real environment and negatively impact the player experience. However, since collision anxiety (CA) is a relatively young concept, it is unclear how factors like users’ VR expertise or specific game design choices may affect it. We defined expected CA profiles for five commercial VR games and conducted a longitudinal study examining how growing VR expertise and VR game design influence the users’ CA. After six weeks and a total of 154 VR sessions, results indicate that CA differs between applications and generally decreases as VR expertise increases. Based on our results, we propose design implications, providing researchers and designers with guidelines on when to expect and how to avoid fear of colliding.
Augmented Reality (AR) is a promising medium for guiding users through tasks, yet its impact on fostering deeper task understanding remains underexplored. This paper investigates the impact of reflective prompts---strategic questions that encourage users to challenge assumptions, connect actions to outcomes, and consider hypothetical scenarios---on task comprehension and performance. We conducted a two-phase study: a formative survey and co-design sessions (N=9) to develop reflective prompts, followed by a within-subject evaluation (N=16) comparing AR instructions with and without these prompts in coffee-making and circuit assembly tasks. Our results show that reflective prompts significantly improved objective task understanding and resulted in more proactive information acquisition behaviors during task completion. These findings highlight the potential of incorporating reflective elements into AR instructions to foster deeper engagement and learning. Based on data from both studies, we synthesized design guidelines for integrating reflective elements into AR systems to enhance user understanding without compromising task performance.
Modern augmented reality (AR) devices with advanced display and sensing capabilities pose significant privacy risks to users and bystanders. While previous context-aware adaptations focused on usability and ergonomics, we explore the design space of privacy-driven adaptations that allow users to meet their dynamic needs. These techniques offer granular control over AR sensing capabilities across various AR input, output, and interaction modalities, aiming to minimize degradations to the user experience. Through an elicitation study with 10 AR researchers, we derive 62 privacy-focused adaptation techniques that preserve key AR functionalities and classify them into system-driven, user-driven, and mixed-initiative approaches to create an adaptation catalog. We also contribute a visualization tool that helps AR developers navigate the design space, validating its effectiveness in design workshops with six AR developers. Our findings indicate that the tool allowed developers to discover new techniques, evaluate tradeoffs, and make informed decisions that balance usability and privacy concerns in AR design.