The ACM CHI Conference on Human Factors in Computing Systems (https://chi2021.acm.org/)

XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces
João Marcelo. Evangelista Belo (Aarhus University, Aarhus, Denmark)Anna Maria. Feit (ETH Zurich, Zurich, Switzerland)Tiare Feuchtner (Aarhus University, Aarhus, Denmark)Kaj Grønbæk (Aarhus University, Aarhus, Denmark)
Arm discomfort is a common issue in Cross Reality applications involving prolonged mid-air interaction. Solving this problem is difficult because of the lack of tools and guidelines for 3D user interface design. Therefore, we propose a method to make existing ergonomic metrics available to creators during design by estimating the interaction cost at each reachable position in the user's environment. We present XRgonomics, a toolkit to visualize the interaction cost and make it available at runtime, allowing creators to identify UI positions that optimize users' comfort. Two scenarios show how the toolkit can support 3D UI design and dynamic adaptation of UIs based on spatial constraints. We present results from a walkthrough demonstration, which highlight the potential of XRgonomics to make ergonomics metrics accessible during the design and development of 3D UIs. Finally, we discuss how the toolkit may address design goals beyond ergonomics.
ThermoCaress: A Wearable Haptic Device with Illusory Moving Thermal Stimulation
Yuhu Liu (The University of Tokyo, Tokyo, Japan)Satoshi Nishikawa (The University of Tokyo, Tokyo, Japan)Young ah Seong (Hosei University, Tokyo, Japan)Ryuma Niiyama (The University of Tokyo, Tokyo, Japan)Yasuo Kuniyoshi (The University of Tokyo, Tokyo, Japan)
We propose ThermoCaress, a haptic device to create a stroking sensation on the forearm using pressure force and present thermal feedback simultaneously. In our method, based on the phenomenon of thermal referral, by overlapping a stroke of pressure force, users feel as if the thermal stimulation moves although the position of temperature source is static. We designed the device to be compact and soft, using microblowers and inflatable pouches for presenting pressure force and water for presenting thermal feedback. Our user study showed that the device succeeded in generating thermal referrals and creating a moving thermal illusion. The results also suggested that cold temperature enhance the pleasantness of stroking. Our findings contribute to expanding the potential of thermal haptic devices.
Touch&Fold: A Foldable Haptic Actuator for Rendering Touch in Mixed Reality
Shan-Yuan Teng (University of Chicago, Chicago, Illinois, United States)Pengyu Li (University of Chicago, Chicago, Illinois, United States)Romain Nith (University of Chicago, Chicago, Illinois, United States)Joshua Fonseca (University of Chicago, Chicago, Illinois, United States)Pedro Lopes (University of Chicago, Chicago, Illinois, United States)
We propose a nail-mounted foldable haptic device that provides tactile feedback to mixed reality (MR) environments by pressing against the user’s fingerpad when a user touches a virtual object. What is novel in our device is that it quickly tucks away when the user interacts with real-world objects. Its design allows it to fold back on top of the user’s nail when not in use, keeping the user’s fingerpad free to, for instance, manipulate handheld tools and other objects while in MR. To achieve this, we engineered a wireless and self-contained haptic device, which measures 24×24×41 mm and weighs 9.5 g. Furthermore, our foldable end-effector also features a linear resonant actuator, allowing it to render not only touch contacts (i.e., pressure) but also textures (i.e., vibrations). We demonstrate how our device renders contacts with MR surfaces, buttons, low- and high-frequency textures. In our first user study, we found that participants perceived our device to be more realistic than a previous haptic device that also leaves the fingerpad free (i.e., fingernail vibration). In our second user study, we investigated the participants’ experience while using our device in a real-world task that involved physical objects. We found that our device allowed participants to use the same finger to manipulate handheld tools, small objects, and even feel textures and liquids, without much hindrance to their dexterity, while feeling haptic feedback when touching MR interfaces.
Proxemics and Social Interactions in an Instrumented Virtual Reality Workshop
Julie R.. Williamson (University of Glasgow, Glasgow, United Kingdom)Jie Li (Centrum Wiskunde & Informatica, Amsterdam, Netherlands)David A.. Shamma (Centrum Wiskunde & Informatica, Amsterdam, Netherlands)Vinoba Vinayagamoorthy (BBC Research & Development, London, United Kingdom)Pablo Cesar (CWI, Amsterdam, Netherlands)
Virtual environments (VEs) can create collaborative and social spaces, which are increasingly important in the face of remote work and travel reduction. Recent advances, such as more open and widely available platforms, create new possibilities to observe and analyse interaction in VEs. Using a custom instrumented build of Mozilla Hubs to measure position and orientation, we conducted an academic workshop to facilitate a range of typical workshop activities. We analysed social interactions during a keynote, small group breakouts, and informal networking/hallway conversations. Our mixed-methods approach combined environment logging, observations, and semi-structured interviews. The results demonstrate how small and large spaces influenced group formation, shared attention, and personal space, where smaller rooms facilitated more cohesive groups while larger rooms made small group formation challenging but personal space more flexible. Beyond our findings, we show how the combination of data and insights can fuel collaborative spaces' design and deliver more effective virtual workshops.
Can Playing with Toy Blocks Reflect Behavior Problems in Children?
Xiyue Wang (Tohoku University, Sendai, Japan)Kazuki Takashima (Tohoku University, Sendai, Japan)Tomoaki Adachi (Miyagi Gakuin Women's University, Sendai, Miyagi, Japan)Yoshifumi Kitamura (Tohoku University, Sendai, Japan)
Although children’s behavioral and mental problems are generally diagnosed in clinical settings, the prediction and awareness of children’s mental wellness in daily settings are getting increased attention. Toy blocks are both accessible in most children’s daily lives and provide physicality as a unique non-verbal channel to express their inner world. In this paper, we propose a toy block approach for predicting a range of behavior problems in young children (4-6 years old) measured by the Child Behavior Checklist (CBCL). We defined and classified a set of quantitative play actions from IMU-embedded toy blocks. Play data collected from 78 preschoolers revealed that specific play actions and patterns indicate total problems, internalizing problems, and aggressive behavior in children. The results align with our qualitative observations, and suggest the potential of predicting the clinical behavior problems of children based on short free-play sessions with sensor-embedded toy blocks.
Improving Viewing Experiences of First-Person Shooter Gameplays with Automatically-Generated Motion Effects
Gyeore Yun (POSTECH, Pohang, Korea, Republic of)Hyoseung Lee (POSTECH, Pohang, Gyeongsangbuk-do, Korea, Republic of)Sangyoon Han (Pohang University of Science and Technology (POSTECH), Pohang, Korea, Republic of)Seungmoon Choi (Pohang University of Science and Technology (POSTECH), Pohang, Gyeongbuk, Korea, Republic of)
In recent times, millions of people enjoy watching video gameplays at an eSports stadium or home. We seek a method that improves gameplay spectator or viewer experiences by presenting multisensory stimuli. Using a motion chair, we provide the motion effects automatically generated from the audiovisual stream to the viewers watching a first-person shooter (FPS) gameplay. The motion effects express the game character’s movement and gunfire action. We describe algorithms for the computation of such motion effects developed using computer vision techniques and deep learning. By a user study, we demonstrate that our method of providing motion effects significantly improves the viewing experiences of FPS gameplay. The contributions of this paper are with the motion synthesis algorithms integrated for FPS games and the empirical evidence for the benefits of experiencing multisensory gameplays.
Flower Jelly Printer: Slit Injection Printing for Parametrically Designed Flower Jelly
Mako Miyatake (The University of Tokyo, Tokyo, Japan)Koya Narumi (The University of Tokyo, Tokyo, Japan)Yuji Sekiya (The University of Tokyo, Bunkyo-ku, Tokyo, Japan)Yoshihiro Kawahara (The university of Tokyo, Bunkyo-ku, Tokyo, Japan)
Flower jellies, a delicate dessert in which a flower-shaped jelly floats inside another clear jelly, fascinate people with both their beauty and elaborate construction. In efforts to simplify the challenging fabrication and enrich the design space of this dessert, we present Flower Jelly Printer: a printing device and design software for digitally fabricating flower jellies. Our design software lets users play with parameters and preview the resulting forms until achieving their desired shapes. We also developed slit injection printing that directly injects colored jelly into a base jelly, and shared several design examples to show the breadth of design possibilities. Finally, the user study with novice and experienced users demonstrates that our system benefits creators of all experience levels by iterative design and precise fabrication. We hope to enable more people to design and create their own flower jellies while expanding access and the design space for digitally fabricated foods.
Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation
Riku Arakawa (The University of Tokyo, Hongo, Japan)Hiromu Yakura (University of Tsukuba, Tsukuba, Japan)
Explicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human-AI symbiosis.
Teardrop Glasses: Pseudo Tears Induce Sadness in You and Those Around You
Shigeo Yoshida (The University of Tokyo, Tokyo, Japan)Takuji Narumi (the University of Tokyo, Tokyo, Japan)Tomohiro Tanikawa (the University of Tokyo, Tokyo, Japan)Hideaki Kuzuoka (The University of Tokyo, Bunkyo-ku, Tokyo, Japan)Michitaka Hirose (The University of Tokyo, Tokyo, Japan)
Emotional contagion is a phenomenon in which one's emotions are transmitted among individuals unconsciously by observing others' emotional expressions. In this paper, we propose a method for mediating people's emotions by triggering emotional contagion through artificial bodily changes such as pseudo tears. We focused on shedding tears because of the link to several emotions besides sadness. In addition, it is expected that shedding tears would induce emotional contagion because it is observable by others. We designed an eyeglasses-style wearable device, Teardrop glasses, that release water drops near the wearer's eyes. The drops flow down the cheeks and emulate real tears. The study revealed that artificial crying with pseudo tears increased sadness among both wearers and those observing them. Moreover, artificial crying attenuated happiness and positive feelings in observers. Our findings show that actual bodily changes are not necessary for inducing emotional contagion as artificial bodily changes are also sufficient.
LightTouch Gadgets: Extending Interactions on Capacitive Touchscreens by Converting Light Emission to Touch Inputs
Kaori Ikematsu (Yahoo Japan Corporation, Tokyo, Japan)Kunihiro Kato (Tokyo University of Technology, Tokyo, Japan)Yoshihiro Kawahara (The University of Tokyo, Tokyo, Japan)
We present LightTouch, a 3D-printed passive gadget to enhance touch interactions on unmodified capacitive touchscreens. The LightTouch gadgets simulate finger operations such as tapping, swiping, and multi-touch gestures by means of conductive materials and light-dependent resistors (LDR) embedded in the object. The touchscreen emits visible light and the LDR senses the level of this light, which changes its resistance value. By controlling the screen brightness, it intentionally connects or disconnects the path between the GND and the touchscreen, thus allowing the touch inputs to be controlled. In contrast to conventional physical extensions for touchscreens, our technique requires neither continuous finger contact on the conductive part nor the use of batteries. As such, it opens up new possibilities for touchscreen interactions beyond the simple automation of touch inputs, such as establishing a communication channel between devices, enhancing the trackability of tangibles, and inter-application operations.
TiltChair: Manipulative Posture Guidance by Actively Inclining the Seat of an Office Chair
Kazuyuki Fujita (Tohoku University, Sendai, Miyagi, Japan)Aoi Suzuki (Research Institute of Electrical Communication, Tohoku University, Sendai, Japan)Kazuki Takashima (Tohoku University, Sendai, Japan)Kaori Ikematsu (Yahoo Japan Corporation, Tokyo, Japan)Yoshifumi Kitamura (Tohoku University, Sendai, Japan)
We propose TiltChair, an actuated office chair that physically manipulates the user's posture by actively inclining the chair's seat to address problems associated with prolonged sitting. The system controls the inclination angle and motion speed with the aim of achieving manipulative but unobtrusive posture guidance. To demonstrate its potential, we first built a prototype of TiltChair with a seat that could be tilted by pneumatic control. We then investigated the effects of the seat's inclination angle and motions on task performance and overall sitting experience through two experiments. The results show that the inclination angle mainly affects the difficulty of maintaining one's posture, while the motion speed affected the conspicuousness and subjective acceptability of the motion. However, these seating conditions did not affect objective task performance. Based on these results, we propose a design space for facilitating effective seat-inclination behavior using the three dimensions of angle, speed, and continuity. Furthermore, we discuss promising applications.
Bad Breakdowns, Useful Seams, and Face Slapping: Analysis of VR Fails on YouTube
Emily Dao (Monash University, Melbourne, Victoria, Australia)Andreea Muresan (University of Copenhagen, Copenhagen, Denmark)Kasper Hornbæk (University of Copenhagen, Copenhagen, Denmark)Jarrod Knibbe (University of Melbourne, Melbourne, Australia)
Virtual reality (VR) is increasingly used in complex social and physical settings outside of the lab. However, not much is known about how these settings influence use, nor how to design for them. We analyse 233 YouTube videos of VR Fails to: (1) understand when breakdowns occur, and (2) reveal how the seams between VR use and the social and physical setting emerge. The videos show a variety of fails, including users flailing, colliding with surroundings, and hitting spectators. They also suggest causes of the fails, including fear, sensorimotor mismatches, and spectator participation. We use the videos as inspiration to generate design ideas. For example, we discuss more flexible boundaries between the real and virtual world, ways of involving spectators, and interaction designs to help overcome fear. Based on the findings, we further discuss the ‘moment of breakdown’ as an opportunity for designing engaging and enhanced VR experiences.
TexYZ: Embroidering Enameled Wires for Three Degree-of-Freedom Mutual Capacitive Sensing
Roland Aigner (University of Applied Sciences Upper Austria, Hagenberg, Austria)Andreas Pointner (University of Applied Sciences Upper Austria, Hagenberg, Austria)Thomas Preindl (University of Applied Sciences Upper Austria, Hagenberg, Austria)Rainer Danner (University of Applied Sciences Upper Austria, Hagenberg, Austria)Michael Haller (University of Applied Sciences Upper Austria, Hagenberg, Austria)
In this paper, we present TexYZ, a method for rapid and effortless manufacturing of textile mutual capacitive sensors using a commodity embroidery machine. We use enameled wire as a bobbin thread to yield textile capacitors with high quality and consistency. As a consequence, we are able to leverage the precision and expressiveness of projected mutual capacitance for textile electronics, even when size is limited. Harnessing the assets of machine embroidery, we implement and analyze five distinct electrode patterns, examine the resulting electrical features with respect to geometrical attributes, and demonstrate the feasibility of two promising candidates for small-scale matrix layouts. The resulting sensor patches are further evaluated in terms of capacitance homogeneity, signal-to-noise ratio, sensing range, and washability. Finally, we demonstrate two use case scenarios, primarily focusing on continuous input with up to three degrees-of-freedom.
Oh, Snap! A Fabrication Pipeline to Magnetically Connect Conventional and 3D-Printed Electronics
Martin Schmitz (Technical University of Darmstadt, Darmstadt, Germany)Jan Riemann (Technical University of Darmstadt, Darmstadt, Germany)Florian Müller (TU Darmstadt, Darmstadt, Germany)Steffen Kreis (TU Darmstadt, Darmstadt, Germany)Max Mühlhäuser (TU Darmstadt, Darmstadt, Germany)
3D printing has revolutionized rapid prototyping by speeding up the creation of custom-shaped objects. With the rise of multi-material 3Dprinters, these custom-shaped objects can now be made interactive in a single pass through passive conductive structures. However, connecting conventional electronics to these conductive structures often still requires time-consuming manual assembly involving many wires, soldering or gluing. To alleviate these shortcomings, we propose Oh, Snap!: a fabrication pipeline and interfacing concept to magnetically connect a 3D-printed object equipped with passive sensing structures to conventional sensing electronics. To this end, Oh, Snap! utilizes ferromagnetic and conductive 3D-printed structures, printable in a single pass on standard printers. We further present a proof-of-concept capacitive sensing board that enables easy and robust magnetic assembly to quickly create interactive 3D-printed objects. We evaluate Oh, Snap! by assessing the robustness and quality of the connection and demonstrate its broad applicability by a series of example applications.
Elbow-Anchored Interaction: Designing Restful Mid-Air Input
Rafael Veras (Huawei, Markham, Ontario, Canada)Gaganpreet Singh (Huawei, Markham, Ontario, Canada)Farzin Farhadi-Niaki (Huawei, Markham, Ontario, Canada)Ritesh Udhani (University of Manitoba, Winnipeg, Manitoba, Canada)Parth Pradeep. Patekar (University of Manitoba, Winnipeg, Manitoba, Canada)Wei Zhou (Huawei Technologies, Markham, Ontario, Canada)Pourang Irani (University of Manitoba, Winnipeg, Manitoba, Canada)Wei Li (Huawei Canada, Markham, Ontario, Canada)
We designed a mid-air input space for restful interactions on the couch. We observed people gesturing in various postures on a couch and found that posture affects the choice of arm motions when no constraints are imposed by a system. Study participants that sat with the arm rested were more likely to use the forearm and wrist, as opposed to the whole arm. We investigate how a spherical input space, where forearm angles are mapped to screen coordinates, can facilitate restful mid-air input in multiple postures. We present two controlled studies. In the first, we examine how a spherical space compares with a planar space in an elbow-anchored setup, with a shoulder-level input space as baseline. In the second, we examine the performance of a spherical input space in four common couch postures that set unique constraints to the arm. We observe that a spherical model that captures forearm movement facilitates comfortable input across different seated postures.
Large Scale Analysis of Multitasking Behavior During Remote Meetings
Hancheng Cao (Stanford University, Stanford, California, United States)Chia-Jung Lee (Amazon, Seattle, Washington, United States)Shamsi Iqbal (Microsoft Research, Redmond, Washington, United States)Mary Czerwinski (Microsoft Research, Redmond, Washington, United States)Priscilla N Y. Wong (UCL Interaction Centre, London, United Kingdom)Sean Rintel (Microsoft Research, Cambridge, United Kingdom)Brent Hecht (Microsoft, Redmond, Washington, United States)Jaime Teevan (Microsoft, Redmond, Washington, United States)Longqi Yang (Microsoft, Redmond, Washington, United States)
Virtual meetings are critical for remote work because of the need for synchronous collaboration in the absence of in-person interactions. In-meeting multitasking is closely linked to people's productivity and wellbeing. However, we currently have limited understanding of multitasking in remote meetings and its potential impact. In this paper, we present what we believe is the most comprehensive study of remote meeting multitasking behavior through an analysis of a large-scale telemetry dataset collected from February to May 2020 of U.S. Microsoft employees and a 715-person diary study. Our results demonstrate that intrinsic meeting characteristics such as size, length, time, and type, significantly correlate with the extent to which people multitask, and multitasking can lead to both positive and negative outcomes. Our findings suggest important best-practice guidelines for remote meetings (e.g., avoid important meetings in the morning) and design implications for productivity tools (e.g., support positive remote multitasking).
Interoceptive Interaction: An Embodied Metaphor Inspired Approach to Designing for Meditation
Claudia Daudén Roquet (Lancaster University, Lancaster, United Kingdom)Corina Sas (Lancaster University, Lancaster, United Kingdom)
Meditation is a mind-body practice with considerable wellbeing benefits that can take different forms. Novices usually start with focused attention meditation that supports regulation of attention towards an inward focus or internal bodily sensations and away from external stimuli or distractors. Most meditation technologies employ metaphorical mappings of meditative states to visual or soundscape representations to support awareness of mind wandering and attention regulation, although the rationale for such mappings is seldom articulated. Moreover, such external modalities also take the focus attention away from the body. We advance the concept of interoceptive interaction and employed the embodied metaphor theory to explore the design of mappings to the interoceptive sense of thermoception. We illustrate this concept with WarmMind, an on-body interface integrating heat actuators for mapping meditation states. We report on an exploratory study with 10 participants comparing our novel thermal metaphors for mapping meditation states with comparable ones, albeit in aural modality, as provided by Muse meditation app. Findings indicate a tension between the highly discoverable soundscape’s metaphors which however hinder attention regulation, and the ambiguous thermal metaphors experienced as coming from the body and supported attention regulation. We discuss the qualities of embodied metaphors underpinning this tension and propose an initial framework to inform the design of metaphorical mappings for meditation technologies.
IdeaBot: Investigating Social Facilitation in Human-Machine Team Creativity
Angel Hsing-Chi Hwang (Cornell University, Ithaca, New York, United States)Andrea Stevenson Won (Cornell University, Ithaca, New York, United States)
The present study investigates how human subjects collaborate with a computer-mediated chatbot in creative idea generation tasks. In three text-based between-group studies, we tested whether the perceived identity (i.e.,whether the bot is perceived as a machine or as a human) or the conversational style of a teammate would moderate the outcomes of participants’ creative production. In Study 1, participants worked with either a chatbot or a human confederate. In Study 2, all participants worked with a human teammate but were informed that their partner was either a human or a chatbot. Conversely, all participants worked with a chatbot in Study 3, but were told the identity of their partner was either a chatbot or a human. We investigated differences in idea generation outcomes and found that participants consistently contributed more ideas and with ideas of higher quality when they perceived their teamworking partner as a bot. Furthermore, when the conversational style of the partner was robotic, participants with high anxiety in group communication reported greater creative self-efficacy in task performance. Finally, whether the perceived dominance of a partner and the pressure to come up with ideas during the task mediated positive outcomes of idea generation also depends on whether the conversational style of the bot partner was robot- or human-like. Based on our findings, we discussed implications for future design of artificial agents as active team players in collaboration tasks.
Interaction Illustration Taxonomy: Classification of Styles and Techniques for Visually Representing Interaction Scenarios
Axel Antoine (Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, Lille, France)Sylvain Malacria (Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, Lille, France)Nicolai Marquardt (University College London, London, United Kingdom)Géry Casiez (Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, Lille, France)
Static illustrations are ubiquitous means to represent interaction scenarios. Across papers and reports, these visuals demonstrate people's use of devices, explain systems, or show design spaces. Creating such figures is challenging, and very little is known about the overarching strategies for visually representing interaction scenarios. To mitigate this task, we contribute a unified taxonomy of design elements that compose such figures. In particular, we provide a detailed classification of Structural and Interaction strategies, such as composition, visual techniques, dynamics, representation of users, and many others -- all in context of the type of scenarios. This taxonomy can inform researchers' choices when creating new figures, by providing a concise synthesis of visual strategies, and revealing approaches they were not aware of before. Furthermore, to support the community for creating further taxonomies, we also provide three open-source software facilitating the coding process and visual exploration of the coding scheme.
Physiological and Perceptual Responses to Athletic Avatars while Cycling in Virtual Reality
Martin Kocur (University of Regensburg, Regensburg, Germany)Florian Habler (University of Regensburg, Regensburg, Germany)Valentin Schwind (Frankfurt University of Applied Sciences, Frankfurt, Germany)Paweł W. Woźniak (Utrecht University, Utrecht, Netherlands)Christian Wolff (University of Regensburg, Regensburg, Bavaria, Germany)Niels Henze (University of Regensburg, Regensburg, Germany)
Avatars in virtual reality (VR) enable embodied experiences and induce the Proteus effect - a shift in behavior and attitude to mimic one's digital representation. Previous work found that avatars associated with physical strength can decrease users' perceived exertion when performing physical tasks. However, it is unknown if an avatar's appearance can also influence the user's physiological response to exercises. Therefore, we conducted an experiment with 24 participants to investigate the effect of avatars' athleticism on heart rate and perceived exertion while cycling in VR following a standardized protocol. We found that the avatars' athleticism has a significant and systematic effect on users' heart rate and perceived exertion. We discuss potential moderators such as body ownership and users' level of fitness. Our work contributes to the emerging area of VR exercise systems.
The Role of Social Presence for Cooperation in Augmented Reality on Head Mounted Devices
Niklas Osmers (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)Michael Prilla (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)Oliver Blunk (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)Gordon George. Brown (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)Marc Janßen (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)Nicolas Kahrl (Clausthal University of Technology, Clausthal-Zellerfeld, Germany)
With growing interest regarding cooperation support using Augmented Reality (AR), social presence has become a popular measure of its quality. While this concept is established throughout cooperation research, its role in AR is still unclear: Some work uses social presence as an indicator for support quality, while others found no impact at all. To clarify this role, we conducted a literature review of recent publications that empirically investigated social presence in cooperative AR. After a thorough selection procedure, we analyzed 19 publications according to factors influencing social presence and the impact of social presence on cooperation support. We found that certain interventions support social presence better than others, that social presence has an influence on user’s preferences and that the relation between social presence and cooperation quality may depend on the symmetry of the cooperation task. This contributes to existing research by clarifying the role of social presence for cooperative AR and deriving corresponding design recommendations.
Experiencing Simulated Confrontations in Virtual Reality
Patrick Dickinson (University of Lincoln, Lincoln, United Kingdom)Arthur Jones (University of Lincoln, Lincoln, Lincolnshire, United Kingdom)Wayne Christian (Lincoln University, Lincoln, Lincolnshire, United Kingdom)Andrew Westerside (University of Lincoln, Lincoln, United Kingdom)Francis Mulloy (University of Lincoln, Lincoln, United Kingdom)Kathrin Gerling (KU Leuven, Leuven, Belgium)Kieran Hicks (University of Lincoln, Lincoln, Lincolnshire, United Kingdom)Liam Wilson (University of Lincoln, Lincoln, United Kingdom)Adrian Parke (University of the West of Scotland, Glasgow, United Kingdom)
The use of virtual reality (VR) to simulate confrontational human behaviour has significant potential for use in training, where the recreation of uncomfortable feelings may help users to prepare for challenging real-life situations. In this paper we present a user study (n=68) in which participants experienced simulated confrontational behaviour performed by a virtual character either in immersive VR, or on a 2D display. Participants reported a higher elevation in anxiety in VR, which correlated positively with a perceived sense of physical space. Character believability was influenced negatively by visual elements of the simulation, and positively by behavioural elements, which complements findings from previous work. We recommend the use of VR for simulations of confrontational behaviour, where a realistic emotional response is part of the intended experience. We also discuss incorporation of domain knowledge of human behaviours, and carefully crafted motion-captured sequences, to increase users' sense of believability.
SoniBand: Understanding the Effects of Metaphorical Movement Sonifications on Body Perception and Physical Activity
Judith Ley-Flores (Universidad Carlos III de Madrid, Leganes, Madrid, Spain)Laia Turmo Vidal (Uppsala University, Uppsala, Sweden)Nadia Berthouze (University College London, London, United Kingdom)Aneesha Singh (University College London, London, United Kingdom)Frederic Bevilacqua (STMS IRCAM-CNRS-Sorbonne Université, Paris, France)Ana Tajadura-Jiménez (Universidad Carlos III de Madrid / University College London, Madrid / London, Spain)
Negative body perceptions are a major predictor of physical inactivity, a serious health concern. Sensory feedback can be used to alter such body perception; movement sonification, in particular, has been suggested to affect body perception and levels of physical activity (PA) in inactive people. We investigated how metaphorical sounds impact body perception and PA. We report two qualitative studies centered on performing different strengthening/flexibility exercises using SoniBand, a wearable that augments movement through different sounds. The first study involved physically active participants and served to obtain a nuanced understanding of the sonifications’ impact. The second, in the home of physically inactive participants, served to identify which effects could support PA adherence. Our findings show that movement sonification based on metaphors led to changes in body perception (e.g., feeling strong) and PA (e.g., repetitions) in both populations, but effects could differ according to the existing PA-level. We discuss principles for metaphor-based sonification design to foster PA.
More Kawaii than a Real-Person Streamer: Understanding How the Otaku Community Engages with and Perceives Virtual YouTubers
Zhicong Lu (City University of Hong Kong, Hong Kong, China)Chenxinran Shen (University of Toronto, Toronto, Ontario, Canada)Jiannan Li (University of Toronto, Toronto, Ontario, Canada)Hong Shen (Carnegie Mellon University , Pittsburgh, Pennsylvania, United States)Daniel Wigdor (University of Toronto, Toronto, Ontario, Canada)
Live streaming has become increasingly popular, with most streamers presenting their real-life appearance. However, Virtual YouTubers (VTubers), virtual 2D or 3D avatars that are voiced by humans, are emerging as live streamers and attracting a growing viewership in East Asia. Although prior research has found that many viewers seek real-life interpersonal interactions with real-person streamers, it is currently unknown what makes VTuber live streams engaging or how they are perceived differently than real-person streamers. We conducted an interview study to understand how viewers engage with VTubers and perceive the identities of the voice actors behind the avatars (i.e., Nakanohito). The data revealed that Virtual avatars bring unique performative opportunities which result in different viewer expectations and interpretations of VTuber behavior. Viewers intentionally upheld the disembodiment of VTuber avatars from their voice actors. We uncover the nuances in viewer perceptions and attitudes and further discuss the implications of VTuber practices to the understanding of live streaming in general.
Sketchnote Components, Design Space Dimensions, and Strategies for Effective Visual Note Taking
Rebecca Zheng (University College London, London, United Kingdom)Marina Fernández Camporro (University College London, London, United Kingdom)Hugo Romat (ETH, Zurich, Switzerland)Nathalie Henry Riche (Microsoft Research, Redmond, Washington, United States)Benjamin Bach (Edinburgh University, Edinburgh, United Kingdom)Fanny Chevalier (University of Toronto, Toronto, Ontario, Canada)Ken Hinckley (Microsoft Research, Redmond, Washington, United States)Nicolai Marquardt (University College London, London, United Kingdom)
Sketchnoting is a form of visual note taking where people listen to, synthesize, and visualize ideas from a talk or other event using a combination of pictures, diagrams, and text. Little is known about the design space of this kind of visual note taking. With an eye towards informing the implementation of digital equivalents of sketchnoting, inking, and note taking, we introduce a classification of sketchnote styles and techniques, with a qualitative analysis of 103 sketchnotes, and situated in context with six semi-structured follow up interviews. Our findings distill core sketchnote components (content, layout, structuring elements, and visual styling) and dimensions of the sketchnote design space, classifying levels of conciseness, illustration, structure, personification, cohesion, and craftsmanship. We unpack strategies to address particular note taking challenges, for example dealing with constraints of live drawings, and discuss relevance for future digital inking tools, such as recomposition, styling, and design suggestions.
“Grip-that-there”: An Investigation of Explicit and Implicit Task Allocation Techniques for Human-Robot Collaboration
Karthik Mahadevan (University of Toronto, Toronto, Ontario, Canada)Mauricio Sousa (University of Toronto, Toronto, Ontario, Canada)Anthony Tang (University of Toronto, Toronto, Ontario, Canada)Tovi Grossman (University of Toronto, Toronto, Ontario, Canada)
In ad-hoc human-robot collaboration (HRC), humans and robots work on a task without pre-planning the robot's actions prior to execution; instead, task allocation occurs in real-time. However, prior research has largely focused on task allocations that are pre-planned - there has not been a comprehensive exploration or evaluation of techniques where task allocation is adjusted in real-time. Inspired by HCI research on territoriality and proxemics, we propose a design space of novel task allocation techniques including both explicit techniques, where the user maintains agency, and implicit techniques, where the efficiency of automation can be leveraged. The techniques were implemented and evaluated using a tabletop HRC simulation in VR. A 16-participant study, which presented variations of a collaborative block stacking task, showed that implicit techniques enable efficient task completion and task parallelization, and should be augmented with explicit mechanisms to provide users with fine-grained control.
Phonetroller: Visual Representations of Fingers for Precise Touch Input when using a Phone in VR
Fabrice Matulic (Preferred Networks Inc., Tokyo, Japan)Aditya Ganeshan (Preferred Networks Inc., Tokyo, Japan)Hiroshi Fujiwara (Preferred Networks Inc., Tokyo, Japan)Daniel Vogel (University of Waterloo, Waterloo, Ontario, Canada)
Smartphone touch screens are potentially attractive for interaction in virtual reality (VR). However, the user cannot see the phone or their hands in a fully immersive VR setting, impeding their ability for precise touch input. We propose mounting a mirror above the phone screen such that the front-facing camera captures the thumbs on or near the screen. This enables the creation of semi-transparent overlays of thumb shadows and inference of fingertip hover points with deep learning, which help the user aim for targets on the phone. A study compares the effect of visual feedback on touch precision in a controlled task and qualitatively evaluates three example applications demonstrating the potential of the technique. The results show that the enabled style of feedback is effective for thumb-size targets, and that the VR experience can be enriched by using smartphones as VR controllers supporting precise touch input.
Visualizing Examples of Deep Neural Networks at Scale
Litao Yan (Harvard University, Cambridge, Massachusetts, United States)Elena L.. Glassman (Harvard University, Cambridge, Massachusetts, United States)Tianyi Zhang (Harvard University, Cambridge, Massachusetts, United States)
Many programmers want to use deep learning due to its superior accuracy in many challenging domains. Yet our formative study with ten programmers indicated that, when constructing their own deep neural networks (DNNs), they often had a difficult time choosing appropriate model structures and hyperparameter values. This paper presents ExampleNet---a novel interactive visualization system for exploring common and uncommon design choices in a large collection of open-source DNN projects. ExampleNet provides a holistic view of the distribution over model structures and hyperparameter settings in the corpus of DNNs, so users can easily filter the corpus down to projects tackling similar tasks and compare design choices made by others. We evaluated ExampleNet in a within-subjects study with sixteen participants. Compared with the control condition (i.e., online search), participants using ExampleNet were able to inspect more online examples, make more data-driven design decisions, and make fewer design mistakes.
Coping with Digital Wellbeing in a Multi-Device World
Alberto Monge Roffarello (Politecnico di Torino, Torino, Italy)Luigi De Russis (Politecnico di Torino, Torino, Italy)
While Digital Self-Control Tools (DSCTs) mainly target smartphones, more effort should be put into evaluating multi-device ecosystems to enhance digital wellbeing as users typically use multiple devices at a time. In this paper, we first review more than 300 DSCTs by demonstrating that the majority of them implements a single-device conceptualization that poorly adapts to multi-device settings. Then, we report on the results from an interview and a sketching exercise (N=20) exploring how users make sense of their multi-device digital wellbeing. Findings show that digital wellbeing issues extend beyond smartphones, with the most problematic behaviors deriving from the simultaneous usage of different devices to perform uncorrelated tasks. While this suggests the need of DSCTs that can adapt to different and multiple devices, our work also highlights the importance of learning how to properly behave with technology, e.g., through educational courses, which may be more effective than any lock-out mechanism.
Gaze-Supported 3D Object Manipulation in Virtual Reality
Difeng Yu (The University of Melbourne, Melbourne, VIC, Australia)Xueshi Lu (Xi'an Jiaotong-Liverpool University, Suzhou, Jiangsu, China)Rongkai Shi (Xi'an Jiaotong-Liverpool University, Suzhou, Jiangsu, China)Hai-Ning Liang (Xi'an Jiaotong-Liverpool University, Suzhou, Jiangsu, China)Tilman Dingler (The University of Melbourne, Melbourne, VIC, Australia)Eduardo Velloso (The University of Melbourne, Melbourne, VIC, Australia)Jorge Goncalves (The University of Melbourne, Melbourne, VIC, Australia)
This paper investigates integration, coordination, and transition strategies of gaze and hand input for 3D object manipulation in VR. Specifically, this work aims to understand whether incorporating gaze input can benefit VR object manipulation tasks, and how it should be combined with hand input for improved usability and efficiency. We designed four gaze-supported techniques that leverage different combination strategies for object manipulation and evaluated them in two user studies. Overall, we show that gaze did not offer significant performance benefits for transforming objects in the primary working space, where all objects were located in front of the user and within the arm-reach distance, but can be useful for a larger environment with distant targets. We further offer insights regarding combination strategies of gaze and hand input, and derive implications that can help guide the design of future VR systems that incorporate gaze input for 3D object manipulation.
HapticSeer: A Multi-channel, Black-box, Platform-agnostic Approach to Detecting Video Game Events for Real-time Haptic Feedback
Yu-Hsin Lin (National Taiwan University, Taipei City, Taiwan)Yu-Wei Wang (National Taiwan University, Taipei City, Taiwan)Pin-Sung Ku (National Taiwan University, Taipei City, Taiwan)Yun-Ting Cheng (National Taiwan University, Taipei City, Taiwan)Yuan-Chih Hsu (National Taiwan University, Taipei City, Taiwan)Ching-Yi Tsai (National Taiwan University, Taipei City, Taiwan)Mike Y.. Chen (National Taiwan University, Taipei City, Taiwan)
Haptic feedback significantly enhances virtual experiences. However, supporting haptics currently requires modifying the codebase, making it impractical to add haptics to popular, high-quality experiences such as best selling games, which are typically closed-source. We present HapticSeer, a multi-channel, black-box, platform-agnostic approach to detecting game events for real-time haptic feedback. The approach is based on two key insights: 1) all games have 3 types of data streams: video, audio, and controller I/O, that can be analyzed in real-time to detect game events, and 2) a small number of user interface design patterns are reused across most games, so that event detectors can be reused effectively. We developed an open-source HapticSeer framework and implemented several real-time event detectors for commercial PC and VR games. We validated system correctness and real-time performance, and discuss feedback from several haptics developers that used the HapticSeer framework to integrate research and commercial haptic devices.
No More Handshaking: How have COVID-19 pushed the expansion of computer-mediated communication in Japanese idol culture?
Hiromu Yakura (University of Tsukuba, Tsukuba, Japan)
In Japanese idol culture, meet-and-greet events where fans were allowed to handshake with an idol member for several seconds were regarded as its essential component until the spread of COVID-19. Now, idol groups are struggling in the transition of such events to computer-mediated communication because these events had emphasized meeting face-to-face over communicating, as we can infer from their length of time. I anticipated that investigating this emerging transition would provide implications because their communication has a unique characteristic that is distinct from well-studied situations, such as workplace communication and intimate relationships. Therefore, I first conducted a quantitative survey to develop a precise understanding of the transition, and based on its results, had semi-structured interviews with idol fans about their perceptions of the transition. The survey revealed distinctive approaches, including one where fans gathered at a venue but were isolated from the idol member by an acrylic plate and talked via a video call. Then the interviews not only provided answers to why such an approach would be reasonable but also suggested the existence of a large gap between conventional offline events and emerging online events in their perceptions. Based on the results, I discussed how we can develop interaction techniques to support this transition and how we can apply it to other situations outside idol culture, such as computer-mediated performing arts.
Understanding the Design Space of Embodied Passwords based on Muscle Memory
Rosa van Koningsbruggen (Bauhaus-Universität Weimar, Weimar, Germany)Bart Hengeveld (Eindhoven University of Technology, Eindhoven, Netherlands)Jason Alexander (University of Bath, Bath, United Kingdom)
Passwords have become a ubiquitous part of our everyday lives, needed for every web-service and system. However, it is challenging to create safe and diverse alphanumeric passwords, and to recall them, imposing a cognitive burden on the user. Through consecutive experiments, we explored the movement space, affordances and interaction, and memorability of a tangible, handheld, embodied password. In this context, we found that: (1) a movement space of 200 mm × 200 mm is preferred; (2) each context has a perceived level of safety, which—together with the affordances and link to familiarity—influences how the password is performed. Furthermore, the artefact’s dimensions should be balanced within the design itself, with the user, and the context, but there is a trade-off between the perceived safety and ergonomics; and (3) the designed embodied passwords can be recalled for at least a week, with participants creating unique passwords which were reproduced consistently.
Interaction Pace and User Preferences
Alix Goguey (Université Grenoble Alpes, Grenoble, France)Carl Gutwin (University of Saskatchewan, Saskatoon, Saskatchewan, Canada)Zhe Chen (University of Canterbury, Christchurch, New Zealand)Pang Suwanaposee (University of Canterbury, Christchurch, New Zealand)Andy Cockburn (University of Canterbury, Christchurch, New Zealand)
The overall pace of interaction combines the user's pace and the system's pace, and a pace mismatch could impair user preferences (e.g., animations or timeouts that are too fast or slow for the user). Motivated by studies of speech rate convergence, we conducted an experiment to examine whether user preferences for system pace are correlated with user pace. Subjects first completed a series of trials to determine their user pace. They then completed a series of hierarchical drag-and-drop trials in which folders automatically expanded when the cursor hovered for longer than a controlled timeout. Results showed that preferences for timeout values correlated with user pace -- slow-paced users preferred long timeouts, and fast-paced users preferred short timeouts. Results indicate potential benefits in moving away from fixed or customisable settings for system pace. Instead, systems could improve preferences by automatically adapting their pace to converge towards that of the user.
Falx: Synthesis-Powered Visualization Authoring
Chenglong Wang (University of Washington, Seattle, Washington, United States)Yu Feng (University of California, Santa Barbara, Santa Barbara, California, United States)Rastislav Bodik (University of Washington, Seattle, Washington, United States)Isil Dillig (University of Texas, Austin, Austin, Texas, United States)Alvin Cheung (University of California, Berkeley, Berkeley, California, United States)Amy J. Ko (University of Washington, Seattle, Washington, United States)
Modern visualization tools aim to allow data analysts to easily create exploratory visualizations. When the input data layout conforms to the visualization design, users can easily specify visualizations by mapping data columns to visual channels of the design. However, when there is a mismatch between data layout and the design, users need to spend significant effort on data transformation. We propose Falx, a synthesis-powered visualization tool that allows users to specify visualizations in a similarly simple way but without needing to worry about data layout. In Falx, users specify visualizations using examples of how concrete values in the input are mapped to visual channels, and Falx automatically infers the visualization specification and transforms the data to match the design. In a study with 33 data analysts on four visualization tasks involving data transformation, we found that users can effectively adopt Falx to create visualizations they otherwise cannot implement.
HulaMove: Using Commodity IMU for Waist Interaction
Xuhai Xu (University of Washington, Seattle, Washington, United States)Jiahao Li (University of Washington, Seattle, Washington, United States)Tianyi Yuan (Department of Industrial Engineering, Beijing, China)Liang He (University of Washington, Seattle, Washington, United States)Xin Liu (University of Washington, Seattle, Washington, United States)Yukang Yan (Tsinghua University, Beijing, China)Yuntao Wang (Tsinghua University, Beijing, China)Yuanchun Shi (Tsinghua University, Beijing, China)Jennifer Mankoff (University of Washington, Seattle, Washington, United States)Anind K. Dey (University of Washington, Seattle, Washington, United States)
We present HulaMove, a novel interaction technique that leverages the movement of the waist as a new eyes-free and hands-free input method for both the physical world and the virtual world. We first conducted a user study (N=12) to understand users’ ability to control their waist. We found that users could easily discriminate eight shifting directions and two rotating orientations, and quickly confirm actions by returning to the original position (quick return). We developed a design space with eight gestures for waist interaction based on the results and implemented an IMU-based real-time system. Using a hierarchical machine learning model, our system could recognize waist gestures at an accuracy of 97.5%. Finally, we conducted a second user study (N=12) for usability testing in both real-world scenarios and virtual reality settings. Our usability study indicated that HulaMove significantly reduced interaction time by 41.8% compared to a touch screen method, and greatly improved users’ sense of presence in the virtual world. This novel technique provides an additional input method when users’ eyes or hands are busy, accelerates users’ daily operations, and augments their immersive experience in the virtual world.
What Are You Doing With Your Phone?: How Social Class Frames Parent-Teen Tensions around Teens’ Smartphone Use
Phoebe K.. Chua (University of California, Irvine, Irvine, California, United States)Melissa Mazmanian (University of California, Irvine, Irvine, California, United States)
Social class contexts shape parents’ guiding principles around teens’ smartphone use. These contexts can affect how parents coach and censor their teens’ smartphone use and can create tensions in the home. Through 174 interviews (87 parent-teen dyads), we find that upper-middle-class families generally adopt an orientation toward scaffolded achievements and working-class families tend to embrace an orientation toward empowered self-sufficiency. We further find that these class-based orientations contribute to parent-teen tensions. For upper-middle-class families, tensions arise when parents insist that teens should use smartphones to get help with academic and enrichment activities and teens disagree about whether their phone-related activities align with this goal. In contrast, we find that conflict can occur in working-class families when teens use their smartphones to get assistance and parents interpret such activity as teens being lazy or not self-sufficient. These findings highlight the role of social class contexts in shaping families’ orientations toward teens’ smartphone use and phone-related tensions.
A Critical Assessment of the Use of SSQ as a Measure of General Discomfort in VR Head-Mounted Displays
Teresa Hirzle (Ulm University, Ulm, Germany)Maurice Cordts (Ulm University, Ulm, Germany)Enrico Rukzio (University of Ulm, Ulm, Germany)Jan Gugenheimer (Institut Polytechnique de Paris, Paris, France)Andreas Bulling (University of Stuttgart, Stuttgart, Germany)
Based on a systematic literature review of more than 300 papers published over the last 10 years, we provide indicators that the simulator sickness questionnaire (SSQ) is extensively used and widely accepted as a general discomfort measure in virtual reality (VR) research – although it actually only accounts for one category of symptoms. This results in important other categories (digital eye strain (DES) and ergonomics) being largely neglected. To contribute to a more comprehensive picture of discomfort in VR head-mounted displays, we further conducted an online study (N=352) on the severity and relevance of all three symptom categories. Most importantly, our results reveal that symptoms of simulator sickness are significantly less severe and of lower prevalence than those of DES and ergonomics. In light of these findings, we critically discuss the current use of SSQ as the only discomfort measure and propose a more comprehensive factor model that also includes DES and ergonomics.
TapNet: The Design, Training, Implementation, and Applications of a Multi-Task Learning CNN for Off-Screen Mobile Input
Michael Xuelin Huang (Google, Mountain View, California, United States)Yang Li (Google Research, Mountain View, California, United States)Nazneen Nazneen (Google, Mountain View, California, United States)Alexander Chao (Google, Mountain View, California, United States)Shumin Zhai (Google, Mountain View, California, United States)
To make off-screen interaction without specialized hardware practical, we investigate using deep learning methods to process the common built-in IMU sensor (accelerometers and gyroscopes) on mobile phones into a useful set of one-handed interaction events. We present the design, training, implementation and applications of TapNet, a multi-task network that detects tapping on the smartphone. With phone form factor as auxiliary information, TapNet can jointly learn from data across devices and simultaneously recognize multiple tap properties, including tap direction and tap location. We developed two datasets consisting of over 135K training samples, 38K testing samples, and 32 participants in total. Experimental evaluation demonstrated the effectiveness of the TapNet design and its significant improvement over the state of the art. Along with the datasets, codebase, and extensive experiments, TapNet establishes a new technical foundation for off-screen mobile input.
Visuo-haptic Illusions for Linear Translation and Stretching using Physical Proxies in Virtual Reality
Martin Feick (Saarland Informatics Campus, Saarbrücken, Germany)Niko Kleer (Saarland Informatics Campus, Saarbrücken, Germany)André Zenner (Saarland University, Saarland Informatics Campus, Saarbrücken, Germany)Anthony Tang (University of Toronto, Toronto, Ontario, Canada)Antonio Krüger (DFKI, Saarland Informatics Campus, Saarbrücken, Germany)
Providing haptic feedback when manipulating virtual objects is an essential part of immersive virtual reality experiences; however, it is challenging to replicate all of an object’s properties and characteristics. We propose the use of visuo-haptic illusions alongside physical proxies to enhance the scope of proxy-based interactions with virtual objects. In this work, we focus on two manipulation techniques, linear translation and stretching across different distances, and investigate how much discrepancy between the physical proxy and the virtual object may be introduced without participants noticing. In a study with 24 participants, we found that manipulation technique and travel distance significantly affect the detection thresholds, and that visuo-haptic illusions impact performance and accuracy. We show that this technique can be used to enable functional proxy objects that act as stand-ins for multiple virtual objects, illustrating the technique through a showcase VR-DJ application.
Project Tasca : Enabling Touch and Contextual Interactions with a Pocket-based Textile Sensor
Te-Yen Wu (Microsoft Research, Redmond, Washington, United States)Zheer Xu (Microsoft Research, Redmond, Washington, United States)Xing-Dong Yang (Dartmouth College, Hanover, New Hampshire, United States)Steve Hodges (Microsoft Research, Cambridge, United Kingdom)Teddy Seyed (Microsoft Research, Redmond, Washington, United States)
We present Project Tasca, a pocket-based textile sensor that detects user input and recognizes everyday objects that a user carries in the pockets of a pair of pants (e.g., keys, coins, electronic devices, or plastic items). By creating a new fabric-based sensor capable of detecting in-pocket touch and pressure, and recognizing metallic, non-metallic, and tagged objects inside the pocket, we enable a rich variety of subtle, eyes-free, and always-available input, as well as context-driven interactions in wearable scenarios. We developed our prototype by integrating four distinct types of sensing methods, namely: inductive sensing, capacitive sensing, resistive sensing, and NFC in a multi-layer fabric structure into the form factor of a jeans pocket. Through a ten-participant study, we evaluated the performance of our prototype across 11 common objects including hands, 8 force gestures, and 30 NFC tag placements. We yielded 92.3% personal cross-validation accuracy for object recognition, 96.4% accuracy for gesture recognition, and 100% accuracy for detecting NFC tags at close distance . We concluded by demonstrating the interactions enabled by our pocket-based sensor in several applications.
Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture
Rebecca Krosnick (University of Michigan, Ann Arbor, Michigan, United States)Fraser Anderson (Autodesk Research, Toronto, Ontario, Canada)Justin Matejka (Autodesk Research, Toronto, Ontario, Canada)Steve Oney (University of Michigan, Ann Arbor, Michigan, United States)Walter S.. Lasecki (University of Michigan, Ann Arbor, Michigan, United States)Tovi Grossman (University of Toronto, Toronto, Ontario, Canada)George Fitzmaurice (Autodesk Research, Toronto, Ontario, Canada)
When users complete tasks on the computer, the knowledge they leverage and their intent is often lost because it is tedious or challenging to capture. This makes it harder to understand why a colleague designed a component a certain way or to remember requirements for software you wrote a year ago. We introduce think-aloud computing, a novel application of the think-aloud protocol where computer users are encouraged to speak while working to capture rich knowledge with relatively low effort. Through a formative study we find people shared information about design intent, work processes, problems encountered, to-do items, and other useful information. We developed a prototype that supports think-aloud computing by prompting users to speak and contextualizing speech with labels and application context. Our evaluation shows more subtle design decisions and process explanations were captured in think-aloud than via traditional documentation. Participants reported that think-aloud required similar effort as traditional documentation.
"It's About Missing Much More Than the People": How Students use Digital Technologies to Alleviate Homesickness
Ryan M.. Kelly (The University of Melbourne, Melbourne, VIC, Australia)Yueyang Cheng (University of Melbourne, Melbourne, VIC, Australia)Dana McKay (University of Melbourne, Parkville, VIC, Australia)Greg Wadley (The University of Melbourne, Melbourne, Vic, Australia)George Buchanan (University of Melbourne, Melbourne, Victoria, Australia)
Homesickness, which refers to feelings of distress caused by separation from home, is prevalent among university-aged students. Chronic homesickness can exacerbate mood problems, erode academic performance and lead to dropout from school. The present research examines how students use digital technologies to resolve the experience of missing home. Qualitative interviews and diaries with 50 students at major Australian universities revealed that technologies play a significant role in alleviating homesickness. Specifically, students use technologies to acquire social contact, find help and support, build co-presence to recreate their home, connect with culture, experience distant places, and regulate emotions. However, the use of technology sometimes led to increased emotional labour, frequent exposure to homesickness triggers, and heightened perceptions of distance. We conclude by discussing possible design implications for new technologies that allow students to alleviate homesickness by experiencing their home from afar.
Squish This: Force Input on Soft Surfaces for Visual Targeting Tasks
Bruno Fruchard (Saarland Informatics Campus, Saarbrücken, Germany)Paul Strohmeier (Saarland University, Saarbrücken, Germany)Roland Bennewitz (INM - Leibniz Institute for New Materials, Saarbrucken, Germany)Jürgen Steimle (Saarland University, Saarland Informatics Campus, Saarbrücken, Germany)
Today's typical input device is flat, rigid and made of glass. However, advances in sensing technology and interaction design suggest thinking about input on other surface, including soft materials. While touching rigid and soft materials might feel similar, they clearly feel different when pressure is applied to them. Yet, to date, studies only investigated force input on rigid surfaces. We present a first systematic evaluation of the effects of compliance on force input. Results of a visual targeting task for three levels of softness indicate that high force levels appear more demanding for soft surfaces, but that performance is otherwise similar. Performance remained very high (~ for 20 force levels) regardless of the compliance, suggesting force input was underestimated so far. We infer implications for the design of force input on soft surfaces and conclude that interaction models used on rigid surfaces might be used on soft surfaces.
Gesture Knitter: A Hand Gesture Design Tool for Head-Mounted Mixed Reality Applications
George B. Mo (University of Cambridge, Cambridge, United Kingdom)John J. Dudley (University of Cambridge, Cambridge, United Kingdom)Per Ola Kristensson (University of Cambridge, Cambridge, United Kingdom)
Hand gestures are a natural and expressive input method enabled by modern mixed reality headsets. However, it remains challenging for developers to create custom gestures for their applications. Conventional strategies to bespoke gesture recognition involve either hand-crafting or data-intensive deep-learning. Neither approach is well suited for rapid prototyping of new interactions. This paper introduces a flexible and efficient alternative approach for constructing hand gestures. We present Gesture Knitter: a design tool for creating custom gesture recognizers with minimal training data. Gesture Knitter allows the specification of gesture primitives that can then be combined to create more complex gestures using a visual declarative script. Designers can build custom recognizers by declaring them from scratch or by providing a demonstration that is automatically decoded into its primitive components. Our developer study shows that Gesture Knitter achieves high recognition accuracy despite minimal training data and delivers an expressive and creative design experience.
Reward Seeking or Loss Aversion?: Impact of Regulatory Focus Theory on Emotional Induction in Children and Their Behavior Towards a Social Robot
Maha Elgarf (KTH Royal Institute of Technology, Stockholm, Sweden)Natalia Calvo-Barajas (Uppsala Univesity, Uppsala, Sweden)Ana Paiva (University of Lisbon, Lisbon, Portugal)Ginevra Castellano (Uppsala University, Uppsala, Sweden)Christopher Peters (KTH Royal Institute of Technology, Stockholm, Sweden)
According to psychology research, emotional induction has positive implications in many domains such as therapy and education. Our aim in this paper was to manipulate the Regulatory Focus Theory to assess its impact on the induction of regulatory focus related emotions in children in a pretend play scenario with a social robot. The Regulatory Focus Theory suggests that people follow one of two paradigms while attempting to achieve a goal; by seeking gains (promotion focus - associated with feelings of happiness) or by avoiding losses (prevention focus - associated with feelings of fear). We conducted a study with 69 school children in two different conditions (promotion vs. prevention). We succeeded in inducing happiness emotions in the promotion condition and found a resulting positive effect of the induction on children's social engagement with the robot. We also discuss the important implications of these results in both educational and child robot interaction fields.
Remote and Collaborative Virtual Reality Experiments via Social VR Platforms
David Saffo (Northeastern University, Boston, Massachusetts, United States)Sara Di Bartolomeo (Northeastern University, Boston, Massachusetts, United States)Caglar Yildirim (Northeastern University, Boston, Massachusetts, United States)Cody Dunne (Northeastern University, Boston, Massachusetts, United States)
Virtual reality (VR) researchers struggle to conduct remote studies. Previous work has focused on working around limitations imposed by traditional crowdsourcing methods. However, the potential for leveraging social VR platforms for HCI evaluations is largely unexplored. These platforms have large VR-ready user populations, distributed synchronous virtual environments, and support for user-generated content. We demonstrate how social VR platforms can be used to practically and ethically produce valid research results by replicating two studies using one such platform (VRChat): a quantitative study on Fitts’ law and a qualitative study on tabletop collaboration. Our replication studies exhibited analogous results to the originals, indicating the research validity of this approach. Moreover, we easily recruited experienced VR users with their own hardware for synchronous, remote, and collaborative participation. We further provide lessons learned for future researchers experimenting using social VR platforms. This paper and all supplemental materials are available at osf.io/c2amz.
PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition
Yasha Iravantchi (University of Michigan, Ann Arbor, Michigan, United States)Karan Ahuja (Carnegie Mellon University, Pittsburgh, Pennsylvania, United States)Mayank Goel (Carnegie Mellon University, Pittsburgh, Pennsylvania, United States)Chris Harrison (Carnegie Mellon University, Pittsburgh, Pennsylvania, United States)Alanson Sample (The University of Michigan, Ann Arbor, Michigan, United States)
Sound presents an invaluable signal source that enables computing systems to perform daily activity recognition. However, microphones are optimized for human speech and hearing ranges: capturing private content, such as speech, while omitting useful, inaudible information that can aid in acoustic recognition tasks. We simulated acoustic recognition tasks using sounds from 127 everyday household/workplace objects, finding that inaudible frequencies can act as a substitute for privacy-sensitive frequencies. To take advantage of these inaudible frequencies, we designed a Raspberry Pi-based device that captures inaudible acoustic frequencies with settings that can remove speech or all audible frequencies entirely. We conducted a perception study, where participants "eavesdropped" on PrivacyMic’s filtered audio and found that none of our participants could transcribe speech. Finally, PrivacyMic’s real-world activity recognition performance is comparable to our simulated results, with over 95% classification accuracy across all environments, suggesting immediate viability in performing privacy-preserving daily activity recognition.
Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making
Gabriel Lima (KAIST, Daejeon, Korea, Republic of)Nina Grgić-Hlača (Max Planck Institute for Software Systems, Saarbrücken, Germany)Meeyoung Cha (Institute for Basic Science (IBS), Daejeon, Korea, Republic of)
How to attribute responsibility for autonomous artificial intelligence (AI) systems' actions has been widely debated across the humanities and social science disciplines. This work presents two experiments (N=200 each) that measure people's perceptions of eight different notions of moral responsibility concerning AI and human agents in the context of bail decision-making. Using real-life adapted vignettes, our experiments show that AI agents are held causally responsible and blamed similarly to human agents for an identical task. However, there was a meaningful difference in how people perceived these agents' moral responsibility; human agents were ascribed to a higher degree of present-looking and forward-looking notions of responsibility than AI agents. We also found that people expect both AI and human decision-makers and advisors to justify their decisions regardless of their nature. We discuss policy and HCI implications of these findings, such as the need for explainable AI in high-stakes scenarios.
"Warm Bodies'': A Post-Processing Technique for Animating Dynamic Blood Flow on Photos and Avatars
Daniel McDuff (Microsoft, Seattle, Washington, United States)Ewa Nowara (Rice University, Houston, Texas, United States)
What breathes life into an embodied agent or avatar? While body motions such as facial expressions, speech and gestures have been well studied, relatively little attention has been applied to subtle changes due to underlying physiology. We argue that subtle pulse signals are important for creating more lifelike and less disconcerting avatars. We propose a method for animating blood flow patterns, based on a data-driven physiological model that can be used to directly augment the appearance of synthetic avatars and photo-realistic faces. While the changes are difficult for participants to "see", they significantly more frequently select faces with blood flow as more anthropomorphic and animated than faces without blood flow. Furthermore, by manipulating the frequency of the heart rate in the underlying signal we can change the perceived arousal of the character.