注目の論文一覧

We explore a temperature illusion that uses low-powered electronics and enables the miniaturization of simple warm and cool sensations. Our illusion relies on the properties of certain scents, such as the coolness of mint or hotness of peppers. These odors trigger not only the olfactory bulb, but also the nose's trigeminal nerve, which has receptors that respond to both temperature and chemicals. To exploit this, we engineered a wearable device based on micropumps and an atomizer that emits up to three custom-made "thermal" scents directly to the user's nose. Breathing in these scents causes the user to feel warmer or cooler. We demonstrate how our device renders warmth and cooling sensations in virtual experiences. In our first study, we evaluated six candidate "thermal" scents. We found two hot-cold pairs, with one pair being less identifiable by odor. In our second study, pParticipants rated VR experiences with our device trigeminal stimulants as significantly warmer or cooler than the baseline conditions. Lastly, we believe this offers an alternative to existing thermal feedback devices, which unfortunately rely on power-hungry heat-lamps or Peltier-elements.

Design thinking is an iterative, human-centered approach to innovation. Its success rests on collaboration within a multidisciplinary project team going through cycles of divergent and convergent ideations. In these teams, nondesigners risk diminishing the divergent reach because they are generally reluctant to sketch, thus missing out on theambiguous, imprecise early conceptual divergent phases. We hypothesized that LEGO^(®) could advantageously be a substitute to sketching. In this comparative study, 44 nondesigners randomly paired in 22 dyads did two conceptual ideations of healthcare landing pages, one using pen/paper (spontaneously writing words on sticky notes) and the other using LEGO, assessed through Torrance and Guilford frameworks for divergent thinking. Results show that LEGO interfaces gathered significantly higher divergent thinking scores because their concepts were significantly more elaborated. Furthermore, when using LEGO, teams who generated more elements were likely to also generate more ideas, more categories of ideas and more original ideas.

3D printing technology can be used to rapidly prototype the look and feel of 3D objects. However, the objects produced are passive. There has been increasing interest in making these objects interactive, yet they often require assembling components or complex calibration. In this paper, we contribute AirTouch, a technique that enables designers to fabricate touch-sensitive objects with minimal assembly and calibration using pneumatic sensing. AirTouch-enabled objects are 3D printed as a single structure using a consumer-level 3D printer. AirTouch uses pre-trained machine learning models to identify interactions with fabricated objects, meaning that there is no calibration required once the object has completed printing. We evaluate our technique using fabricated objects with various geometries and touch sensitive locations, obtaining accuracies of at least 90% with 12 interactive locations.

Customised avatars are a powerful tool to increase identification, engagement and intrinsic motivation in digital games. We investigated the effects of customisation in a self-competitive VR exergame by modelling players and their previous performance in the game with customised avatars. In a first study we found that, similar to non-exertion games, customisation significantly increased identification and intrinsic motivation, as well as physical performance in the exergame. In a second study we identified a more complex relationship with the customisation style: idealised avatars increased wishful identification but decreased exergame performance compared to realistic avatars. In a third study, we found that 'enhancing' realistic avatars with idealised characteristics increased wishful identification, but did not have any adverse effects. We discuss the findings based on feedforward and self-determination theory, proposing notions of intrinsic identification (fostering a sense of self) and extrinsic identification (drawing away from the self) to explain the results.

While virtual objects are likely to be a part of future interfaces, we lack knowledge of how the dynamic position of virtual objects influences users' posture. In this study, we investigated users' posture change following the unobtrusive and swift motions of a content window in virtual reality (VR). In two perception studies, we estimated the perception threshold on undetectable slow motions and displacement during an eye blink. In a formative study, we compared users' performance, posture change as well as subjective responses on unobtrusive, swift, and no motions. Based on the result, we designed concept applications and explored potential design space of moving virtual content for unobtrusive posture change. With our study, we discuss the interfaces that control users and the initial design guidelines of unobtrusive posture manipulation.

Contemporary voice assistants require that objects of inter-est be specified in spoken commands. Of course, users are often looking directly at the object or place of interest – fine-grained, contextual information that is currently unused. We present WorldGaze, a software-only method for smartphones that provides the real-world gaze location of a user that voice agents can utilize for rapid, natural, and precise interactions. We achieve this by simultaneously opening the front and rear cameras of a smartphone. The front-facing camera is used to track the head in 3D, including estimating its direction vector. As the geometry of the front and back cameras are fixed and known, we can raycast the head vector into the 3D world scene as captured by the rear-facing camera. This allows the user to intuitively define an object or region of interest using their head gaze. We started our investigations with a qualitative exploration of competing methods, before developing a functional, real-time implementation. We conclude with an evaluation that shows WorldGaze can be quick and accurate, opening new multimodal gaze+voice interactions for mobile voice agents.

Social signals are crucial when we decide if we want to interact with someone online. However, social signals are typically limited to the few that platform designers provide, and most can be easily manipulated. In this paper, we propose a new idea called synthesized social signals (S3s): social signals computationally derived from an account's history, and then rendered into the profile. Unlike conventional social signals such as profile bios, S3s use computational summarization to reduce receiver costs and raise the cost of faking signals. To demonstrate and explore the concept, we built Sig, an extensible Chrome extension that computes and visualizes S3s. After a formative study, we conducted a field deployment of Sig on Twitter, targeting two well-known problems on social media: toxic accounts and misinformation. Results show that Sig reduced receiver costs, added important signals beyond conventionally available ones, and that a few users felt safer using Twitter as a result. We conclude by reflecting on the opportunities and challenges S3s provide for augmenting interaction on social platforms.

This paper introduces FaceHaptics, a novel haptic display based on a robot arm attached to a head-mounted virtual reality display. It provides localized, multi-directional and movable haptic cues in the form of wind, warmth, moving and single-point touch events and water spray to dedicated parts of the face not covered by the head-mounted display.The easily extensible system, however, can principally mount any type of compact haptic actuator or object. User study 1 showed that users appreciate the directional resolution of cues, and can judge wind direction well, especially when they move their head and wind direction is adjusted dynamically to compensate for head rotations. Study 2 showed that adding FaceHaptics cues to a VR walkthrough can significantly improve user experience, presence, and emotional responses.

Past research regarding on-body interaction typically requires custom sensors, limiting their scalability and generalizability. We propose EarBuddy, a real-time system that leverages the microphone in commercial wireless earbuds to detect tapping and sliding gestures near the face and ears. We develop a design space to generate 27 valid gestures and conducted a user study (N=16) to select the eight gestures that were optimal for both human preference and microphone detectability. We collected a dataset on those eight gestures (N=20) and trained deep learning models for gesture detection and classification. Our optimized classifier achieved an accuracy of 95.3%. Finally, we conducted a user study (N=12) to evaluate EarBuddy's usability. Our results show that EarBuddy can facilitate novel interaction and that users feel very positively about the system. EarBuddy provides a new eyes-free, socially acceptable input method that is compatible with commercial wireless earbuds and has the potential for scalability and generalizability

Collecting accurate and precise emotion ground truth labels for mobile video watching is essential for ensuring meaningful predictions. However, video-based emotion annotation techniques either rely on post-stimulus discrete self-reports, or allow real-time, continuous emotion annotations (RCEA) only for desktop settings. Following a user-centric approach, we designed an RCEA technique for mobile video watching, and validated its usability and reliability in a controlled, indoor (N=12) and later outdoor (N=20) study. Drawing on physiological measures, interaction logs, and subjective workload reports, we show that (1) RCEA is perceived to be usable for annotating emotions while mobile video watching, without increasing users' mental workload (2) the resulting time-variant annotations are comparable with intended emotion attributes of the video stimuli (classification error for valence: 8.3%; arousal: 25%). We contribute a validated annotation technique and associated annotation fusion method, that is suitable for collecting fine-grained emotion annotations while users watch mobile videos.

Kirigami Haptic Swatches demonstrate how kirigami and origami based structures enable sophisticated haptic feedback through simple cut-and-fold fabrication techniques. We leverage four types of geometric patterns: rotational erection system (RES), split-fold waterbomb (SFWB), the overlaid structure of SFWB and RES (SFWB+RES), and cylindrical origami, to render different sets of haptic feedback (i.e. linear, bistable, bouncing snap-through, and rotational force behaviors, respectively). In each structure, not only the form factor but also the force feedback properties can be tuned through geometric parameters. We experimentally analyzed and modeled the structures, and implemented software to automatically generate 2D patterns for desired haptic properties. We also demonstrate five example applications including an assistive custom keyboard, rotational switch, multi-sensory toy, task checklist, and phone accessories. We believe the Kirigami Haptic Swatches helps tinkerers, designers, and even researchers to create interactions that enrich our haptic experience.

Helping others can have a positive effect on both the giver and the receiver. However, supporting someone with depression can be complicated and overwhelming. To address this, we proposed a Facebook-based social bot displaying depressive symptoms and disclosing vulnerable experiences that allows users to practice providing reactions online. We investigated how 55 college students interacted with the social bot for three weeks and how these support-giving experiences affected their mental health and stigma. By responding to the bot, the participants reframed their own negative experiences, reported reduced feelings of danger regarding an individual with depression and increased willingness to help the person, and presented favorable attitudes toward seeking treatment for depression. We discuss design opportunities for accessible social bots that could help users to keep practicing peer support interventions without fear of negative consequences.

We present Jubilee, an open-source hardware machine with automatic tool-changing and interchangeable bed plates. As digital fabrication tools have become more broadly accessible, tailoring those machines to new users and novel workflows has become central to HCI research. However, the lack of hardware infrastructure makes custom application development cumbersome. We identify a need for an extensible platform to allow HCI researchers to develop workflows for fabrication, material exploration, and other applications. Jubilee addresses this need. It can automatically and repeatably change tools in the same operation. It can be built with a combination of simple 3D-printed and readily available parts. It has several standard head designs for a variety of applications including 3D printing, syringe-based liquid handling, imaging, and plotting. We present Jubilee with a comprehensive set of assembly instructions and kinematic mount templates for user-designed tools and bed plates. Finally we demonstrate Jubilee's multi-tool workflow functionality with a series of example applications.

E-textile microinteractions advance cord-based interfaces by enabling the simultaneous use of precise continuous control and casual discrete gestures. We leverage the recently introduced I/O Braid sensing architecture to enable a series of user studies and experiments which help design suitable interactions and a real-time gesture recognition pipeline. Informed by a gesture elicitation study with 36 participants, we developed a user-dependent classifier for eight discrete gestures with 94% accuracy for 12 participants. In a formal evaluation we show that we can enable precise manipulation with the same architecture. Our quantitative targeting experiment suggests that twisting is faster than existing headphone button controls and is comparable in speed to a capacitive touch surface. Qualitative interview feedback indicates a preference for I/O Braid's interaction over that of in-line headphone controls. Our applications demonstrate how continuous and discrete gestures can be combined to form new, integrated e-textile microinteraction techniques for real-time continuous control, discrete actions and mode switching.

Recent advances in Virtual Reality (VR) technology, such as larger fields of view, have made VR increasingly immersive. However, a larger field of view often results in a user focusing on certain directions and missing relevant content presented elsewhere on the screen. With HiveFive, we propose a technique that uses swarm motion to guide user attention in VR. The goal is to seamlessly integrate directional cues into the scene without losing immersiveness. We evaluate HiveFive in two studies. First, we compare biological motion (from a prerecorded swarm) with non-biological motion (from an algorithm), finding further evidence that humans can distinguish between these motion types and that, contrary to our hypothesis, non-biological swarm motion results in significantly faster response times. Second, we compare HiveFive to four other techniques and show that it not only results in fast response times but also has the smallest negative effect on immersion.

Short online videos have become the dominating media on social platforms. However, finding suitable music to accompany videos can be a challenging task to some video creators, due to copyright constraints, limitations in search engines, and required audio-editing expertise. One possible solution to these problems is to use AI music generation. In this paper we present a user interface (UI) paradigm that allows users to input a song to an AI engine and then interactively regenerate and mix AI-generated music. To arrive at this design, we conducted user studies with a total of 104 video creators at several stages of our design and development process. User studies supported the effectiveness of our approach and provided valuable insights about human-AI interaction as well as the design and evaluation of mixed-initiative interfaces in creative practice.

We present Miniature Haptics, a new approach to providing realistic haptic experiences by applying miniaturized haptic feedback to hand-based, embodied avatars. By shrinking haptics to a much smaller scale, Miniature Haptics enables the exploration of new haptic experiences that are not practical to create at the full, human-body scale. Using Finger Walking in Place (FWIP) as an example avatar embodiment and control method, we first explored the feasibility of Miniature Haptics then conducted a human factors study to understand how people map their full-body skeletal model to their hands. To understand the user experience of Miniature Haptic, we developed a miniature football haptic display, and results from our user study show that Miniature Haptics significantly improved the realism and enjoyment of the experience and is preferred by users (p < 0.05). In addition, we present two miniature motion platforms supporting the haptic experiences of: 1) rapidly changing ground height for platform jumping games such as Super Mario Bros and 2) changing terrain slope. Overall, Miniature Haptics makes it possible to explore novel haptic experiences that have not been practical before.

Co-creative systems have been widely explored in the field of computational creativity. However, existing AI partners of these systems are mostly virtual agents. As sketching on paper with embodied robots could be more engaging for designers' early-stage ideation and collaborative practices, we envision the possibility of Cobbie, a mobile robot that ideates iteratively with designers by generating creative and diverse sketches. To evaluate the differences in co-creativity and user experience between the co-creative robots and virtual agents, we conducted a comparative experiment and analyzed the data collected from quantitative scales, observation, and semi-structured interview. The results reveal that Cobbie is more satisfying in motivating exploration, provoking unexpected ideas and engaging designers in the collaborative ideation process. Based on these findings, we discussed the prospects of co-creative robots for future developments of human-AI collaborative systems.

Many people, especially those in sedentary occupations, fail to achieve the recommended levels of physical activity. Virtual reality (VR) games have the potential to overcome this because they are fun and also can be physically demanding. This paper explores whether a VR game studio can help workers in sedentary jobs to get valuable levels of exercise. We studied how 11 participants used our VR game studio in a sedentary workplace over 8-weeks and their perceptions of the experience. We analysed the physical exertion in the VR game studio, comparing this to their step counts from a smartwatch. All participants achieved valuable levels of physical activity and mood benefits. Importantly, for 6 participants, only with the VR game studio did they meet recommended activity levels. Our key contributions are insights about the use of a workplace VR game studio and its health benefits.

With the increasing ubiquity of personal devices, social acceptability of human-machine interactions has gained relevance and growing interest from the HCI community. Yet, there are no best practices or established methods for evaluating social acceptability. Design strategies for increasing social acceptability have been described and employed, but so far not been holistically appraised and evaluated. We offer a systematic literature analysis (N=69) of social acceptability in HCI and contribute a better understanding of current research practices, namely, methods employed, measures and design strategies. Our review identified an unbalanced distribution of study approaches, shortcomings in employed measures, and a lack of interweaving between empirical and artifact-creating approaches. The latter causes a discrepancy between design recommendations based on user research, and design strategies employed in artifact creation. Our survey lays the groundwork for a more nuanced evaluation of social acceptability, the development of best practices, and a future research agenda.

Participating in competitive races can be a thrilling experience for athletes, involving a rush of excitement and sensations of flow, achievement, and self-fulfilment. However, for non-athletes, the prospect of competition is often a scary one which affects intrinsic motivation negatively, especially for less fit, less competitive individuals. We propose a novel method making the positive racing experience accessible to non-athletes using a high-intensity cycling VR exergame: by recording and replaying all their previous gameplay sessions simultaneously, including a projected future performance, players can race against a crowd of "ghost" avatars representing their individual fitness journey. The experience stays relevant and exciting as every race adds a new competitor. A longitudinal study over four weeks and a cross-sectional study found that the new method improves physical performance, intrinsic motivation, and flow compared to a non-competitive exergame. Additionally, the longitudinal study provides insights into the longer-term effects of VR exergames.

Subtitles play a crucial role in cross-lingual distribution of multimedia content and help communicate information where auditory content is not feasible (loud environments, hearing impairments, unknown languages). Established methods utilize text at the bottom of the screen, which may distract from the video. Alternative techniques place captions closer to related content (e.g., faces) but are not applicable to arbitrary videos such as documentations. Hence, we propose to leverage live gaze as indirect input method to adapt captions to individual viewing behavior. We implemented two gaze-adaptive methods and compared them in a user study (n=54) to traditional captions and audio-only videos. The results show that viewers with less experience with captions prefer our gaze-adaptive methods as they assist them in reading. Furthermore, gaze distributions resulting from our methods are closer to natural viewing behavior compared to the traditional approach. Based on these results, we provide design implications for gaze-adaptive captions.

Personalization of eating such that everyone consumes only what they need allows improving our management of food waste. In this paper, we explore the use of food 3D printing to create perceptual illusions for controlling the level of perceived satiety given a defined amount of calories. We present FoodFab, a system that allows users to control their food intake through modifying a food's internal structure via two 3D printing parameters: infill pattern and infill density. In two experiments with a total of 30 participants, we studied the effect of these parameters on users' chewing time that is known to affect people's feeling of satiety. Our results show that we can indeed modify the chewing time by varying infill pattern and density, and thus control perceived satiety. Based on the results, we propose two computational models and integrate them into a user interface that simplifies the creation of personalized food structures.

One of the great benefits of virtual reality (VR) is the implementation of features that go beyond realism. Common "unrealistic" locomotion techniques (like teleportation) can avoid spatial limitation of tracking, but minimize potential benefits of more realistic techniques (e.g. walking). As an alternative that combines realistic physical movement with hyper-realistic virtual outcome, we present JumpVR, a jump-based locomotion augmentation technique that virtually scales users' physical jumps. In a user study (N=28), we show that jumping in VR (regardless of scaling) can significantly increase presence, motivation and immersion compared to teleportation, while largely not increasing simulator sickness. Further, participants reported higher immersion and motivation for most scaled jumping variants than forward-jumping. Our work shows the feasibility and benefits of jumping in VR and explores suitable parameters for its hyper-realistic scaling. We discuss design implications for VR experiences and research.

On-face wearables are currently limited to piercings, tattoos, or interactive makeup that aesthetically enhances the user, and have been minimally used for scent-delivery methods. However, on-face scent interfaces could provide an advantage for personal scent delivery in comparison with other modalities or body locations since they are closer to the nose. In this paper, we present the mechanical and industrial design details of a series of form factors for on-face olfactory wearables that are lightweight and can be adhered to the skin or attached to glasses or piercings. We assessed the usability of three prototypes by testing with 12 participants in a within-subject study design while they were interacting in pairs at a close personal distance. We compare two of these designs with an "off-face" olfactory necklace and evaluate their social acceptance, comfort as well as perceived odor intensity for both the wearer and observer.

Mid-air haptic (MAH) feedback, providing touch feedback through ultrasound, has been considered an attractive substitute for the absence of physical touch during gesture-based interaction. Although the impact of MAH feedback on workload has already received some attention, the impact on other qualities of the user experience, including general attractiveness and experienced pleasure have been less investigated. In this preregistered study, involving 32 participants, we observed an added value of MAH feedback, on top of visual feedback, by increasing the attractiveness and experienced pleasure during gesture-based interaction, but not by decreasing workload. The added value regarding pleasure and attractiveness disappeared however after statistically controlling for perceived novelty. This paper highlights the importance of statistically controlling for novelty when testing the user experience of new technology during first-time use.

Integration of soft haptic devices into garments can improve their usability and wearability for daily computing interactions. In this paper, we introduce PneuSleeve, a fabric-based, compact, and highly expressive forearm sleeve which can render a broad range of haptic stimuli including compression, skin stretch, and vibration. The haptic stimuli are generated by controlling pneumatic pressure inside embroidered stretchable tubes. The actuation configuration includes two compression actuators on the proximal and distal forearm, and four uniformly distributed linear actuators around and tangent to the forearm. Further, to ensure a suitable grip force, two soft mutual capacitance sensors are fabricated and integrated into the compression actuators, and a closed-loop force controller is implemented. We physically characterize the static and dynamic behavior of the actuators, as well as the performance of closed-loop control. We quantitatively evaluate the psychophysical characteristics of the six actuators in a set of user studies. Finally, we show the expressiveness of PneuSleeve by evaluating combined haptic stimuli using subjective assessments.

Cognitive tasks are increasingly being gamified in an attempt to leverage the motivational power of games; however, they are sensitive to manipulation and literature is divided on how adding game elements affects participant performance and experience. We applied two popular gamification approaches (points/feedback and theme/narrative) to a typical cognitive task (the dot probe) and measured performance and experience in two studies (N1=287, N2=321). Similar to prior work, we confirm in Study1 that points increase reaction time and error rate, and positive affect. We replicated these results in Study2, and expanded our analysis to investigate participant experience. Our findings suggest that theme creates expectations of an interesting game, which gamified tasks fail to deliver, whereas points maintain enjoyment better throughout the task itself. Important for the development of gamified cognitive tasks, our findings suggest that novel approaches to gameful assessment may be better than the status quo.

Haptic technology is maturing, with expectations and evidence that it will contribute to user experience (UX). However, we have very little understanding about how haptic technology can influence people's experience. Researchers and designers need a way to understand, communicate, and evaluate haptic technology's effect on UX. From a literature review and two studies – one with haptics novices, the other with expert hapticians – we developed a theoretical model of the factors that constitute a good haptic experience (HX). We define HX and propose its constituent factors: design parameters of Timeliness, Density, Intensity, and Timbre; the cross-cutting concern of Personalization; usability requirements of Utility, Causality, Consistency, and Saliency; and experiential factors of Harmony, Expressivity, Autotelics, Immersion, and Realism as guiding constructs important for haptic experience. This model will help guide design and research of haptic systems, inform language around haptics, and provide the basis for evaluative instruments, such as checklists, heuristics, or questionnaires.

Sketchnoting is the process of creating a visual record with combined text and imagery of an event or presentation. Although analogue tools are still the most common method for sketchnoting, the use of digital tools is increasing. We conducted a study to better understand the current practices, techniques, compromises and opportunities of creating both pen&paper and digital sketchnotes. Our research combines insights from semi-structured interviews with the findings from a within-subjects observational study where ten participants created real time sketchnotes of two video presentations on both paper and digital tablet. We report our key findings, categorised into six themes: insights into sense of space; trade-offs with flexibility; choice paradox and cognitive load; matters of perception, accuracy and texture; issues around confidence; and practicalities. We discuss those findings, the potential and limitations of different methods, and implications for the design of future digital sketchnoting tools.

We introduce Gripmarks, a system that enables users to opportunistically use objects they are already holding as input surfaces for mixed reality head-mounted displays (HMD). Leveraging handheld objects reduces the need for users to free up their hands or acquire a controller to interact with their HMD. Gripmarks associate a particular hand grip with the shape primitive of the physical object without the need of object recognition or instrumenting the object. From the grip pose and shape primitive we can infer the surface of the object. With an activation gesture, we can enable the object for use as input to the HMD. With five gripmarks we demonstrate a recognition rate of 94.2%; we show that our grip detection benefits from the physical constraints of holding an object. We explore two categories of input objects 1) tangible surfaces and 2) tangible tools and present two representative applications. We discuss the design and technical challenges for expanding the concept.

In 3D environments, objects can be difficult to select when they overlap, as this affects available target area and increases selection ambiguity. We introduce Outline Pursuits which extends a primary pointing modality for gaze-assisted selection of occluded objects. Candidate targets within a pointing cone are presented with an outline that is traversed by a moving stimulus. This affords completion of the selection by gaze attention to the intended target's outline motion, detected by matching the user's smooth pursuit eye movement. We demonstrate two techniques implemented based on the concept, one with a controller as the primary pointer, and one in which Outline Pursuits are combined with head pointing for hands-free selection. Compared with conventional raycasting, the techniques require less movement for selection as users do not need to reposition themselves for a better line of sight, and selection time and accuracy are less affected when targets become highly occluded.

Touches between the fingers of an unencumbered hand represent a ready-to-use, eyes-free and expressive input space suitable for interacting with wearable devices such as smart glasses or watches. While prior work has focused on touches to the inner surface of the hand, touches to the nails, a practical site for mounting sensing hardware, have been comparatively overlooked. We extend prior implementations of single touch sensing nails to a full set of five and explore their potential for wearable input. We present design ideas and an input space of 144 touches (taps, flicks and swipes) derived from an ideation workshop. We complement this with data from two studies characterizing the subjective comfort and objective characteristics (task time, accuracy) of each touch. We conclude by synthesizing this material into a set of 29 viable nail touches, assessing their performance in a final study and illustrating how they could be used by presenting, and qualitatively evaluating, two example applications.

Playtesting of games often relies on a mixed-methods approach to obtain more holistic insights about and, in turn, improve the player experience. However, triangulating the different data sources and visualizing them in an integrated manner such that they contextualize each other still proves challenging. Despite its potential value for gauging player behaviour, this area of research continues to be underexplored. In this paper, we propose a visualization approach that combines commonly tracked movement data with - from a visualization perspective rarely considered - gaze behaviour and emotional responses. We evaluated our approach through a qualitative expert study with five professional game developers. Our results show that both the individual visualization of gaze, emotions, and movement but especially their combination are valuable to understand and form hypotheses about player behaviour. At the same time, our results stress that careful attention needs to be paid to ensure that the visualization remains legible and does not obfuscate information.

Driving is a task that is often affected by emotions. The effect of emotions on driving has been extensively studied. Anger is an emotion that dominates in such investigations. Despite the knowledge on strong links between scents and emotions, few studies have explored the effect of olfactory stimulation in a context of driving. Such an outcome provides HCI practitioners very little knowledge on how to design for emotions using olfactory stimulation in the car. We carried out three studies to select scents of different valence and arousal levels (i.e. rose, peppermint, and civet) and anger eliciting stimuli (i.e. affective pictures and on-road events). We used this knowledge to conduct the fourth user study investigating how the selected scents change the emotional state, well-being, and driving behaviour of drivers in an induced angry state. Our findings enable better decisions on what scents to choose when designing interactions for angry drivers.

The subjective experience of emotion is notoriously difficult to interpersonally communicate. We believe that technology can challenge this notion through the design of neuroresponsive systems for interpersonal communication. We explore this through "Neo-Noumena", a communicative neuroresponsive system that uses brain-computer interfacing and artificial intelligence to read one's emotional states and dynamically represent them to others in mixed reality through two head-mounted displays. In our study five participant pairs were given Neo-Noumena for three days, using the system freely. Measures of emotional competence demonstrated a statistically significant increase in participants' ability to interpersonally regulate emotions. Furthermore, participant interviews revealed themes regarding Spatiotemporal Actualization, Objective Representation, and Preternatural Transmission. We also suggest design strategies for future augmented emotion communication systems. We intend that work gives guidance towards a future in which our ability to interpersonally communicate emotion is augmented beyond traditional experience.

Socializing is an important part of why people choose to play games and is at the core of many game mechanics. Anxiety and fear about social interactions can lead to withdrawal from socializing in the physical world, yet players with social anxiety preferentially choose MMORPGs — a highly social genre — raising questions of whether social anxiety expresses differently during in-game interactions. In the present study (N=181), we explore whether and how social anxiety translates into MMORPGs. By developing a measure of in-game social anxiety, we find that although fear and apprehension of socializing in the physical and game worlds are related, they differently affect preferences, behaviours, and experiences. Social anxiety in the physical world drives reasons for playing, whereas in-game anxiety affects behaviours, reducing participation in activities related to socializing and difficult in-game challenges. Our findings can inform the design of social games and the links between social anxiety and social gaming.

Many online games suffer when players drop off due to lost connections or quitting prematurely, which leads to match terminations or game-play imbalances. While rule-based outcome evaluations or substitutions with bots are frequently used to mitigate such disruptions, these techniques are often perceived as unsatisfactory. Deep learning methods have successfully been used in deep player behavior modelling (DPBM) to produce non-player characters or bots which show more complex behavior patterns than those modelled using traditional AI techniques. Motivated by these findings, we present an investigation of the player-perceived awareness, believability and representativeness, when substituting disconnected players with DPBM agents in an online-multiplayer action game. Both quantitative and qualitative outcomes indicate that DPBM agent substitutes perform similarly to human players and that players were unable to detect substitutions. Notably, players were in fact able to detect substitution with agents driven by more traditional heuristics.

Sketching is a practical and useful skill that can benefit communication and problem solving. However, it remains a difficult skill to learn because of low confidence and motivation among students and limited availability for instruction and personalized feedback among teachers. There is an need to improve the educational experience for both groups, and we hypothesized that integrating technology could provide a variety of benefits. We designed and developed an intelligent tutoring system for sketching fundamentals called Sketchtivity, and deployed it in to six existing courses at the high school and university level during the 2017-2018 school year. 268 students used the tool and produced more than 116,000 sketches of basic primitives. We conducted semi-structured interviews with the six teachers who implemented the software, as well as nine students from a course where the tool was used extensively. Using grounded theory, we found ten categories which unveiled the benefits and limitations of integrating an intelligent tutoring system for sketching fundamentals in to existing pedagogy.

Studies combining psychotherapy with psychedelic drugs (ΨDs) have demonstrated positive outcomes that are often associated with ΨDs' ability to induce 'mystical-type' experiences (MTEs) – i.e., subjective experiences whose characteristics include a sense of connectedness, transcendence, and ineffability. We suggest that both PsiDs and virtual reality can be situated on a broader spectrum of psychedelic technologies. To test this hypothesis, we used concepts, methods, and analysis strategies from ΨD research to design and evaluate 'Isness', a multi-person VR journey where participants experience the collective emergence, fluctuation, and dissipation of their bodies as energetic essences. A study (N=57) analyzing participant responses to a commonly used ΨD experience questionnaire (MEQ30) indicates that Isness participants reported MTEs comparable to those reported in double-blind clinical studies after high doses of psilocybin and LSD. Within a supportive setting and conceptual framework, VR phenomenology can create the conditions for MTEs from which participants derive insight and meaning.

Ruan et al. found transcribing short phrases with speech recognition nearly 200% faster than typing on a smartphone. We extend this comparison to a novel composition task, using a protocol that enables a controlled comparison with transcription. Results show that both composing and transcribing with speech is faster than typing. But, the magnitude of this difference is lower with composition, and speech has a lower error rate than keyboard during composition, but not during transcription. When transcribing, speech outperformed typing in most NASA-TLX measures, but when composing, there were no significant differences between typing and speech for any measure except physical demand.

Capturing 3D foot models is important for applications such as manufacturing customized shoes and creating clubfoot orthotics. In this paper, we propose a novel prototype, Sensock, to offer a fully wearable solution for the task of 3D foot reconstruction. The prototype consists of four soft stretchable sensors, made from silk fibroin yarn. We identify four characteristic foot girths based on the existing knowledge of foot anatomy, and measure their lengths with the resistance value of the stretchable sensors. A learning-based model is trained offline and maps the foot girths to the corresponding 3D foot shapes. We compare our method with existing solutions using red–green–blue (RGB) or RGBD (RGB-depth) cameras, and show the advantages of our method in terms of both efficiency and accuracy. In the user experiment, we find that the relative error of Sensock is lower than 0.55%. It performs consistently across different trials and is considered comfortable and suitable for long-term wearing.

On-the-go text-editing is difficult, yet frequently done in everyday lives. Using smartphones for editing text forces users into a heads-down posture which can be undesirable and unsafe. We present EYEditor, a heads-up smartglass-based solution that displays the text on a see-through peripheral display and allows text-editing with voice and manual input. The choices of output modality (visual and/or audio) and content presentation were made after a controlled experiment, which showed that sentence-by-sentence visual-only presentation is best for optimizing users' editing and path-navigation capabilities. A second experiment formally evaluated EYEditor against the standard smartphone-based solution for tasks with varied editing complexities and navigation difficulties. The results showed that EYEditor outperformed smartphones as either the path OR the task became more difficult. Yet, the advantage of EYEditor became less salient when both the editing and navigation was difficult. We discuss trade-offs and insights gained for future heads-up text-editing solutions.

Influence of interaction fidelity and rendering quality on perceived user experience have been largely explored in Virtual Reality (VR). However, differences in interaction choices triggered by these rendering cues have not yet been explored. We present a study analysing the effect of thermal visual cues and contextual information on 50 participants' approach to grasp and move a virtual mug. This study comprises 3 different temperature cues (baseline empty, hot and cold) and 4 contextual representations; all embedded in a VR scenario. We evaluate 2 different hand representations (abstract and human) to assess grasp metrics. Results show temperature cues influenced grasp location, with the mug handle being predominantly grasped with a smaller grasp aperture for the hot condition, while the body and top were preferred for baseline and cold conditions.

People often operate their smartphones with only one hand, using just their thumb for touch input. With today's larger smartphones, this leads to a reachability issue: Users can no longer comfortably touch everywhere on the screen without changing their grip. We investigate using the head tracking in modern smartphones to address this reachability issue. We developed three interaction techniques, pure head (PH), head + touch (HT), and head area + touch (HA), to select targets beyond the reach of one's thumb. In two user studies, we found that selecting targets using HT and HA had higher success rates than the default direct touch (DT) while standing (by about 9%) and walking (by about 12%), while being moderately slower. HT and HA were also faster than one of the best techniques, BezelCursor (BC) (by about 20% while standing and 6% while walking), while having the same success rate.

Studies in psychology have shown that framing effects, where the positive or negative attributes of logically equivalent choices are emphasised, influence people's decisions. When outcomes are uncertain, framing effects also induce patterns of choice reversal, where decisions tend to be risk averse when gains are emphasised and risk seeking when losses are emphasised. Studies of these effects typically use potent framing stimuli, such as the mortality of people suffering from diseases or personal financial standing. We examine whether these effects arise in users' decisions about interface features, which typically have less visceral consequences, using a crowd-sourced study based on snap-to-grid drag-and-drop tasks (n = 842). The study examined several framing conditions: those similar to prior psychological research, and those similar to typical interaction choices (enabling/disabling features). Results indicate that attribute framing strongly influences users' decisions, that these decisions conform to patterns of risk seeking for losses, and that patterns of choice reversal occur.

In this paper we explore how the qualities of Autonomous Sensory Meridian Response (ASMR) media–its pairing of sonic and visual design, ability to subvert fast-paced technology for slow experiences, production of somatic responses, and attention to the everyday–might reveal new design possibilities for interactions with wearable technology. We recount our year-long design inquiry into the subject which began with an interview with a "live" ASMR creator and design probes, a series of first-person design exercises, and resulted in the creation of two interactive garments for attending, noticing, and becoming enchanted with our our everyday surroundings. We conclude by suggesting that these ASMR inspired designs cultivate personal, intimate, embodied, and felt practices of attention within our everyday, mundane, environments.

We present GazeConduits, a calibration-free ad-hoc mobile interaction concept that enables users to collaboratively interact with tablets, other users, and content in a cross-device setting using gaze and touch input. GazeConduits leverages recently introduced smartphone capabilities to detect facial features and estimate users' gaze directions. To join a collaborative setting, users place one or more tablets onto a shared table and position their phone in the center, which then tracks users present as well as their gaze direction to determine the tablets they look at. We present a series of techniques using GazeConduits for collaborative interaction across mobile devices for content selection and manipulation. Our evaluation with 20 simultaneous tablets on a table shows that GazeConduits can reliably identify which tablet or collaborator a user is looking at.

User experience estimation of VR exergame players by recognising their affective state could enable us to personalise and optimise their experience. Affect recognition based on psychophysiological measurements has been successful for moderate intensity activities. High intensity VR exergames pose challenges as the effects of exercise and VR headsets interfere with those measurements. We present two experiments that investigate the use of different sensors for affect recognition in a VR exergame. The first experiment compares the impact of physical exertion and gamification on psychophysiological measurements during rest, conventional exercise, VR exergaming, and sedentary VR gaming. The second experiment compares underwhelming, overwhelming and optimal VR exergaming scenarios. We identify gaze fixations, eye blinks, pupil diameter and skin conductivity as psychophysiological measures suitable for affect recognition in VR exergaming and analyse their utility in determining affective valence and arousal. Our findings provide guidelines for researchers of affective VR exergames.

Virtual and augmented reality head-mounted displays (HMDs) are currently heavily relying on spatially tracked input devices (STID) for interaction. These STIDs are all prone to the phenomenon that a discrete input (e.g. button press) will disturb the position of the tracker, resulting in a different selection point during ray-cast interaction (Heisenberg Effect of Spatial Interaction). Besides the knowledge of its existence, there is currently a lack of a deeper understanding of its severity, structure and impact on throughput and angular error during a selection task. In this work, we present a formal evaluation of the Heisenberg effect and the impact of body posture, arm position and STID degrees of freedom on its severity. In a Fitt's Law inspired user study (N=16), we found that the Heisenberg effect is responsible for 30.45% of the overall errors occurring during a pointing task, but can be reduced by 25.4% using a correction function.