Voice messages, by nature, prevent users from gauging the emotional tone without fully diving into the audio content. This hinders the shared emotional experience at the pre-retrieval stage. Research scarcely explored "Emotional Teasers"—pre-retrieval cues offering a glimpse into an awaiting message's emotional tone without disclosing its content. We introduce EmoWear, a smartwatch voice messaging system enabling users to apply 30 animation teasers on message bubbles to reflect emotions. EmoWear eases senders' choice by prioritizing emotions based on semantic and acoustic processing. EmoWear was evaluated in comparison with a mirroring system using color-coded message bubbles as emotional cues (N=24). Results showed EmoWear significantly enhanced emotional communication experience in both receiving and sending messages. The animated teasers were considered intuitive and valued for diverse expressions. Desirable interaction qualities and practical implications are distilled for future design. We thereby contribute both a novel system and empirical knowledge concerning emotional teasers for voice messaging.
doi.org/10.1145/3613904.3642101
By combining voice and touch interactions, multimodal interfaces can surpass the efficiency of either modality alone. Traditional multimodal frameworks require laborious developer work to support rich multimodal commands where the user’s multimodal command involves possibly exponential combinations of actions/function invocations. This paper presents ReactGenie, a programming framework that better separates multimodal input from the computational model to enable developers to create efficient and capable multimodal interfaces with ease. ReactGenie translates multimodal user commands into NLPL (Natural Language Programming Language), a programming language we created, using a neural semantic parser based on large-language models. The ReactGenie runtime interprets the parsed NLPL and composes primitives in the computational model to implement complex user commands. As a result, ReactGenie allows easy implementation and unprecedented richness in commands for end-users of multimodal apps. Our evaluation showed that 12 developers can learn and build a non-trivial ReactGenie application in under 2.5 hours on average. In addition, compared with a traditional GUI, end-users can complete tasks faster and with less task load using ReactGenie apps.
doi.org/10.1145/3613904.3642517
The extensive use of computers for text entry has been linked to increased stress, depression, and sleep disturbances, adversely affecting performance. Recent trends involve using scent diffusers to counter these effects. However, the impact of scents on text entry performance is not well-studied. Our empirical study investigated the effects of self-selected pleasant scents on text composition and transcription performance. Results showed that while composing, users were slower with a scent present, potentially due to heightened focus on text quality. Scent did not alter accuracy or text length. In transcription tasks, although scent did not alter typing speed, it adversely affected accuracy, likely due to its impact on concentration levels. Despite these mixed results, users felt more effective and enjoyed the scent, indicating a preference for its continued use. This study opens avenues for further research into scents' influence on computer-based tasks, potentially contributing to the evolving field of olfactory displays.
doi.org/10.1145/3613904.3642445
Inspired by physical possessions displayed in the home, we define "digital knick-knacks" as standalone audiovisual digital possessions or embellishments contained within non-game digital environments. "Neko", a cat that chases the cursor, is a historical example. We propose a taxonomy to define and generate digital knick-knacks based on key publications on consumer behaviour and personal possessions, augmented by results of a brainstorming session with 9 HCI researchers. Using the taxonomy, we prototype three classes of digital knick-knack exemplars: an ambient noise machine, a virtual pet, and a virtual picture frame. In a 10-day diary study, 10 participants design their own variants of the prototypes, and report on their experience using them on a personal device. Our analysis shows how digital knick-knacks can bring value to users, and we suggest implications for designing playful digital embellishments.
doi.org/10.1145/3613904.3642695
When computers unexpectedly delay or thwart goal attainment, frustration ensues. The central studies of the extent, content, and impact of such frustration were done more than 15 years ago. We revisit this issue after computers have become more mature and computer use is more extensive. To this end, we had 234 crowdsourced participants log the frustrating episodes they experienced with their computers during one hour of computer use. The average time lost due to frustrating episodes was between 11% and 20% of the one-hour period. Though this is less time lost than in the earlier studies, frustration remains a common user experience. While shorter, the median level of frustration during the episodes was high (7 on a 9-point scale). The frustration level correlated with task importance and time lost but was unaffected by computer experience and largely unaffected by computer self-efficacy. In addition, participants indicated that 84% of the episodes had happened before, that 87% could happen again, and that they were unable to resolve 26% of the episodes. This high rate of recurrence and lack of control likely added to the frustration level. The episodes spanned various issues pertaining to performance (49%), usability (36%), and utility (16%).