We present a Research through Design project that explores how whispering influences the ways people experience and interact with voice assistants. The research project includes a co-speculation workshop and the use of a design probe, which culminated in the production of a design fiction short film. Our design-led inquiry contributes with experiential qualities of whispering with voice assistants: creepiness, trust, and intimacy. Furthermore, we present how whispering opens up new dimensions of how and when voice interaction could be used. We propose that designers of whispering voice assistants should reflect on how they facilitate the experiential qualities of creepiness, trust, and intimacy, and reflect on the potential challenges whispering brings to the relation between a user and a voice assistant.
AI assistants that use a voice user interface (VUI), such as AI speakers, have become popular in family homes. However, it is still unclear what roles the AI speaker can support within the family unit. We investigated the roles of an AI speaker as a family-shared technology. By conducting a one-week participatory user study, we discovered that family members' co-ownership toward the AI speaker was the key in the development of its family-oriented roles. Our findings showed seven domains of user expectations on these roles, and we realized that all the expectations can be represented as family cohesion. In addition, privacy awareness was emphasized regarding personal supports. Finally, we discuss a new perspective for AI speaker design and offer two suggestions: 1) leveraging human-likeness to develop its potential roles of supporting the unit of a family and 2) interpreting the home context to seamlessly connect family and personal supporting roles.
Smart speakers with voice agents are becoming increasingly common. However, the agent's voice always emanates from the device, even when that information is contextually and spatially relevant elsewhere. Digital Ventriloquism allows smart speakers to render sound onto everyday objects, such that it appears they are speaking and are interactive. This can be achieved without any modification of objects or the environment. For this, we used a highly directional pan-tilt ultrasonic array. By modulating a 40 kHz ultrasonic signal, we can emit sound that is inaudible "in flight" and demodulates to audible frequencies when impacting a surface through acoustic parametric interaction. This makes it appear as though the sound originates from an object and not the speaker. We ran a study in which we projected speech onto five objects in three environments, and found that participants were able to correctly identify the source object 92% of the time and correctly repeat the spoken message 100% of the time, demonstrating our digital ventriloquy is both directional and intelligible.
https://doi.org/10.1145/3313831.3376503
We present a qualitative study with 16 deaf and hard of hearing (DHH) participants examining reactions to smartwatch-based visual + haptic sound feedback designs. In Part 1, we conducted a Wizard-of-Oz (WoZ) evaluation of three smartwatch feedback techniques (visual alone, visual + simple vibration, and visual + tacton) and investigated vibrational patterns (tactons) to portray sound loudness, direction, and identity. In Part 2, we visited three public or semi-public locations where we demonstrated sound feedback on the smartwatch in situ to examine contextual influences and explore sound filtering options. Our findings characterize uses for vibration in multimodal sound awareness, both for push notification and for immediately actionable sound information displayed through vibrational patterns (tactons). In situ experiences caused participants to request sound filtering — particularly to limit haptic feedback — as a method for managing soundscape complexity. Additional concerns arose related to learnability, possibility of distraction, and system trust. Our findings have implications for future portable sound awareness systems.
AI-enabled smart-home agents that automate household routines are increasingly viable, but the design space of how and what such systems should communicate with their users remains underexplored. Through a user-enactment study, we identified various interpretations of and feelings toward such a system's confidence in its automated acts. That confidence and their own mental models influenced what and how the participants wanted the system to communicate, as well as how they would assess, diagnose, and subsequently improve it. Automated acts resulted from false predictions were not generally considered improper, provided that they were perceived as reasonable or potentially useful. The participants' improvement strategies were of four general types, all of which will be discussed. Factors affecting their preferred levels of involvement in automated acts and their interest in system confidence were also identified. We conclude by making practical design recommendations for the user-system communication design spaces of smart-home routine assistants.