Indirect speech acts (ISAs) are a natural pragmatic feature of human communication, allowing requests to be conveyed implicitly while maintaining subtlety and flexibility. Although advancements in speech recognition have enabled natural language interactions with robots through direct, explicit commands—providing clarity in communication—the rise of large language models presents the potential for robots to interpret ISAs. However, empirical evidence on the effects of ISAs on human-robot collaboration (HRC) remains limited. To address this, we conducted a Wizard-of-Oz study (N=36), engaging a participant and a robot in collaborative physical tasks. Our findings indicate that robots capable of understanding ISAs significantly improve human's perceived robot anthropomorphism, team performance, and trust. However, the effectiveness of ISAs is task- and context-dependent, thus requiring careful use. These results highlight the importance of appropriately integrating direct and indirect requests in HRC to enhance collaborative experiences and task performance.
Videoconferencing is integral to modern work and living. Recently, technologists have sought to leverage data captured -- e.g. from cameras and microphones -- to augment communication. This might mean capturing communication information about verbal (e.g. speech, chat messages), or non-verbal exchanges (e.g. body language, gestures, tone of voice) and using this to mediate -- and potentially improve -- communication. However, such tracking has implications for user experience and raises wider concerns (e.g. privacy). To design tools which account for user needs and preferences, this study investigates perspectives on communication tracking through a global survey and interviews, exploring how daily behaviours and the impact of specific features influence user perspectives. We examine user preferences on non-verbal communication tracking, preferred methods of how this information is conveyed and to whom this should be communicated. Our findings aim to guide the development of non-verbal communication tools which augment videoconferencing that prioritise user needs.
Non-verbal signals, including co-speech gestures, play a vital role in human communication by conveying nuanced meanings beyond verbal discourse. While researchers have explored co-speech gestures in human-like conversational agents, limited attention has been given to non-humanoid alternatives. In this paper, we propose using swarm robotic systems as conversational agents and introduce a foundational set of swarm-based co-speech gestures, elicited from non-technical users and validated through an online study. This work outlines the key software and hardware requirements to advance research in co-speech gesture generation with swarm robots, contributing to the future development of social robotics and conversational agents.
In emerging "driver-less" automated vehicles (AVs), the intuitive communication that exists between human drivers and passengers no longer exists, which can lead to reduced trust and acceptance in passengers if they are unclear about what the AV intends to do. This paper contributes the foundational understanding of how passengers naturally decode drivers' non-verbal cues about their intended action to inform intuitive Human-Machine Interface (HMI) designs that try to emulate those cues. Our study investigates what cues passengers perceive, their saliency, and interpretation through a mixed-method approach combining field observations, experience sampling, and auto-confrontation interviews with 30 driver-passenger pairs. Analysis of posture, head/eye movements, and vestibular sensations revealed four categories of intention cues: awareness, interaction, vestibular, and habitual. These findings provide empirical foundations for designing AV interfaces that mirror natural human communication patterns. We discuss implications for designing anthropomorphic HMIs that could enhance trust, predictability, and user experience in AVs.
The integration of emotion-aware systems in vehicles is accelerated by new technologies, including advancements in AI and ubiquitous sensing technologies. As the automotive industry shifts from technology-centred, feature-driven approaches to human-centred design, this research focuses on how to effectively incorporate emotion features into user-centred design to enhance effective human-vehicle interaction in practices. By conducting an interview study with 31 industrial design practitioners, supplemented by insights from engineers and AI experts involved in the early-stage design and development of novel in-vehicle user interfaces and systems, we examined current practices, and sampled their challenges, attitudes and expectations related to emotion-aware systems. Our findings provide critical insights to the design space of emotion-aware systems from both user and AI perspectives, inform efforts to support design practices in this evolving area, and identify opportunities for future innovation in emotion-aware in-vehicle design. Based on our findings, we propose adaptations to design practices and recommendations for further research.
Maintaining engagement in immersive meetings is challenging, particularly when users must catch up on missed content after disruptions. While transcription interfaces can help, table-fixed panels have the potential to distract users from the group, diminishing social presence, while avatar-fixed captions fail to provide past context. We present EngageSync, a context-aware avatar-fixed transcription interface that adapts based on user engagement, offering live transcriptions and LLM-generated summaries to enhance catching up while preserving social presence. We implemented a live VR meeting setup for a 12-participant formative study and elicited design considerations. In two user studies with small (3 avatars) and mid-sized (7 avatars) groups, EngageSync significantly improved social presence (𝑝 < .05) and time spent gazing at others in the group instead of the interface over table-fixed panels. Also, it reduced re-engagement time and increased information recall (𝑝 < .05) over avatar-fixed interfaces, with stronger effects in mid-sized groups (𝑝 < .01).
Despite the spread of technologies in the physical world and the normalization of virtual experiences, non-verbal communication with radically non-anthropomorphic avatars remains an underexplored frontier. We present an interaction system in which two participants must learn to communicate with each other non-verbally through a digital filter that morphs their appearance. In a collaborative escape room, the Visitor must teach a non-anthropomorphic physical robot to play, while the Controller, in a different location, embodies the robot with an altered perception of the environment and the Visitor’s companion in VR. This study addresses the design of the activity, the robot, and the virtual environment, with a focus on how the Visitor’s morphology is translated in VR. Results show that participants were able to develop emergent and effective communication strategies, with the Controller naturally embodying its avatar’s narrative, making this system a promising testbed for future research on human-technology interaction, entertainment, and embodiment.