The consumption of music is increasingly reliant on the personalisation, recommendation, and automated curation features of music streaming services. Using algorithm experience (AX) as a lens, we investigated the user experience of the algorithmic recommendation and automated curation features of several popular music streaming services. We conducted interviews and participant-observation with 15 daily users of music streaming services, followed by a design workshop. We found that despite the utility of increasingly algorithmic personalisation, listeners experienced these algorithmic and recommendation features as impersonal in determining their background listening, music discovery, and playlist curation. While listener desire for more control over recommendation settings is not new, we offer a number of novel insights about music listening to nuance this understanding, particularly through the notion of vibe.
https://doi.org/10.1145/3544548.3581492
Machine learning is applied in a multitude of sectors with very impressive results. This success is due to the availability of an ever-growing amount of data acquired by omnipresent sensor devices and platforms on the internet. But there is a scarcity of labeled data which is required for most ML methods. However, generation of labeled data requires much time and resources. In this paper, we propose a portable, Open Source, simple and responsive manual Tool for 2D multiple object Tracking Annotation (TmoTA). Besides responsiveness, our tool design provides several features like view centering and looped playback that speed up the annotation process. We evaluate our proposed tool by comparing TmoTA with the widely used manual labeling tools CVAT, Label Studio, and two semi-automated tools Supervisely and VATIC with respect to object labeling time and accuracy. The evaluation includes a user study and pre-case studies showing that the annotation time per object frame can be reduced by 20\% to 40\% over the first 20 annotated objects compared to the manual labeling tools.
https://doi.org/10.1145/3544548.3581185
Robots are becoming increasingly omnipresent in our daily lives, supporting us and carrying out autonomous tasks. In Human-Robot Interaction, human actors benefit from understanding the robot's motion intent to avoid task failures and foster collaboration. Finding effective ways to communicate this intent to users has recently received increased research interest. However, no common language has been established to systematize robot motion intent. This work presents a scoping review aimed at unifying existing knowledge. Based on our analysis, we present an intent communication model that depicts the relationship between robot and human through different intent dimensions (intent type, intent information, intent location). We discuss these different intent dimensions and their interrelationships with different kinds of robots and human roles. Throughout our analysis, we classify the existing research literature along our intent communication model, allowing us to identify key patterns and possible directions for future research.
https://doi.org/10.1145/3544548.3580857
Human data labeling is an important and expensive task at the heart of supervised learning systems. Hierarchies help humans understand and organize concepts. We ask whether and how concept hierarchies can inform the design of annotation interfaces to improve labeling quality and efficiency. We study this question through annotation of vaccine misinformation, where the labeling task is difficult and highly subjective. We investigate 6 user interface designs for crowdsourcing hierarchical labels by collecting over 18,000 individual annotations. Under a fixed budget, integrating hierarchies into the design improves crowdsource workers' F1 scores. We attribute this to (1) Grouping similar concepts, improving F1 scores by +0.16 over random groupings, (2) Strong relative performance on high-difficulty examples (relative F1 score difference of +0.40), and (3) Filtering out obvious negatives, increasing precision by +0.07. Ultimately, labeling schemes integrating the hierarchy outperform those that do not - achieving mean F1 of 0.70.
https://doi.org/10.1145/3544548.3581431
Explainability (XAI) has matured in recent years to provide more human-centered explanations of AI-based decision systems. While static explanations remain predominant, interactive XAI has gathered momentum to support the human cognitive process of explaining. However, the evidence regarding the benefits of interactive explanations is unclear. In this paper, we map existing findings by conducting a detailed scoping review of 48 empirical studies in which interactive explanations are evaluated with human users. We also create a classification of interactive techniques specific to XAI and group the resulting categories according to their role in the cognitive process of explanation: "selective", "mutable" or "dialogic". We identify the effects of interactivity on several user-based metrics. We find that interactive explanations improve perceived usefulness and performance of the human+AI team but take longer. We highlight conflicting results regarding cognitive load and overconfidence. Lastly, we describe underexplored areas including measuring curiosity or learning or perturbing outcomes.
https://doi.org/10.1145/3544548.3581314
We propose a conceptual perspective on prompts for Large Language Models (LLMs) that distinguishes between (1) diegetic prompts (part of the narrative, e.g. “Once upon a time, I saw a fox...”), and (2) non-diegetic prompts (external, e.g. “Write about the adventures of the fox.”). With this lens, we study how 129 crowd workers on Prolific write short texts with different user interfaces (1 vs 3 suggestions, with/out non-diegetic prompts; implemented with GPT-3): When the interface offered multiple suggestions and provided an option for diegetic prompting, participants preferred choosing from multiple suggestions over controlling them via non-diegetic prompts. When participants provided non-diegetic prompts it was to ask for inspiration, topics or facts. Single suggestions in particular were guided both with diegetic and non-diegetic information. This work informs human-AI interaction with generative models by revealing that (1) writing non-diegetic prompts requires effort, (2) people combine diegetic and non-diegetic prompting, and (3) they use their draft (i.e. diegetic information) and suggestion timing to strategically guide LLMs.
https://doi.org/10.1145/3544548.3580969