Recent advancements in text-to-music generative AI (GenAI) have significantly expanded access to music creation. However, deaf and hard of hearing (DHH) individuals remain largely excluded from these developments. This study explores how music GenAI could enhance the music-making experience of DHH individuals, who often rely on hearing people to translate sounds and music. We developed a multimodal music-making assistive tool informed by focus group interviews. This tool enables DHH users to create and edit music independently through language interaction with music GenAI, supported by integrated visual and tactile feedback. Our findings from the music-making study revealed that the system empowers them to engage in independent and proactive music-making activities, increasing their confidence, fostering musical expression, and positively shifting their attitudes toward music. Contributing to inclusive art by preserving the unique sensory characteristics of DHH individuals, this study demonstrates how music GenAI can benefit a marginalized community, fostering independent creative expression.
https://dl.acm.org/doi/10.1145/3706598.3714298
This paper explores the relationship between music and keyboard typing behavior. In particular, we focus on how it affects keystroke-based authentication systems. To this end, we conducted an online experiment (N=43), where participants were asked to replicate paragraphs of text while listening to music at varying tempos and loudness levels across two sessions. Our findings reveal that listening to music leads to more errors and faster typing if the music is fast. Identification through a biometric model was improved when music was played either during its training or testing. This hints at the potential of music for increasing identification performance and a tradeoff between this benefit and user distraction. Overall, our research sheds light on typing behavior and introduces music as a subtle and effective tool to influence user typing behavior in the context of keystroke-based authentication.
https://dl.acm.org/doi/10.1145/3706598.3713222
Prompt-based music generative artificial intelligence (GenAI) offers an efficient way to engage in music creation through language. However, it faces limitations in conveying artistic intent with language alone, highlighting the need for more research on AI-creator interactions. This study evaluates three different interaction modes (prompt-based, preset-based, and motif-based) of commercialized music AI toots with 17 participants of varying musical expertise to examine how prompt-based GenAI can improve creative intention. Our findings revealed that user groups preferred prompt-based music GenAI for distinct purposes: experts used it to validate musical concepts, novices to generate reference samples, and nonprofessionals to transform abstract ideas into musical compositions. We identified its potential for enhancing compositional efficiency and creativity through intuitive interaction, while also noting limitations in handling temporal and musical nuances solely through prompts. Based on these insights, we present design guidelines to ensure users can effectively engage in the creative process, considering their musical expertise.
https://dl.acm.org/doi/10.1145/3706598.3713762
Music notation programs force composers to follow the many rules of the staff notation when writing music and constantly seek to optimize symbol placement, making numerous adjustments automatically. Even though this impedes their creative process, many composers still use them throughout their workflow, for lack of a better option. We introduce EuterPen, a music notation program prototype that selectively relaxes both syntactic and structural constraints while editing a score. Composers can input and manipulate music symbols with increased flexibility, leveraging the affordances of pen and touch. They can make space on, between and around staves to insert additional content such as digital ink, pictures and audio samples. We describe the iterative design process that led to EuterPen: prototyping phases, a participatory design workshop, and a series of interviews. Feedback from the participating professional composers indicates that EuterPen offers a compelling and promising approach to music writing.
https://dl.acm.org/doi/10.1145/3706598.3713488
This paper reports on a field study of the WavData Lamp: an interactive lamp that can physically visualize people’s music listening data by changing light colors and outstretching its form enclosure. We deployed five WavData Lamps to five participants' homes for two months to investigate their composite relation with a data-physicalized thing. Findings reveal that their music-listening norms were determined by the instantiated materiality of the Lamp in the early days. With a tilted form enclosure, the WavData Lamp successfully engendered rich actions and meanings of the cohabiting participants and their family members. In the end, the participants described their experiences of entangling with and living with the Lamp as a form of collaboration. Reflecting on these empirical insights explicitly extends the intrinsic meaning of the composite relation and offers rich implications to promote further HCI explorations and practices.
https://dl.acm.org/doi/10.1145/3706598.3713489
Music videos have traditionally been the domain of experts, but with text-to-video generative AI models, AI artists can now create them more easily. However, accurately reflecting the desired music-visual mise-en-scène remains challenging without specialized knowledge, highlighting the need for supportive tools. To address this, we conducted a design workshop with seven music video experts, identified design goals, and developed MVPrompt—a tool for generating music-visual mise-en-scène prompts. In a user study with 24 AI artists, MVPrompt outperformed the Baseline, effectively supporting the collaborative creative process. Specifically, the Visual Theme stage facilitated the exploration of tone and manner, while the Visual Scene & Grammar stage refined prompts with detailed mise-en-scène elements. By enabling AI artists to specify mise-en-scène creatively, MVPrompt enhances the experience of making music video scenes with text-to-video generative AI.
https://dl.acm.org/doi/10.1145/3706598.3713876
FFAME (Filtering Familiar Audio for Movement Exploration) is a novel sonification framework aiming to facilitate movement in individuals with chronic back pain. Our personalised, music-based approach contrasts and extends prior work with predetermined tonal sonification. FFAME progressively filters selected music based on angles of the trunk. Through a qualitative analysis of reported experience of 15 participants with chronic pain and 5 physiotherapists, we identify how sonification parameters and musical characteristics affect movement and meaning-making. Music-based movement sonification proved impactful across multiple dimensions: (1) encouraging movement, (2) escaping pain-related rumination, (3) externalizing pain experiences, and (4) scaffolding physical activities. Drawing on enactivism and related philosophies, the study highlights how the semantic indeterminacy of music, combined with real-time movement sonification, created a rich, open-ended environment that supported user agency and exploration. Sonification for pain management can be creative and expressive, enabling people with pain to extend challenging movements and build movement confidence.
https://dl.acm.org/doi/10.1145/3706598.3713601