AMUSE: Human-AI Collaborative Songwriting with Multimodal Inspirations

要旨

Songwriting is often driven by multimodal inspirations, such as imagery, narratives, or existing music, yet songwriters remain unsupported by current music AI systems in incorporating these multimodal inputs into their creative processes. We introduce Amuse, a songwriting assistant that transforms multimodal (image, text, or audio) inputs into chord progressions that can be seamlessly incorporated into songwriters' creative process. A key feature of Amuse is its novel method for generating coherent chords that are relevant to music keywords in the absence of datasets with paired examples of multimodal inputs and chords. Specifically, we propose a method that leverages multimodal language models to convert multimodal inputs into noisy chord suggestions and uses a unimodal chord model to filter the suggestions. A user study with songwriters shows that Amuse effectively supports transforming multimodal ideas into coherent musical suggestions, enhancing users' agency and creativity throughout the songwriting process.

受賞
Best Paper
著者
Yewon Kim
KAIST, Daejeon, Korea, Republic of
Sung-Ju Lee
KAIST, Daejeon, Korea, Republic of
Chris Donahue
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
DOI

10.1145/3706598.3713818

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713818

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Creative Tools

Annex Hall F205
7 件の発表
2025-04-30 01:20:00
2025-04-30 02:50:00
日本語まとめ
読み込み中…