Automated Conversion of Music Videos into Lyric Videos

要旨

Musicians and fans often produce lyric videos, a form of music videos that showcase the song's lyrics, for their favorite songs. However, making such videos can be challenging and time-consuming as the lyrics need to be added in synchrony and visual harmony with the video. Informed by prior work and close examination of existing lyric videos, we propose a set of design guidelines to help creators make such videos. Our guidelines ensure the readability of the lyric text while maintaining a unified focus of attention. We instantiate these guidelines in a fully automated pipeline that converts an input music video into a lyric video. We demonstrate the robustness of our pipeline by generating lyric videos from a diverse range of input sources. A user study shows that lyric videos generated by our pipeline are effective in maintaining text readability and unifying the focus of attention.

著者
Jiaju Ma
Stanford University, Stanford, California, United States
Anyi Rao
Stanford University, Stanford, California, United States
Li-Yi Wei
Adobe Research, San Jose, California, United States
Rubaiat Habib Kazi
Adobe Research, Seattle, Washington, United States
Hijung Valentina Shin
Adobe Research, Cambridge, Massachusetts, United States
Maneesh Agrawala
Stanford University, Stanford, California, United States
論文URL

https://doi.org/10.1145/3586183.3606757

動画

会議: UIST 2023

ACM Symposium on User Interface Software and Technology

セッション: Masterful Media: Audio and Video Authoring Tools

Gold Room
6 件の発表
2023-10-30 23:20:00
2023-10-31 00:40:00