Crowdsourced Think-Aloud Studies

要旨

The think-aloud (TA) protocol is a useful method for evaluating user interfaces, including data visualizations. However, TA studies are time-consuming to conduct and hence often have a small number of participants. Crowdsourcing TA studies would help alleviate these problems, but the technical overhead and the unknown quality of results have restricted TA to synchronous studies. To address this gap we introduce CrowdAloud, a system for creating and analyzing asynchronous, crowdsourced TA studies. CrowdAloud captures audio and provenance (log) data as participants interact with a stimulus. Participant audio is automatically transcribed and visualized together with events data and a full recreation of the state of the stimulus as seen by participants. To gauge the value of crowdsourced TA studies, we conducted two experiments: one to compare lab-based and crowdsourced TA studies, and one to compare crowdsourced TA studies with crowdsourced text prompts. Our results suggest that crowdsourcing is a viable approach for conducting TA studies at scale.

著者
Zach Cutler
University of Utah, Salt Lake City, Utah, United States
Lane Harrison
Worcester Polytechnic Institute, Worcester, Massachusetts, United States
Carolina Nobre
University of Toronto, Toronto, Ontario, Canada
Alexander Lex
University of Utah, Salt Lake City, Utah, United States
DOI

10.1145/3706598.3714305

論文URL

https://dl.acm.org/doi/10.1145/3706598.3714305

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Crowdsourcing and Tech in the Wild

Annex Hall F204
7 件の発表
2025-04-29 23:10:00
2025-04-30 00:40:00
日本語まとめ
読み込み中…