A Comparative Analysis of Information Gathering by Chatbots, Questionnaires, and Humans in Clinical Pre-Consultation

要旨

Information gathering is an important capability that allows chatbots to understand and respond to users' needs, yet the effectiveness of LLM-powered chatbots at this task remains underexplored. Our work investigates this question in the context of clinical pre-consultation, wherein patients provide information to an intermediary before meeting with a physician to facilitate communication and reduce consultation inefficiencies. We conducted a study at a walk-in clinic with 45 patients who interacted with one of three conversational agents: a chatbot, a questionnaire, and a Wizard-of-Oz. We analyzed patients' messages using metrics adapted from Grice's maxims to assess the quality of information gathered at each conversation turn. We found that the Wizard and LLM were more successful than the questionnaire because they modified questions and asked follow-ups when participants provided unsatisfactory answers. However, the LLM did not ask nearly as many follow-up questions as the Wizard, particularly when participants provided unclear answers.

著者
Brenna Li
University of Toronto, Toronto, Ontario, Canada
Saba Tauseef
Independent Researcher, Brampton, Ontario, Canada
Khai N.. Truong
University of Toronto, Toronto, Ontario, Canada
Alex Mariakakis
University of Toronto, Toronto, Ontario, Canada
DOI

10.1145/3706598.3713613

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713613

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: LLM for Health

Annex Hall F206
6 件の発表
2025-04-30 18:00:00
2025-04-30 19:30:00
日本語まとめ
読み込み中…