Talk to the Hand: an LLM-powered Chatbot with Visual Pointer as Proactive Companion for On-Screen Tasks

要旨

This paper presents Pointer Assistant, a novel human-AI interaction technique for on-screen tasks. The design features a chatbot displayed as an extra mouse pointer, alongside the user's, which proactively gives feedback on user actions while directing them to relevant areas on the screen and responding to the user's direct chat messages. The effectiveness of the design's key characteristics, pointer form and proactivity, was investigated in a study involving 220 participants in a financial budget planning task. Results demonstrated that the pointer design and interaction reduced task load while improving satisfaction with the experience, and increased the number of budget categories ideated during the task compared to the traditional passive chat log design. Participants viewed Pointer Assistant as a fun, innovative, and helpful visual guide while noting that its assertiveness can be improved. Future developments could offer even further enhancements to the user experience of human-AI collaboration and task outcomes.

著者
Thanawit Prasongpongchai
KASIKORN Business-Technology Group, Nonthaburi, Thailand
Pat Pataranutaporn
Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
Monchai Lertsutthiwong
KASIKORN Business-Technology Group, Nonthaburi, Thailand
Pattie Maes
Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
DOI

10.1145/3706598.3715579

論文URL

https://dl.acm.org/doi/10.1145/3706598.3715579

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Knowledge Work

G416+G417
6 件の発表
2025-04-30 23:10:00
2025-05-01 00:40:00
日本語まとめ
読み込み中…