ACKnowledge: A Computational Framework for Human Compatible Affordance-based Interaction Planning in Real-world Contexts

要旨

Intelligent agents coexisting with humans often need to interact with human-shared objects in environments. Thus, agents should plan their interactions based on objects' affordances and the current situation to achieve acceptable outcomes. How to support intelligent agents' planning of affordance-based interactions compatible with human perception and values in real-world contexts remains under-explored. We conducted a formative study identifying the physical, intrapersonal, and interpersonal contexts that count to household human-agent interaction. We then proposed ACKnowledge, a computational framework integrating a dynamic knowledge graph, a large language model, and a vision language model for affordance-based interaction planning in dynamic human environments. In evaluations, ACKnowledge generated acceptable planning results with an understandable process. In real-world simulation tasks, ACKnowledge achieved a high execution success rate and overall acceptability, significantly enhancing usage-rights respectfulness and social appropriateness over baselines. The case study's feedback demonstrated ACKnowledge's negotiation and personalization capabilities toward an understandable planning process.

著者
Ziqi Pan
The Hong Kong University of Science and Technology, Hong Kong, China
Xiucheng Zhang
Sun Yat-sen University, Zhuhai, China
Zisu Li
The Hong Kong University of Science and Technology, Hong Kong SAR, Hong Kong, China
Zhenhui Peng
Sun Yat-sen University, Zhuhai, Guangdong Province, China
Mingming Fan
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Xiaojuan Ma
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
DOI

10.1145/3706598.3713791

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713791

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Recommendation and Personalization

G401
7 件の発表
2025-04-29 20:10:00
2025-04-29 21:40:00
日本語まとめ
読み込み中…