Principles of Safe AI Companions for Youth: Parent and Expert Perspectives

要旨

AI companions are increasingly popular among teenagers, yet current platforms lack safeguards to address developmental risks and harmful normalization. Despite growing concerns, little is known about how parents and developmental psychology experts assess these interactions or what protections they consider necessary. We conducted 26 semi-structured interviews with parents and experts, who reviewed real-world youth–AI companion conversation snippets. We found that stakeholders assessed risks contextually, attending to factors such as youth maturity, AI character age, and how AI characters modeled values and norms. We also identified distinct logics of assessment: parent participants flagged single events, such as a mention of suicide or flirtation, as high risk, whereas expert participants looked for patterns over time, such as repeated references to self-harm or sustained dependence. Both groups proposed interventions, with parents favoring broader oversight and experts preferring cautious, crisis-only escalation paired with youth-facing safeguards. These findings provide directions for embedding safety into AI companion design.

著者
Yaman Yu
University of Illinois at Urbana Champaign, Champaign, Illinois, United States
Fnu Mohi
University of Illinois at Urbana-Champaign, Champaign, Illinois, United States
Aishi Debroy
University of Illinois Urbana-Champaign (UIUC), Champaign, Illinois, United States
Xin Cao
University of Illinois at Urbana Champaign, Champaign, Illinois, United States
Karen Rudolph
University of Illinois at Urbana-Champaign, Champaign, Illinois, United States
Yang Wang
University of Illinois at Urbana-Champaign, Champaign, Illinois, United States

会議: CHI 2026

ACM CHI Conference on Human Factors in Computing Systems

セッション: AI Governance and Safety

P1 - Room 119
7 件の発表
2026-04-16 18:00:00
2026-04-16 19:30:00