Soliciting Stakeholders’ Fairness Notions in Child Maltreatment Predictive Systems

Abstract

Recent work in fair machine learning has proposed dozens of technical definitions of algorithmic fairness and methods for enforcing these definitions. However, we still lack an understanding of how to develop machine learning systems with fairness criteria that reflect relevant stakeholders' nuanced viewpoints in real-world contexts. To address this gap, we propose a framework for eliciting stakeholders' subjective fairness notions. Combining a user interface that allows stakeholders to examine the data and the algorithm's predictions with an interview protocol to probe stakeholders' thoughts while they are interacting with the interface, we can identify stakeholders' fairness beliefs and principles. We conduct a user study to evaluate our framework in the setting of a child maltreatment predictive system. Our evaluations show that the framework allows stakeholders to comprehensively convey their fairness viewpoints. We also discuss how our results can inform the design of predictive systems.

Authors
Hao-Fei Cheng
University of Minnesota, Minneapolis, Minnesota, United States
Logan Stapleton
University of Minnesota, Minneapolis, Minnesota, United States
Ruiqi Wang
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Paige Bullock
Kenyon College, Gambier, Ohio, United States
Alexandra Chouldechova
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Zhiwei Steven Wu
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Haiyi Zhu
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
DOI

10.1145/3411764.3445308

Paper URL

https://doi.org/10.1145/3411764.3445308

Video

Conference: CHI 2021

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2021.acm.org/)

Session: Computational Human-AI Conversation

[A] Paper Room 02, 2021-05-11 17:00:00~2021-05-11 19:00:00 / [B] Paper Room 02, 2021-05-12 01:00:00~2021-05-12 03:00:00 / [C] Paper Room 02, 2021-05-12 09:00:00~2021-05-12 11:00:00
Paper Room 02
14 items in this session
2021-05-11 08:00:00
2021-05-11 10:00:00
Japanese summary
Loading...