Human-Computer Interaction (HCI) scholarship has studied how Artificial Intelligence (AI) can be leveraged to support care work(ers) by recognizing, reducing, and redistributing workload. Assessment of AI's impact on workers requires scrutiny and is a growing area of inquiry within human-centered evaluations of AI. We add to these conversations by unpacking the sociotechnical gap between the broader aspirations of workers from an AI-based system and the narrower existing definitions of success. We conducted a mixed-methods study and drew on Amartya Sen's Capability Approach to analyze the gap. We shed light on the social factors---on top of performance on evaluation metrics---that guided the AI model choice and determined whose wellbeing must be evaluated while conducting such evaluations. We argue for assessing broader achievements enabled through AI's use when conducting human-centered evaluations of AI. We discuss and recommend the dimensions to consider while conducting such evaluations.
https://dl.acm.org/doi/10.1145/3706598.3713278
The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)