Coping with AI: not agAIn!

Paper session

会議の名前
CHI 2020
How to Trick AI: Users' Strategies for Protecting Themselves from Automatic Personality Assessment
要旨

Psychological targeting tries to influence and manipulate users' behaviour. We investigated whether users can protect themselves from being profiled by a chatbot, which automatically assesses users' personality. Participants interacted twice with the chatbot: (1) They chatted for 45 minutes in customer service scenarios and received their actual profile (baseline). (2) They then were asked to repeat the interaction and to disguise their personality by strategically tricking the chatbot into calculating a falsified profile. In interviews, participants mentioned 41 different strategies but could only apply a subset of them in the interaction. They were able to manipulate all Big Five personality dimensions by nearly 10%. Participants regarded personality as very sensitive data. As they found tricking the AI too exhaustive for everyday use, we reflect on opportunities for privacy protective designs in the context of personality-aware systems.

キーワード
chatbot
automatic personality assessment
personality
著者
Sarah Theres Völkel
Ludwig Maximilian University of Munich, Munich, Germany
Renate Haeuslschmid
Madeira Interactive Technologies Institute & Ludwig Maximilian University of Munich, Funchal, Madeira Island, Portugal
Anna Werner
Ludwig Maximilian University of Munich, Munich, Germany
Heinrich Hussmann
Ludwig Maximilian University of Munich, Munich, Germany
Andreas Butz
Ludwig Maximilian University of Munich, Munich, Germany
DOI

10.1145/3313831.3376877

論文URL

https://doi.org/10.1145/3313831.3376877

Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism
要旨

Understanding how racial information impacts human decision making in online systems is critical in today's world. Prior work revealed that race information of criminal defendants, when presented as a text field, had no significant impact on users' judgements of recidivism. We replicated and extended this work to explore how and when race information influences users' judgements, with respect to the saliency of presentation. Our results showed that adding photos to the race labels had a significant impact on recidivism predictions for users who identified as female, but not for those who identified as male. The race of the defendant also impacted these results, with black defendants being less likely to be predicted to recidivate compared to white defendants. These results have strong implications for how system-designers choose to display race information, and cautions researchers to be aware of gender and race effects when using Amazon Mechanical Turk workers.

キーワード
bias, recidivism
race
gender
crowd work
Mechanical Turk
legal
human-AI collaboration
著者
Keri Mallari
University of Washington, Seattle, WA, USA
Kori Inkpen
Microsoft Research, Redmond, WA, USA
Paul Johns
Microsoft Research, Redmond, WA, USA
Sarah Tan
Cornell University, Ithaca, NY, USA
Divya Ramesh
University of Michigan, Ann Arbor, MI, USA
Ece Kamar
Microsoft Research, Redmond, WA, USA
DOI

10.1145/3313831.3376257

論文URL

https://doi.org/10.1145/3313831.3376257

Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences
要旨

Algorithmic decision-making systems are increasingly used throughout the public and private sectors to make important decisions or assist humans in making these decisions with real social consequences. While there has been substantial research in recent years to build fair decision-making algorithms, there has been less research seeking to understand the factors that affect people's perceptions of fairness in these systems, which we argue is also important for their broader acceptance. In this research, we conduct an online experiment to better understand perceptions of fairness, focusing on three sets of factors: algorithm outcomes, algorithm development and deployment procedures, and individual differences. We find that people rate the algorithm as more fair when the algorithm predicts in their favor, even surpassing the negative effects of describing algorithms that are very biased against particular demographic groups. We find that this effect is moderated by several variables, including participants' education level, gender, and several aspects of the development procedure. Our findings suggest that systems that evaluate algorithmic fairness through users' feedback must consider the possibility of "outcome favorability" bias.

キーワード
perceived fairness
algorithmic decision-making
algorithmoutcome
algorithm development
著者
Ruotong Wang
Carnegie Mellon University, Pittsburgh, PA, USA
F. Maxwell Harper
Amazon, Seattle, WA, USA
Haiyi Zhu
Carnegie Mellon University, Pittsburgh, PA, USA
DOI

10.1145/3313831.3376813

論文URL

https://doi.org/10.1145/3313831.3376813

Interpreting Interpretability: Understanding Data Scientists' Use of Interpretability Tools for Machine Learning
要旨

Machine learning (ML) models are now routinely deployed in domains ranging from criminal justice to healthcare. With this newfound ubiquity, ML has moved beyond academia and grown into an engineering discipline. To that end, interpretability tools have been designed to help data scientists and machine learning practitioners better understand how ML models work. However, there has been little evaluation of the extent to which these tools achieve this goal. We study data scientists' use of two existing interpretability tools, the InterpretML implementation of GAMs and the SHAP Python package. We conduct a contextual inquiry (N=11) and a survey (N=197) of data scientists to observe how they use interpretability tools to uncover common issues that arise when building and evaluating ML models. Our results indicate that data scientists over-trust and misuse interpretability tools. Furthermore, few of our participants were able to accurately describe the visualizations output by these tools. We highlight qualitative themes for data scientists' mental models of interpretability tools. We conclude with implications for researchers and tool designers, and contextualize our findings in the social science literature.

受賞
Honorable Mention
キーワード
Interpretability
Machine learning
User-centric evaluation
著者
Harmanpreet Kaur
University of Michigan, Ann Arbor, MI, USA
Harsha Nori
Microsoft Research, Seattle, WA, USA
Samuel Jenkins
Microsoft Research, Redmond, WA, USA
Rich Caruana
Microsoft Research, Redmond, WA, USA
Hanna Wallach
Microsoft Research, New York City, NY, USA
Jennifer Wortman Vaughan
Microsoft Research, New York, NY, USA
DOI

10.1145/3313831.3376219

論文URL

https://doi.org/10.1145/3313831.3376219

Mental Models of AI Agents in a Cooperative Game Setting
要旨

As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of AI in a cooperative word guessing game. We run think-aloud studies in which people play the game with an AI agent; through thematic analysis we identify features of the mental models developed by participants. In a large-scale study we have participants play the game with the AI agent online and use a post-game survey to probe their mental model. We find that those who win more often have better estimates of the AI agent's abilities. We present three components for modeling AI systems, propose that understanding the underlying technology is insufficient for developing appropriate conceptual models (analysis of behavior is also necessary), and suggest future work for studying the revision of mental models over time.

受賞
Best Paper
キーワード
Artificial intelligence
mental models
conceptual models
games
word games
AI agents
think-aloud
著者
Katy Ilonka Gero
Columbia University, New York City, NY, USA
Zahra Ashktorab
IBM Research AI, Yorktown Heights, NY, USA
Casey Dugan
IBM Research AI, Cambridge, MA, USA
Qian Pan
IBM Research AI, Cambridge, MA, USA
James Johnson
IBM Research AI, Cambridge, MA, USA
Werner Geyer
IBM Research AI, Cambridge, MA, USA
Maria Ruiz
IBM Watson, Cambridge, MA, USA
Sarah Miller
IBM Watson, Cambridge, MA, USA
David R. Millen
IBM Watson, Cambridge, MA, USA
Murray Campbell
IBM Research AI, Yorktown, NY, USA
Sadhana Kumaravel
IBM Research AI, Yorktown, NY, USA
Wei Zhang
IBM Research AI, Yorktown, NY, USA
DOI

10.1145/3313831.3376316

論文URL

https://doi.org/10.1145/3313831.3376316