Trade-offs for Substituting a Human with an Agent in a Pair Programming Context: The Good, the Bad, and the Ugly


Pair programming has a documented history of benefits, such as increased code quality, productivity, self-efficacy, knowledge transfer, and reduced gender gap. Research uncovered problems with pair programming related to scheduling, collocating, role imbalance, and power dynamics. We investigated the trade-offs of substituting a human with an agent to simultaneously provide benefits and alleviate obstacles in pair programming. We conducted gender-balanced studies with human-human pairs in a remote lab with 18 programmers and Wizard-of-Oz studies with 14 programmers, then analyzed results quantitatively and qualitatively. Our comparative analysis of the two studies showed no significant differences in productivity, code quality, and self-efficacy. Further, agents facilitated knowledge transfer; however, unlike humans, agents were unable to provide logical explanations or discussions. Human partners trusted and showed humility towards agents. Our results demonstrate that agents can act as effective pair programming partners and open the way towards new research on conversational agents for programming.

Honorable Mention
Sandeep Kaur. Kuttal
University of Tulsa, Tulsa, Oklahoma, United States
Bali Ong
University of Tulsa, Tulsa, Oklahoma, United States
Kate Kwasny
University of Tulsa, Tulsa, Oklahoma, United States
Peter Robe
University of Tulsa, Tulsa, Oklahoma, United States




会議: CHI 2021

The ACM CHI Conference on Human Factors in Computing Systems (

セッション: Human, ML & AI

[A] Paper Room 14, 2021-05-10 17:00:00~2021-05-10 19:00:00 / [B] Paper Room 14, 2021-05-11 01:00:00~2021-05-11 03:00:00 / [C] Paper Room 14, 2021-05-11 09:00:00~2021-05-11 11:00:00
Paper Room 14
13 件の発表
2021-05-10 17:00:00
2021-05-10 19:00:00