User engagement (UE) is widely discussed in HCI articles, but its definition, reliability, and application remain elusive. This research conducts a systematic literature review of 241 articles from 1993 to 2023 to analyze how UE is defined and measured within the domain of HCI. Our findings reveal significant definitional inconsistencies that hinder UE’s practical application in HCI research and system design. Based on our findings, we recommend using UE as a categorical label rather than a unified construct until more systematic frameworks are established. We also highlight the need for divergent views of UE across HCI research communities as a valuable avenue to pursue. This divergent view approach can help HCI researchers focus on specific, measurable aspects of UE that align with specific community practices and norms. Our findings also suggest that until such a framework emerges, researchers should be aware of its limitations when using UE as a research construct.
https://dl.acm.org/doi/10.1145/3706598.3713505
Science is a complex system comprised of many scientists who individually make decisions that, due to the size and nature of the academic system, largely do not affect the system as a whole. However, certain decisions at the meso-level of research communities, such as the Human-Computer Interaction (HCI) community, may result in deep and long-lasting behavioral changes in scientists. In this article, we provide empirical evidence on how a change in editorial policies introduced at the ACM CHI Conference in 2016 destabilized the CHI research community and launched it on an expansive path, denoted by a year-by-year increase in the mean number of references included in CHI articles. If this near-linear trend continues undisrupted, an article at CHI 2030 will include on average almost 130 references. The trend toward more citations reflects a citation culture where quantity is prioritized over quality, contributing to both author and peer reviewer fatigue. Our exploratory analysis highlights the profound impact of meso-level policy adjustments on the evolution of scientific fields and disciplines, urging all stakeholders to carefully consider the broader implications of such changes.
How creative is HCI research? Although creativity has been a notable theme in HCI, the landscape of the creativity of HCI research itself remains unclear. In this paper, we address this by measuring the disruptiveness of HCI research, one important dimension distinguishing the level of creativity, through a large-scale data-driven bibliometric analysis. By quantitatively tracing its evolution over the past 40 years, we find that the disruptiveness of HCI is decreasing sharply, even at a faster speed than the global average across all fields. We characterize the patterns shown by the themes, knowledge use, and authorship of disruptive papers in HCI, and identify how they associate with disruptiveness, e.g., the positive relationship between author freshness and disruptiveness. Based on our results, we discuss practical implications to improve and secure disruptiveness and creativity in HCI.
https://dl.acm.org/doi/10.1145/3706598.3713917
Positionality acknowledges that researchers’ subjectivities, values and experiences influence approaches to and outcomes of research. It underlines and promotes self-awareness and explicit demonstration of reflexivity. To understand how positionality is conceptualised and used in HCI, we conducted two studies: (i) a scoping review of positionality and reflexivity statements in CHI papers from the last 11 years and (ii) a survey of HCI researchers (n=75). Our findings show that positionality statements are often viewed as a box-ticking exercise and their influence on the research is seldom discussed. They are also often restricted to more sensitive areas of research and may impact marginalised identities. We argue that positionality statements may be valuable but not as markers of methodological rigour; their content should be at the discretion of authors and methodologically consistent. Our contributions include a current snapshot of positionality in HCI and reflections on its current role and future directions in HCI.
https://dl.acm.org/doi/10.1145/3706598.3713280
Large language models (LLMs) have been positioned to revolutionize HCI, by reshaping not only the interfaces, design patterns, and sociotechnical systems that we study, but also the research practices we use. To-date, however, there has been little understanding of LLMs' uptake in HCI. We address this gap via a systematic literature review of 153 CHI papers from 2020-24 that engage with LLMs. We taxonomize: (1) domains where LLMs are applied; (2) roles of LLMs in HCI projects; (3) contribution types; and (4) acknowledged limitations and risks. We find LLM work in 10 diverse domains, primarily via empirical and artifact contributions. Authors use LLMs in five distinct roles, including as research tools or simulated users. Still, authors often raise validity and reproducibility concerns, and overwhelmingly study closed models. We outline opportunities to improve HCI research with and on LLMs, and provide guiding questions for researchers to consider the validity and appropriateness of LLM-related work.
https://dl.acm.org/doi/10.1145/3706598.3713726
The ACM CHI Conference has a tradition of citing its intellectual heritage. At the same time, we know CHI is highly diverse and evolving. In this highly dynamic context, it is not clear how the CHI community continues to appreciate its milestones (within and outside of CHI). We present an investigation into how the community's citations to milestones have evolved over 43 years of CHI Proceedings (1981-2024). Forgetting curves plotted for each year suggest that milestones are slowly fading from the CHI community's collective memory. However, the picture is more nuanced when we trace citations to the top-cited milestones over time. We identify three distinct types of milestones cited at CHI, a typology of milestone contributions, and define the Milestone Coefficient as a metric to assess the impact of milestone papers on a continuous scale. Further, our findings suggest the potential presence of a Matthew effect at CHI. We discuss the broader ramifications for the CHI community and the field of HCI.
Alternative Research Outcomes (AROs) go beyond traditional academic publications, taking diverse forms such as documentaries, DIY tutorials, or exhibitions. With growing recognition of the need for more inclusive and contextually appropriate research dissemination, AROs are particularly relevant in HCI and design research. Yet, little has been discussed on why it is important to work on AROs. What are key qualities of AROs? How can the HCI community benefit from learning more about creating AROs? By analyzing six case studies, we propose four qualities of AROs and demonstrate how they emerge in the timeline of a research project. We argue AROs can be adapted to diverse audience needs and share research insights that may extend beyond the original research goals. Our work contributes to a deeper understanding of how AROs can support inclusive research dissemination practices, enabling HCI researchers to engage broader audiences and extend the relevance of their work.
https://dl.acm.org/doi/10.1145/3706598.3713884