Small, Medium, Large? A Meta-Study of Effect Sizes at CHI to Aid Interpretation of Effect Sizes and Power Calculation

要旨

Statistical reporting, especially of effect sizes, is at the root of many methodological issues in quantitative research at CHI. Effect sizes are necessary for assessing practical relevance of results, a-priori power analysis, and meta-analyses, but currently, they are often not reported. Interpretations in the context of the study and the research field are also rare. To aid to researchers in reporting and contextualizing their effect sizes within their research field as well as choosing effect sizes for power analysis, we conducted a meta-study of quantitative CHI papers. We extracted statistics from all quantitative CHI papers published between 2019-2023 (N=1692). Based on effect sizes and the papers' CCS categories, we present effect size distributions in 12 CHI research fields. Through an additional qualitative analysis of 67 quantitative CHI'23 publications, we identify five categories of approaches that researchers take when interpreting effect size: Comparing test-specific values, assigning size labels, using a statistical or methodological reference frame, comparing different observations and interpreting for the big picture.

著者
Anna-Marie Ortloff
University of Bonn, Bonn, Germany
Florin Martius
University of Bonn, Bonn, Germany
Mischa Meier
Fraunhofer FKIE, Bonn, Germany
Theo Raimbault
University of Bonn, Bonn, Germany
Lisa Geierhaas
University of Bonn, Bonn, Germany
Matthew Smith
University of Bonn, Bonn, Germany
DOI

10.1145/3706598.3713671

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713671

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: HCI Methods

G416+G417
7 件の発表
2025-04-28 20:10:00
2025-04-28 21:40:00
日本語まとめ
読み込み中…