Sensors in offices mainly measure environmental data, missing qualitative insights into office workers' perceptions. This opens the opportunity for active individual participation in data collection. To promote reflection on office well-being while overcoming experience sampling challenges in terms of privacy, notification, and display overload, and in-the-moment data collection, we developed Click-IO. Click-IO is a tangible, privacy-sensitive, mobile experience sampling tool that collects contextual information. We evaluated Click-IO for 20-days. The system enabled real-time reflections for office workers, promoting self-awareness of their environment and well-being. Its non-digital design ensured privacy-sensitive feedback collection, while its mobility facilitated in-the-moment feedback. Based on our findings, we identify design recommendations for the development of mobile experience sampling tools. Moreover, the integration of contextual data with environmental sensor data presented a more comprehensive understanding of individuals' experiences. This research contributes to the development of experience sampling tools and sensor integration for understanding office well-being.
doi.org/10.1145/3613904.3642295
HCI research applies ethnographic methods to understand and represent practices that involve the use of interactive systems. A subdomain of this work is interpretivist ethnography, which positions the researcher’s perspectival view [37] as central to ethnographic research and its epistemic contribution. Given this we ask: How might ethnographic researchers in HCI surface the meaning-making role of their subjectivities in research? We reflect on our prior ethnographic fieldwork on small-scale sustainable farms in Indianapolis, Indiana to bring the ethnographic “I” into focus by articulating our reflections as “impressionist tales'' [64:101-124]. We ground this pursuit in sociologist Andrea Doucet’s concept of “gossamer walls” to surface researcher’s three reflexive relationships 1) with herself; 2) with participants; and 3) with her epistemic communities [34]. We build on and contribute to postmodern ethnography in HCI to clarify the epistemic virtues and methodological best practices of a more unapologetically subjective ethnographic practice in HCI.
doi.org/10.1145/3613904.3642727
Researchers are increasingly facilitating qualitative research studies online. While this has made research more accessible for participation, there have been notable encounters with “fraudulent” participants. By fraudulent, we refer to individuals who are deceptive about meeting the inclusion criteria, their identity, or experiences. Fraudulent participants have generated new challenges for researchers who have to interact 1:1 with these individuals, face ethical dilemmas on appropriate next steps, diagnose and prevent the issue from happening again, and deal with their own identity as a scholar. In this study, we interview 16 HCI researchers to understand and learn from their experiences. We contribute: (1) an understanding of how HCI qualitative researchers deal with fraudulent participants; (2) a guide for qualitative HCI researchers on how to handle fraudulence; and (3) a reflection on how the HCI research community might better improve our science and training efforts.
doi.org/10.1145/3613904.3642732
When fielding satisfaction questions, survey platforms offer the option to randomly reverse the response options. In this paper, we provide evidence that the use of this option leads to biased results. In Study 1, we show that reversing vertically oriented response options leads to significantly lower satisfaction ratings – from 90 to 82 percent in our case. Study 2 had survey respondents verify their response and found that on a reversed scale, the very-dissatisfied option was selected unintentionally in about half of the cases. The cause, shown by Study 3, is that survey respondents expect the positive option at the top and do not always pay sufficient attention to the question, combined with the similar spelling of satisfied and dissatisfied. To prevent unintentional responses from biasing the results, we recommend keeping the positive option at the top in vertically-oriented scales with visually-similar endpoint labels.
doi.org/10.1145/3613904.3642397
The Experience Sampling Method (ESM) is widely used to collect emotion self-reports to train machine learning models for emotion inference. However, as ESM studies are time-consuming and burdensome, participants often withdraw in between. This unplanned withdrawal compels the researchers to discard the dropout participants’ data, significantly impacting the quality and quantity of the self-reports. To address this problem, we leverage only the self-reporting similarity across participants (unlike prior works that apply different machine learning approaches on additional modalities) for missing self-report estimation. In specific, we propose a Multi-task Learning (MTL) framework, MUSE, that constructs the missing self-reports of the dropout participants. We evaluate MUSE in two in-the-wild studies (N1=24, N2=30) of 6-week and 8-week duration, during which the participants reported four emotions (happy, sad, stressed, relaxed) using a smartphone application. The evaluation reveals that MUSE estimates the missing emotion self-reports with an average AUCROC of 84% (Study I) and 82% (Study II). A follow-up evaluation of MUSE for an emotion inference (downstream) task reveals no significant difference in emotion inference performance when estimated self-reports are used. These findings underscore the utility of MUSE in estimating missing self-reports in ESM studies and the applicability of MUSE for downstream tasks (e.g., emotion inference).
doi.org/10.1145/3613904.3642833