Red-teaming—where adversarial prompts are crafted to expose harmful behaviors and assess risks—offers a dynamic approach to surfacing underlying stereotypical bias in large language models. Because such subtle harms are best recognized by those with lived experience, involving targets of stereotyping as red-teamers is essential. However, critical challenges remain in leveraging their lived experience for red-teaming while safeguarding psychological well-being. We conducted an empirical study of participatory red-teaming with 20 individuals stigmatized by stereotypes against nonprestigious college graduates in South Korea’s rigid educational meritocracy. Through mixed-methods analysis, we found participants transformed experienced discrimination into strategic expertise for identifying biases, while facing psychological costs such as stress and negative reflections on group identity. Notably, red-team participation enhanced their sense of agency and empowerment through their role as guardians of the AI ecosystem. We discuss the implications for designing participatory red-teaming that prioritizes both the ethical treatment and the empowerment of stigmatized groups.
Bystander intervention, or upstanding, is an effective antidote to cyberbullying, but entails many challenges (e.g. self-efficacy, not knowing what to do or how to upstand). Through two studies, this paper investigates collaborative upstanding, examining how a conversational partner (human or AI) can guide bystanders through these challenges in-situ. In a paired role-play study (n=24), we found that bystanders faced significant challenges in how to intervene. Even after deciding to act, how-to challenges often reignited doubts about their self-efficacy and responsibility. Using these insights, we designed ConCUR, a chatbot that (1) encourages bystanders to co-author an upstanding message, leading them to confront how-to challenges sooner, and (2) addresses how-to challenges simultaneously with other challenges that are introduced through a flexible process. Our second study (n=20) suggests such a chatbot is effective in promoting upstanding behavior in the lab setting. We discuss the implications of in-situ collaborative upstanding to upstanding education research, framing upstanding as an iterative and flexible process rather than sequential.
With generative AI enabling easier production of sexually abusive content, deepfake sexual abuse has intensified, making anyone with visual data be a potential victim or perpetrator. Current moderation systems for non-consensual intimate imagery (NCII) are platform-centric, reactive, and poorly aligned with the workflows of real-time monitors and survivor supporters. To address this gap, we held participatory design workshops with 10 activists affiliated with victim advocacy and survivors experienced in combating deepfake sexual abuse in South Korea. Their insights revealed distinctive challenges, including ambiguity in content classification, barriers to evidence collection, and increased workloads and safety risks during monitoring. Participants suggested features for proactive protection, long-term case tracking, and cross-platform coordination, while emphasizing the need for conversations about data ownership and platform accountability. Based on these findings, we discuss design implications for system and policy that foster multi-stakeholder collaboration to prevent harm, strengthen cross-platform response, and reduce secondary trauma for activists.
Counterspeech offers a non-repressive approach to moderate hate speech in online communities. Research has examined how counterspeech chatbots restrain hate speakers and support targets, but their impact on bystanders remains unclear. Therefore, we developed a counterspeech strategy framework and built \textit{Civilbot} for a mixed-method within-subjects study. Bystanders generally viewed Civilbot as credible and normative, though its shallow reasoning limited persuasiveness. Its behavioural effects were subtle: when performing well, it could guide participation or act as a stand-in; when performing poorly, it could discourage bystanders or motivate them to step in. Strategy proved critical: cognitive strategies that appeal to reason, especially when paired with a positive tone, were relatively effective, while mismatch of contexts and strategies could weaken impact. Based on these findings, we offer design insights for mobilizing bystanders and shaping online discourse, highlighting when to intervene and how to do so through reasoning-driven and context-aware strategies.
Non-consensual intimate imagery (NCII), also known as image-based sexual abuse (IBSA), is mediated through online platforms. Victim-survivors must turn to platforms to collect evidence and request content removal. Platforms act as the crime scene, judge, and jury, determining whether perpetrators face consequences and if harmful material is removed. We present a study of NCII victim-survivors' online reporting experiences, drawing on trauma-informed interviews with 13 participants. We find that platform reporting processes are hostile, opaque, and ineffective, often forcing complex harms into narrow interfaces, responding inconsistently, and failing to result in meaningful action. Leveraging institutional betrayal theory, we show how platforms' structures and practices compound harm, and, in doing so, surface concrete intervention points for redesigning reporting systems and shaping policy to better support victim-survivors.
Short-form video platforms have become major channels for misinformation, with their rich multimodal features making false claims highly believable. HCI research shows that providing corrections in the same modality as the misinformation can be an effective solution. However, since corrections and misinformation convey contradicting information, the order in which one is exposed to them can impact what one believes. We conducted a between-subjects mixed-methods experiment where participants (N=120) rated the credibility of misinformation statements before and after viewing misinformation videos paired with correction videos. Corrections were shown either before, during, or after misinformation. Across all three timings, corrections reduced belief in misinformation, but post-exposure corrections proved most effective and mid-exposure corrections least effective. These findings suggest that correction mechanisms should appear after misinformation exposure, while avoiding mid-exposure interruptions that reduce impact. We outline design recommendations for integrating correction videos into short-form video platforms to improve resilience against misinformation.
Hate speech remains a pressing challenge on social media, where platform moderation often fails to protect targeted users. Personal moderation tools that let users decide how content is filtered can address some of these shortcomings. However, it remains an open question on which screens (e.g., the comments, the reels tab, or the home feed) users want personal moderation and which features they value most. To address these gaps, we conducted a three-wave Delphi study with 40 activists who experienced hate speech. We combined quantitative ratings and rankings with open questions about required features. Participants prioritized personal moderation for conversational and algorithmically curated screens. They valued features allowing for reversibility and oversight across screens, while input-based, content-type specific, and highly automated features are more screen specific. We discuss the importance of personal moderation and offer user-centered design recommendations for personal moderation on Instagram.