Much of the research in online moderation focuses on punitive actions. However, emerging research has shown that positive reinforcement is effective at encouraging desirable behavior on online platforms. We extend this research by studying the ``creator heart'' feature on YouTube, quantifying their primary effects on comments that receive hearts and on videos where hearts have been given out by creators. Overall, creator hearts increased creator agency over feed presentation in YouTube comments sections, and also served as an incentive mechanism to drive user engagement. We find that creator hearts increased the visibility of comments, and increased the amount of positive engagement they received from other users. We also find that the presence of a creator-hearted comment soon after a video is published can incentivize viewers to comment, increasing the total engagement with the video over time. We discuss how creators can use hearts to shape behavior in their communities by highlighting, rewarding, and incentivizing desired behaviors from users. We discuss avenues for extending our study to understanding positive signals from moderators and curators on other platforms.
https://dl.acm.org/doi/10.1145/3706598.3713521
Recent advancements in AI have significantly enhanced collaboration between humans and writing assistants. However, empirical evidence is still lacking on how this collaboration unfolds in scientific writing, especially considering the variety of tools researchers can use nowadays. We conducted observations and retrospective interviews to investigate how 19 computer science researchers collaborated with intelligent writing assistants while working on their ongoing projects. We adopted a design-in-use lens to analyze the collected data, exploring how researchers adapt writing assistants during their use to overcome challenges and meet their specific needs and preferences. Our findings identify issues such as workflow disruptions and over-reliance on AI, and reveal five distinct design-in-use styles---teaching, resisting, repurposing, orchestrating, and complying---each consisting of different practices used by researchers. This study contributes to understanding the evolving landscape of human-AI co-writing in scientific research and offers insights for designing more effective writing assistants.
https://dl.acm.org/doi/10.1145/3706598.3713205
Social media platform design often incorporates explicit signals of positive feedback. Some moderators provide positive feedback with the goal of positive reinforcement, but are often unsure of their ability to actually influence user behavior. Despite its widespread use and theory touting positive feedback as crucial for user motivation, its effect on recipients is relatively unknown. This paper examines how positive feedback impacts Reddit users and evaluates its differential effects to understand who benefits most from receiving positive feedback. Through a causal inference study of 11M posts across 4 months, we find that users who received positive feedback made more frequent (2\% per day) and higher quality (57\% higher score; 2\% fewer removals per day) posts compared to a set of matched control users. Our findings highlight the need for platforms, communities, and moderators to expand their perspective on moderation and complement punitive approaches with positive reinforcement strategies to foster desirable behavior online.
https://dl.acm.org/doi/10.1145/3706598.3713830
Existing tools for laypeople to create personal classifiers often assume a motivated user working uninterrupted in a single, lengthy session. However, users tend to engage with social media casually, with many short sessions on an ongoing, daily basis. To make creating personal classifiers for content curation easier for such users, tools should support rapid initialization and iterative refinement. In this work, we compare three strategies---(1) example labeling, (2) rule writing, and (3) large language model (LLM) prompting---for end users to build personal content classifiers. From an experiment with 37 non-programmers tasked with creating personalized moderation filters, we found that participants preferred different initializing strategies in different contexts, despite LLM prompting's better performance. However, all strategies faced challenges with iterative refinement. To overcome challenges in iterating on their prompts, participants even adopted hybrid approaches such as providing examples as in-context examples or writing rule-like prompts.
https://dl.acm.org/doi/10.1145/3706598.3713691
Many communities, including the scientific community, develop implicit writing norms. Understanding them is crucial for effective communication with that community. Writers gradually develop an implicit understanding of norms by reading papers and receiving feedback on their writing. However, it is difficult to both externalize this knowledge and apply it to one's own writing. We propose two new writing support concepts that reify document and sentence-level patterns in a given text corpus: (1) an ordered distribution over section titles and (2) given the user's draft and cursor location, many retrieved contextually relevant sentences. Recurring words in the latter are algorithmically highlighted to help users see any emergent norms. Study results (N=16) show that participants revised the structure and content using these concepts, gaining confidence in aligning with or breaking norms after reviewing many examples. These results demonstrate the value of reifying distributions over other authors’ writing choices during the writing process.
https://dl.acm.org/doi/10.1145/3706598.3713974
LLM-based applications are helping people write, and LLM-generated text is making its way into social media, journalism, and our classrooms. However, the differences between LLM-generated and human-written text remain unclear. To explore this, we hired professional writers to edit paragraphs in several creative domains. We first found these writers agree on undesirable idiosyncrasies in LLM-generated text, formalizing it into a seven-category taxonomy (e.g. clichés, unnecessary exposition). Second, we curated the LAMP corpus: 1,057 LLM-generated paragraphs edited by professional writers according to our taxonomy. Analysis of LAMP reveals that none of the LLMs used in our study (GPT4o, Claude-3.5-Sonnet, Llama-3.1-70b) outperform each other in terms of writing quality, revealing common limitations across model families. Third, building on existing work in automatic editing we evaluated methods to improve LLM-generated text. A large-scale preference annotation confirms that although experts largely prefer text edited by other experts, automatic editing methods show promise in improving alignment between LLM-generated and human-written text.
https://dl.acm.org/doi/10.1145/3706598.3713559
This work sheds light on whether and how creative writers' needs are met by existing research and commercial writing support tools (WST). We conducted a need finding study to gain insight into the writers' process during creative writing through a qualitative analysis of the response from an online questionnaire and Reddit discussions on \textit{r/Writing}. Using a systematic analysis of 115 tools and 67 research papers, we map out the landscape of how digital tools facilitate the writing process. Our triangulation of data reveals that research predominantly focuses on the writing activity and overlooks pre-writing activities and the importance of visualization. We distill 10 key takeaways to inform future research on WST and point to opportunities surrounding underexplored areas. Our work offers a holistic and up-to-date account of how tools have transformed the writing process, guiding the design of future tools that address writers' evolving and unmet needs.
https://dl.acm.org/doi/10.1145/3706598.3713161