Generative AI significantly enhances player agency in interactive narratives (IN) by enabling just-in-time content generation that adapts to player actions. While delegating generation to AI makes IN more interactive, it becomes challenging for authors to control the space of possible narratives - within which the final story experienced by the player emerges from their interaction with AI. In this paper, we present WhatELSE, an AI-bridged IN authoring system that creates narrative possibility spaces from example stories. WhatELSE provides three views (narrative pivot, outline, and variants) to help authors understand the narrative space and corresponding tools leveraging linguistic abstraction to control the boundaries of the narrative space. Taking innovative LLM-based narrative planning approaches, WhatELSE further unfolds the narrative space into executable game events. Through a user study (N=12) and technical evaluations, we found that WhatELSE enables authors to perceive and edit the narrative space and generates engaging interactive narratives at play-time.
We introduce Toyteller, an AI-powered storytelling system where users generate a mix of story text and visuals by directly manipulating character symbols like they are toy-playing. Anthropomorphized symbol motions can convey rich and nuanced social interactions; Toyteller leverages these motions (1) to let users steer story text generation and (2) as a visual output format that accompanies story text. We enabled motion-steered text generation and text-steered motion generation by mapping motions and text onto a shared semantic space so that large language models and motion generation models can use it as a translational layer. Technical evaluations showed that Toyteller outperforms a competitive baseline, GPT-4o. Our user study identified that toy-playing helps express intentions difficult to verbalize. However, only motions could not express all user intentions, suggesting combining it with other modalities like language. We discuss the design space of toy-playing interactions and implications for technical HCI research on human-AI interaction.
Travel is a powerful yet fleeting experience that can shape personal perspectives and support self-reflection. To recapture the essence of travel, we explored the use of VR as a medium for immersive re-experiencing with an emphasis on storytelling. We developed TravelGalleria, a VR authoring tool that allows users to curate personalized digital galleries. TravelGalleria encourages creative expression, enabling users to use audio narration, annotations, spatially arranged photos, and more to recount their travel stories. A probing user study with TravelGalleria (n = 20) showed promising trends toward emotional resonance and introspective learning. Our findings illustrate how our tool supports users in remembering, reliving, and deriving new insights regarding past experiences, as they were able to reconnect with emotions and themes central to their travels. We discuss these findings in the context of meaningful digital experiences and storytelling in reflective digital practices, highlighting design suggestions and open areas for future research.
While several digital tools exist for children’s creative storytelling, few have explored how generative AI can enhance storytelling quality. Our formative research identified five design requirements for AI-powered storytelling tools for elementary students. We developed a system named StoryPrompt that enables children to co-create stories and comics with AI, boosting literacy and creativity. Pilot tests with children and HCI experts demonstrated good usability and positive learning experiences. In a mixed-methods evaluation with 40 children from Grades 2-6, we found that StoryPrompt significantly improved storytelling creativity and richness, compared to the storyboard method. Observations indicated more purposeful planning and strategic use of AI-generated words and images, facilitating efficient exploration of storytelling alternatives. While children preferred AI images, they recognized the limitations in representing storytelling details. Teacher interviews highlighted the system’s motivational potential and classroom flexibility. We discuss the benefits and considerations of using generative AI to enhance creative storytelling for children.
While much prior work on computational visual storytelling analyzes image content, it largely overlooks formal elements. This raises the question: how might particular cinematographic techniques shape a system's interpretation and narration of imagery? To investigate this question, we generate 60 responses from a Vision Language Model using a multi-faceted prompt paired with different still frames from Man with a Movie Camera (1929), a silent documentary film renowned for its innovative cinematography. We present three themes that highlight roles of cinematography in computational visual storytelling: (1) how AI discerns drama and power from camera shots and angles that portray social reality; (2) how AI (mis)interprets lighting and focus techniques that compose ambiguous reality; and (3) how AI navigates visual effects that render surreality. In turn, we look toward cinematic controls to reimagine users as directors of visual storytelling systems and discuss how expressive AI can support speculating about the past.
Generative AI is increasingly used to create images from text, but its role in documentary photography remains under-explored. This paper investigates how generative AI can be integrated into documentary practice while maintaining ethical standards. Through interviews with six documentary photographers, we explored their views on AI’s potential to support community-driven storytelling. While AI presents opportunities for creative expression and community involvement, concerns about trust, authenticity, and decontextualization of images persist. Photographers expressed doubts about AI’s ability to accurately represent lived experiences, fearing it could compromise narrative integrity. Our findings suggest that AI tools should be designed to enhance collaboration and transparency in storytelling, complementing rather than replacing traditional documentary methods. This study contributes to the ongoing discourse on AI in photography, advocating for the development of tools that preserve the ethical foundations of documentary storytelling while empowering communities.
Engaging with people's lived experiences is foundational for HCI research and design. This paper introduces a novel narrative elicitation method to empower people to easily articulate ‘micro-narratives’ emerging from their lived experiences, irrespective of their writing ability or background. Our approach aims to enable at-scale collection of rich, co-created datasets that highlight target populations' voices with minimal participant burden, while precisely addressing specific research questions. To pilot this idea, and test its feasibility, we: (i) developed an AI-powered prototype, which leverages LLM-chaining to scaffold the cognitive steps necessary for users’ narrative articulation; (ii) deployed it in three mixed-methods studies involving over 380 users; and (iii) consulted with established academics as well as C-level staff at (inter)national non-profits to map out potential applications. Both qualitative and quantitative findings show the acceptability and promise of the micro-narrative method, while also identifying the ethical and safeguarding considerations necessary for any at-scale deployments.