Generative AI (GenAI) systems offer unprecedented opportunities for transforming professional and personal work, yet present challenges around prompting, evaluating and relying on outputs, and optimizing workflows. We argue that metacognition—the psychological ability to monitor and control one’s thoughts and behavior—offers a valuable lens to understand and design for these usability challenges. Drawing on research in psychology and cognitive science, and recent GenAI user studies, we illustrate how GenAI systems impose metacognitive demands on users, requiring a high degree of metacognitive monitoring and control. We propose these demands could be addressed by integrating metacognitive support strategies into GenAI systems, and by designing GenAI systems to reduce their metacognitive demand by targeting explainability and customizability. Metacognition offers a coherent framework for understanding the usability challenges posed by GenAI, and provides novel research and design directions to advance human-AI interaction.
doi.org/10.1145/3613904.3642902
Bio-inspired design (BID) fosters innovative solutions in engineering by drawing inspiration from biology. Learning BID is crucial for developing multidisciplinary innovation skills of designers and engineers. While current BID education has attempted to enhance learners' understanding and analogical reasoning skills in BID, it often relies much on teachers' expertise. When learners turn to learn independently through some educational tools, there are challenges in understanding and reasoning practice in such complex multidisciplinary environment, as well as evaluating learning outcomes comprehensively. Addressing these challenges, we introduce a Large Language Models (LLMs)-driven BID education method based on a structured ontology, as well as three strategies: enhancing understanding through LLMs-enpowered "learning by asking", assisting reasoning by providing hints and feedback, and assessing learning outcomes through benchmarking against existing BID knowledge. Implementing the method, we developed BIDTrainer, an interactive BID education tool. User studies indicate that learners using BIDTrainer understood BID cases better, reason faster with higher interactivity than the baseline, and BIDTrainer assessed the learning outcomes consistent with experts.
doi.org/10.1145/3613904.3642887
The viral launch of new generative AI (GAI) systems, such as ChatGPT and Text-to-Image (TTL) generators, sparked questions about how they can be effectively incorporated into writing education. However, it is still unclear how teachers, parents, and students perceive and suspect GAI systems in elementary school settings. We conducted a workshop with twelve families (parent-child dyads) with children ages 8-12 and interviewed sixteen teachers in order to understand each stakeholder's perspectives and opinions on GAI systems for learning and teaching writing. We found that the GAI systems could be beneficial in generating adaptable teaching materials for teachers, enhancing ideation, and providing students with personalized, timely feedback. However, there are concerns over authorship, students’ agency in learning, and uncertainty concerning bias and misinformation. In this article, we discuss design strategies to mitigate these constraints by implementing an adults-oversight system, balancing AI-role allocation, and facilitating customization to enhance students’ agency over writing projects.
doi.org/10.1145/3613904.3642438
With the pervasive and evolving use of tracking and AI to make inferences about online platform users, it has become imperative for adolescents---a key demographic using such platforms---to develop a deep understanding of these practices to protect their privacy. Traditionally, K-12 cybersecurity education has largely been confined to extracurricular activities, limiting underrepresented students' access. To resolve this shortcoming, we partnered with a rural-identifying middle school to deliver AI-related privacy education in classrooms. Using Design-Based Research methodology, we identified students' AI-related privacy learning needs and developed six education modules. This paper focuses on the design, classroom implementation, and evaluation of module \#2, covering the privacy threats of Tracking and Pervasive Personalization (TaPP). Student assessment outcomes show they developed transferable foundational knowledge of the privacy implications of tracking and personalization after participating in the TaPP module. Our findings demonstrate the benefits of integrating AI-related privacy education into existing K-12 curricula.
doi.org/10.1145/3613904.3642460
Fostering students' interests in learning is considered to have many positive downstream effects. Large language models have opened up new horizons for generating content tuned to one's interests, yet it is unclear in what ways and to what extent this customization could have positive effects on learning. To explore this novel dimension, we conducted a between-subjects online study (n=272) featuring different variations of a generative AI vocabulary learning app that enables users to personalize their learning examples. Participants were randomly assigned to control (sentence sourced from pre-existing text) or experimental conditions (generated sentence or short story based on users’ text input). While we did not observe a difference in learning performance between the conditions, the analysis revealed that generative AI-driven context personalization positively affected learning motivation. We discuss how these results relate to previous findings and underscore their significance for the emerging field of using generative AI for personalized learning.
doi.org/10.1145/3613904.3642393