Conversational agents have been used to support student learning for some time, but the emergence of Large Language Models (LLMs) poses a novel opportunity to enhance their capabilities in collaborative settings. LLM-powered agents can provide timely interventions in collaborative conversations when a teacher is unable to assist the students. However, the use of LLMs in such tools raises many ethical questions and concerns, especially for use with young, impressionable populations. In this work, we present the human-centered design and evaluation of an LLM-based agent aimed to facilitate small group collaboration in middle- and high-school classrooms. Fifty-eight groups of dyads and triads (145 participants), aged 12-17, collaborated in a jigsaw activity and were assigned to be assisted by our agent or not. The results showed decreased self-reported ratings of social loafing and increased use of language related to respectful collaboration in interactions with the agent compared to those without.
https://dl.acm.org/doi/10.1145/3706598.3713349
Students struggle with accurately assessing their own performance, especially given little training to do so. We propose an AI-powered training tool to help students improve “metacognitive calibration,” or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students’ use of metacognition-informed learning behaviors. We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment. The AI-driven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384, p = .019), and this effect was significantly mediated by learning behaviors. Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p = .049). These insights highlight the value of AI-powered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.
https://dl.acm.org/doi/10.1145/3706598.3713960
Teaching literature under interdisciplinary contexts (e.g., science, art) that connect reading materials has become popular in elementary schools. However, constructing such contexts is challenging as it requires teachers to explore substantial amounts of interdisciplinary content and link it to the reading materials. In this paper, we develop LitLinker via an iterative design process involving 13 teachers to facilitate the ideation of interdisciplinary contexts for teaching literature. Powered by a large language model (LLM), LitLinker can recommend interdisciplinary topics and contextualize them with the literary elements (e.g., paragraphs, viewpoints) in the reading materials. A within-subjects study (N=16) shows that compared to an LLM chatbot, LitLinker can improve the integration depth of different subjects and reduce workload in this ideation task. Expert interviews (N=9) also demonstrate LitLinker’s usefulness for supporting the ideation of interdisciplinary contexts for teaching literature. We conclude with concerns and design considerations for supporting interdisciplinary teaching with LLMs.
https://dl.acm.org/doi/10.1145/3706598.3714111
Serious games, particularly board games, have long been employed in production management education to teach various concepts. While they have demonstrated educational effectiveness, their integration with emerging Industry 4.0 technologies remains limited. Furthermore, there is a lack of empirical research on how industry practitioners apply these digitization technologies in the workplace. To bridge this gap, we designed a course that integrates digital technologies into a traditional board game. We conducted two studies to evaluate both knowledge gains within the classroom and knowledge transfer back into the manufacturing industry. Our results show an improved understanding of the synergies between production management principles and Industry 4.0 technologies, as well as the real-world challenges students face when attempting to transfer this knowledge. Our work contributes pedagogical and practical perspectives on how technology-enhanced serious games can extend learning in and beyond the classroom.
https://dl.acm.org/doi/10.1145/3706598.3714169
Although remote learning is widely used for delivering and capturing knowledge, it has limitations in teaching hands-on skills that require nuanced instructions and demonstrations of precise actions, such as massage. Furthermore, scheduling conflicts between instructors and learners often limit the availability of real-time feedback, reducing learning efficiency. To address these challenges, we developed a synthesis tool utilizing an LLM-powered Virtual Teaching Assistant (VTA). This tool integrates multimodal instructions that convey precise data, such as stroke patterns and pressure control, while providing real-time feedback for learners and summarizing their performance for instructors. Our case study with instructors and learners demonstrated the effectiveness of these multimodal instructions and the VTA in enhancing massage teaching and learning. We then discuss the tools' use in other hands-on skills instruction and cognitive process differences in various courses.
https://dl.acm.org/doi/10.1145/3706598.3713677
With the rise of online learning, many novice tutors lack experience engaging students remotely. We introduce TutorUp, a Large Language Model (LLM)-based system that enables novice tutors to practice engagement strategies with simulated students through scenario-based training. Based on a formative study involving two surveys (N1=86, N2=102) on student engagement challenges, we summarize scenarios that mimic real teaching situations. To enhance immersion and realism, we employ a prompting strategy that simulates dynamic online learning dialogues. TutorUp provides immediate and asynchronous feedback by referencing tutor-students online session dialogues and evidence-based teaching strategies from learning science literature. In a within-subject evaluation (N=16), participants rated TutorUp significantly higher than a baseline system without simulation capabilities regarding effectiveness and usability. Our findings suggest that TutorUp provides novice tutors with more effective training to learn and apply teaching strategies to address online student engagement challenges.
https://dl.acm.org/doi/10.1145/3706598.3713589