Professional writing skills are essential for crafting job application materials where applicants showcase their qualifications to recruiters and employers. Lettersmith is a digital tool that supports writing through scaffolded annotation, an instructional approach combining an expert-informed checklist, annotated examples, and self-tagging. We evaluated the efficacy of the instructional mechanisms that make up scaffolded annotation, as well as the use of ChatGPT, in facilitating writing cognitive processes and writing quality. Through a lab experiment with 146 first-year college students writing and revising a cover letter, we found that the combined mechanisms of scaffolded annotation within Lettersmith promoted a stronger understanding of the writing genre. Specifically, the use of a checklist combined with another writing support, like an example or self-tagging, was particularly effective for improving writing quality. Unstructured use of ChatGPT did not improve writing cognitive processes or writing quality more than Lettersmith.
ACM CHI Conference on Human Factors in Computing Systems