Expressing design intent using natural language prompts requires designers to verbalize the ambiguous visual details concisely, which can be challenging or even impossible. To address this, we introduce Brickify, a visual-centric interaction paradigm — expressing design intent through direct manipulation on design tokens. Brickify extracts visual elements (e.g., subject, style, and color) from reference images and converts them into interactive and reusable design tokens that can be directly manipulated (e.g., resize, group, link, etc.) to form the visual lexicon. The lexicon reflects users’ intent for both what visual elements are desired and how to construct them into a whole. We developed Brickify to demonstrate how AI models can interpret and execute the visual lexicon through an end-to-end pipeline. In a user study, experienced designers found Brickify more efficient and intuitive than text-based prompts, allowing them to describe visual details, explore alternatives, and refine complex designs with greater ease and control.
Although awareness of and urgency around the environmental impact of energy consumption in digital infrastructures such as data centers are gradually increasing, many academic efforts still struggle to translate research into practical, real-world applications for reducing digital carbon footprints. Recent studies have highlighted incorporating environmental interventions such as sustainable interaction design (SID) into digital product development practices holds significant potential to reduce their carbon footprint, but integrating sustainability perspectives into everyday design and development practices remains limited in the industry. In this study, we report on the results of in-depth interviews with eight practitioners who have attempted to embed environmental interventions into their practices, capturing their experiences that highlight complex challenges and motivational enablers within the organizational context. Based on these findings, we propose implications for the broader engagement in sustainability-centered design and development practices that resonate with the organizational complexities.
With recent advancements in the capabilities of Text-to-Image (T2I) AI models, product designers have begun experimenting with them in their work. However, T2I models struggle to interpret abstract language and the current user experience of T2I tools can induce design fixation rather than a more iterative, exploratory process. To address these challenges, we developed Inkspire, a sketch-driven tool that supports designers in prototyping product design concepts with analogical inspirations and a complete sketch-to-design-to-sketch feedback loop. To inform the design of Inkspire, we conducted an exchange session with designers and distilled design goals for improving T2I interactions. In a within-subjects study comparing Inkspire to ControlNet, we found that Inkspire supported designers with more inspiration and exploration of design ideas, and improved aspects of the co-creative process by allowing designers to effectively grasp the current state of the AI to guide it towards novel design intentions.
Product design is an iterative process involving several kinds of drawing techniques. Analytic drawing, which involves the use of guidelines or scaffolds to draw the object's shape curves, aids in achieving precision and accuracy. Freehand drawing allows designers to add details without guidance. The set of scaffold, shape, and detail curves are heavily interrelated. As a result, once a draft set of curves is completed, modifications are extremely difficult. This impedes iterative exploration.
We propose to use scaffold manipulation in virtual reality to assist designers in exploring and modifying their product designs. Our key insight is that the same scaffolds designers create for analytic drawing provide an intuitive set of handles. Given a scaffolded 3D product sketch as input, our VR-based system allows designers to directly manipulate the scaffold lines and add detail strokes in any order. Whenever scaffold lines are edited, our system solves for a scaffold line configuration that preserves inter-scaffold relationships. The shape and detail curves are then deformed to match the new scaffold lines. This allows exploratory product design in which a simple template scaffolded 3D drawing is modified and detailed---and further modified---to create a variety of designs. We validated our approach with professional product designers.
A design method card deck helps designers understand and provoke thinking by presenting each method in a simple format and allow designers to switch between methods seamlessly by maintaining the same simple format across the deck. However, recent observations have shown designers hesitate to use a card deck due to the lack of support, while other tools have provided identified support with generative AI. Through a formative study, we identified the specific support designers need when applying the design method cards and intentions in integrating generative AI. Accordingly, we developed the intelligent design method card deck, I-Card, which integrates generative AI to provide applicable design methods, design knowledge and data support, and interactive and dynamic support. A user study demonstrates that I-Card improved the design efficiency and applicability by offering personalized guidance, enhanced decision-making with comprehensive data generation and provided more design inspiration via interactive support.
Generative AI has enabled novice designers to quickly create professional-looking visual representations for product concepts. However, novices have limited domain knowledge that could constrain their ability to write prompts that effectively explore a product design space. To understand how experts explore and communicate about design spaces, we conducted a formative study with 12 experienced product designers and found that experts — and their less-versed clients — often use visual references to guide co-design discussions rather than written descriptions. These insights inspired DesignWeaver, an interface that helps novices generate prompts for a text-to-image model by surfacing key product design dimensions from generated images into a palette for quick selection. In a study with 52 novices, DesignWeaver enabled participants to craft longer prompts with more domain-specific vocabularies, resulting in more diverse, innovative product designs. However, the nuanced prompts heightened participants' expectations beyond what current text-to-image models could deliver. We discuss the implications of AI-based product design support tools.
Product metaphors, which involve creating products that convey meaning through metaphorical associations, are a powerful tool in product design. However, according to our formative study, novice designers often struggle to establish coherent links between target and source, to manage the complexity of diverse mapping possibilities and to balance product usability with metaphorical expression. To address these challenges, we introduce ProductMeta, a creativity support tool designed to support novice designers in exploring and developing metaphorical product designs. ProductMeta incorporates domain knowledge and decomposes the design process into iterative modules and framework-based interfaces, fostering both divergent and convergent thinking. Through user studies, we demonstrate that ProductMeta enables novice designers to generate diverse and contextually relevant design ideas by facilitating structured exploration. We conclude with design implications for human-AI co-creation.