InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs

要旨

Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In this paper, we introduce InstructPipe, an AI assistant for prototyping machine learning (ML) pipelines with text instructions. We contribute two large language model (LLM) modules and a code interpreter as part of our framework. The LLM modules generate pseudocode for a target pipeline, and the interpreter renders the pipeline in the node-graph editor for further human-AI collaboration. Both technical and user evaluation (N=16) shows that InstructPipe empowers users to streamline their ML pipeline workflow, reduce their learning curve, and leverage open-ended commands to spark innovative ideas.

受賞
Honorable Mention
著者
Zhongyi Zhou
Google, Tokyo, Japan
Jing Jin
Google, Mountain View, California, United States
Vrushank Phadnis
Google, Mountain View, California, United States
Xiuxiu Yuan
Google, Mountain View, California, United States
Jun Jiang
Google, Sunnyvale, California, United States
Xun Qian
Google, Mountain View, California, United States
Kristen Wright
Google, Mountain View, California, United States
Mark Sherwood
Google, Mountain View, California, United States
Jason Mayes
Google, Mountain View, California, United States
Jingtao Zhou
Google, Mountain View, California, United States
Yiyi Huang
Google, Sunnyvale, California, United States
Zheng Xu
Google Research, Seattle, Washington, United States
Yinda Zhang
Google, Mountain View, California, United States
Johnny Lee
Google, Redmond, Washington, United States
Alex Olwal
Google Inc., Mountain View, California, United States
David Kim
Google Research, Zurich, Switzerland
Ram Iyengar
Google, Mountain View, California, United States
Na Li
Google, Palo Alto, California, United States
Ruofei Du
Google, San Francisco, California, United States
DOI

10.1145/3706598.3713905

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713905

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Programming and Interaction

G304
7 件の発表
2025-05-01 18:00:00
2025-05-01 19:30:00
日本語まとめ
読み込み中…