2023 · 35 citations · 3 references
ArtsDesignInteraction DesignInteractive DesignUser Experience DesignInterface DesignRapid PrototypingExploratory Case StudyFramesInteraction PrototypingInterface PrototypingMock-up CreationPrototypingPrototyping ToolsUser Interface Prototyping
Large Language Models have enabled novices without machine learning (ML) experience to quickly prototype ML functionalities with prompt programming. This paper investigates incorporating prompt-based prototyping into designing functional user interface (UI) mock-ups. To understand how infusing LLM prompts into UI mock-ups might affect the prototyping process, we conduct a exploratory study with five designers, and find that this capability might significantly speed up creating functional prototypes, inform designers earlier on how their designs will integrate ML, and enable user studies with functional prototypes earlier. From these findings, we built PromptInfuser, a Figma plugin for authoring LLM-infused mock-ups. PromptInfuser introduces two novel LLM-interactions: input-output, which makes content interactive and dynamic, and frame-change, which directs users to different frames depending on their natural language input. From initial observations, we find that PromptInfuser has the potential to transform the design process by tightly integrating UI and AI prototyping in a single interface.
3
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Qian Yang, Aaron Steinfeld, Carolyn Penstein Rosé et al. · 2020 · 573 citations · Full text
Computing, Computer Science, Design +13
Planning for Natural Language Failures with the AI Playbook
Matthew Hong, Adam Fourney, Derek DeBellis et al. · 2021 · 55 citations