Towards Generating UI Design Feedback with LLMs

Abstract

Feedback on user interface (UI) mockups is crucial for the design process, and designers often seek and leverage feedback to improve their UIs. However, human feedback is not always readily available. Given the recent emergence of LLMs, which have been shown to be proficient in rule-based reasoning, we explore the potential of LLMs to provide feedback automatically. In particular, we investigate automating heuristic evaluation, which currently entails a human expert assessing how well a UI adheres to a given set of design guidelines. We build an LLM-based heuristic evaluation plugin for Figma, which designers can use to evaluate their UI mockups. The plugin queries the LLM with the guidelines and a JSON representation of the UI mockup and then renders the identified guideline violations as constructive suggestions for design improvements. Future work is needed to study what types of usability problems can be successfully identified by LLM-driven heuristic evaluation.