A methodology for objectively evaluating error messages

Barbara S. Isa, James M. Boyle, Alan S. Neal, Roger M. Simons

1983 · 13 citations · 1 references

DOIFull text

Open access

Concepts

TL;DR

Message quality critically influences user acceptance and can reduce development time, cost, and aid user learning. The Message Test Program, adaptable to various product contexts, allows evaluation of error messages even before code is available, supporting batch, interactive, stand‑alone, online help, and reference‑manual scenarios. The methodology enables controlled usability evaluations that objectively assess error messages and deliver specific feedback on user difficulties and improvement opportunities.

Abstract

Message quality is a critical factor in influencing user acceptance of a program product. Good error messages can reduce the time and cost to create and maintain software, as well as help users learn about the product. We have developed a methodology for conducting controlled usability evaluations of error messages. The Message Test Program is easily modified to adapt to different product situations, and messages can be evaluated even before working code exists. The Message Test Program can be used to test error messages for a batch product, as well as messages for an interactive product. It can also be used for stand-alone messages, for products that offer on-line help, or messages that provide additional information in a reference manual. Message testing enables us to objectively evaluate error messages and provide specific feedback about the difficulties users encounter and how error messages can be improved.

References

1