Evaluating pattern catalogs

Abstract

Patterns and pattern catalogs (pattern languages) have been proposed as a mechanism for re-use. Traditionally, patterns have been used to foster design re-use, and generative design patterns have been used to achieve both design and code re-use. In theory, a pattern catalog could be created and used to provide re-usable patterns within a project and across a group of related projects. This idea raises a natural question. How can we measure the effectiveness of a pattern catalog or compare the effectiveness of different pattern catalogs? In this paper, we define four metrics that can be used to measure the effectiveness of pattern catalogs. We illustrate these metrics by applying them to a case study that uses a pattern catalog of generative design patterns to generate scripting code for computer games. The metrics are general enough to assess any pattern catalog, independent of application domain or whether the patterns are generative or descriptive.