The Evolution of Benchmarking as a Computer Performance Evaluation Technique

Abstract

The practice of benchmarking as an assessment tool in computer performance evaluation is traced in this article from the early 1960s to the present. Included here are discussions of standard benchmark problems (used in Standard EDP Reports), application benchmarks, workload mapping, synthetic jobs, and the standard benchmark library concept. Advantages and disadvantages of the various techniques are given, along with a discussion of critical issues raised during the past two decades.