Designing a test suite for empirically-based middleware performance prediction

Abstract

One of the major problems in building large-scale enterprise systems is anticipating the performance of the eventual solution before it has been built. This problem is especially germane to modern Internet-based e-business applications, where failure to provide high performance and scalability can lead to application and business failure. The fundamental software engineering problem is compounded by many factors, including application diversity, architectural trade-offs and options, COTS component integration requirements, and differences in performance of various software and hardware infrastructures. In the ForeSight project, a practical solution to this problem, based on empirical testing is being investigated. The approach constructs useful models that act as predictors of the performance and the effects of architectural trade-offs for component-based systems such as CORBA, COM+ and J2EE. This paper focuses on describing the issues involved in designing and executing a test suite that is efficient to characterize the behavior and performance profile of a J2EE application server product. The aims of the test suite are described, along with its design and some illustrative empirical results to show it's effectiveness.