IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use

TLDR

Recent research at IBM has focused on subjective usability measurement, evaluating the psychometric properties of questionnaires for scenario‑based usability evaluation at both global and scenario levels. The article aims to discuss the psychometric characteristics of IBM usability questionnaires. It also provides the questionnaires along with administration and scoring instructions. The ASQ shows excellent internal consistency (α .90–.96), the PSSUQ/CSUQ has α .97, and factor analysis reveals three reliable factors—System Usefulness, Information Quality, and Interface Quality—confirming the questionnaire’s structure and supporting its use for measuring user satisfaction in scenario‑based studies.

Abstract

This article describes recent research in subjective usability measurement at IBM, focused on evaluating the psychometric properties of questionnaires designed for use in scenario‐based usability evaluation. The questionnaires address evaluation at both a global overall system level and at a more detailed scenario level. The primary goals of this article are to (a) discuss the psychometric characteristics of IBM questionnaires that measure user satisfaction with computer system usability, and (b) provide the questionnaires, with administration and scoring instructions. For scenario‐level measurement, the 3‐item After‐Scenario Questionnaire (ASQ) has excellent internal consistency, with coefficient alphas across a set of scenarios ranging from .90 to .96. For more global assessment, the Post‐Study System Usability Questionnaire (PSSUQ) also has excellent internal consistency, with an overall coefficient alpha of .97. Preliminary principal factor analysis of 48 PSSUQ questionnaires suggested the presence of three factors named, after varimax rotation, System Usefulness, Information Quality, and Interface Quality, with corresponding coefficient alphas of .96, .91, and .91. Evaluation of 377 PSSUQ questionnaires (modified to allow mailing to respondents in their offices and referred to as the Computer System Usability Questionnaire, or CSUQ) confirmed the structure of the preliminary principal factor analysis. Consequently, usability practitioners can use these questionnaires to help them measure users’ satisfaction with the usability of computer systems in the context of scenario‐based usability studies.