Improving the quality of the NCQA (National Committee for Quality Assurance) Annual Member Health Care Survey Version 1.0.

Abstract

The National Committee for Quality Assurance (NCQA) developed a standardized survey instrument in 1995 designed to measure enrollee satisfaction with the care and services received from health plans across the United States. After the survey was administered for a large number of health plans and thousands of responses were received, some areas for survey improvement have emerged. The objective of this research was to evaluate the NCQA Annual Member Health Care Survey Version 1.0 (the standard form) relative to an alternate survey form created in cooperation with the HMO Group, Maritz Marketing Research, Inc., and Healthcare Research Systems, Ltd. The alternate form of the NCQA survey was constructed to test several theories of measurement improvement via rewording of items, reordering of items, deletion of items, and addition of items. The most important findings of the research project are reported herein. Ten geographically dispersed member health maintenance organizations (HMOs) of The HMO Group took part in the project. A split-half design was used to test the standard and alternate survey forms concurrently. Surveys were administered by using the NCQA-recommended mail methodology of survey and cover letter, reminder card, and second survey. Assuming a 50% response rate, a target of 400 responses (200 for each survey form) per HMO was planned. The window for responding was allowed to remain open 4 weeks beyond the mailing of the second survey to achieve the desired response rate. A total of 4,056 responses were collected (2,022 for the standard form and 2,034 for the alternate form). It was found that the addition of "No Experience" and "No Opinion" response options to the majority of satisfaction items reduced the random error associated with informed responses and produced statistically significant higher correlations with the global satisfaction items relative to the standard form items. Only four of the eight Short Form 12 summary scales, General Health, Reported Health Transition, Mental Health, and Social Functioning, were useful for adjusting data (covariation). The entire set of comorbidities (chronic disease checklist) could be eliminated without losing significant data adjustment capability. The multiple linear regression models generated by using the global satisfaction items on the alternate form had higher adjusted R2 values than the standard-form models. The alternate-form item Overall Value correlated highly with cost items and general satisfaction item, making it a useful global satisfaction variable for predictive modeling.