Reliability and Validity of the Acute Care Confidence Survey

Abstract

Purpose: Presently, no standard measurement tool exists to assess students' self-confidence and predict student performance on acute care and inpatient clinical experiences. This pilot study was designed to develop and evaluate the construct validity, reliability (test-retest, internal consistency) and responsiveness of a new tool to assess students' self-confidence as a measure of preparedness for inpatient clinical experiences and to determine whether students' confidence measured by this instrument can be a predictor of student performance in the inpatient setting. Subjects: A total of 11 clinical experts in physical therapy (mean age, 36.5 years; mean years in practice, 11.5; 9% with ABPTS certification) participated in the survey development. A convenience sample of 102 DPT students from a single university (24 men and 78 women), mean age 22.8 years (range, 21.5-29.5 years), participated in the study. Methods: Focus groups were held with clinical experts to review the survey content before reliability and validity testing. Once content validity was established, students completed a baseline survey and internal consistency was assessed. A second (1 week later) and third (10 months later) administration of the survey was performed before clinical placements for analyses of aspects of test-retest reliability (n = 81) and responsiveness (n = 58). Clinical performance scores of 23 students were compared with their Acute Care Confidence Survey (ACCS) scores to evaluate responsiveness. Results: The final ACCS consists of 15 items divided into 4 subscales. No floor or ceiling effects (≤15%) were found on subscale levels at baseline or retest administrations. An exploratory factor analysis of the final version showed eigenvalues: F1 value was 6.935; for the instruct subscale, F1 value was 1.602; for the judgment subscale, F1 value was 3.255; for the manual subscale, F1 value was 1.190; and for the mobility subscale, F1 value was 3.015. Sufficient homogeneity was found for the full scale (α = 0.91) and 3 subscales (α = 0.75, instruct; α = 0.86, judgment; and α = 0.83, mobility) but not for the manual subscale (α = 0.22). Test-retest reliability was excellent for all subscales (intraclass correlation coefficient, 0.78-0.91). A low-to-moderate correlation was found between midterm Clinical Performance Instrument scores and overall ACCS scores (r = 0.5; P = .02). Low-to-moderate correlations were also found for 4 individual questions: judgment regarding appropriate bed mobility instruction (r = 0.56; P = .006) judgment regarding treatment frequency (r = 0.6; P = .0007), judgment regarding assistance with transfers (r = 0.5; P = .011), and ability to justify physical therapy to a physician (r = 0.45; P = .033). Conclusion: The ACCS may be a valid and reliable tool to measure students' self-confidence in the inpatient setting. Students with higher scores on specific ACCS subscales were determined to have increased performance on inpatient clinical placements than students with lower ACCS scores. Further testing is needed.