VI-RADS: Multiinstitutional Multireader Diagnostic Accuracy and Interobserver Agreement Study

Abstract

<b>BACKGROUND.</b> The Vesical Imaging Reporting and Data System (VI-RADS), based on multiparametric MRI (mpMRI), was developed to provide accurate information for the diagnosis of muscle-invasive bladder cancer (MIBC). <b>OBJECTIVE.</b> The purpose of our study was to evaluate the interobserver agreement and diagnostic performance of VI-RADS among readers with different levels of experience. <b>METHODS.</b> This retrospective study included 91 consecutive patients who underwent mpMRI before transurethral resection of bladder tumor (TURBT) from July 2010 through August 2018. After attending a training session, seven radiologists (five radiologists experienced in bladder MRI and two inexperienced radiologists) reviewed and scored all MRI examinations according to VI-RADS. The interobserver agreement was assessed by kappa statistics. ROC analysis was used to evaluate the diagnostic performance for MIBC. AUCs were estimated. <b>RESULTS.</b> Among 91 patients (72 men and 19 women; mean age ± SD, 73.2 ± 10.2 years), 48 (52.7%) had MIBC and 43 (47.3%) had non-muscle-invasive bladder cancer. Sixty-eight patients were treated with TURBT, and 23 were treated with radical cystectomy. Interobserver agreement was moderate to substantial (κ = 0.60-0.80) among the experienced readers, substantial (κ = 0.67) between the two inexperienced readers, and moderate to substantial (κ = 0.55-0.75) between the experienced and inexperienced readers. The pooled AUC was 0.88 (range, 0.82-0.91) for experienced readers and 0.84 (range, 0.83-0.85) for inexperienced readers, and 0.87 for all readers. Using a VI-RADS score of 4 or greater as the cutoff value for MIBC, the pooled sensitivity and specificity were 74.1% (range, 66.0-80.9%) and 94.1% (range, 88.6-97.7%) for experienced readers and 63.9% (range, 59.6-68.1%) and 86.4% (range, 84.1-88.6%) for inexperienced readers. Using a VI-RADS score of 3 or greater as the cutoff value, the pooled sensitivity and specificity were 83.4% (range, 80.9-85.1%) and 77.3% (range, 61.4-88.6%) for experienced readers and 82.0% (range, 80.9-83.0%) and 73.9% (range, 72.7-75.0%) for inexperienced readers. <b>CONCLUSION.</b> We observed moderate to substantial interobserver agreement and a pooled AUC of 0.87 among radiologists of different levels of expertise using VI-RADS. <b>CLINICAL IMPACT.</b> VI-RADS could help determine the depth and range of excision in TURBT, decreasing the risk of complications and enhancing the accuracy of pathologic diagnosis.