Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework

TLDR

Prognosis research seeks to identify factors linked to health outcomes, yet judging the overall quality of evidence in systematic reviews is difficult because primary studies often fail to adhere to established standards, leading to confusion. The authors propose adapting the GRADE framework, originally designed for intervention studies, to assess the quality of prognostic evidence. They modify GRADE by adding six factors that can lower evidence quality—phase of investigation, study limitations, inconsistency, indirectness, imprecision, publication bias—and two that can raise it—moderate or large effect size and exposure–response gradient, with criteria for evaluating each factor’s impact during reviews. These recommendations await further investigation and testing.

Abstract

Prognosis research aims to identify factors associated with the course of health conditions. It is often challenging to judge the overall quality of research evidence in systematic reviews about prognosis due to the nature of the primary studies. Standards aimed at improving the quality of primary studies on the prognosis of health conditions have been created, but these standards are often not adequately followed causing confusion about how to judge the evidence. This article presents a proposed adaptation of Grading of Recommendations Assessment, Development and Evaluation (GRADE), which was developed to rate the quality of evidence in intervention research, to judge the quality of prognostic evidence. We propose modifications to the GRADE framework for use in prognosis research along with illustrative examples from an ongoing systematic review in the pediatric pain literature. We propose six factors that can decrease the quality of evidence (phase of investigation, study limitations, inconsistency, indirectness, imprecision, publication bias) and two factors that can increase it (moderate or large effect size, exposure-response gradient). We describe criteria for evaluating the potential impact of each of these factors on the quality of evidence when conducting a review including a narrative synthesis or a meta-analysis. These recommendations require further investigation and testing.