A new system for grading recommendations in evidence based guidelines

TLDR

SIGN produces evidence‑based NHS guidelines through multidisciplinary teams, systematic reviews, and evidence‑linked grading, yet its prior AHRQ‑derived system—designed mainly for RCT‑based effectiveness questions—has been criticized for neglecting study quality, diverse evidence types, and for being misinterpreted by users. The authors introduce a revised system for determining levels of evidence and grades to overcome these limitations. Experience over more than five years of guideline development revealed significant weaknesses in the existing grading system.

Abstract

The Scottish Intercollegiate Guidelines Network (SIGN) develops evidence based clinical guidelines for the NHS in Scotland. The key elements of the methodology are (a) that guidelines are developed by multidisciplinary groups; (b) they are based on a systematic review of the scientific evidence; and (c) recommendations are explicitly linked to the supporting evidence and graded according to the strength of that evidence. Until recently, the system for grading guideline recommendations was based on the work of the US Agency for Healthcare Research and Quality (formerly the Agency for Health Care Policy and Research). 1 2 However, experience over more than five years of guideline development led to a growing awareness of this system's weaknesses. Firstly, the grading system was designed largely for application to questions of effectiveness, where randomised controlled trials are accepted as the most robust study design with the least risk of bias in the results. However, in many areas of medical practice randomised trials may not be practical or ethical to undertake; and for many questions other types of study design may provide the best evidence. Secondly, guideline development groups often fail to take adequate account of the methodological quality of individual studies and the overall picture presented by a body of evidence rather than individual studies or they fail to apply sufficient judgment to the overall strength of the evidence base and its applicability to the target population of the guideline. Thirdly, guideline users are often not clear about the implications of the grading system. They misinterpret the grade of recommendation as relating to its importance, rather than to the strength of the supporting evidence, and may therefore fail to give due weight to low grade recommendations. #### Summary points A revised system of determining levels of evidence and grades …