Systematic comparison of study quality criteria

Abstract

Approaches for the systematic review and evaluation of chemical toxicity are currently being reconsidered, with a specific focus on the evaluation of individual studies and their integration into the overall body of evidence. This renewed interest has arisen, in part, as a result of several prominent reviews of these approaches by special committees of the National Research Council (NRC), among others. We conducted a critical evaluation of several available frameworks for evaluating study quality. We assessed the criteria separately for human, animal, and in vitro studies as well as for systematic reviews. We then evaluated commonalities across disciplines. We also considered the potential implications of applying criteria frameworks and how they bear on fundamental risk assessment questions. We found that the available frameworks within each discipline differed in terms of their intended purpose and level of guidance for decision making. All the frameworks across disciplines shared common themes, however, including the adequate reporting of specific details of study conditions and design/protocol, selection and randomization of study groups (where applicable), outcome assessment methods and applicability (e.g., validity and reliability), avoidance of selective reporting, and the consideration of potential confounders or bias. We identified the most informative study quality considerations, which will enable researchers to implement more objective and standardized methods for evaluating studies and, ultimately, improve risk assessment methods.