Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey

TLDR

To develop and test optimal Medline search strategies for retrieving sound clinical studies on prevention or treatment of health disorders. An analytical survey of 161 clinical journals indexed in Medline in 2000 evaluated 4,862 unique terms across 18,404 combinations for sensitivity, specificity, precision, and accuracy. Only 24.2 % of 6,568 treatment articles met criteria, and optimal multi‑term combinations achieved 99.3 % sensitivity at 70.4 % specificity—an absolute 4.1 % sensitivity gain over single terms but a 23.7 % specificity loss—while maximizing specificity yielded 97.4 %, comparable to the best single term, and the new strategies outperformed existing ones except two with slightly higher specificity but lower sensitivity.

Abstract

<h3>Abstract</h3> <b>Objective</b> To develop and test optimal Medline search strategies for retrieving sound clinical studies on prevention or treatment of health disorders. <b>Design</b> Analytical survey. <b>Data sources</b> 161 clinical journals indexed in Medline for the year 2000. <b>Main outcome measures</b> Sensitivity, specificity, precision, and accuracy of 4862 unique terms in 18 404 combinations. <b>Results</b> Only 1587 (24.2%) of 6568 articles on treatment met criteria for testing clinical interventions. Combinations of search terms reached peak sensitivities of 99.3% (95% confidence interval 98.7% to 99.8%) at a specificity of 70.4% (69.8% to 70.9%). Compared with best single terms, best multiple terms increased sensitivity for sound studies by 4.1% (absolute increase), but with substantial loss of specificity (absolute difference 23.7%) when sensitivity was maximised. When terms were combined to maximise specificity, 97.4% (97.3% to 97.6%) was achieved, about the same as that achieved by the best single term (97.6%, 97.4% to 97.7%). The strategies newly reported in this paper outperformed other validated search strategies except for two strategies that had slightly higher specificity (98.1% and 97.6% <i>v</i> 97.4%) but lower sensitivity (42.0% and 92.8% <i>v</i> 93.1%). <b>Conclusion</b> New empirical search strategies have been validated to optimise retrieval from Medline of articles reporting high quality clinical studies on prevention or treatment of health disorders.