Methodology for ACOEM's Occupational Medicine Practice Guidelines—2017 Revision

Abstract

Methodology transparency is a core principle of high-quality clinical practice guideline development. Transparent, high-quality methods applied rigorously yield reproducible results and instill confidence in the user that the recommendations are made with the highest medical evidentiary support. Thus, transparency is a central value underlying criteria and methods published by multiple bodies which review quality of guidelines, including the US Institute of Medicine (IOM),1 and the international GRADE Working Group.2 The American College of Occupational and Environmental Medicine (ACOEM) has published transparent, evidence-based practice guidelines for the treatment of common health disorders among workers since 1997.3,4 The continually updated methods used for evidence search, critical appraisal, synthesis, and recommendation development were documented in the introduction to each edition and in other publications.5,6 The ACOEM Guideline Methodology Committee published an updated methodology in 2008,7 which ACOEM has used in publishing newer guidelines.8,9 Since the 2008 update, a number of advances in creating quality guidelines have been developed and are embodied in IOM's Clinical Practice Guidelines We Can Trust,1 and a series of publications from the GRADE Working Group.2 Tools and standards for the assessment of clinical practice guidelines (AGREE II10) and systematic reviews (AMSTAR11 and IOM1) have also been published. The updated ACOEM methodology, described fully in Methodology for ACOEM's Occupational Medicine Practice Guidelines—2017 Revision,12 is summarized herein and includes these advances in systematic review methods and guideline development. Summary tables comparing ACOEM's methodology to the standards described by AMSTAR, GRADE, AGREE II, and IOM are available in the complete methodology document.12 PURPOSE AND SCOPE The purpose of the ACOEM Occupational Medicine Practice Guidelines (Guidelines) is to improve the quality of occupational medical care and disability management through identification of evidence-based best practices for key areas of occupational health care.10,13 The primary goals of the Guidelines are to suggest treatment most helpful in aiding recovery and restoring function after an illness or injury and to help focus delivery of the most effective treatment for a given condition at the earliest feasible point. Additionally, the Guidelines present information to improve the efficiency and accuracy of the diagnostic process, identify effectiveness and risks of treatment, and enhance patient autonomy by providing information needed for shared decision making.14 The Guidelines address the key domains of occupational health care practice including prevention, health promotion, diagnosis, causation determination, illness and injury treatment, and disability management. Examples of broad clinical questions that the Guidelines may address in these areas are listed in Table 1.10TABLE 1: Clinical Questions in the Key Domains of Occupational Medicine PracticeThe Guidelines include foundation guidelines covering the basic clinical assessment processes used, as well as guidelines for specific occupationally-related conditions and treatments, for example, for musculoskeletal, respiratory disorders, opioids, eye conditions, and traumatic brain injury. Clinical questions used to direct the evidence search and assessment are framed in a systematized format covering population of interest, intervention comparison and outcome (PICO).15 The questions for etiology, diagnosis, and prognosis are framed in a modified PICO format, emphasizing such factors as specific exposures or trauma for etiology or prevention, natural history for prognosis, and the reproducibility and performance of diagnostic tests accepted reference standards for clinical assessment. PATIENT POPULATION ACOEM Guidelines are meant to apply to working-age adults (age 18 to 65).10 Although the primary focus is on injuries and illnesses that may have been caused by work, the Guidelines include common topics that affect the ability to work but are unlikely to be work-related. As many workers are now older than 65, guidance has been expanded to include all workers. As a practical matter, many studies include older adults and are incorporated in the evidence-base unless clear rationale for exclusion is found. This results in guidelines that may have substantially wider applicability than the target population. TARGET AUDIENCE Primary target users of the ACOEM Guidelines are physicians and other health care providers, and the health care organizations in which they deliver care.10,13 As workers’ compensation is an insurance system and a legal framework, the following are also members of the intended audience: patients, clinical case managers, insurers, third-party administrators, insurance claims managers, utilization reviewers, attorneys, judges, regulators, and policy makers. ORGANIZATIONAL STRUCTURE The ACOEM Board of Directors appoints one physician to chair the guidelines development processes, act as the Guidelines editor-in-chief, and chair the Evidence-based Practice Committee (EBPC).10 A summary of the function, responsibility, and objective statements for the committees and panels involved with ACOEM's guideline-related activities follows: The EBPC is comprised of the chairs of the Evidence-based Practice Panels (EBPPs or Panels). EBPC meeting may be attended by others who have been involved with previous ACOEM Guideline activities (eg, panel members and similar individuals). The EBPC is charged with coordinating updates of the ACOEM Guidelines, and also assists with identification of additional guideline topics and clinical questions that should be considered. Multidisciplinary EBPPs are distinct panels of experts for each body part, system, or skill area covered by the Guidelines.16 Currently, there are panels for asthma, interstitial lung diseases, low back, cervical/thoracic spine, hand/wrist/forearm, elbow, shoulder, hip/groin, knee, foot/ankle, eye, chronic pain, opioids, traumatic brain injury, mental health, and disability prevention and management. Panels have an appointed chair and may have a co-chair. Prospective panel members apply for membership or are invited based on expertise, productivity, and/or prior performance. They are then screened for expertise and conflicts of interest (COI). Appropriate applicants are appointed and trained to develop and/or update evidence-based practice recommendations. Panels approve clinical questions to frame the literature search, review critical analyses of the literature based on this approved methodology, and develop, review and approve evidence-based recommendations for clinical practice, care management, and disability management. Panels are often subdivided into areas of practice or research interest at the discretion of the panel chair in discussion with the editor-in-chief (eg, medical management, therapy, chiropractic, interventional treatments, surgical care, psychological approaches) particularly when the panel has a large scope of work (eg, low back or chronic pain). Research Team members are trained and draft preliminary clinical questions in PICO format for each guideline. The Research Team develops and documents search strategies and methods for each guideline topic and then conducts exhaustive systematic literature searches for each guideline topic and summarize studies in evidence tables. The Research Team then critically appraises, grades, and critiques each study, and finally drafts background text, rationale statements, and recommendations for each guideline topic and forwards these to the EBPP. The Guideline Methodology Committee (GMC) developed the initial methodology for production and revision of the Guidelines and other evidence-based products. On an ongoing basis, the GMC refines, clarifies, and updates the methodology based on state-of-the-art internationally accepted methods. To ensure transparency, it publishes documents that describe and explain the methodology used for ACOEM evidence-based materials and products.7 The GMC ensures adherence to state-of-the-art methods by assigning methodologists to each panel and also approving panel members after reviewing applications, curriculum vitae, and COI information from interested individuals. It then trains panels in this methodology and guideline development process.17–20 PROCESS FOR DEVELOPMENT AND REVISION OF THE GUIDELINES AND OTHER ACOEM EVIDENCE-BASED PRODUCTS The process for development of ACOEM Guidelines and evidence-based products was the work of the GMC and includes participation of the EBPC, review and formulation of recommendations by the Panels, stakeholder input, external peer review, and review by the ACOEM Board. Members of the guideline development groups are selected from applications of ACOEM members and nominees from relevant interest groups and professional organizations. All panel members are required to complete an application and a questionnaire to: (i) outline qualifications and interests; (ii) disclose potential COI; and (iii) agree to confidentiality procedures. Summaries of disclosures for all panel members are available online. All members of guideline development groups are required to complete training in ACOEM's evidence-based medicine methodology.7,12 Prioritization of Topics for Review and Recommendation The editor-in-chief and Research Team in collaboration with each panel chair, works with the panels to identify clinical questions about important, useful, common, expensive, and/or controversial work-related diagnoses, tests, and procedures.17 Topics may be forwarded from panel members, peer reviewers, Board members, external stakeholders, and others. The following procedures are followed: Research Team identifies the most common occupational health problems, tests, and treatments in terms of frequency, cost, time off work, apparent benefits/harms, and rapid increases in utilization. Diagnoses are grouped into homogeneous diagnostic groups. Tests and treatments are identified as groups, if similar or synonymous, or individually if the criteria for use and evidence of effectiveness are likely to be relatively unique. Panels solicit suggestions on areas to include/examine, including identification of commonly used, and/or emerging diagnoses, work linkages, and commonly used (but not necessarily safe or effective) tests, medications or procedures from panel members and other stakeholders. PICO questions are finalized for a given guideline by the panel. REVIEW AND FORMULATION OF RECOMMENDATIONS BY THE EVIDENCE-BASED PRACTICE PANELS Panels with assistance from the Research Team, formulate recommendations for guidelines as follows: Literature Evaluation: Literature Search and Study Selection The Research Team conducts exhaustive systematic literature reviews for each guideline topic, using multiple electronic search services.10 Relevant citations in articles are reviewed. In order to identify all high- and moderate-quality original research studies, the literature search is broad and comprehensive, although limited to English publications. Animal studies are excluded other than for treatment of chemical ocular injuries for which there are few human studies. When searching the US National Library of Medicine database, Medical Subject Headings (MeSH) are used to identify studies relevant to tests, treatments, and diagnoses in question. A combination of MeSH and other terms is used to determine the method that will yield the most relevant studies. Gray literature is included primarily through searches of Google Scholar to identify publication biases. Ongoing literature surveillance is also used to assure currency of guidelines recommendations, as well as to provide literature to be incorporated during the next comprehensive update. Treatment-Related Study Searches For treatment-related study searches, randomized controlled trials (RCTs), and randomized crossover trials, quality guidelines, meta-analyses, and systematic reviews are the primary foci of these comprehensive, exhaustive literature searches.21 Prospective and retrospective cohort studies are searched if there are no RCTs or systematic reviews identified. High-quality guidelines, meta-analyses, and systematic reviews are sought primarily for verification of search completeness; they are independently assessed for reproducibility of conclusions. When there is a discrepancy identified, re-review of a topic is conducted to verify conclusions. Both evidence of effectiveness (ie, comparisons with placebo, sham, or control of known level of efficacy) and comparative effectiveness (ie, comparisons between two or more active treatments) are sought. RCTs and randomized crossover trials are all selected for critical appraisal and quality grading. For evidence of harms, case reports, case series, retrospective cohort studies, and arms of RCTs are sought. For risk factor assessments, prospective and retrospective cohort studies are preferentially sought, with case-control or cross-sectional studies selected where cohort studies are absent. In some cases, studies with lower grades of evidence may be selected to examine current practice patterns or for other reasons. To ensure that all relevant, higher-quality studies are identified, the Research Team performs hand searches of reference lists in related articles. Diagnostic or Screening Searches For diagnostic study searches, all study design types are searched. Searches for these topics primarily focus on large, comparative trials looking at two or more diagnostic tests that are being compared. Ideally, one is the “gold standard” test for that condition. Key terms (eg, “Sensitivity and Specificity” [MeSH] OR “Predictive Value of Tests” [MeSH] OR “Gold-standard” OR accurate OR accuracy OR precision OR precise OR test) are used to identify the accuracy of the new test. Diagnostic studies are then summarized in evidence tables. Quality grading of diagnostic studies’ methods follows a different scheme than the grading scheme used for treatment. Highest scores are given to studies that compare the new test to a gold standard, if one exists. Timing of testing in relation to the progression of the disease state is also evaluated. Another criterion for judging the quality of studies of diagnostic methods is availability in the publication of data to determine or explicit documentation of specificity and sensitivity. Studies that compare a new test to an established gold standard are evaluated first. Studies that compare a new test to another test, but not a gold standard, are also evaluated. In order to ensure all relevant studies are included in the review, researchers also utilize literature identified, including from ongoing literature surveillance, stakeholders, panel members, and reference lists from previously identified studies. Search Term Documentation Search strategies and methods, including specific databases, search terms, number of studies found (eg, regarding treatment efficacy searches including RCTs and crossover trials) are documented. A search results section is included as a footnote for each evidence table. This section includes databases searched, limits on publication dates and languages, search terms used, number of studies found from all databases searched, total number of articles screened, number meeting inclusion criteria, number critically appraised, and total number of studies included. Tracking logs that document the search process, search terms, limitations, etc, are also published in order to maintain transparency. The Research Team reviews the abstracts of all citations found in the bibliographic search and identifies studies relevant to the topic that might meet the inclusion criteria (eg, in English, RCTs that address treatment questions, relevant literature for adverse effects, and comparative studies for diagnostic or screening tests) as adequate evidence and that could be used as the basis for evidence-based guidance statements. Researchers then retrieve the full text of these articles and perform a second screening process of the study in order to determine which studies meet the inclusion criteria to be considered as adequate evidence for these purposes. For those studies accepted as providing adequate evidence, individual article quality ratings are included in the evidence tables. Literature Evaluation: Critical Review of Studies The Research Team reviews in detail each study that meets inclusion criteria and summarizes important information from each in an evidence table.10 Evidence tables include first author's last name, year of publication, study design, quality rating score, population sample, age, sex, treatment comparison, follow-up time, results, conclusions, and comments relevant to the study.22 Potential COI and study sponsorships are also recorded. The evidence presented in the tables is limited to primary studies. In most cases, quality systematic reviews, meta-analyses, and professional guidelines are reviewed for comparison and assessment of reproducibility. The relative ranking of study designs for theoretical robustness is included in the complete methodology document.12Table 2 summarizes the level of confidence levels for the different study designs. While study design should confer various levels of confidence in the reproducibility of the results, how studies are conducted and analyzed is variable and must be specifically of for Study Research Team critically appraises, grades, and critiques each each study using a quality and in complete studies, based on are as or evidence and the in the quality assessment (eg, scores than or to or are or The highest for studies is Research Team complete the evidence researchers with each study for Studies are for and and assessed for robustness and of the from presented the body of quality evidence is and it is recommendations are then and to the panel. In all cases, a Research Team physician performs a review for clinical and Panels perform additional quality DEVELOPMENT OF GUIDELINES AND Panels review and draft recommendations by the Research Panels review evidence tables and draft recommendations, and original studies if review, panels determine the of evidence ratings for each topic in complete and recommendations for all clinical Table the used for evidence-based recommendations. a is recommendations are forwarded to the panel in for additional recommendation is and as not and not in complete is primarily through of is sought for all recommendations and in each guideline. When is not a is statements may be for health adverse effects, and relative of each test or treatment are considered and when should recommendation is to to which condition it of of and or is since evidence tables have included information about potential that are published. It is that there are studies of conducted in clinical where there is to the clinical there is no clear method to address this of In this may as comparisons were between a and and results may be and in other studies, comparison groups may have been (eg, of and a in of the or the results independently or the results with studies. In studies have been to have results and lower than studies conducted by Studies that include the population are to develop most recommendations. is given to the to which may or may not be to ACOEM's of clinical and should be in guidelines and clinical ACOEM's Guidelines recommendation includes an explicit with evidence, an evidence and of is by a that the about the evidence found on that for example, the rationale for the specific explain how in the recommendations the panel and the evidence and evidence of effectiveness or accuracy potential and relative for example, if the quality of the evidence was the may on how they and the evidence in a and and to recommendations are then and approved REVIEW ACOEM conducts external peer review of the Guidelines to: assure all relevant high-quality literature related to the topics has been assure that the important evidence from the literature relevant to the Guidelines has been solicit on the and recommendation statements are and with the and information on the and from external topic experts may also review the methodology used as well as of the critically evidence and the recommendations in each of peer are with for those not Panels review external peer comments and to the In for 2 a of all guidelines is at the for To the and of and organizations who use or are by the use of clinical practice guidelines in and workers’ compensation system, ACOEM ongoing from health care utilization reviewers, case managers, administrators, attorneys, regulators, and policy to comments through a on ACOEM's ACOEM also on the clinical questions for each guideline. The Guidelines are by utilization review managers, case managers, state workers’ compensation etc, who on use of the Guidelines in practice or management activities to determine if they are to and The Guidelines may be modified based on from testing if suggestions In the Guidelines conducted a test and to address during this REVIEW BY THE GMC AND ACOEM OF the evidence-based development process, a from the GMC works with the and Research Team to ensure this methodology is in the literature process and in development of and recommendation statements. ACOEM Board members may on the guidelines during the external review and these comments are reviewed by the and included in a to other external peer Panels and the Research Team have complete from ACOEM and the PROCESS ACOEM reviews the literature to identify in the evidence-base by updates of the Guidelines include a full review of previous recommendations. updates conducted to panel review of new evidence and revision of the recommendations. in literature FOR PRACTICE AND The ACOEM Guidelines are developed and by ACOEM members, and are for research and and control quality of the The Guidelines are published through has a new and which to ACOEM diagnostic and treatment recommendations and the ACOEM has also developed and criteria for each guideline complete methodology In ACOEM's process was developed to ensure the reproducibility and transparency of the guideline development This process and the criteria for and relevant evidence from the published literature are based on accepted and of and evidence-based practice guideline development which have been and of this methodology results in and recommendations for clinical practice that have the potential to improve the health and function of workers and other The the assistance of the ACOEM Research Team at the of and