A large number of taxonomies are used to rate the quality of an individual study and the strength of a recommendation based on a body of evidence. We have developed a new grading scale that will be used by several family medicine and primary care journals (required or optional), with the goal of allowing readers to learn one taxonomy that will apply to many sources of evidence. Our scale is called the Strength of Recommendation Taxonomy. It addresses the quality, quantity, and consistency of evidence and allows authors to rate individual studies or bodies of evidence. The taxonomy is built around the information mastery framework, which emphasizes the use of patient-oriented outcomes that measure changes in morbidity or mortality. An A-level recommendation is based on consistent and good quality patient-oriented evidence; a B-level recommendation is based on inconsistent or limited quality patient-oriented evidence; and a C-level recommendation is based on consensus, usual practice, opinion, disease-oriented evidence, or case series for studies of diagnosis, treatment, prevention, or screening. Levels of evidence from 1 to 3 for individual studies also are defined. We hope that consistent use of this taxonomy will improve the ability of authors and readers to communicate about the translation of research into practice.
Review articles (or overviews) are highly valued by physicians as a way to keep up to date with the medical literature. Sometimes, though, these articles are based more on the authors’ personal experience, or anecdotes, or incomplete surveys of the literature than on a comprehensive collection of the best available evidence. As a result, there is an ongoing effort in the medical publishing field to improve the quality of review articles through the use of more explicit grading of the strength of evidence on which recommendations are based.1–4
Several journals, including American Family Physician and Journal of Family Practice, have adopted evidence-grading scales that are used in some of the articles published in those journals. Other organizations and publications have also developed evidence-grading scales. The diversity of these scales can be confusing for readers. More than 100 grading scales are in use by various medical publications.5 A level B recommendation in one journal may not mean the same thing as a level B recommendation in another. Even within journals, different evidence-grading scales sometimes are used in different articles within the same issue of a journal. Journal readers do not have the time, energy, or interest to interpret multiple grading scales, and more complex scales are difficult to integrate into daily practice.
Therefore the editors of the US family medicine and primary care journals (ie, American Family Physician, Family Medicine, Journal of Family Practice, Journal of the American Board of Family Practice, and BMJ-USA) and the Family Practice Inquiries Network (FPIN) came together to develop a unified taxonomy for the strength of recommendations based on a body of evidence. The new taxonomy should include the following attributes: (1) be uniform in most family medicine journals and electronic databases; (2) allow authors to evaluate the strength of recommendation of a body of evidence; (3) allow authors to rate the level of evidence for an individual study; (4) be comprehensive and allow authors to evaluate studies of screening, diagnosis, therapy, prevention, and prognosis; (5) be easy to use and not too time-consuming for authors, reviewers, and editors who may be content experts but not experts in critical appraisal or clinical epidemiology; and (6) be straightforward enough that primary care physicians can readily integrate the recommendations into daily practice.
A number of relevant terms must be defined for clarification.
These outcomes include intermediate, histopathologic, physiologic, or surrogate results (ie, blood sugar, blood pressure, flow rate, coronary plaque thickness) that may or may not reflect improvements in patient outcomes.
These are outcomes that matter to patients and help them live longer or better lives, including reduced morbidity, reduced mortality, symptom improvement, improved quality of life, or lower cost.
The validity of an individual study is based on an assessment of its study design. According to some methodologies,6 levels of evidence can refer not only to individual studies but also to the quality of evidence from multiple studies about a specific question or the quality of evidence supporting a clinical intervention. For purposes of maintaining simplicity and consistency in this proposal, we use the term level of evidence to refer to individual studies.
The strength (or grade) of a recommendation for clinical practice is based on a body of evidence (typically more than one study). This approach takes into account the level of evidence of individual studies, the type of outcomes measured by these studies (patient-oriented or disease-oriented), the number, consistency, and coherence of the evidence as a whole, and the relationship between benefits, harms, and costs.
These guidelines are recommendations for practice that involve a comprehensive search of the literature, an evaluation of the quality of individual studies, and recommendations that are graded to reflect the quality of the supporting evidence. All search, critical appraisal, and grading methods should be described explicitly and be replicable by similarly skilled authors.
Consensus guidelines are recommendations for practice based on expert opinions that typically do not include a systematic search, an assessment of the quality of individual studies, or a system to label the strength of recommendations explicitly.
This evidence is presented in publications of original research, involving collection of original data or the systematic review of other original research publications. It does not include editorials, opinion pieces, or review articles (other than systematic reviews or meta-analyses).
A nonsystematic overview of a topic is a review article. In most cases, it is not based on an exhaustive, structured review of the literature and does not evaluate the quality of included studies systematically.
A systematic review is a critical assessment of existing evidence that addresses a focused clinical question, includes a comprehensive literature search, appraises the quality of studies, and reports results in a systematic manner. If the studies report comparable quantitative data and have a low degree of variation in their findings, a meta-analysis can be performed to derive a summary estimate of effect.
In March 2002, the Agency for Healthcare Research and Quality (AHRQ) published a report that summarized the state of the art in methods of rating the strength of evidence.5 The report identified a large number of systems for rating the quality of individual studies: 20 for systematic reviews, 49 for randomized controlled trials, 19 for observational studies, and 18 for diagnostic test studies. It also identified 40 scales that graded the strength of a body of evidence consisting of one or more studies.
The authors of the AHRQ report proposed that any system for grading the strength of evidence should consider 3 key elements: quality, quantity, and consistency. Quality is the extent to which the identified studies minimize the opportunity for bias and is synonymous with the concept of validity. Quantity is the number of studies and subjects included in those studies. Consistency is the extent to which findings are similar between different studies on the same topic. Only 7 of the 40 systems identified and addressed all 3 of these key elements.6–11
The authors of this article represent the major family medicine journals in the United States and a large family practice academic consortium. Our process began with a series of electronic mail exchanges, was developed during a meeting of the editors, and continued through another series of electronic mail exchanges.
We decided that our taxonomy for rating the strength of a recommendation should address the 3 key elements identified in the AHRQ report: quality, quantity, and consistency of evidence. We also were committed to creating a grading scale that could be applied by authors with varying degrees of expertise in evidence-based medicine and clinical epidemiology and interpreted by physicians with little or no formal training in these areas. We believed that the taxonomy should address the issue of patient-oriented evidence versus disease-oriented evidence explicitly and be consistent with the information mastery framework proposed by Slawson and Shaughnessy.2
After considering these criteria and reviewing the existing taxonomies for grading the strength of a recommendation, we decided that a new taxonomy was needed to reflect the needs of our specialty. Existing grading scales were focused on a particular kind of study (ie, prevention or treatment), were too complex, or did not take into account the type of outcome.
Our proposed taxonomy is called the Strength of Recommendations Taxonomy (SORT). It is shown in Figure 1. The taxonomy includes ratings of A, B, or C for the strength of recommendation for a body of evidence. The table in the center of Figure 1 explains whether a body of evidence represents good or limited-quality evidence and whether evidence is consistent or inconsistent. The quality of individual studies is rated 1, 2, or 3; numbers are used to distinguish ratings of individual studies from the letters A, B, and C used to evaluate the strength of a recommendation based on a body of evidence. Figure 2 provides information about how to determine the strength of recommendation for management recommendations, and Figure 3 explains how to determine the level of evidence for an individual study. These 2 algorithms should be helpful to authors preparing manuscripts for submission to family medicine journals. The algorithms are to be considered general guidelines, and special circumstances may dictate assignment of a different strength of recommendation (eg, a single, large, well-designed study in a diverse population may warrant an A-level recommendation).
The Strength of Recommendation Taxonomy (SORT). SR, systematic review; RCT, randomized controlled trial.