Guest Editorial: Guidelines for the Guidelines -- Impact and Controversies

This issue of LipidSpin is dedicated to “Evaluating Published Clinical Trials and Translating Them into Practice.” Clinical trials are the basis for advancing our knowledge of effective therapies in medicine. Well-conducted clinical trials, particularly those with results that can be replicated, traditionally have been used to develop practice guidelines.

Practice guidelines have far-reaching implications. They provide medical practitioners with guidance on the standard of care derived from evidence-based medicine. They also have economic impact, as they can promote and validate certain therapies and destroy others. They have administrative impacts, because they can form the basis of establishing payment, authorization, and regulatory policies by administrative agencies and health insurance plans. They have legal implications, because they can form the basis for medical standards of care used in peer review and medical malpractice issues.

Over the past 15 years, a plethora of practice guidelines have emerged, many of which suffered from shortcomings in the development process. In response, the Institute of Medicine (IOM) assembled an expert committee to provide recommendations to ensure that practice guidelines would truly represent an unbiased and evidence-based compilation of knowledge and standards of therapy. A year in the making, in 2011, the IOM committee’s report, “Clinical Practice Guidelines We Can Trust,” was released.¹

The IOM claims that its new standards will minimize the chances that important health decisions are based on information that may be biased or erroneous. However, since their release, the IOM standards for practice guidelines have been met with praise, criticism, and controversy.

Review of IOM Standards
After an exhaustive process of research, review, and even public comment, the IOM committee outlined eight standards for clinical practice guideline development. (Table 1) The new standards addressed a variety of problems that were identified by the IOM committee. However, two major issues were given special emphasis.

One was the sheer number of guidelines available to clinicians. Many had conflicting recommendations and a process of development that was not transparent. A charge of the committee was to ensure that future guidelines are developed in a trustworthy, evidence-based process.

The second concern was addressing conflicts of interest. The committee found that many guidelines have been developed with fairly egregious conflicts of interest, such as instances when drug companies substantially fund guideline development. A more subtle, but potentially equally important, conflict of interest was recognized in which clinical experts involved in developing practice guidelines had inherent biases because of their affiliation with industry.

Validations of IOM Standards
The assertions of the IOM committee regarding the problems it identified in clinical practice guidelines have been supported by objective and independent reviews. A recent report reviewed 130 randomly chosen clinical practice guidelines.² Only 44 percent met the median number of IOM standards. Another report found that, in a review of 149 practice guidelines, only 46 percent incorporated a grading or classification system for their recommendations.³

Both of these studies found that more than half of the guidelines reviewed did not reveal conflict-of-interest statements. Of those that did, more than 70 percent disclosed that the writing committee chairperson(s) had conflicts of interest. Guidelines from non-U.S. groups and medical specialty societies were the least likely to include conflict-of-interest information or meet the other IOM standards.

These reports also found that the mean age of guidelines was more than five years. Thus, practice guidelines are not updated with sufficient frequency to reflect changes in clinical knowledge. Furthermore, guidelines were more likely to emphasize benefits of treatment rather than potential harms.

Criticisms of IOM Standards
The IOM standards have been criticized as being impractical and inflexible. To be labeled trustworthy, the IOM states, a practice guideline must meet all eight standards. If most clinical practice guidelines already are not meeting IOM standards, then how practical are the IOM standards? We know we cannot have high-level evidence for everything we do in medicine. Do we leave clinicians unsupported in instances where high- quality evidence does not exist? A second criticism of the IOM report is the underlining belief system that working with industry to develop and commercialize new therapies is somehow inherently bad and should be managed or eliminated. The problem with this philosophy is that it fails to recognize the expertise and value from industry-academia collaboration.

Lastly, the IOM report fails to substantially meet its own standards.4 Subjected to scrutiny, the IOM document completely passed only two of its own standards, partially passed two and failed four. Therefore, one could question whether the new IOM standards were, in fact, trustworthy.

Impact of IOM Standards on Lipid Guidelines
After much delay and anticipation, last year the American College of Cardiology/ American Heart Association (ACC/AHA) released the report of their expert panel task force, which was intended to be an updated set of guidelines for cholesterol management in the U.S.5 They were expected to provide a comprehensive review on both the science and clinical evidence linking atherosclerosis to disordered lipid metabolism and by incorporating recent clinical trials of lipid intervention; they also were expected to provide the most up-to-date recommendations for clinicians to evaluate and optimize cardiovascular risk as it relates to lipids. For most clinicians, the ACC/AHA report fell far short of these expectations, and ended up being another set of specialty society guidelines.

The ACC/AHA report acknowledged that it was influenced by the IOM’s 2011 report on the development of trustworthy clinical guidelines. The expert panel’s recommendations were derived primarily from randomized controlled trials (RCTs). The virtual exclusion of evidence other than that from RCTs restricted the scope of the new guidelines. No recommendations were formulated when sufficient evidence was not available. Thus, the new guidelines left clinicians in the position of having to use their own clinical judgment to arrive at many clinical decisions instead of having science-based and expert guidance to inform these clinical choices.

In addition to criticisms of its method- ology, the ACC/AHA report has been soundly criticized for many of its clinical recommendations, or lack thereof. One major criticism is that, with respect to lipid-lowering therapies, only statins were considered. Furthermore, the concept of low-density lipoprotein (LDL) goals for treatment was completely ignored. By not making LDL the centerpiece of the new guidelines, they largely disregarded the traditional lipid hypothesis.

More controversy arose from the new Pooled Cohort Risk assessment as a replacement for the Framingham risk calculator.⁶ The ACC/AHA risk algorithm has been purported to overestimate 10-year risk. As a result, an estimated 30 million to 40 million lower-risk Americans will be eligible for drug therapy.

The new guidelines place increased emphasis on lifestyle modifications. However, an interesting aspect of the new guidelines is that lifestyle is highly promoted without randomized clinical trial evidence. Thus, it seems that RCTs are only applied to drug therapy.

The new guidelines are silent with respect to patients older than 75 and those younger than 40. They do not comment on genetic dyslipidemias, metabolic syndrome, or primary prevention. They do not adequately address potential harm associated with statin use. Nontraditional risk factors, such as C-reactive protein (CRP), calcium score, ApoB, and carotid intimal medial thickness, generally have been dismissed.

If the ACC/AHA document was intended to adhere to the IOM standards regarding conflict of interest, it failed to do so.⁷ Of the 13 authors in the main treatment guideline panel who were not NHLBI staffers, seven had financial relationships with pharmaceutical companies that manufacture statins. Of the 10 expert reviewers for the panel, three had financial relationships with pharmaceutical companies that manufacture statins. Of the 11 people on the risk prediction panel, five had financial relationships with pharmaceutical companies that manufactures statins.

Response to IOM Standards
The National Lipid Association (NLA) has failed to endorse the new ACC/AHA guidelines. Likewise, the American Association of Clinical Endocrinologists (AACE) not only declined to endorse the guidelines but also recommended that its members continue to use AACE guidelines, which generally agree with Adult Treatment Panel 3 (ATP 3).

As a result, the NLA has put forth its new Recommendations for Patient- Centered Management of Dyslipidemia.⁸ The evidence base considered in the development of new consensus recommendations emphasized results from RCTs but also included subgroup assessments and pooled analysis for multiple trials, epidemiologic, genetic, metabolic, and mechanistic investigations. The panel acknowledged that the primary results from RCTs represent the strongest evidence from which to draw conclusions about the benefits and risks of treatment strategies. However, RCT evidence has limitations and often is incomplete or of uncertain relevance to patients with characteristics that may differ in important ways from those who participated in RCTs.

When it comes to practice guidelines, a reasonable question to pose to the IOM is whether, in our quest for trustworthy, we have compromised effectiveness. 

Disclosure statement: Dr. Sheikh received honorarium as a member of the speakers bureau from Amarin and Boehringer Ingelheim.

References are listed on page 35.