EssayproSolutions

Guideline clearing houses (like the US Agency for Healthcare Research and Quality (AHRQ) Guideline Clearing House, http://www.guideline.gov) that identify and systematically define guidelines on a number of areas have emerged to assist users locate and pick guidelines [3, 4]. Large-scale guideline producing organizations have emerged at both the national (UK National Institute for Health and Clinical Excellence) and state (Scottish Intercollegiate Guidelines Network) levels (such as the Ontario Cancer Guideline Program). We’ve also contributed to reports for the WHO [5] and professional associations (Schünemann HJ, Woodhead M, Anzueto A, Buist AS. A handbook for professional societies and other recommendation makers: an ATS/ERS Workshop Report (in process). The Guidelines International Network (http://www.g-i-n.net/) currently has a high visibility society for such organizations and individuals. In this context, it appears appropriate to update and extend our prior study on generating clinical practice recommendations in three papers. On the basis of our background paper [6] for the IOM study ‘Clinical Practice Guidelines We Can Trust’ [7].

To begin, we addressed the target audience(s), defining themes for recommendations, guideline group makeup, and the mechanisms by which guideline groups work. Moving from evidence to suggestions is discussed in this second article. To improve guideline implementability and how recommendations approach dealing with patients with co-morbid disorders, we will examine these topics in the third article.

Guidelines evidence and outcome selection

Different clinical questions are typically considered in guidelines, such as risk factors for conditions, diagnostic criteria for conditions, prognostic factors with and without treatment, benefits and harms of various diagnostic or treatment options, and patients’ experiences with healthcare interventions. For each question, multiple study approaches give the most credible data. This affects the search, appraisal, and synthesis stages of knowledge syntheses used to generate guideline recommendations. These include explicit decisions on the specific questions to be answered and the outcomes to be assessed at the outset of the analytic process, having a clear understanding of the analytic logic of the recommendations and using this model to keep the group’s analytic work ‘on track.’ An analytic framework accomplishes these organizational principles [8-13].

Creating a framework

The analytic framework of a guideline is important. During this key step, a group decides which questions to answer, what evidence to use, and how to analyze it. Scientific facts, expert opinion, clinical experience, and other pertinent information are analyzed using decision rules to provide recommendations. The guideline’s analytic logic, the basis for the suggestions, captures the process’s outcome.

A framework for analysis

First, define the important questions. What information does the group need to make a recommendation? It starts with outlining the criteria for persuading the group to adopt a clinical conduct. The alternatives available depend on the group’s stance and the issue. A decision is made by certain groups based on existing practice, consensus, or clinical experience. Many organisations rely on scientific data, yet their definitions of effectiveness vary. Various indices of morbidity and death define benefits. Some groups just evaluate benefits, while others consider costs and other consequences. It is critical that guideline writers define desired objectives as precisely as possible. That the practice to be ‘clinically effective’ is not enough. What particular outcomes must be influenced to make a suggestion? Select health, intermediate, and surrogate outcomes for consideration.

Physical morbidity (e.g., dyspnea, blindness, frailty), emotional well-being, and death are all examples of health outcomes (e.g., survival, life expectancy). Eddy characterizes these as ‘outcomes that people can feel and care about’ [8]. A surrogate result is an impact that is equal to or may operate as a proxy for a health outcome. However, these intermediate and surrogate outcomes frequently have established pathophysiologic links with the primary outcomes. As a result of coronary angioplasty, arterial patency is established, avoiding further ischemia. Surrogate outcomes for myocardial ischemia, renal insufficiency, and obstructive pulmonary illness include electrocardiographic abnormalities and pulmonary function testing. The use of intermediate and surrogate outcomes is common since they are typically the only validated outcome measures available in existing studies. The group should be convinced that the maneuver will effect one of these outcomes.

The complicated interrelationships between these outcomes are best shown graphically or tabularly. Figure 1 depicts an analytic approach created by the US Preventive Services Task Force for an osteoporosis screening recommendation [14]. First articulated in the late 1980s, causal routes [10], causal models [11], influence diagrams (12), and evidence models (13), are all diagrammatic approaches. The graphic is built by first identifying the group’s main outcomes. This list of outcomes highlights the important factors the group must consider in order to assess appropriateness and provide a recommendation. The group’s preferred intermediate or surrogate outcomes are then added to the graphic. Figure 1 shows the important premises in the analytic logic that must be validated by the review process to justify the proposal. KQ1 asks if risk factor evaluation or bone measurement testing reduces fracture morbidity and death. They concern the accuracy of risk factor assessment and bone measurement testing, the possible risks of testing, and treatment of those diagnosed as abnormal.

1

1

Framework and KQs (Keywords). Update for the US Preventive Services Task Force on Osteoporosis Screening Nelson HD, et al., published in Annals of Internal Medicine, July 5, 2010. Permission to reprint.

Image size

A visual analytic framework that specifies the hypothesized link between intermediate, surrogate, and health outcomes serves several uses. It compels analysts to make clear, a priori judgements regarding desired results. It lets others know if crucial results were missed. It expresses the group’s views on the validity of intermediate and surrogate health outcomes. The analyst’s ideas regarding pathophysiologic linkages were disclosed in the diagram. They let others know if the right questions were asked right away.

This form of analytic framework looks like flowcharts, algorithms, and other images, but its substance and function are very different. The visual analytic framework is prospective: the group determines the criteria it will use to make a recommendation. Frameworks are not algorithms. The ‘arrows’ and outcomes in algorithms illustrate clinical choices, test findings, or pathophysiologic events in the workup and treatment of patients [15, 16].

Adding evidence

The visual analytic approach provides a ‘road map’ for examining the evidence. While they present a list of questions to answer, they do not specify which sorts of evidence should be sought to deliver the information. After answering these questions, the literature study can proceed to examine admissible evidence for support for the analytic framework’s specific links. The evidence for the links is typically mixed, with some backed by randomized controlled trials and others by alternative evidence types.

Given the increasing availability of systematic reviews of various types of research addressing various topics, guideline makers should first look for suitable systematic reviews for each question. Whitlock et al. discuss the methodological and practical problems involved in utilising current systematic studies to establish guidelines [17].

Analytic logic completion

The data provided in the analytic framework is often merely the beginning of a deeper study. The final graphic communicates nothing about the research’ conclusions, their consistency, or the data’s quality. Detailed evidence analysis includes basic narrative summaries, evidence tables, meta-analyses, and modeling. The visual analytic framework is not a graphics gadget. In the analytic logic, it does not describe what the evidence demonstrates.

The analytic framework’s information helps write a reasoning statement. Thus, the justification statement might explain the outcomes evaluated (including patient preferences), the group’s beliefs regarding the relationship between intermediate and surrogate outcomes and health outcomes, and the evidence found to support the links. If the evaluation identified no evidence-based links, the justification statement might honestly indicate how opinion, theory, or clinical experience influenced the suggestion. This ‘truth in advertising’ helps physicians, policymakers, and other guideline users trust the justification statement’s assumptions. For example, saying that a strategy is supported by ‘randomized controlled trials’ when the data only supports one relationship in the explanation. The analytic logic links help organizations to pinpoint the crucial assumptions they differ on.

Finally, by highlighting unsupported links, the analytic approach reveals the most critical outcomes for researchers to assess a healthcare practice’s efficacy. In an era of limited research expenditures, this data is vital to setting priorities and directing outcomes research toward the core concerns. The framework’s outcomes can also be used to evaluate the guidelines’ impact on quality of care.

Values-based guideline development

The guideline committee does not provide recommendations based on empirical facts. When the research clearly shows that an intervention is useless or dangerous, the need to address values and preferences is less significant. However, there are two primary situations in which personal preferences and subjective judgements are required.

First, when the information is ambiguous, judgements regarding the existence and significance of an intervention are generally subjective. For example, several randomized controlled studies have assessed the efficacy of mammography screening for breast cancer, with extensive empirical data on impact size [18]. Despite this, experts have disagreed for two decades over the quality of the data and the probable mortality reduction from mammography at various ages [19]. In the face of scientific ambiguity, other factors often, and properly, take precedence. When developing guidelines, experts weigh in on the ailment being addressed and its severity, the possible hazards of an intervention, and the potential risks of inactivity. Groups’ descriptions of the facts and recommendations are necessarily colored by these assessments [20]. Some organisations choose neutrality, citing a lack of evidence to make a recommendation [21]. However, where the disease is serious or the potential damage is minimal, the committee may suggest the intervention despite lack of proof. In the case of possible hazards, a group may advise against an intervention until additional data is available [22]. It is better if guideline creators are explicit about value judgements [23]. The reasoning for finding that the evidence is strong, weak, or ambiguous should be described. Concerns regarding study design or result evaluation should be stated, not only to justify the group’s position but also to inform future research. Knowing that guideline groups frequently mention contamination of the control group as a flaw in intervention studies may motivate future research to seek novel solutions.

Second, even when the incidence or impact size is obvious from the data, the assessment of benefits vs hazards is generally subjective [24]. People can arrive to different judgments about net benefit when given the same evidence regarding likelihood of benefits and hazards [25, 26]. For example, an oncologist or patient worried about the cancer’s danger may be less concerned with urine incontinence than a clinician or patient interested about quality of life. These subjective judgements are neither right nor incorrect, but they do impact net benefit findings and a group’s recommendation of an intervention.

Groups have two alternatives when faced with challenging tradeoffs. First, people can decide for themselves whether the advantages exceed the risks. With its in-depth understanding of the clinical problem and the supporting research, the organization can predict how most patients will behave. The advantage of this technique is that the group has command of subtleties that most patients cannot absorb or most busy practitioners cannot convey. This technique has the problem of intrinsic paternalism and collective misjudgment [27]. On may also encourage shared or informed decision-making, in which the patient discusses the tradeoffs with their clinician and makes an individual decision based on personal preferences [28, 29]. In doing so, the group avoids taking a policy position. They understand that determining whether benefits exceed hazards is a subjective choice that can only be made by the patient and practitioner individually, not by a guideline group [30]. As a result, recommendations become a tool for patient empowerment, participation, and activation [31, 32]. Instead of making a recommendation, the group should provide guidelines for the patient-clinician conversation. The guideline group may be able to outline the items the patient and clinician should review, the relevant evidence, the role of decision aids, and other suggestions for incorporating personal preferences into the decision-making process.

Including economics in guideline development

Incorporating economic factors into guidelines has not been widely accepted. ‘Health interventions are not free, people are not infinitely wealthy, and [healthcare] program budgets are limited.’ A dollar will be paid for every dollar spent on healthcare. These payments will not be hidden, disguised, or laundered’ [8]. Opportunity costs are universal. While considerations of effectiveness may be applicable across healthcare systems, cost and value are more likely to be system-specific. So a cost-effectiveness guideline may be less transferable than a clinical effectiveness guideline.

Aspirational recommendations from the 1992 IOM report [33] included including cost implications of alternative preventive, diagnostic, and management strategies for each clinical situation. The stated rationale was that this data would help potential users assess the consequences of various practices. But they later admitted that ‘this recommendation poses major methodological and practical challenges.’ Despite new practical experience, this position has not changed. Because cost issues are more likely to be health system-specific than clinical evidence issues, many guideline developers do not do this unless explicitly mandated by organizations like the UK’s National Institute for Health and Clinical Excellence (NICE).

The NICE guideline development manual states that ‘only rarely will the health economic literature be comprehensive and conclusive enough that no further analysis is required.’ Usually, more economic research is required.’ Economic evidence may be limited in its general applicability to the clinical guideline context, but it can be useful in framing general cost-effectiveness bounds for management options for a clinical condition and providing an explicit source for some assumptions.

The methods of incorporating economic factors shape guideline development [34]. Choosing how to summarize data and whether or not there are common outcomes across studies is an early decision in the development of each of the guidelines. If common outcomes are available, quantitative techniques (meta-analysis or meta-regression) can be used to produce summary relative and absolute benefit estimates, which can then be combined with effectiveness and cost into a cost-effectiveness statistic. It is more difficult to do this with broad clinical areas (e.g., type 2 diabetes management) than narrower areas (e.g., choosing a drug to treat depression).

If the evidence summary is qualitative (a narrative review of studies), common descriptors (e.g., study design, study population, intervention, intervention duration) can be used to compare studies. Unless the evidence summary is dominated by one study with appropriate outcomes, cost-effectiveness estimates may not be possible. A reader can decide how much weight to give to each dimension of evidence in a guideline that uses qualitative evidence summary methods (not amenable to meta-analysis). ‘As the treatments appear equivalent, clinicians should offer the cheapest preparation that patients can tolerate and comply with,’ for example.

For single-decision guidelines, economic data may be incorporated into a formal decision analysis framework. Statistics from other sources are combined with intermediate clinical outcome data to explore the overall costs and consequences of treatment alternatives. It is theoretically possible to map clinical data to generic quality of life scores, model disease progression, and estimate cost per QALY for each treatment decision. However, it differs from the above methods in several ways. First, while a multi-disciplinary guideline development group may help shape questions, values, and assumptions that go into a model, the model itself produces the ‘right decision’. Second, the data are aggregated into a single metric, whose components (and associated uncertainty) are opaque. Third, the complexity of modeling a single decision often calls into question the method’s applicability to more complex clinical decisions with multiple interdependencies. A decision analysis-driven guideline’s appropriate application is currently unclear and requires further research.

Recommendations for

Wording suggestions

The wording of recommendations is important in developing recommendations that will positively influence care. ‘Patients with condition name > should be offered the intervention if clinically appropriate,’ or ‘clinicians should follow-up patients given the intervention every four weeks, or sooner if necessary,’ because clinicians using the guideline may struggle with, or be unsure of, what constitutes ‘clinically appropriate’. Grol et al. found that vague or nonspecific guideline recommendations were followed 35% of the time while clear recommendations were followed 67% [36]. Researchers found that specific rather than vague guidelines produced more appropriate and less inappropriate orders for electrodiagnostic tests [37]. Using psychological research, Michie and Johnston concluded that rewriting guidelines in behaviorally specific terms is the most cost effective intervention [38].

But there is no standard for recommendation wording [39]. The results of Hussain et alcomprehensive .’s evaluation of over 1275 randomly selected recommendations from the National Guideline Clearinghouse [40] reflect the lack of a standard. The recommendations were inconsistent within and across guidelines, and 31.6 percent were non-actionable. Over half (52.6%) did not indicate the recommendation’s strength.

The National Guideline Clearinghouse Editorial Board “encourages [guideline] developers to formulate recommendation statements that are actionable” [41]. NICE (National Institutes for Health and Clinical Excellence Handbook) [42] states that recommendations should be clear and concise, but contain enough information to be understood without additional material.

These qualities are desired not only in guidelines but also in decision support tools (e.g., electronic medical records prompts, standing orders, and checklists). However, evidence-based guidelines developers may find the science inadequate to justify such precision. In such cases, ambiguity may be more accurate than false precision. A Papanicolaou smear every one to three years, for example, and mammographic screening every year or every other year can reduce mortality [43]. There is insufficient evidence to define any screening interval or risk groups. Arbitrarily constructing a precise answer may satisfy demands for ‘clear’ guidelines, but it departs from the evidence. It also puts clinicians and patients at risk by dictating care practices that are perfectly reasonable. The tension between providing clear and precise guidance and not going beyond the supporting science is always present in evidence-based guideline development.

The evidence suggests that consumers prefer symbols to numbers to indicate the strength of recommendations [44, 45]. Based on the NGC database, Hussain et al. propose six criteria for presenting and formulating recommendations (Table 1).

Table 1: Criteria for presenting and formulating recommendations

Full table

What methods exist for grading evidence and recommendation strength?

The Canadian Task Force on Periodic Health Examination began grading healthcare recommendations over 30 years ago [46]. AHRQ published a systematic review of existing systems to grade evidence quality and recommendations in 2002 [47]. The AHRQ review considered 40 systems that graded the strength of evidence until 2000. The authors agreed that quality (the sum of quality ratings for individual studies, based on how bias was minimized) and quantity (effect size, number of studies, and sample size or power) were important domains and elements for grading the strength of evidence systems (for any given topic, the extent to which similar findings are reported using similar and different study designs).

The Canadian Optimal Medication Prescribing and Utilization Service (COMPUS), a division of the Canadian Agency for Drugs and Technology in Health (CADTH), expanded AHRQ’s work until 2005 [48]. 11 review articles identified nearly 50 evidence grading systems. Experts in evidence evaluation methodology helped identify ten additional instruments or grading systems not previously identified. The identified instruments and systems were evaluated using the AHRQ evaluation grids. The highest scoring instruments were the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) working group and the SIGN approaches [48]. A second round of expert consultation and stakeholder input from all interested parties confirmed the selection of these instruments. However, SIGN—while providing a detailed system for assessing the quality of individual studies—provided no clear guidance for summarizing the quality of evidence across studies and for moving from the research evidence to recommendations. SIGN therefore recently adopted GRADE that laid out these steps more explicitly.

GRADE

A number of publications describe the GRADE approach and its development [44, 49-57]. The GRADE working group (http://www.gradeworkinggroup.org) [49] emphasizes the link between the quality of a body of evidence and the recommendation, but recognizes that other factors beyond the quality of evidence contribute to the strength of a recommendation, such as patient values and preferences [58, 59].

GRADE considers eight factors in the assessments of the quality of evidence for each important outcome (Table 2). (Table 2). Concerns about any of five factors can lower the confidence in an estimate of effect and study quality: study design and execution (risk of bias); consistency of the evidence across studies; directness of the evidence (including concepts of generalizability, transferability and external validity); the precision of the estimate of the effect; and publication bias. The presence of any of the following three factors can increase the quality of evidence: a strong or very strong association; a dose-effect relationship; and all plausible residual confounding may be working to reduce the demonstrated effect or increase the effect if no effect was observed. The overall quality of evidence is determined by the lowest quality of evidence for each of the critical outcomes. However, when outcomes point in the same direction (all critical outcomes suggesting benefit), then the overall quality of evidence reflects the quality of the better evidence (e.g., two critical outcomes showing convincing benefit are of low quality and a third of very low quality, the overall quality is not reduced from low to very low) (e.g., two critical outcomes showing convincing benefit are of low quality and a third of very low quality, the overall quality is not reduced from low to very low).

Table 2 A summary of the GRADE approach to grading the quality of evidence for each outcome

Full table

A substantial conceptual difference between GRADE and other approaches is the handling of expert opinion. GRADE specifically acknowledges that expertise is required for interpretation of any form of evidence (‘judgments’) but considers that opinion is an interpretation of—sometimes unsystematic—evidence, but not a form of evidence.

Factors that influence recommendations

Four factors influence whether a panel makes a recommendation for or against a management strategy. These four factors include: the quality of the available supporting body of evidence; the magnitude of the difference between the benefits and undesirable downsides or harms; the certainty about or variability in values and preferences of patients; and the resource expenditure associated with the management options.

Quality of evidence

The quality of evidence reflects the confidence or certainty in the estimates of effects related to an outcome. If guideline panels are uncertain of the magnitude of the benefits and harms of an intervention, it is unlikely they can make a strong recommendation for that intervention (see section on quality of evidence) (see section on quality of evidence). Thus, even when there is an apparent large gradient in the balance of advantages and disadvantages, guideline developers will be appropriately reluctant to offer a strong recommendation for an intervention if the quality of the evidence is low.

The balance between benefits and undesirable downsides

When the benefits of following the recommendation clearly outweigh the downsides, it is more likely that the recommendation will be strong. When the desirable and undesirable consequences are closely balanced, a weaker recommendation is warranted. While most original studies and systematic reviews present the magnitudes of effect of outcomes in relative terms (e.g., relative risk, hazard ratio, odds ratio), weighing the magnitude of the difference between the benefits and downsides to develop a recommendation also requires the knowledge of the likely absolute effects for a specific population or situation. If the guideline panel judges that the balance between desirable and undesirable effects varies by baseline risk, it can issue separate recommendations for groups with different baselines risks when tools for risk stratification are available for the guideline users [60, 61]. Often, when values and preferences or attitude towards the resource use may differ from those assumed by guideline developers, patients, clinicians, and policy makers may choose to examine the magnitude of effects of management options on the outcomes of interest themselves, rather than relying on judgments of those making the recommendation.

Uncertainty or variability of patient values and preferences

Different patients can take different views about what outcome constitutes benefit or harm and clinicians’ understanding of importance of particular outcomes for patients can differ from that of the patients. Recognizing patients’ liberty requires explicit consideration of their values and preferences (autonomy). Every management strategy has benefits and drawbacks, thus a trade-off is always required. A recommendation’s strength depends on how patients and panel members perceive certain advantages, dangers, and inconveniences. But patient preferences and values data are typically scarce. GRADE advises guideline panels to fully declare their beliefs and preferences, as well as their weightings. This clear explanation helps interpret suggestions, especially weak ones when the optimal course of action is unclear.

Costs or resource use

When weighing the benefits and drawbacks of competing management systems, resource utilization might be one of the results. However, as stated above, expenses vary greatly over time and space. Moreover, the resource’s implications vary greatly. A year’s prescription of a medicine may pay for one nurse’s salary in the US, ten nurses in Romania, and thirty nurses in India. Costlier interventions are therefore less likely to be strongly recommended, although the context of the suggestion is essential. It is important to specify the context in which a recommendation is made, as well as the standpoint from which it is made (patient, third party payer or society).

Making suggestions

Those giving suggestions may be more or less certain that following their advice will benefit patients [62]. They tell consumers of guidelines (e.g., physicians, patients and their families, policymakers) how confident they are in their recommendations. While the GRADE system utilizes two grades of recommendation strength—strong or weak (also known as conditional)—reflecting confidence in the clarity of that balance or lack thereof (Table 3). This dichotomy helps to clarify the information and increase comprehension. The GRADE method uses phrases like ‘conditional,’ ‘qualified,’ and ‘discretionary’ to communicate reduced confidence in the balance of advantages and drawbacks.

Table 3: Implications of the GRADE approach’s two recommendation strength grades

Full table

They may describe the available data (e.g., chromones are successful in treating allergic rhinitis), but not suggest what action should be taken (e.g., given all other treatment choices, should chromones be utilized in treating allergic rhinitis?) [40]. GRADE advocates using the active voice when recommending specific actions. For example, numerous GRADE guidelines used the words ‘we recommend…’ and ‘we indicate…’ to distinguish strong from weak suggestions. Strong recommendations might be stated as ‘clinicians should…’ or ‘we conditionally suggest….’ Defining the strength of recommendations in languages other than English can be difficult.

Should guideline committees make recommendations based on poor evidence?

In the face of poor evidence, most experts believe that all guideline committees should have the option of not recommending. But better evidence may never be found, and clinicians require direction regardless of the underlying evidence’s quality. Best judgment guideline panels should offer explicit and unambiguous recommendations (however conditional in the face of low-quality evidence) and transparently explain their decisions. Some argue that when the evidence is ‘insufficient,’ no recommendations should be given. The USPSTF utilizes ‘insufficient evidence to suggest’. It is suggested that a guideline panel making a recommendation on low- or very low-quality is too dangerous.

Research suggestions

There are no set standards for directing panels in deciding whether or not to conduct research. To be valid, a suggestion for the use of interventions in research must meet the criteria in Table 4 [63, 64]. The research suggestions should include specific study questions, patient-important outcomes to be assessed, and other pertinent factors [65]. Because most guidelines are written for physicians, the research suggestions may seem out of place amid the practice recommendations. An appendix or special part of the guideline for researchers and research funding agencies might include research recommendations. Similarly, executive summaries should be formatted.

Table 4: Criteria for a sensible suggestion for using treatments in research

Full table

Summary

On the other hand, integrating values into a guideline, taking economic factors into account, and going from evidence to recommendations are all explored in this study. To improve guideline implementability and how recommendations approach dealing with patients with co-morbid disorders, we will examine these concerns in the third and final article in the series.

References

Field MJ, Lohr KN, Committee to Advise the Public Health Service on Clinical Practice Guidelines IoM: NAP, 1990, WASHINGTON

GSI

Clinical guidelines: establishing guidelines. 10.1136/bmj.318.7183.593

CAS

Article

PubMed

PubMed

GSI

The AGREE Collaboration Writing Group: Development and validation of an international assessment tool for assessing the quality of clinical practice guidelines: the AGREE project. Quality and Safety in Health Care.

Article

GSI

AGREE II: enhancing guideline creation, reporting, and assessment in health care. J Clin Epidemiol 2010; 63: 1308-1311.

Article

PubMed

GSI