HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS REGISTER		Author Keyword(s) Vol Page [Advanced]

This Article Extract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this link to a friend Similar articles in this journal Similar articles in PubMed Add article to my folders Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Berwick, D M Search for Related Content PubMed PubMed Citation Articles by Berwick, D M Related Collections Guidelines Quality improvement Related Articles

Broadening the view of evidence-based medicine

Correspondence to:
Dr D M Berwick
Institute for Healthcare Improvement, 20 University Road, 7th Floor, Cambridge, MA 02138, USA; dberwick{at}ihi.org

Keywords: quality improvement; evidence; publication; guidelines

Scholars in the last half of the 20th century forged our modern commitment to evidence in evaluating clinical practices. They were courageous people, iconoclasts for their time, insisting that the scientific method was a necessary and plausible tool for judging the value of what we did for and to patients. Scientific evaluation of clinical practice was necessary, they argued, because unguided human observers are frail meters of truth—too prone to see what they expect to see, too likely to confuse effort with results or to attribute outcomes to visible causes rather than hidden ones, too trusting in small numbers and local opinion. Only formal scientific designs and strong statistical methods, they claimed, can protect the human mind from its own biases and adjust for hidden uncontrolled influences, sorting signals from noise. Scientific evaluation of practice is plausible, they argued, because the hypothetico-deductive method and proper statistical theory can be applied, with only modest adjustments, to the world of clinical process, just as it can be in a laboratory. And they taught us how to do that.

Their arguments were not welcomed at first. Today they are heroes, honored in the history of clinical science, but Archie Cochrane,¹ Alvin Feinstein,² Frederick Mosteller,³ Tom Chalmers,⁴ David Sackett,⁵ and others had first to play the role of outsiders, essentially pestering the center of health care to get serious about evaluating its work. They had thick skins, these critics, because they were—and had to be—change agents.

But they had evidence for their assertions, and they systematically accumulated more as they built their case. The risks of unguided impression were documented well in studies of the emergence and persistence of clinical practice of little or no value, once studied. Gastric freezing,⁶ radical mastectomy,⁷ theophylline for asthma,⁸ and dozens of other common practices wilted under the microscope of properly designed clinical trials, proving no better than simpler practices or out-and-out harmful. Beliefs and evidence simply did not always correspond.

A normative framework emerged for judging the value of evidence, a heraldry made clear in works such as the monumental volumes on effective care in pregnancy and childbirth compiled by Iain Chalmers and his colleagues,⁹ the reports of the Canadian Task Force on preventive medicine,¹⁰ Mosteller’s magisterial Institute of Medicine report on assessing medical technologies,¹¹ Feinstein’s texts,² Sackett’s series in the Canadian Medical Association Journal,¹² and in the United States Task Force on Clinical Preventive Services.¹³ The Crown Prince of methods was the randomized, double blind, prospective, controlled clinical trial—the "RCT"—which stood second to no other method in protecting the scientist and the reader against bias, confounding, and other generators of false conclusion. Below the RCT stood methods of less nobility, graded in their evidence value from the properly designed cohort and case-control studies of epidemiology to the lowly case series, the suspect expert opinion, and the bestial anecdote. Systematic reviews that allow concatenation of results across multiple studies of varying design and quality, and meta-analysis that uses high powered statistics to combine quantitative findings from across multiple comparison studies, have further ramped up the power of controlled protocol-driven evidence.¹⁴

The leaders of evaluative clinical science fostered their young, and a generation of new scholars emerged in healthcare academia, founding their careers on evaluation of practice and on the progression of methods for evaluation. One of the most impressive success stories of 20th century medicine was how these people and these views—the entire field of "clinical epidemiology" and "evaluative clinical sciences"—not only survived but thrived, and eventually placed its leaders—scholars of the caliber of John Eisenberg, Christine Cassel, Harold Sox, and many others—in positions of the highest influence in departments of medicine, journal editorships, and professional societies, honoring work that a few decades before would not even have been understood to be about health care.

The benefits of evidence-based medicine, thus defined, have been immense. Patients today can count on a growing proportion of the tests, diagnostic processes, surgical procedures, and other costs and risks in care to have been subjected to proper systematic evaluation. The very definition of "quality" in health care has now come to incorporate the use of scientific evidence in practice; that is what the Institute of Medicine meant in its call for improvement of "effectiveness" as a key aim for improving care.¹⁵ Gaps between science and practice remain wide, but we seem increasingly committed to closing them. That is good.

But, we now have a problem: we have overshot the mark. We have transformed the commitment to "evidence-based medicine" of a particular sort into an intellectual hegemony that can cost us dearly if we do not take stock and modify it. And because peer reviewed publication is the sine qua non of scientific discovery, it is arguably true that hegemony is exercised by the filter imposed by the publication process. The failure of the publication filter to accommodate the kind of discovery that drives most improvement in health care—and the failure of those working in healthcare improvement to reconfigure the filter appropriately—is the message of the paper on publication guidelines by Davidoff and Batalden in this issue of QSHC.¹⁶ This paper is important, not only because it addresses the narrower issue of publication standards but also because it provides important support for an epistemology of a new and broadened understanding of the evidence needed for the improvement of care.

The argument for that epistemology is not a simple one, but its intuitive force is somewhat easier to uncover with a simple question: "How much of the knowledge that you use in your successful negotiation of daily life did you acquire from formal scientific investigation—yours or someone else’s?"

Did you learn Spanish by conducting experiments? Did you master your bicycle or your skis using randomized trials? Are you a better parent because you did a laboratory study of parenting? Of course not. And yet, do you doubt what you have learned?

Broadly framed, much of human learning relies wisely on effective approaches to problem solving, learning, growth, and development that are different from the types of formal science so well explicated and defended by the scions of evidence-based medicine. Although they are far from RCTs in design, some of those approaches offer good defences against misinterpretation, bias, and confounding. In the world of clinical care, especially in the quest for improvement of clinical processes, is it plausible that those approaches—the ones we use in everyday life—might have value too, used well and consciously, to help us learn?

The answer is "Yes". And yet, the very success of the movement toward formal scientific methods that has matured into the modern commitment to evidence-based medicine now creates a wall that excludes too much of the knowledge and practice that can be harvested from experience, itself, reflected upon. The iconoclasts of the past now have power, and they can define who will be seen as iconoclasts of the present.

There is a way out. It involves curiosity. The methods of observation and reflection on the basis of which most human learning occurs and, frankly, on the basis of which many modern industries and enterprises are building their futures, are systematic, theoretically grounded, often quantitative, and powerful. They do not include RCTs, but they honor RCTs in their proper place. They perhaps deserve some honor in return, or at least the serious open minded scrutiny that marks true scholarship.

My close friend and mentor Tom Nolan PhD uses a felicitous term to denote these methods of learning: "pragmatic science".¹⁷ Here are a few of the elements of the methods of pragmatic science:

• tracking effects over time, especially with graphs (rather than summarizing with statistics that do not retain the information involved in sequences);
  
• using local knowledge—the knowledge of local workers—in measurement (rather than relegating measurement to people least familiar with the subject matter and work);
  
• integrating detailed process knowledge into the work of interpretation (inviting observers to comment on what they notice rather than "blinding" them to protect them against what they know);
  
• using small samples and short experimental cycles to learn quickly (rather than overpowering studies and delaying new theories with samples larger than needed at the time); and
  
• employing powerful multifactorial designs (rather than univariate ones when the better questions for the time are formative, not summative).
  

Pragmatic science of this type is alive and well. It thrives in the halls of continual improvement of care now engaging the energies of thousands of healthcare leaders worldwide. It thrives in brilliant texts by theoreticians who have been teachers in sectors of the economy other than health care.^18,19 But, to our great expense, it remains largely trapped on the far side of a publication wall well guarded by academicians who may, I think, have overlearned the crucial lessons of the courageous clinical methodologists of the past few decades. Today’s evaluation methodologists guard not only the portals of our journals, but also our curricula and the minds of our young professionals. Health care has much to gain if those portals now open again to a new wave of disciplined methods of learning from reflective practice, and disciplined methods of sharing the learning through transparent, accurate, and complete published reports—such as the use of publication guidelines—as explained and defended here by Davidoff and Batalden. Health care is too important and too fragile to deny it the benefits of disseminating the hard won fruits of systemic learning, however this learning takes place.

REFERENCES

1. Cochrane AL. Effectiveness and efficiency: random reflections on health services. London: Royal Society of Medicine Press, 1999.
2. Feinstein AR. Clinimetrics. New Haven: Yale University Press, 1987.
3. Bunker JP, Barnes BA, Mosteller F. Costs, risks, and benefits of surgery. Oxford: Oxford University Press, 1977.
4. Chambers TC. Meta-analysis in clinical medicine. Trans Am Clin Climatol Assoc 1987;99:144–50.[Medline]
5. Sackett DL, Rosenberg WM. On the need for evidence-based medicine. Health Econ 1995;4:249–54.[Medline]
6. Edmonson JM. Gastric freezing: the view a quarter century later. J Lab Clin Med 1989;114:613–4.[Medline]
7. Fisher B, Anderson S, Bryant J, et al. Twenty-year follow-up of a randomized trial comparing total mastectomy, lumpectomy and lumpectomy plus irradiation for the treatment of invasive breast cancer. N Engl J Med 2002;347:1270–1.[Free Full Text]
8. National Asthma Education and Prevention Program. Key clinical activities for quality asthma care: recommendation of the National Asthma Education and Prevention Program. Morbidity and Mortality Weekly Report: Recommendations and Reports 2003;52(RR-6).
9. Chalmers I, Enkin M, Keirse M. Effective care in pregnancy and childbirth. New York: Oxford University Press, 1989.
10. Woolf SH, Battista RN, Anderson GM, et al. Assessing the clinical effectiveness of preventive maneuvers: analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. J Clin Epidemiol 1990;43:891–905.[CrossRef][Medline]
11. Institute of Medicine. Assessing medical technologies. Washington, DC: National Academy Press, 1985.
12. Sackett DL. The hypertensive patient: 1–6. Can Med Assoc J 1979;120:1319–20 pp 1477-8 121:7–11 pp 145-8 pp 259-61 pp 397-8.[Medline]
13. US Preventative Services Task Force. The guide to clinical preventative services: report of the United States Preventative Services Task Force. 1st edn. New York: Lippincott Williams & Wilkins, 1989.
14. Egger M, Smith GD, Altman DG. Systematic reviews in health care: meta-analysis in context. London: BMJ Books, 2001.
15. Institute of Medicine. Crossing the quality chasm. Washington, DC: National Academy Press, 2001.
16. Davidoff F, Batalden P. Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project, Qual Saf Health Care 2005;14:319–25.[Abstract/Free Full Text]
17. Brock WA, Nolan KM, Nolan TW. Pragmatic science: accelerating the improvement of critical care. New Horizons 1998;6:61–8.[Medline]
18. Juran JM, Godfrey AB. Juran’s quality handbook. 5th edn. New York: McGraw-Hill, 1999.
19. Langley GL, Nolan KM, Nolan TW, et al. The improvement guide. San Francisco: Jossey-Bass Publishers, 1996.

This article has been cited by other articles:

				M. Coory, S. Duckett, and K. Sketcher-Baker Using control charts to monitor quality of hospital care with administrative data Int. J. Qual. Health Care, February 1, 2008; 20(1): 31 - 39. [Abstract] [Full Text] [PDF]

				A. D. Auerbach, C. S. Landefeld, and K. G. Shojania The Tension between Needing to Improve Care and Knowing How to Do It N. Engl. J. Med., August 9, 2007; 357(6): 608 - 613. [Full Text] [PDF]

				S. M. Shortell, T. G. Rundall, and J. Hsu Improving Patient Care by Linking Evidence-Based Medicine and Evidence-Based Management JAMA, August 8, 2007; 298(6): 673 - 676. [Full Text] [PDF]

				P Pronovost and R Wachter Proposed standards for quality improvement research and publication: one step forward and two steps back. Qual. Saf. Health Care, June 1, 2006; 15(3): 152 - 153. [Full Text] [PDF]

				P B Marck, J A Kwan, B Preville, M Reynes, W Morgan-Eckley, R Versluys, L Chivers, B O'Brien, J Van der Zalm, M Swankhuizen, et al. Building safer systems by ecological design: using restoration science to develop a medication safety intervention. Qual. Saf. Health Care, April 1, 2006; 15(2): 92 - 97. [Abstract] [Full Text] [PDF]

This Article Extract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this link to a friend Similar articles in this journal Similar articles in PubMed Add article to my folders Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Berwick, D M Search for Related Content PubMed PubMed Citation Articles by Berwick, D M Related Collections Guidelines Quality improvement Related Articles