When Gary Klein told us that he had co-authored a paper critical of evidence-based medicine (EBM), it was music to my ears. After we finished taping the show, I quickly dug into his bibliography and found the paper in question, written by Klein and his team: “Can We Trust Best Practices? Six Cognitive Challenges of Evidence-Based Approaches.”
Despite the general appeal of the title, the paper treats exclusively of EBM because the authors view EBM as a prime example of a “modernist” and “rationalist” love affair with “cognitive engineering insights and methods” that focus on the use of data to improve quality in a wide variety of human fields of activity. As they note, EBM is important because it is in healthcare that such a rationalist approach has been most extensively developed.
The paper is a great complement to the critique brought to bear by Mark Tonelli, another of our podcast guests (episode 30). Tonelli and Klein make many of the same points, but they argue from different intellectual perspectives, so there is value in studying them both.
Klein: The importance of intuition and pattern recognition
In their critique of EBM, Klein and colleagues identify of number of situations where a formal and more algorithmic analysis of a situation falls short.
A prime example is in establishing the diagnosis for a particular patient. Evidence-based medicine may give the relevant steps to follow when someone is afflicted with sepsis, but determining that a given patient is indeed septic requires experience and reliance on intuition and pattern recognition (challenge #1 in the paper).
Of course, the EBM crowd will cry foul against such an alleged straw man criticism: “EBM does not deny the importance of clinical judgment!” they might say, citing David Sackett’s authoritative definition of EBM.
But what happens when clinical judgment and formal clinical research evidence are at odds (challenge #3 in the paper)? I have yet to see self-proclaimed followers of EBM who will not ditch the evidence in favor of their intuition. In such a cases, however, instead of admitting the primacy of clinical judgment, EBMists invariably justify their behavior by criticizing the evidence as insufficient or tainted!
Another challenge that Klein and colleagues highlight is the difficulty of applying simple rules to complex patients (challenge #4). This is frequently but mistakenly conceived as a problem of “generalization” of research findings.
Rather, as Sandra Tanenbaum has aptly remarked a few years ago, the problem is not of generalizing but of “particularizing” the knowledge gained from clinical research to the individual patient. Because every single patient is at once “similar to” and “different from” the patients included in the clinical research, the physician must decide whether the similarities and differences are sufficiently pronounced to justify applying or rejecting “the evidence.” This can only be a work of judgment based in experience and intuition.
Other challenges include revising treatment plans when those are not working (challenge #5) and adopting remedies that have not been vetted by empirical research (challenge #6)
Tonelli: Medical decision-making as a form of argumentation
Tonelli’s critique of EBM shares much in common with Klein’s, but he also brings a unique perspective to the problem.
In a provocative paper entitled “Evidence-Free Medicine: Foregoing Evidence in Clinical Decision Making,” Tonelli remarks that
Clinical medicine is not a deductive discipline, but rather utilizes informal logic more familiar to argumentation than to scientific investigation. Clinical medicine is a casuistic (case-based) enterprise, personal and prudential, requiring clinicians to weigh and negotiate between multiple potential facts, values, and reasons in order to arrive at the best choice for a particular individual in need of healing. Clinical judgment requires the use of various kinds of reasons and reasoning. Done well, clinical decision making closely resembles the structure of argumentation with careful consideration of many facts, warrants, backings, and rebuttals, ultimately resulting in a conclusion that is only probably, never demonstrably, correct.
Drawing on the reflections of philosophers such as Stephen Toulmin, Albert Jonsen, and Edmund Pellegrino, Tonelli makes the persuasive case that it’s in the nature of medical decision making that the factors that end up providing a warrant for the medical decision a doctor chooses to make cannot be pre-specified.
Instead, the physician is constantly weighing a multiplicity of factors that bear on the decision. Those factors may come from 5 broad categories of evidence which he classifies as follows:
- Empirical evidence derived from clinical research
- Experiential evidence derived from personal clinical experience or the experience of others (expert opinion)
- Pathophysiologic rationale
- Patient values and preferences
- System features, including resource availability, societal and professional values, and legal and cultural concerns.
None of these categories can be ranked a priori as weighing more or less than another. In any particular case, the physician may come to a decision based on one or more factors, drawn from one or more of these categories. The decision is ultimately a matter of judgment and cannot be replaced—or even influenced—by a rule or algorithm.
Klein and Tonelli have given us much to reflect on. I really encourage you to listen to the 2 podcast episodes (see links below) and to read their work. This is of prime importance for a good practice of medicine.