Ready, Aim, Judge
As a new patient rolls past the nurses’ station and into the intensive care unit, the transportation crew drops a thick packet of paper onto the counter with a sharp slap. The physician on duty eyeballs the new arrival while critical care nurses swap out monitor leads, check IVs, and document the patient’s every scar and blemish on an endless flow sheet. Seeing no urgent concerns, the physician turns his attention to the mound of paper that taunts him from the counter.
He sits at his workstation and flips through the packet like it’s the world’s longest restaurant menu, and he has no appetite. Every few seconds he pauses, rolls his eyes, and scoffs like he is trying to expel a nonexistent hairball. He finally pushes the stack of paper away having reached his conclusion: those fools at the transferring hospital sent us a mess.
“Thank God this poor guy is now at a hospital with real doctors,” he says and those within earshot cluck their agreement. After all, anyone looking over the patient’s weeks-long hospital course can see the mistakes. And those errors would have never occurred if the patient were in this ICU, right?
In healthcare, we often subject our colleagues to these ad hoc trials. After all, retrospection is the foundation of our training. We take a patient’s history; we piece together clues from the past to illuminate the present. And when we aim that laser-focus on a patient who has already received medical care, we inevitably unearth the flaws of our colleagues.
But is our interpretation of those prior actions an accurate assessment of the past? Or is it a cognitive trap, an illusion created by a minefield of biases?
And if hindsight itself is flawed, what undue damage are we causing in the name of a faulty principle? Have we created an environment of punitive collaboration? Have we opened the door to professional mistrust and toxic communication? Or something even worse?
Let’s look at two factors that set up the retrospective firing squad we so often perpetuate in medicine: a chronic overconfidence in our own predictive abilities and a cause-and-effect mentality that lends itself to high levels of hindsight bias.
It’s not me, it’s you…
Any time we look back at a decision and think, “Wow, that person is an idiot,” we are really saying, “I would have known better.” Of course, in order to think this, we first must make an assumption about the difficulty of the decision we are judging.
So, how good are we at predicting the outcomes of clinical events? And how much does our confidence in our predictions correlate with their accuracy?
It turns out: not very well at all.
In a 1993 study, researchers asked a group of physicians to predict hemodynamic measurements (pulmonary capillary wedge pressure, cardiac index, systemic vascular resistance) in a group of real patients prior to right heart catheterization. These physicians also ranked their confidence in their guesses prior to learning the real values.
Overall, the physicians did a poor job of predicting the right answers but, more importantly, their level of confidence in their guesses had no correlation with their accuracy—they felt just as good about their correct guesses as their incorrect ones. Experienced physicians were no better at estimating values than their rookie colleagues, but the experienced doctors were, of course, more confident in their predictions.
A 2013 study in JAMA found similar results when it presented physicians with clinical vignettes and asked for a diagnosis and a confidence rating in that diagnosis. As you might expect, diagnostic accuracy was much lower for the more difficult clinical cases. But physicians’ confidence in their predictions was uniform despite the case difficulty; they felt just as confident when they were right as when they were wrong.
Nurses seem to face the same problem. Nurses and nursing students presented with real clinical vignettes were asked to predict the likelihood of a dangerous event in each situation. Like our first example, although experienced nurses reported much higher levels of confidence in their answers, they were no better at predicting adverse events than their inexperienced counterparts.
This overconfidence across the spectrum of healthcare workers—especially experienced ones—marks a poor starting point from which to judge the behavior of others. It’s the first misstep in the logic trap of retrospective judgment.
But it isn’t the only mental trickery at play.
Now that you mention it…
Have you ever watched a movie with a friend and heard him say something like, “Man, that smudge on the screen is really bothering me?” Even if you had never noticed it before, the imperfection your friend pointed out is now impossible to ignore. No matter how you try to tell yourself you don’t see it, you can’t scrub the knowledge from your brain. It might ruin the entire film.
Knowledge of the present does the same thing when we try to evaluate events in hindsight. It is impossible to know how a past situation looked in the past because we can’t scrub the present from our mind.
What does that look like in a clinical context?
A 2010 study tested this concept in a group of radiologists. The physicians were shown brain CT scans of a group of patients who presented with ischemic stroke. The radiologists were told to interpret the images without a medical history or any subsequent diagnostic studies. They were specifically asked to estimate the chances an acute stroke was present and, if so, to identify the location of the stroke.
After a washout period, the studies were shuffled and shown to the same radiologists again. This time, however, they were told the presenting symptoms and given a follow-up MRI to read. How do you think they did? The radiologists were, of course, far better at finding strokes on CT scans when they had additional information and knew the outcome of a follow-up MRI.
But before you dismiss that as an obvious outcome, ask yourself how this scenario is any different than a clinician comparing the judgment of a colleague to his own judgment an hour, or a day, or a week later. The passage of time inevitably yields additional information that, in turn, increases the likelihood of a “good decision.”
Combine this advantage with the fact that we chronically overestimate our own predictive ability, and we have created a playing field that, in retrospect, could not possibly do justice to our colleagues of the past.
But is there yet another cognitive effect tilting the scales against our judgments of yesterday? And does the psychology behind it call into question one of our oldest institutions?
We’ll explore that in Hindsight is Blind, Part II.
The Contralateral 