top of page

When Data Misinforms

 

The statistics were grim. World War II Bomber crews faced an average life expectancy of 11 missions. 55,573 crew members were killed out of a total of 125,000 aircrew, (44.4% death rate). The average age of death was 23 years. Further, 8,403 members were wounded in action and 9,838 became prisoners of war. Of the 12,731 B-17 bombers built, 4,735 were lost – a loss rate of over 37%.


The returning bombers were analyzed. The commanders saw the problem clearly. Put more armor where most bullet holes are. They were wrong.


Abraham Wald was a mathematician looked at the problem differently. He noted these planes were the ones which returned. These bullet holes marked the non-fatal damage areas. If it were possible to study the planes shot down, they would likely be found to have an inverse pattern. This flawed way of thinking became known as survivorship bias.


Survivorship bias is the logical error made when we concentrate on the things that made it past a certain selection process, while overlooking those that didn’t. Where you learn lessons only from successful outcomes, because the failures have no voice and remain unseen.


The book, Moneyball, speaks about how Billy Beane revolutionized baseball by creating a championship team from players nobody wanted. Beane proved institutional baseball was valuing the wrong things. More important than hitting home runs is avoiding being put out. Finding a way to get on base became the most valued commodity. No getting on base was the failure.


We face bias in medicine, especially as it is applied to metrics. Valuing the wrong things and not asking the right questions results in clinics extending enormous resources to meeting, collecting and reporting data that makes little, if any, difference in outcomes or patient experience.


Instead of asking providers and staff to provide unending questionnaires to patients and collect data mandated by CMS and OHP, we would do well to ask patients what matters most to them. These are the questions I believe may come out of that:


1. Did I have access to care?


2. Did I receive appropriate care?


3. Was I treated with respect?


I have found that third concern is powerful. It can be difficult to provide due to demanding unpleasant patients with unrealistic expectations. I have found these individuals are a minority that cast a wide shadow in our mind. I have also found the most effective approach to modify unpleasant behavior is to maintain a respectful, though firm, personage. This elevates the interaction and sets a tone for adjusted expectations.


Much of our population are unaccustomed to receiving this dignity in their job, at home, or in life. Sometimes, we see the patients nobody else wanted. There is a grace in surprising with kindness. An aura of respect imparts dignity to both of us. Even when it may seem undeserved, we find a way to treat the wrong person right.


After a sense of trust is established, we may find our way to the story the behavior is speaking. It always starts with asking the right questions. I wish there was a metric for that.

 

Tim Powell MD

Recent Posts

See All

A Friend Like That

My younger sister, Esther, underwent coronary artery bypass surgery this August. It should have been me. Esther weighs about 110 pounds,...

Playing Fair

“There is no such thing as new truth; error might be old or new, but truth is as old as the universe.” Frederick Douglass, 1863 “Popular...

Where You’re Going Matters

Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat. “I...

Comments


bottom of page