Selection Bias

Conflating a certification, license, or degree with actual ability is easy. We look on the wall, see the certificate, and assume that the paper implies ability. These days we especially pay attention to online reviews. A glowing five-star review with a paragraph of praise is worth money. If only reality was so easily aligned with glamour. Use of other criteria seems to vanish when compared to just a few simple guides.

A cross-cultural study done years ago looked at top-rated medical practitioners in various cultures, not just western ones. The finding boiled down to a few basic features.

Across cultures, there seemed to be three common traits between the folk practitioner in Mexico, Papua, and the MD in New York. Things they had in common? All possessed:

  1. a very effective bedside manner
  2.  expensive fees
  3. a piece of impressive technology that they used in treatment.

So, note that I described the practitioners as top-rated, not necessarily effective. 

But I don’t want to pick on the medical profession. Unfortunately, we apply the same criteria to selecting services from financial advisors and other “professionals.” Related standards select schools, certification programs, and consumer products.

Here’s my evaluation. In a world where there were never so many research tools available for the considerate and intelligent user to manipulate, There was never so much lousy research. So what do people base their analysis on? I suspect that it’s Yelp reviews – how effective the bedside manner was; Cost – is it expensive and therefore better; and how unique the technology is.

So here is my challenge. Broaden your evaluations. Take into account more factors.

It’s a challenge and may take more time, but hey…you’re not chicken…are you?

3 Replies to “Selection Bias”

  1. Ha, great examples! I had one of my over 90 year patients come in once saying, I think I have influenza. I though so too, because she had a fever and a fast heart rate and the history fitted. I called the emergency room to let them know I was sending her (from less then a mile away). The ER doctor says, “Did you do an ECG?” I said, “Oh, no, I don’t have an ECG machine.” “WHAT?!?” he says, “You don’t have an ECG machine! You can’t have a clinic without one!” “Oh, it’s in the shop right now,” I assured him. I did have an AED, on long term loan from a retired flight surgeon. My over 90 year old went to the ER and as she and I both thought, she had influenza, needed oxygen, and was not having a heart attack. I ran my solo clinic for ten years and the best piece of “equipment” I had was my ears and eyes and brain and stethoscope.

    1. The very best physicians in medieval Spain relied on two primary diagnostic tools the pulse, which they divided and carefully analyzed, and the glass urine bottle, because analyzing the color, turbidity, and sedimentation of urine was considered to be one of the best diagnostic tools they had.
      They were dependent on their eyes, ears, nose, brain, and of course the depth of their diagnostic experience. The technology is great, but often as you point out it’s the basic built-in tools that make the correct diagnosis.

Comments are closed.

%d bloggers like this: