Advertisement

 

 

Patient Satisfaction Surveys Are Useless —Take #2

Patient Satisfaction Surveys Are Useless —Take #2
Author Information (click to view)

Skeptical Scalpel

Skeptical Scalpel is a recently retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last two years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages 800 page views per day, and he has over 4500 followers on Twitter.

+


Skeptical Scalpel (click to view)

Skeptical Scalpel

Skeptical Scalpel is a recently retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last two years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages 800 page views per day, and he has over 4500 followers on Twitter.

Advertisement
When the scores fall, task forces are formed and meetings are held. A slight improvement is often seen. But once the scores improve, everyone goes back to business as usual and the scores drop again.

Patient satisfaction surveys are theoretically good things. After all, hospitals and doctors should know what their patients think about them. If a service is not performing well, the results of a patient satisfaction survey would be useful in helping us make improvements. This would be true if patient satisfaction could be measured in a meaningful way. In a previous blog, I explained some of the statistical problems with all patient satisfaction surveys.

Here’s an example of the current state of the art and why these surveys are useless.

untitled3

 

 

 

 

 

 

 

 

 

A hospital posted the above graphic. As provided by a leading company, it shows the trend of patient satisfaction scores for the question “Were you informed about the causes of delays during your emergency department [ED] visit?”

You can see that the scores fluctuate around the average score of the 79th percentile for the year, meaning that 21% of comparator hospitals fared better on this question.

This ED receives more than 25,000 visits per year, or over 2000 visits per month. The number of responses to the survey questionnaires ranged from a low of 2 in June of that year to a high of 42 in October of the year before. This represents a range of 0.1% to 2.1% of all ED visits — far below what even a non-statistician would recognize as a reasonable sample.

Despite efforts to improve scores for this question, no significant change has occurred.

What does it all mean? If you look at the trend line, you will note that it mirrors what occurs in a feedback loop. [Digression: A thermostat is a classic example of a feedback loop in action. If you set the thermostat for 70 degrees, the temperature will fluctuate about that point. Most of the time it will not be 70 degrees in the room as the furnace will run until the temperature is 70 and when the furnace shuts off the temperature will overshoot. Likewise when the temperature falls below 70, it will take time for the furnace to warm the room back up. (Ref. GJS Wilde, Target Risk 2, PDE Publications, Toronto 2001)]. This feedback loop phenomenon has been blogged about on Smartlyheated.com as a classic reaction to the posting of such scores. When the scores fall, task forces are formed and meetings are held. A slight improvement is often seen. [Hawthorne effect: Performance improves when subjects are aware they are being watched.] But once the scores improve, everyone goes back to business as usual and the scores drop again.

And don’t forget, the next month’s responses will not reflect what a task force will do this month, as there is a time lag of at least a few weeks for surveys to be sent and returned.

It’s too bad that the 87.5 percentile score was achieved when the response to the questionnaire was a mere 2 patients.

What do you think happened to the score the next month?

Skeptical Scalpel is a recently retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last two years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages 800 page views per day, and he has over 4500 followers on Twitter.

4 Comments

  1. Would be interesting to see over the next couple of years how prescriptions for antibiotics, pain meds, and other unnessasary medical tests and treatments may spike as physicians and other medical providers attempt to improve their patient satisfaction scores.

    Reply
    • Bonita, many have written about what you just said. I’m pretty sure that the desire to please will override the lack of medical necessity, especially if penalties are attached to bad satisfaction scores.

      Reply
  2. Regarding your summary judgment on evaluative surveys today, and despite your depressing stats, I will persevere in getting my personal witness to the medical crises of my life “out there.” Maybe…just possibly…some poor fool will benefit (if it’s only me.)

    I do know one thing: The mere telling of all my experieces will be catharctic (I hate that word!) for me. If anyone else is affected, I’ll consider it an added perk…

    Reply
    • Thanks for the comment. I agree that the cathartic effect of venting on line might help you. Whether it helps anyone else is hard to say.

      Reply

Trackbacks/Pingbacks

  1. Week in Review | Kroger | Burrus – Medical Malpractice & Health Law Blog - [...] Patient Satisfaction Survey Value Questioned Patient satisfaction surveys may prove useless in the long run. While negative responses inspire…

Submit a Comment

Your email address will not be published. Required fields are marked *

10 − eight =

[ HIDE/SHOW ]