Advertisement

 

 

Discrediting a Paper About Discredited Practices

Author Information (click to view)

Skeptical Scalpel

Skeptical Scalpel is a retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last three years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages over 1300 page views per day, and he has over 8000 followers on Twitter.

+


Skeptical Scalpel (click to view)

Skeptical Scalpel

Skeptical Scalpel is a retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last three years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages over 1300 page views per day, and he has over 8000 followers on Twitter.

Advertisement
Maybe one should not necessarily base an opinion about whether a practice has been discredited or not on a single paper in one journal.

According to a study in the Mayo Clinic Proceedings, 146 papers appearing in the New England Journal of Medicine over the first decade of this century contradicted medical practices previously thought to be effective.

The paper’s findings were widely publicized. There was talk in the New York Times of inertia hindering change and allowing ineffective treatments to continue for years.

The full text of the paper and a supplement containing a brief summary of all 146 discredited practices are available online.

I decided to see for myself if any practices relating to general surgery were included in the paper and found 11.

Two of them seemed highly debatable to me.

Number 43 on the list was a comparison of open mesh to laparoscopic mesh inguinal hernia repair that appeared in NEJM on 4/29/04. This was the critique:

“A laparoscopic approach to repair inguinal hernias with mesh was thought to have lower hernia recurrence rates and less post-operative pain. This multicenter, randomized trial in a VA population found that the laparoscopic approach led to a higher rate of complications and a higher rate of recurrences when repairing primary inguinal hernias.” 

It definitively closed the door on laparoscopic inguinal hernia repair. Or did it?

Follow-up letters to the editor pointed out that the laparoscopic recurrence rate of 10% in the VA study was much higher than in other reported series, and the size of the mesh (~8.0 cm) used in the laparoscopic cohort was much smaller than the 10 cm x 15 cm that most experts recommended.

How has the VA paper affected surgeons’ choice of technique for hernia repair?

A report from the American Journal of Surgery in 2012 found that as of 2008 at the Mayo Clinic, 41% of inguinal herniorrhaphies were performed laparoscopically.

Looking at national resident case logs data for 2012 from the ACGME, 35% of all groin hernia repairs were done laparoscopically.

Despite having been “discredited” in NEJM, laparoscopic inguinal hernia repair is quite alive and well.

The Mayo Clinic Proceedings paper also stated that preoperative biliary drainage for patients with cancer of the head of the pancreas was discredited by another NEJM paper for 1/14/10. Here is what they said about number 131 on their list:

“Jaundice in surgical patients is postulated to increase the rate of postoperative complications. Many surgical centers have employed biliary drainage prior to surgical intervention for cancer of the head of the pancreas, but there is conflicting evidence regarding its effect on morbidity and mortality. This multicenter, randomized trial found that routine preoperative biliary drainage increases the rate of serious complications without a mortality benefit.”

Subsequent letters criticized the study because patients were drained for 6 weeks prior to surgery, which was longer than the norm of 2 weeks, patients with bilirubin levels above 14.6 gm/dL who were most likely to benefit from preop drainage had been excluded, the wrong type of stent was used, and prophylactic antibiotics were not uniformly administered.

Is preop biliary drainage still being used?

A randomized trial from South Korea in the July 2013 American Journal of Surgery showed that preoperative biliary drainage for longer than 2 weeks resulted in twice as many complications as drainage for less than 2 weeks (25.9% vs. 9.1%, respectively). This compares favorably to the 74% complication rate of 6 weeks of drainage found in the 2010 NEJM study.

In the July 2013 American Journal of Gastroenterology, a group from Memorial Sloan Kettering published a retrospective review of over 500 pancreaticoduodenectomy patients, 220 of whom had preop stents. The overall complication rates did not differ whether a stent was used or not.

Again, despite being “discredited,” the use of preoperative biliary drainage continues to be very common.

So, what happened here?

The only surgeon among the authors of the 146 discredited practices paper is a third-year general surgery resident. Maybe he did not have enough experience to evaluate these papers and their impact.

Or maybe one should not necessarily base an opinion about whether a practice has been discredited or not on a single paper in one journal.

The findings about these two topics–hernia repair and biliary drainage–lead me to question just how many of the other supposedly discredited practices are really no longer indicated or used.

Skeptical Scalpel is a retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last three years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages over 1300 page views per day, and he has over 8000 followers on Twitter.

4 Comments

  1. There are two issues here that should not be conflated. 1) is the refuting study a high-quality study, and 2) do clinicians abandon a practice when there is evidence it is of no value? You can’t use the fact that clinicians continue to employ a practice as evidence that the refuting study wasn’t any good. The history of modern medicine is littered with examples of practices that have been shown to be of little or no value to the patient but nevertheless persist in widespread practice, and which are justified by practitioners using low quality studies to claim that a high-quality study is wrong.. Vertebroplasty and kyphoplasty leap to mind. Cardiac stents for asymptomatic patients. Mastectomy for early-stage breast cancer (which should be the woman’s choice, not the clinician’s).

    Reply
    • Thank you for commenting. Let’s look at the study that “refuted” laparoscopic herniorrhaphy. As I said in the post, letters to the editor of the NEJM pointed out significant flaws. I did not mention the fact that only one surgeon in the study was experienced in the technique of laparoscopic repair. There were similar flaws in the biliary drainage paper. Without my specifically saying so, I think you could surmise that the refuting studies were flawed.
      ****
      I agree that doctors have continued to employ practices of little or no value, but the paper discrediting many practices failed to shed any real light on the two practices I wrote about. As Dr. Ashish Jha said on Twitter, “one contradictory study doesn’t discredit entire practice.” And since most of supposed discredited practices involved areas of medicine outside of my field, I leave it to others to say which of those other practices are truly discredited.

      Reply
  2. The article clearly states that it is not claiming that a single paper necessarily invalidates anything. However, it does show the frequency with which existing practices are called into question by newer trials. Additionally, the fact that a practice may remain common does not mean that it is supported by evidence or that it has any validity. But given that many therapies are the standard of care based on no evidence or weak evidence, we ought to at least stop and think when RCTs come out that contradict existing practice.

    Reply
    • The title of the paper is “A Decade of Reversal: An Analysis of 146 Contradicted Medical Practices.” Contradicted means to deny the truth of something. As I said in the post, the New York Times talked about “inertia hindering change and allowing ineffective treatments to continue for years.” I have no problem with your suggestion that we stop and think when contradictory RCTs appear, but the paper and reactions to it seemed to indicate that it was all settled with the “146 Contradicted Medical Practices.” That is simply not so. And I cited evidence that the NEJM contradictory papers had their own flaws.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

eleven + 18 =

[ HIDE/SHOW ]