Are We Too Quick to Alter Our Practice Patterns Because of the Results of a Randomized Controlled Trial?
A closer scrutiny of the methodology and validity of many level 1 randomized controlled trials may dampen the rush to adopt new technologies without additional evidence. Dr John Ioannidis is Professor of Medicine and Health Research and Policy at Stanford University School of Medicine. His publication in 2005, “Why most published research findings are false,” is one of the most cited publications in the field.8 He demonstrated convincingly that 80% of nonrandomized studies turn out to be in error, 25% of randomized controlled trials are also flawed, and flaws were found in 10% of large clinical trials. In randomized controlled trials, he found that it was easy to manipulate results at every step, make a stronger claim, or select what is going to be concluded. A range of errors can often be found, including what questions researchers asked, how they set up the study, which patients were included, which measurements to use, how the data were analyzed, and how the results were presented. In another publication, he studied 49 of the most highly regarded findings in medicine over the prior 13 years, with 45 having claimed to have uncovered effective interventions. Thirty-four of these were retested, with 14% or 41% shown to be wrong or substantially exaggerated.9
Are we not seeing a similar situation with widespread enthusiasm for the surgical management of fractures of the distal end of the radius in older age patients with anatomically shaped volar plates. In this case, there were not even randomized control trials supporting this approach but rather level IV case series. Mirroring the experience with the clavicle fracture, subsequent randomized controlled trials comparing the surgical treatment with closed reduction and cast application have not supported significantly better patient-rated nor surgeon-rated outcomes with surgery.10–12
Our specialty and our patients will continue to benefit from the adoption of new technologies; however, careful scrutiny of published results even at level 1 may well help to avoid unexpected complications.