Acquiring Surgical Knowledge: Sound Methodology and Transparent Reporting

    loading  Checking for direct PDF access through Ovid


Innovation has always played a major role in surgical practice. Traditionally, advances were made through trial and error primarily solving acute clinical problems (ie, hemorrhage, perforation). The need to demonstrate superior clinical effectiveness for alternative surgical strategies required the development of rigorous scientific methods of evaluation, the most robust of which is the randomized controlled trial (RCT).1 Yet weak methodology, bias, and incomplete reporting may undermine our confidence in a study’s conclusions despite the title of RCT. In addition, much of the surgical literature remains observational and of varying quality; thus, the answers to many common surgical problems remain elusive. Regardless of study type, best evidence requires the application of sound methodology and transparent reporting of results.
Potential difficulties with surgical research have been well documented and discussed.2,3 Randomized controlled trials are capable of determining cause and effect but, from a surgical point of view, are often seen as difficult and sometimes impractical because of the complexity of surgical interventions and the required resources. Procedural outcomes are often the result of several related yet distinct components that are difficult to examine in isolation. Additional variables include surgical decision making, skills, preferences, and experience all of which require careful consideration during study design to avoid systematic bias.3 In addition to the presence of bias, confidence in estimates of effect from an RCT are lower if a trial demonstrates inconsistency, indirectness, or imprecision.4
As outlined by Cook,2 there are 3 generic types of surgical trials: exploratory trials where the utility of an intervention is determined, explanatory trials to assess whether an intervention is efficacious when performed by surgeons with specific expertise in a well-defined population, and pragmatic trials to assess whether an intervention is effective when performed by surgeons with differing levels of expertise in a general population with broad inclusion criteria. Progression through this hierarchy ensures complete evaluation of an innovation. The challenge is to determine the optimal time to shift from an exploratory (observational) to an investigative study (RCT). If performed too early, the definitive technique may not be established and the results may reflect a point on the learning curve. If the investigative study is too late and the intervention is widely available, equipoise is lost and recruitment difficult. This paradox has been termed Buxton’s law, “It is always too early until suddenly it’s too late.”5 Another challenge specific to surgical trials is choosing the appropriate comparator.3 Patients and surgeons may be less likely to participate when 2 interventions have very different risk-benefit profiles such as a comparison to a newer technique with less experience, a medical treatment, or a sham operation. Despite these hurdles, there is ample evidence in the surgical literature of well-designed RCTs that have answered important surgical questions. Investigators should persevere in the use of this design whenever appropriate.
Although the RCT design provides the best chance of avoiding bias, it is neither practical nor desirable to answer every surgical question because these studies are resource intensive, time consuming, and not always generalizable to larger populations. Many surgical procedures that are accepted as standard of care have not been subjected to the scrutiny of an RCT, and, at this point, such a trial would be unethical.3 Observational studies can provide relatively strong evidence to support advances in surgery when the results demonstrate large treatment effects or when the effect of any confounding factors would bias the results in opposition to the observed effect.4 Large well-designed observational cohort studies can be used to determine and validate risk assessment for surgical therapies and monitor safety in practice.
    loading  Loading Related Articles