Irony and the Elephant in the Review

    loading  Checking for direct PDF access through Ovid


In the October 2016 issue of Anesthesia & Analgesia, the confounding issue of publication bias (herein defined as the disproportionate publication of positive versus negative study results) on the interpretability of systematic reviews and meta-analyses was twice highlighted. The first was by Hedin et al1 in their own systematic review and meta-analysis of other meta-analyses specifically addressing publication bias and nonreporting in the “top 5” anesthesiology journals. It was also the subject of an accompanying editorial by Dalton et al,2 who highlighted the “elephant in the review” when Hedin et al1 showed that only half of the published systematic reviews in these top 5 anesthesiology journals actually presented any formal assessment of publication bias.
It is rather ironic then, that 40% of the systematic reviews and meta-analyses that were published in that same issue failed to include any reporting of publication bias. This highlights just how difficult it is for journals and their editorial teams to adequately screen even their own articles for this reporting shortcoming.
Despite continuing discussions in the literature as to how authors and journals can address the issue of publication bias,3 it seems to be as problematic today as it ever has been. This was recently highlighted in a study addressing the publication bias of articles originating from the 5918 abstracts presented at the 2001 to 2004 annual meetings of the American Society of Anesthesiologists.4 In that study, only 54% of those abstracts went on to subsequent publication in the 10 years that followed (relative risk for abstracts with positive results proceeding to publication, 1.42; 95% confidence interval, 1.22 to 1.66; P < .001).
Strategies to address the issue of publication bias were also discussed in the same article by Hedin et al.1 For example, they noted that the International Journal of Radiation Oncology Biology Physics is testing a pilot program in which the publication of the methods section of a study proceeds without any data analysis, indicating that regardless of what results were obtained (ie, positive or negative), the study results would still be publishable. Indeed, this 2-tiered publication approach (ie, publish the methods first, and then the results) has also been suggested recently by Law and Lo5 as a means to improve the rates of negative studies being included in journals. In terms of addressing publication bias in meta-analyses, attention to reporting guidelines, such as those promulgated in the PRISMA checklist (, ensuring that all its items (such as PRISMA item No. 15 pertaining to publication bias) are accounted for, is an important mechanism that journal editors can use to help avoid this issue, although most editors (myself included) will admit that a few do slip through the cracks.
In summary, it appears that the issue of publication bias, both in the failure to report negative studies and the failure to account for them in meta-analyses, will remain in our literature for some time. Despite our best efforts, we still have a long way to go to eradicate this confounding influence in the literature. The issue of publication bias deserves continued attention from writers, readers, and editors in both reports of original investigation and in meta-analyses.
    loading  Loading Related Articles