DOI: 10.1097/EDE.0b013e3181e9bbbd

,  

,  

PMID: 20657294

Issn Print: 1044-3983

Publication Date: 2010/09/01


Print

Meta-analysis Protocol Registration: Sed quis custodiet ipsos custodes? [But Who Will Guard the Guardians?]

Bahi Takkouche; Guy Norman

+ Author Information

Author Information: a

    loading  Checking for direct PDF access through Ovid

Excerpt

A tidal wave of guidelines is flooding the epidemiologic world. Many have in common the requirement that research protocols should be registered when still in an embryonic phase. Having hit the world of clinical trials, this wave of requirements has reached observational studies, and, most recently, meta-analyses. The PRISMA (Preferred Reporting Items Systematic Reviews and Meta-Analysis) guidelines demand the prior registration of the protocol of any systematic review and meta-analysis, requiring that this protocol should be made accessible before any hands-on work is done.1 Such registration is claimed to be similar in rationale to the registration of clinical trials. It will “reduce the risk of an excessive number of reviews addressing the same question” and allow tracking of any subgroup analysis that has been carried out without prespecification in the protocol.
We believe that this requirement is unjustified and rather insidious. Certainly, the registration of clinical trials is necessary: these are experimental studies with human beings as subjects, and the scientific community needs to know what is being done to monitor possible adverse events. But why does a meta-analysis need to be registered? What is to be gained by this institutionalization of the meta-analytic approach?
We would raise the following questions. Who decides whether an existing review is too old and deserves a new try, or whether it remains valid? Who gives the registration number and under what conditions? Who decides whether a new meta-analysis will or will not be allowed to investigate aspects that were not included in a previous meta-analysis on the same topic (inclusion of a new subgroup analysis, inclusion of studies published in languages other than English, etc.)? Are we not likely to see powerful groups racing to register tens or hundreds of potential topics for meta-analysis, on the off-chance that they may eventually be able to perform them? And perhaps most importantly: who controls the people who control the register and make crucial decisions about other researchers' work? Sed quis custodiet ipsos custodes?
Meta-analyses are published only after having passed through 2 filters: peer review, and an editorial decision. We believe these filters suffice to decide whether a meta-analysis is good enough and novel enough to deserve publication. An additional committee or register does not increase the quality of what is published: it only weakens the decision capacity of editorials and reviewers, and increases bureaucracy.
We also dispute the view that subgroup analyses and similar approaches should be used only if they were prespecified. Why should a discovery driven by the preliminary results of the meta-analysis be less reliable than one that was prefigured in the initial protocol? In a recent meta-analysis,2 we showed that exposure to pets is a strong risk factor for asthma in countries in which, for cultural or religious reasons, pets are not frequently kept at home. This result is probably because frequent exposure to pet allergens may induce tolerance, and a low prevalence of pet-keeping means that few people become tolerant. We see nothing questionable about reaching this conclusion after we became aware that some of the studies we had retrieved had been carried out in Arab and African countries, where pets are not generally welcome in the home.
Our discontent with this specific requirement would have remained purely theoretical had we not recently encountered comments from reviewers of a meta-analysis submitted to a major journal, demanding proof that we had published the protocol and adhered to the PRISMA guidelines. In fact, we are not aware of any register for meta-analyses, although it seems that journals are increasingly requesting prepublication of protocols.
    loading  Loading Related Articles