The growth in comparative effectiveness research and evidence-based medicine has increased attention to systematic reviews and meta-analyses. Meta-analysis synthesizes and contrasts evidence from multiple independent studies to improve statistical efficiency and reduce bias. Assessing heterogeneity is critical for performing a meta-analysis and interpreting results. As a widely used heterogeneity measure, the I2 statistic quantifies the proportion of total variation across studies that is caused by real differences in effect size. The presence of outlying studies can seriously exaggerate the I2 statistic. Two alternative heterogeneity measures, the
have been recently proposed to reduce the impact of outlying studies. To evaluate these measures’ performance empirically, we applied them to 20,599 meta-analyses in the Cochrane Library. We found that the
have strong agreement with the I2, while they are more robust than the I2 when outlying studies appear.