|| Checking for direct PDF access through Ovid
Assessing the population-level impact of any new vaccine is necessary to inform public health decision-making. The most common approach for such assessments is time series analysis, which compares disease rates and trends before and after vaccine introduction. However, such studies are subject to a number of important biases and confounders, which can lead to either overly optimistic or pessimistic assessments of vaccine programs. These studies can be strengthened using best practices from epidemiology and biostatistics, as well as employing tools from other related fields, such as econometrics. We argue that following four basic recommendations could increase the accuracy, reproducibility, and comparability of such studies:Report on outcome specificity used in a study in relation to previously published studies. Relative estimates of vaccine effect are expected to rise with increasing outcome specificity.Apply bias-reducing model selection procedures by using data-driven model selection or model averaging strategies.Assess the likelihood of detecting a vaccine effect by performing simple power calculations with simulated time series.Integrate multiple comparison outcomes to address unmeasured bias and confounding using a data-driven weighting of outcomes (each unaffected by the vaccine introduction) into a “synthetic control.”While there are examples of vaccine impact studies that apply some of these suggestions, this is often not the case and it is rarely seen in combination. Following these best practices is particularly important for studies where the relative impact of the vaccine is expected to be modest or where there is a strong possibility that factors other than vaccination could influence disease rates. For example, studies of vaccine impact against influenza, rotavirus, human papillomavirus, and pneumococcus would all benefit from consistent application of these principles.1–7To highlight problems that can occur in vaccine impact studies, and to propose potential solutions, we examine in detail assessments of pneumococcal conjugate vaccine as an intervention against pneumonia. Pneumococcus is a major cause of pneumonia and invasive pneumococcal disease globally.8,9 Pneumococcal conjugate vaccines are associated with a significant and long-term reduction in the incidence of invasive pneumococcal disease in areas of introduction.10–12 Although this disease is severe and can lead to death and disability, it occurs with substantially lower incidence than pneumonia.8 This means that the high cost of the vaccine cannot always be justified based on its impact on invasive pneumococcal disease alone.13 Therefore, decisions about the use of pneumococcal conjugate vaccines that rely on cost-effectiveness considerations depend critically on the expected population-level impact against pneumonia, a far more common condition. Several randomized controlled trials (RCTs) have assessed the efficacy of pneumococcal conjugate vaccines against different definitions of pneumonia.14–18 However, accurate population-level assessments of pneumococcal conjugate vaccines against pneumonia are needed, as these capture both the direct effects of the vaccine and positive and negative population-level effects such as herd immunity and strain replacement, respectively.19–21 Wide variability in published estimates of vaccine impact against pneumonia highlights the difficulties with the interpretation of such analyses and the need for more robust statistical methods.22–24Assessment of the population-level impact of pneumococcal conjugate vaccines against pneumonia is challenging. While invasive pneumococcal disease is, by definition, caused by pneumococcus, pneumonia can be caused by many different pathogens. Identifying a causative agent for pneumonia is complex and has a high failure rate when attempted, making it difficult to determine the fraction of pneumonia cases caused by pneumococcus.25 This issue substantially complicates both RCTs and population-level studies of vaccine impact against pneumonia.