Background: Ohio has been funded from the Centers for Disease Control and Prevention for the Paul Coverdell National Acute Stroke Prevention Program for over ten years. This data driven program focuses on reducing stroke mortality and morbidity with emphasis on stroke systems of care throughout the continuum. Inter rater reliability is required to confirm reliability and inform quality initiatives. The objective of this study is to determine the annual inter-rater reliability, utilizing Item Specific Percent Agreement (ISPA),after comprehensive abstractor training, in 45 Ohio Coverdell hospitals.
Methods: We developed a comprehensive abstraction training program which included: in-person, all-day training for abstractors and reabstractors; reference binders; Question and Answer webinars; abstraction updates at statewide tri-annual Coverdell meetings and easy access, prompt, technical assistance communications for specific questions. Results are shared with individual hospital identifying specific areas for improvement. We examined 395 randomly selected charts from 45 hospitals to determine the overall state and individual hospital ISPA during 2014 to 2015. Up to 67 national stroke performance measure and Coverdell data elements were reviewed for agreement according to eligibility. Charts were abstracted and reabstracted by hospitals using Get With the Guidelines®-Stroke and securely transmitted for analysis. The number of charts provided by hospitals was determined by the previous year’s stroke volume. The statistical package “Rstudio” was used for analysis.
Results: A total of 13,528 elements were reviewed. The overall state ISPA was 94.9%, compared to 94% during 2013 to 2014. Hospital agreement scores ranged from 78.2% to 99.6%, compared to the previous 83.1% to 99% in the previous year.
Conclusion: The comprehensive abstraction training program developed by Ohio Coverdell has resulted in excellent inter rater reliability scores. ISPA is a feasible method of performing inter-rater reliability.