Quality Assessment Tools: Incremental Advance or Paradigm Shift?
Quality Assessment Tools is a new quality measurement system for women age 18 to 50 years that covers preventive medicine and 17 other acute and chronic conditions. McGlynn et al 3 report that this system detected potentially important quality variation within and between 2 managed care organizations based on a combined sample size of 758 members. More importantly, QA Tools offers significant improvements to currently used reporting systems. These include (1) increased focus on overall care, (2) enhanced flexibility allowing multiple levels of summary measures, and (3) greater efficiency of medical record abstraction. Full appreciation of this work, however, requires a more detailed examination of these 3 improvements, a review of remaining problems, and consideration of the rapidly evolving expectations of our health care system.
First, QA Tools focuses on multiple conditions with several quality measures per condition, instead of the usual single condition or single measure per condition. Further, QA Tools covers a broad scope of diagnoses, encompassing approximately half of the reasons for ambulatory visits and most of the reasons for hospitalization. This breadth discourages health plans from shunting disproportionate resources toward improving only specific quality measures at the expense of other conditions. 4,5 A broad measurement base also decreases the possibility of gaming, or manipulation of patient cohorts to achieve artificially high results. For example, using data from 3 large health maintenance organizations, Hofer et al 6 demonstrated that mean glycosylated hemoglobin levels for poorly performing physicians could be dramatically improved by removing a few selected patients from the profile. Sensitivity to such manipulation remained even after extensive case-mix adjustment.
Second, QA Tools offers multiple levels of summary measures. Summary measures may be more accessible to consumers than an overwhelming array of individual measures. The investigators appropriately speculate that examination at the level of the system, condition, and individual measure may suggest complementary, yet distinct, approaches to improvement. Flexible aggregation exposes poor performance on individual measures or domains that otherwise might remain hidden within composite measures. Conversely, a review of component measures may disclose the cause of a low aggregate score.
Third, QA Tools gains efficiency by using a single data set for patients with different combinations of conditions. An electronic chart abstraction module, customized to the individual record during manual abstraction, guides data collection. This approach allows potentially meaningful differences to be detected with relatively small sample sizes. The resulting efficiency decreases the expense of chart abstraction, although manual abstraction will become less relevant with increasing adoption of electronic medical record systems. 7
Beyond the 3 specific advances described above, QA Tools fulfills most traditional criteria for sound quality measures. A consensus statement of the American Medical Association, the Joint Commission on Accreditation of Health Care Organizations, and the National Committee for Quality Assurance (NCQA) summarized the desirable attributes of performance measures. 8 This statement posits that important quality measures should address target conditions that exert a large population impact and that manifest demonstrable variation in care, thus revealing potential for improvement. Useful measures engender recommendations that are evidence-based, achievable, and under the direct control of the physician or health care organization.