To get where we want to go in American healthcare, we need a more thoroughly supported measure development infrastructure.
—Anne-Marie J. Audet, MD, MSc, SM, vice president, Health System Quality and Efficiency, Commonwealth Fund
The Centers for Medicare & Medicaid Services (CMS) has been publicly reporting performance measures on its Hospital Compare website (www.hospitalcompare.hhs.gov) since 2005, focusing on processes of care, patient outcomes, patient satisfaction, patient safety, and other measures. A recent addition of patient-safety metrics has rekindled skeptical questions about the validity, purpose, and effectiveness of public healthcare quality report cards, while highlighting the need for hospitalists and their institutions to remain vigilant in the struggle to ensure that they are compared and rewarded fairly and appropriately.
Provocative Measures
Last fall, CMS began posting “Serious Complications and Deaths” measures, developed by the Agency for Healthcare Research and Quality (AHRQ). The measures score individual hospitals according to the rates at which their patients suffer from:
- Pneumothorax due to medical treatment;
- Post-operative VTE;
- Post-operative abdominal or pelvic dehiscence; and
- Accidental lacerations from medical treatment.
Four other serious complication measures (pressure ulcers, catheter and bloodstream infections, and hip fractures from falling after surgery) are folded into a separate composite score for each hospital, while another composite score for “Deaths for Certain Conditions” is based on a hospital’s post-admission mortality rate for hip fractures, acute MI, heart failure, stroke, GI bleed, and pneumonia.
National and local media reports have thrust these dramatic metrics into the public eye, putting many hospitals on the spot to explain their putative breaches of patient safety. A closer inspection of the metrics, however, reveals plausible criticisms of their shortcomings.
Methodological Weakness
The new metrics are derived from Medicare claims data instead of medical chart abstractions, which experts say weakens their validity significantly and makes their use for provider profiling questionable. Moreover, claims data are based on records that were never designed to capture the sort of clinical nuances needed for valid and equitable risk adjustment (see “Methodological Challenges to Quality Metrics,” below). “Serious Complications and Deaths” rates based on these data, critics maintain, lack validity for meaningful hospital comparisons because they can exaggerate problems at hospitals that treat a high volume of complicated patients and use more invasive procedures to do so, such as teaching hospitals in academic medical centers.1
The ante gets upped when CMS eventually begins adding patient-safety measures to the Hospital Value-Based Purchasing (HVBP) program, which rewards or punishes hospitals financially, depending on their performance on the metrics. CMS is considering adding the Serious Complications and Deaths measures to the HVPB program in the near future.
As the science of documenting and reporting patient harm struggles to find its footing, physicians and hospitals have to be more vigilant than ever to adopt a unified, organized approach to advocate the most appropriate processes and outcomes for which they will be held accountable, and avoid being cast in a reactive mode when metrics are imposed on them, says Patrick J. Torcson, MD, MMM, FACP, SFHM, chair of SHM’s Performance Measurement and Reporting Committee, and director of hospital medicine at St. Tammany Parish Hospital in Shreveport, La.
Last year, SHM sent comments to then-CMS administrator Don Berwick, expressing concern that the patient-safety measures CMS proposes to include in the HVBP program in fiscal-year 2014 are not endorsed by the National Quality Forum (NQF), that they are derived from billing and payment data that are not intended to be used primarily for clinical purposes, that the outcome measures are not entirely preventable even with the best of care, and that they are not adequately risk-adjusted.