Demystifying Performance Measures for Hospitalists: HCAHPS

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey was first implemented by the Centers for Medicare and Medicaid Services (CMS) to assess and compare performance on patient experience across hospitals, including metrics on communication with doctors and an overall hospital rating. This overall rating accounts for 30% of each hospital’s value-based purchasing score, potentially affecting millions of dollars per year.

As originally envisioned, the HCAHPS survey was intended to improve transparency through public reporting and fair comparisons of hospitals, however, questions persist around survey response rates, attribution, relation to quality outcomes, and more. Despite these questions, 48.7% of adult-only hospitalist groups include patient experience in their performance-incentive measures-compensation plans.¹ The authors acknowledge that certain vendors may offer additional data analysis to improve upon some of these factors, but HCAHPS as the key driver of patient experience metrics, and specifically its application as a performance metric for hospitalist programs, will be our focus for this measure review.

Attribution is problematic. For each patient, one score incorporates all physician interactions during a hospitalization, largely negating its value as a reliable metric for individual hospitalists and groups, since multiple hospitalists and non-hospitalists affect responses. Novel approaches such as “provider day weighted” scores which share attribution based on billing data, may improve this.² Two recent Journal of Hospital Medicine studies found no correlation between attendings’ assessed communication skills and the patient experience scores attributed to them.^3,4 CMS officials have also cautioned that HCAHPS is “not suitable for evaluating or incentivizing individuals or groups” and that doing so is “contrary to the survey’s design and policy aim.”⁵ We agree that HCAHPS scores cannot reliably be used to gauge individual or group performance.

With the collection and reporting of HCAHPS scores already mandated by CMS, it does not require additional resources to measure. However, beyond the attribution issues outlined above, the measures’ value is also affected by significant lag time in receiving response data, in addition to many hospital-related and demographic factors beyond hospitalists’ control. The American Hospital Association raised concerns about falling response rates (from 33% in 2008 to 26% in 2017) and inadequate capture of patients with lower health literacy levels, suggesting that factoring in social determinants may help control for these confounders when comparing hospitals.⁶ “HCAHPS 2.0” will be implemented in January 2025 with the intent of addressing some of these survey concerns. Despite the convenience of established data collection, these factors markedly limit the reliability and actionability of HCAHPS scores for hospitalists.

“Does it lead to better patient safety or outcomes?” is a pivotal question for any hospital medicine metric. Studies to date have shown mixed results as to whether there are even associations between patient experience and other traditional markers of quality of care; however, none provide evidence to support causation between improved HCAHPS scores and better clinical outcomes.^7-10 Overall, the authors agree with the assessment as stated in one of these studies: “Patient satisfaction is not a surrogate of patient safety and effectiveness” and “does not reflect the safety of care delivered by a hospital.”⁷

Even without affecting other outcomes, improved patient experience itself is still a worthy goal. Communication training is a common consideration, however, studies of interventions such as didactic sessions, individualized feedback, and incentives show only mixed results in improving physician communication domains, though not in the critical overall “rate the hospital” score.^11-14 While hospitalists can affect their patients’ experience, whether they can significantly impact HCAHPS scores is a different question. Our review of the available literature does not substantiate the existence of readily actionable plans demonstrated to improve scores that would support the use of HCAHPS as a hospitalist performance metric. Furthermore, in any initiative of a hospitalist team to improve communication, the success (or failure) in affecting HCAHPS scores would still be confounded by the inclusion of non-hospitalists in the scoring, potentially falsely elevating or lowering scores.

The potential unintended consequences of striving to improve HCAHPS must be considered because it could diminish resources available for targets more clearly tied to quality and safety outcomes. The authors have significant concern that initiatives aimed at raising patient-experience scores could lead to increased expenditures, increased length of stay, and even patient harm through testing, treatments, or firing of hospitalists in acquiescence to unreasonable demands, or through diverting hospitalists’ time and attention from other critical issues. This concern is highlighted in an outpatient study where respondents in the highest patient experience quartile went on to have higher total expenditures and most notably higher mortality rates.¹⁵ Those authors cite the potential role of physicians complying with patients’ requests for discretionary services that are of little or no medical benefit, noting this association may be compounded when physician compensation is strongly linked to patient experience. A recent Journal of Hospital Medicine study found that “patients in higher resource intensity groups were more likely to assign a top decile raw satisfaction rating,” with the potential outcome that “hospitals with high resource utilization could appear more attractive to healthcare consumers.”¹⁶ This finding further supports our concerns that incentivizing HCAHPS could lead to higher resource usage without adding clinical value, and potentially induce harm.

Physician burnout is another pressing concern. A 2021 survey found that 3.5% of physicians were planning to leave medicine altogether, a strong predictor of actual departure, up from 0.6% in 2011 and 1.3% in 2014.¹⁷ Among the “7 Drivers of Burnout” published by Journal of Hospital Medicine is “Being Held Accountable for Measures Beyond Their Control,” with patient experience specifically cited among those.¹⁸ There is evidence that physician burnout correlates with diminished patient experience and that higher physician satisfaction correlates with improved patient experience and higher-quality outcomes.^19,20 Programs aimed at improving hospitalists’ HCAHPS scores may be not only unsuccessful but also counterproductive given the risk of contributing to physician burnout. It is worth considering that improved physician job satisfaction may be one of the keys to successfully improving patient experience.

Conclusion

Emphasis on patient experience has grown as healthcare has become a consumer-driven business. A metric that truly reflected the quality of communication would be valuable in patient safety, and treatment adherence, and as a means for patients to select higher-quality healthcare based on the experiences of others. However, as shared in this review, translating theory to the real world is plagued with pitfalls. While HCAHPS may have value in evaluating a hospital, its applicability to evaluating hospitalists at the group, and more so at the individual, level is severely limited and possibly counterproductive. Based on our review of the literature we strongly recommend against using HCAHPS as a performance measure for hospitalist programs.

This does not mean principles that may contribute to higher hospital HCAHPS scores should be wholly disregarded. At the heart of patient-experience assessment are the worthwhile goals that the patient feels included in health-related decisions, heard with compassion and courtesy by their healthcare team, and that their medical team is working well together. Communication skills can still be enhanced for physicians with good self-awareness and a desire to improve, though effective interventions to accomplish these goals have not yet been clearly demonstrated. Therefore, if selecting a patient-experience metric is still desired, we recommend process measures aimed at improving coordination and communication among team members, improving conflict resolution, potentially improving nurse and physician staffing ratios, and reducing physician burnout. 

Dr. Miller

Dr. Dharapak

Dr. Smith

Dr. Virapongse

Dr. Miller is a 25-year adult hospitalist, long-time site and regional director, and an independent hospital medicine consultant with Chesapeake Hospitalists in Chesapeake, Va. Dr. Dharapak is chief of quality, patient safety, and performance improvement for the department of medicine and associate professor at Icahn School of Medicine, Mount Sinai Beth Israel in New York City. Dr. Smith is a medicine-pediatric hospitalist and the chief medical officer for Tucson Medical Center in Tucson, Ariz. Dr. Virapongse is an associate vice-chair for quality in the department of medicine, the director of quality, patient safety, and experience in the division of hospital medicine, and an associate professor at the University of Colorado School of Medicine in Aurora, Colo. SHM’s Performance Measurement and Reporting Committee periodically contributes articles demystifying performance measures for healthcare professionals.

References

1. Society of Hospital Medicine. 2020 State of Hospital Medicine Report. SHM website. https://store.hospitalmedicine.org/PersonifyEbusiness/Store/Product-Details/productId/7371177.
2. Herzke CA, Michtalik HJ, et al. A method for attributing patient-level metrics to rotating providers in an inpatient setting. J Hosp Med. 2018;13(7):470-5.
3. Velez VJ, Kaw R, et al. Do HCAHPS doctor communication scores reflect the communication skills of the attending on record? A cautionary tale from a tertiary-care medical service. J Hosp Med. 2017;12(6):421-7.
4. Torok H, Ghazarian SR, et al. Development and validation of the tool to assess inpatient satisfaction with care from hospitalists. J Hosp Med. 2014;9(9):553-8.
5. Tefera L, Lehrman WG, et al. Measurement of the patient experience: clarifying facts, myths, and approaches. JAMA. 2016;315(20):2167-8.
6. Salzberg CA, Kahn CN, et al. Modernizing the HCAHPS survey – Recommendations from Patient Experience Leaders. American Heart Association website. https://www.aha.org/guidesreports/2019-07-24-modernizing-hcahps-survey. Published July 24, 2019. Accessed April 6, 2024.
7. Kennedy GD, Tevis SE, et al. Is there a relationship between patient satisfaction and favorable outcomes? Ann Surg. 2014;260(4):592-600.
8. Tevis SE, Schmocker RK, et al. Can patients reliably identify safe, high quality care? J Hosp Adm. 2014;3(5):150-60.
9. Doyle C, Lennox L, et al. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570. doi: 10.1136/bmjopen-2012-001570.
10. Boulding W, Glickman SW, et al. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):41-8.
11. Horton DJ, Yarbrough PM, et al. Improving physician communication with patients as measured by HCAHPS using a standardized communication model. Am J Med Qual. 2017;32(6):617-24.
12. Indovina K, Keniston A, et al. Real-time patient experience surveys of hospitalized medical patients. J Hosp Med. 2016;11(4):251-6.
13. O’Leary KJ, Darling TA, et al. Impact of hospitalist communication-skills training on patient-satisfaction scores. J Hosp Med. 2013;8(6):315-20.
14. Boissy A, Windover AK, et al. Communication skills training for physicians improves patient satisfaction. J Gen Intern Med. 2016;31(7):755-61.
15. Fenton JJ, Jerant AF, et al. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172(5):405-11. 
16. Biondi EA, Hall M, et al. Association between resource utilization and patient satisfaction at a tertiary care medical center. J Hosp Med. 2016;11(11):785-91.
17. Shanafelt TD, Dyrbye LN, et al. Career plans of US physicians after the first 2 years of the COVID-19 pandemic. Mayo Clin Proc. 2023;98(11):1629-40.
18. Society of Hospital Medicine. 7 Drivers of Burnout In Hospital Medicine. SHM website. https://www.hospitalmedicine.org/practice-management/well-being/. Accessed April 6, 2024.
19. O’Donnell EP, Humeniuk KM, et al. The effects of fatigue and dissatisfaction on how physicians perceive their social responsibilities. Mayo Clin Proc. 2015;90(2):194-201.
20. Scheurer D, McKean S, et al. U.S. physician satisfaction: a systematic review. J Hosp Med. 2009;4(9):560-8.