Last week, U.S. News and World Report released its 35th annual hospital rankings.1 While they have remained at the forefront of hospital ranking across the healthcare industry, the methodology has been scrutinized in recent years.2 Certain institutions have opted out of the rankings (e.g., Penn Medicine) and government officials have launched investigations into the questionable practices (e.g., receiving revenue from hospitals that are ranked) of U.S. News.3,4 In light of this pushback, some methodology changes were incorporated in order to rely less on subjective data, incorporate more outpatient and Medicare Advantage data and name the “best” hospitals in aggregate, rather than an ordinal list; but concerns still persist.
While consumers are the intended audience for the U.S. News & World Report rankings, the reality is that health economy executives benchmark their own performance based on these rankings.
In our conversations with clients, we are routinely asked how their organization compares to top-rated hospitals as defined by U.S. News & World Report like Mayo Clinic or Cedars-Sinai. Typically, the goal is to learn from these high-performing peers and elevate their own performance. The problem is that, regardless of adjustments made to ranking methodologies, hospitals continue to compare themselves against organizations that are fundamentally different with respect to the most important performance variable of all: market characteristics (e.g., competitive dynamics, population demographics, payer networks).
Benchmarking is defined as the process of measuring an organization’s performance against those of comparable organizations, with the goal of identifying internal opportunities for improvement.5 Health economy executives must correctly identify their “true peers” since benchmarking against an aspirational peer is ineffective for performance improvement.
Yet, benchmarking methods within healthcare have seen little evolution over time. Historically, traditional hospital benchmarking has not equipped health economy stakeholders with the ability to identify relevant hospital peers. The existing benchmarking resources, which rely primarily on quality measures coupled with subjective criteria, have received criticism from both clinicians and academics, with one group of researchers citing prevalent issues across lists, including limited data, a lack of data auditing procedures and varying methods for compiling and weighting measures.6
Existing hospital rankings and ratings provide ordinal scores or ordered lists (i.e., best to worst) based in part on a variety of quality-centric measures, including HCAHPS, 30-day risk-adjusted mortality rate and readmission rates. Over time, the “best” or “top” hospital lists have become an element in strategic planning despite being designed for consumer use. The U.S. News & World Report rankings aim to help consumers understand the “best” place to receive certain types of healthcare services, while Leapfrog Group scores hospitals on patient safety (Figure 1).7 Healthgrades provides a review of clinical outcomes across multiple conditions to identify the hospitals with the “best” outcomes.8 While CMS Care Compare is intended to educate patients and provide consumer-curated scores, it also is used to incentivize performance, with Federal reimbursement levels (i.e., Medicare, Medicaid) subject to change based on a hospital’s rating score.
The current ratings and rankings lack comparative elements, which leads hospitals, health systems and other health economy stakeholders to make arbitrary and incomplete parallels between a particular hospital and some of the nation’s “top” hospitals. Hospitals do not know how dissimilar they are to some of the nation’s top hospitals on different metrics, nor do they know which hospitals they are most like.
Moreover, the well-documented unaffordability of U.S. healthcare is a critical part of the longstanding “Triple Aim,” but none of the popularized benchmarking methodologies account for measures related to cost of care in tandem with quality (i.e., value). Do some hospitals have comparable quality but starkly different prices? Absolutely. There is no observed correlation between price and quality in healthcare services at the national level, as exemplified with the comparison of negotiated rates and 30-day mortality rates for COPD at acute care hospitals in Chicago (Figure 2).
Additionally, take Cleveland Clinic for example. Within its true peer group of hospitals, as determined mathematically using Trilliant Health’s evidence-based benchmarking tool, known as the SimilarityIndex™ | Hospitals, Cleveland Clinic’s quality score is 55.29 and its average price for a hip and knee replacement is $56,756. Yet, within the same peer group, Penn Medicine, also known as the Hospital of University of Pennsylvania, and Vanderbilt University Medical Center have not only higher quality scores at 62.55 and 71.18, respectively, but lower average prices for the same procedure (Figure 3).
All existing hospital benchmarking sources exclusively compare facilities nationally, rather than identifying peer groups. For hospital executives, understanding how their organization “stacks up” nationally is often interesting, rarely important and never actionable without objective and relevant benchmark hospitals.
Insight into a broader set of measures across comparable organizations are critical components of improving performance in demonstrable and material ways.
U.S. News and World Report has consistently identified a small cohort of “Best Hospitals” over the years. In examining the 2024 list of “Best Hospitals”, four hospitals that were on the list in 2023 are no longer considered to be “in the top”. Between 2016 and 2024, 12 hospitals – Mayo Clinic, Cleveland Clinic, Cedars-Sinai, Johns Hopkins, NYU Langone, Massachusetts General, UC San Francisco, UCLA, New York-Presbyterian, Northwestern, Stanford and Mt. Sinai – appeared each year with little variability for the remaining top eight hospitals (Figure 4). In eight years, fewer than 30 hospitals held the coveted spot of a U.S. News Best Hospital for at least one year.
For example, while Barnes-Jewish Hospital was ranked among the top 20 hospitals for seven of the last nine years ranked by U.S. News & World Report, it received a “C” Hospital Safety Grade from the Leapfrog Group, four out of five stars overall from CMS and only rates among the top 250 U.S. hospitals according to Healthgrades.9,10,11,12 It also ranks 1,615th of the nation's most socially responsible hospitals by the Lown Institute.13 How were those “grades” compiled and are they equally important? The variation across existing hospital “benchmarks” and “raters” reinforces the need to redefine benchmarking based upon objective data and mathematical principles rather than national benchmarking based on subjective data and interpretation (i.e., “perceived prestige”). While Barnes-Jewish Hospital is undoubtedly one of the nation’s top healthcare facilities, its “true peers” and performance across quality, financial and competitive metrics likely vary compared to its perceived peers.
Increasingly, those who pay for healthcare are understanding that quality is a critical element of the value equation, but not the only critical determinant of “value,” which includes price, quality, efficiency, effectiveness, outcomes, process, experience and brand perception. In order to deliver value for money (i.e., when those receiving and paying for healthcare receive the best quality at the best price), even the “best hospitals” cannot rely solely on brand perception. Health economy stakeholders, namely employers, will increasingly demand value for money, which will force providers of all “tiers” to compete on every element of value in healthcare, namely price.
Over 4,000 U.S. hospitals rely on internal benchmarking through patient and physician experience surveys (i.e., Press Ganey), and yet those same hospital executives often ask me how to reconcile their generally positive survey results with unfavorable share of care (“wallet”), which underlies overall hospital financial performance.14
Benchmarking has the power to facilitate healthy competition, promote transparency and shed light on the successes and challenges at the hospital level. It is a critical function of helping hospitals operate at the top of license and deliver high-value care. In a health economy grappling with financial and operational challenges, stakeholders cannot afford to rely solely on traditional "gold standard" sources like U.S. News & World Report. Rather, performance improvement strategies should stem from objective and more comprehensive evidence-based benchmarks.
Accurate benchmarking is essential to competing effectively, which is why we created the SimilarityIndex™ | Hospitals, which enables accurate benchmarking by allowing stakeholders to identify a hospital’s group of empirically similar hospitals (“true peers”) and then compare the performance of the peer group nationally across Quality, Financial, Competitive and Price Indexes. The national and peer index scores provide more tangible considerations for how much a single hospital can and should improve across metrics. Hospitals should first be thinking about how to improve their performance within their peer group, focusing on whether there are strategies utilized by their peers that could be adopted.
Findings from SimilarityIndex™ | Hospitals and our other research reveals that there is no observed correlation between quality and negotiated price at the national, regional or local level, with a wide range of quality and negotiated reimbursement across U.S. hospitals. Beyond improvements in the foundational quality elements of mortality and safety, analyzing price versus quality measures reveals hundreds of examples in which the same quality outcome can be obtained at a wide variety of rates. With the advent of health plan price transparency, value for money will increasingly be the most important measure of America’s “best” hospitals.
Thanks to Katie Patton for her research support.