Measuring Performance

NQF's All-Cause Readmissions Project 

The NQF Board of Directors recently voted to endorse performance measure 1789: Hospital-wide all-cause readmission measure, developed by Yale University and the Centers for Medicare and Medicaid Services (CMS). This endorsement came at the end of a multi-step process, known as the consensus development process or CDP, which NQF follows as its method for endorsing nationally-recognized healthcare performance measures.

Measure # 1789 estimates the hospital-level, risk-standardized rate of unplanned, all-cause readmissions for any eligible condition within 30 days of hospital discharge for patients aged 18 and older. The measure will result in a single summary risk-adjusted readmission rate for conditions or procedures that fall under five specialties: surgery/gynecology, general medicine, cardiorespiratory, cardiovascular, and neurology.
Upon endorsement, the NQF Board requested that this measure be accompanied by the following guidance to help recognize the multifaceted nature of hospital readmissions and the opportunity for broad stakeholder collaboration to address the issue:

"Multiple factors affect readmission rates and other measures including: the complexity of the medical condition and associated therapies; effectiveness of inpatient treatment and care transitions; patient understanding of and adherence to treatment plans; patient health literacy and language barriers; and the availability and quality of post-acute and community-based services, particularly for patients with low income. Readmission measurement should reinforce national efforts to focus all stakeholders' attention and collaboration on this important issue." 

Measure # 1789 has been appealed by a group of seven hospital systems, which is the last step of the CDP. This appeal is now under consideration by the Consensus Standards Approval Committee (CSAC), and their recommendation on whether endorsement should be upheld will be brought to the NQF Board of Directors on Monday, June 25 for their reconsideration. This Board call is open to the public.

NQF Statement

NQF is a transparent, multi-stakeholder, mission-driven organization. Our work relies on seeking multiple stakeholders' input, and in balancing those interests in our decision-making. Our process offers many opportunities for dialogue as we work through stakeholder concerns and questions. For the all-cause readmissions measure, we've observed strong support for measure #1789 at each step of the endorsement process, with the exception of our membership voting which is a very important step that is considered in measure endorsement decision-making.

All-Cause Readmissions Expedited Review Project

Consensus Step  1789: All Cause Hospital-Wide Unplanned Readmission (CMS/Yale) 
Steering Committee Final Vote following Comment  Yes: 14; No: 5
Member Voting  % Councils approving > 50%: 57%;
Average council approval: 67%
CSAC  Yes: 11; No: 2
Board of Directors  Yes: 21; No: 0

In the case of measure #1789, the appellants point out that the stakes are high for them, as organizations that will be required to publicly report on this measure, and may eventually be paid based on results. The various bodies – membership, the steering committee, CSAC, and Board of Directors – clearly understood and heard these concerns. They paid equal attention to the concerns of those who experience readmissions – patients and their families – and those who pay for these costly, often preventable trips to the hospital. These voices are all an important part of NQF deliberations.

The CDP process includes multiple mechanisms for input with committee meetings and calls open to the public. Over the course of the last seven months, a range of NQF expert volunteers, members, staff, and our Board of Directors have worked on this project. In quantifiable terms, engagement looks like this:

Task  Total Number of Hours/Comments/Requests/Sessions 
Steering Committee (in-person meeting, December 2011 call and January 2012 call) 20 hours
Public and NQF member comments 117 comments/43 individuals or organizations
# of Special requests (SC-day two of in-person meeting, December 2011 call, Board call) 3 special requests of the developer for additional information/clarification
# of Member council calls (consumer/purchaser, annual meeting-provider, provider council) 3 member council sessions
CSAC (conference call, March 2012 in-person meeting, June 2012-appeals) 4 hours
Board of Directors (February 2012 call, March 2012 call, June 2012-appeals (tentative)) 4 hours

We recognize that the majority of NQF members chose not to exercise their right to vote on this measure and we are actively seeking to identify ways to encourage more members to vote. While our hope is that every one of NQF's 450 members will engage in every endorsement project we undertake, we also accept that different members choose to engage on projects relevant to their priorities; that our members face many competing external demands in this fast-paced healthcare world that may take them away from engagement in NQF work; and that some members find value in commenting on a project or serving on a steering committee rather than voting. We respect our members' varied approaches to engagement, even as we constantly seek more ways to facilitate it.

A takeaway from this project is that we intend to initiate member focus groups to find out what more we can do to encourage their active involvement in every aspect of our process. NQF's CSAC will also take up this topic in its July meeting.

With respect to the measure itself, the appellants have expressed a view that this measure is "not ready for prime time." Unintended consequences of measurement have been raised as the major concern relating to the use of measure #1789; more plainly, there is worry that patients may be harmed if access to the hospital is impeded as a result of using this measure.

There are several steps NQF has initiated that address these concerns. CMS will be conducting a "dry run" of the measure, and we have asked that they submit their findings to NQF within one year. NQF hopes that these data could be used to further assess issues raised during the review and appeals process, including the adequacy of the planned readmission algorithm and the frequency of incorrect information identified by hospitals and confirmed or resolved. NQF also encourages hospitals to provide feedback in their use of the measure, as part of a "measure use feedback loop," to help foster a vibrant post-measure implementation surveillance dialogue.

NQF plans to ask the project's steering committee to review the dry run results and any other information we receive and determine if there are any concerns with the use of the measure, either in terms of potential harms to patients or the scientific acceptability of the measure properties. The results of this review, as well as the dry run results from CMS, will be posted on the NQF website for public comment. The CSAC and Board will have an opportunity to consider these results at that time and consider whether field testing provides any concerns for continued endorsement. In addition, the CSAC will consider at its July 2012 meeting whether field testing should be a requirement for a certain subset of measures prior to endorsement, and bring a recommendation forward for consideration by the Board at its September 2012 meeting.

Almost one in five Medicare beneficiaries that leave a hospital are readmitted within 30 days, at a cost of approximately $15 billion annually. And that’s only a portion of patients that experience readmissions.  Harm, pain, and anguish cannot be measured in dollars but take an emotional toll.  This is a national problem that cuts right to the heart of where we need to focus our attention.  Measures matter – we can only improve what we can measure – but measures are by no means the full solution.  Measurement is but part of a collective solution to this complicated, multifactorial problem.