• Developers have indicated that it can be challenging to test eMeasures to the extent necessary to meet NQF endorsement criteria – at least until they have been more widely implemented. At the same time, there is interest in developing eMeasures for use in federal programs and obtaining NQF endorsement for those eMeasures. NQF endorsement may provide the impetus to implement measures; however, if a submitted measure with very limited testing does not meet NQF endorsement criteria, it could be prematurely abandoned.

    In 2014, NQF piloted the eMeasure Approval for Trial Use Program for eMeasures that are ready for implementation, but cannot yet be adequately tested to meet NQF endorsement criteria. NQF uses the multistakeholder consensus process to evaluate and approve eMeasures for trial use that address important areas for performance measurement and quality improvement, though they may not have the requisite testing needed for NQF endorsement. These eMeasures must be assessed to be technically acceptable for implementation and the ability to conduct more robust reliability and validity testing that can take advantage of clinical data in EHRs.

    In April 2015, the Consensus Standards Approval Committee (CSAC) agreed to make approval for trial use available for all eMeasures submitted to NQF. Approval for trial use is NOT time-limited endorsement as it carries no endorsement label. Furthermore, although the Trial Use Program is a pathway for measures to be potentially be considered for endorsement, it is not a mechanism by which measures should be considered that do not comply with NQF’s criteria regarding Important to Measure and Report, nor ones in which data to evaluate its Scientific Acceptability cannot reasonably be collected within an electronic health record. Measures approved for trial use will be so indicated on QPS. See Table 12 (PDF) for comparison of endorsement and approval for trial use.

    Approval for Trial Use Candidate Screening

    eMeasure candidates for the Trial Use Program are initially screened by the eMeasure and Maintenance Measure Review team. Criteria for approval for Trial Use include:

    • Must meet all criteria under Importance to Measure and Report (clinical evidence and opportunity for improvement/performance gap.)
    • The eMeasure feasibility assessment must be completed.
    • The measure developer must complete the testing form for Trial Use (DOCX) and use results from either a simulated data set from the BONNIE Tool, or a test data set from another source to demonstrate that the QDM and HQMF are used appropriately and that the measure logic performs as expected.
    • While trial measures are not intended for accountability purposes, there should be a plan for future use and discussion of how the measures will be useful for accountability and improvement.
    • Related and competing measures are identified with a plan for harmonization or justification for developing a competing measure.

    Candidacy for approval for Trial Use are reviewed much like any eMeasure by the standing committee in a project, based on the standard NQF criteria. The main difference between Approval for Trial Use and standard endorsement is that the completion of measure testing is not needed prior to committee review to receive Approval for Trial Use status.

    Maintenance of Trial eMeasures

    1. The trial eMeasure designation automatically expires three years after initial approval if the eMeasures is not submitted for endorsement prior to that time.
      1. The time to submit for endorsement is driven by success with testing. There is no expectation that every trial measure will be submitted for endorsement – some may fail during testing.
      2. When submitted for endorsement, the measure will be evaluated through the multistakeholder process. Ideally, Standing Committees and/or more flexible schedules for submitted measures will prevent delays for the endorsement process.
    2. If submitted for endorsement prior to the three-year expiration, the developer can select from the following options for evaluation and endorsement:
      1. Option 1: Submit and evaluate only the Scientific Acceptability of Measure Properties, including the final eMeasure specification and all testing. If endorsed, endorsement maintenance will be scheduled from the date approved as a trial measure, at which time it will be submitted for endorsement maintenance and subject to evaluation on all criteria.
      2. Option 2: Submit and evaluate on all criteria. If endorsed, a new endorsement date will be identified and endorsement maintenance will be scheduled from the new endorsement date, at which time it will be submitted for endorsement maintenance and subject to evaluation on all criteria.
    3. If submitted for endorsement three or more years after the date of approval as a trial measure, the measure must be submitted and evaluated on all criteria just as any measure being submitted for initial endorsement.
  • Become Part of the Quality Community! Join NQF.