Comments are listed in the order they were submitted.
__________________________________________________________
Name: Craig Law
Organization: Delaware Health Net
Date Entered: 1/7/2008 9:55:16 AM
Comments:
This seems to be very good work. One area not noticed relates to creation of standardized data sets on successful clinical measure completion. The mechanism of coding diligence in completing recommended tasks. A discussion would be valuable in creating standards relating to both providers and patients success in services that are referred, declined, refused, recommended but not completed and how they become part of the construct of performance.
------------------------------------------------------------------------------------------------------
Name: Kevin Scheckelhoff
Organization: McKesson Corporation
Date Entered: 1/9/2008 9:29:52 AM
Comments:
Re: line 327, item #5 re: medication duration calculation. Manipulation/analysis of pharmacy dispensing data has substantial limitations that should be recognized. Outpatient dispensing data is typically only accurate at the time of dispense: should the prescriber alter the dosage in a manner that does not require a new prescription (ie split tablets or double tablets) the dispensing pharmacy is typically not notified and would only become aware of the change if/when presented with a new prescription (unless patient recieving MTM). Same would be true for discontinuation. Use of multiple pharmacies also creates limitation as previous dispensing histories from other pharmacies are not viewable unless part of same organization and medication histories not always complete. There have been some advances in bridging this gap through the use of pharmacy dispensing databases collected when Rx claims are adjudicated online but this also has limitations: many prescriptions do not require adjudication (a trend currently being fueled by $4 generic Rxs offerings paid in cash). All insurers do not allow their electronic claims data to be entered into the common database. The result is information that can be helpful but not comprehensive. Vendors may be able to create means to perform duration calculations but the quality of this output is limited by inputs. Could be problematic if user places too much reliance on med duration calculation that has limited accuracy given the data source.
------------------------------------------------------------------------------------------------------
Name: William Conway
Organization: Henry Ford Health System
Date Entered: 1/9/2008 7:15:32 PM
Comments:
The emphasis on use of a common set of data elements across different care settings is an important recommendation of the report. We recommend creation of common data elements based upon the ISO/IEC 11179 standard. This should be tied in with development of the individual data elements using a community driven approach similar to the one NCI’s caBIG program has adopted. Several data elements may already be available in the NCI’s common data element repository (caDSR). This ISO/IEC 11179 approach will allow anchoring the data elements robustly in semantics from standard vocabularies/ontologies like SNOMED CT. Further, the developers of electronic health record systems should be encouraged to align the semantics of the data elements with HL7 RIM based information models. (provided by Dr. Robert Enberg and Dr. Hemant Shah)
------------------------------------------------------------------------------------------------------
Name: William Conway
Organization: Henry Ford Health System
Date Entered: 1/9/2008 7:15:32 PM
Comments:
Clinical Guidelines and the metrics for clinical performance are inextricably linked. It is notable that the panel took cognizance of this fact and has given it due weight. We advocate taking this approach further to evolve a methodology for developing actionable guidelines. This will ensure that the evaluation of process performance is not restricted to measures based on single data points. Rather, processes are best evaluated by multiple data points describing their characteristics. This will also encourage a movement in the direction of merging of the two apparently dichotomous approaches, those evaluating the outputs of processes and those evaluating the outcomes of patients. (provided by Dr. Robert Enberg and Dr. Hemant Shah)
------------------------------------------------------------------------------------------------------
Name: Jesse Singer
Organization: New York City Department of Health and Mental Hygiene
Date Entered: 1/11/2008 11:53:11 AM
Comments:
Understanding that this may be out of scope of the panel charge, as it is not an area of AQA or HQA approved measures, we feel strongly that, as an IOM priority area, the additional domain of obesity should be considered in the assessment of priority measure status.
The selection criteria for determining priority order for measure selection included impact, improvability and inclusiveness per IOM. Obesity is a current IOM priority area and as such, we feel that obesity should be included as a high priority with the following NQF-endorsed measure:
Body Mass Index (BMI) in adults > 18 years of age:
Percentage of adults with BMI documentation in the past 24 months.
------------------------------------------------------------------------------------------------------
Name: Jesse Singer
Organization: New York City Department of Health and Mental Hygiene
Date Entered: 1/11/2008 11:53:11 AM
Comments:
Understanding that this may be out of scope of the panel charge, as it is not an area of AQA or HQA approved measures, we feel strongly that, as an IOM priority area, the additional domain of care coordination should be considered in the assessment of priority measure status.
The selection criteria for determining priority order for measure selection included impact, improvability and inclusiveness per IOM. Care coordination is a current IOM priority area and as such, we feel that care coordination should be included as a high priority. In an effort to begin to address the concept of care coordination and the patient centered medical home, the corresponding measure should be considered for this purpose:
Patients > 18 years of age seeing assigned PCP:
Percentage of adults who have seen their assigned Primary Care Provider at least once in the last 12 months.
------------------------------------------------------------------------------------------------------
Name: Jesse Singer
Organization: New York City Department of Health and Mental Hygiene
Date Entered: 1/11/2008 11:53:11 AM
Comments:
After reviewing the draft document, we feel the addition of two data types are warranted:
Data Category: “History” should include the Data type: “Visit”, which may include further subtypes beyond the scope of this draft such as visit type and date of visit. This is critical for all measures when you consider a data model that includes measure driven clinical decision support at the point of care.
Data Category: “Medication” should include the Data type: “Patient adherence”, which may include further subtypes beyond the charge of this panel. Although a full discussion of patient medication adherence is beyond the scope of this comment, please see the following references for discussion on this domain:
N Engl J Med 2005;353:487-97
Arch Intern Med. 2007;167:847-852
Arch Intern Med. 2007;167:540-550
Journal of Asthma, 2006;43:521–526
------------------------------------------------------------------------------------------------------
Name: Jesse Singer
Organization: New York City Department of Health and Mental Hygiene
Date Entered: 1/11/2008 11:53:11 AM
Comments:
Re: Measure exclusions beginning line 291:
Given the relative immaturity of Health Information Exchange implementation, the addition of measure specific exclusion criteria raises the question of data validity, especially in other than closed healthcare systems. The incorporation of electronically operable diagnostic testing results and even procedures (which follow the CPT standard), both distantly historical and those done via external entities creates numerous problems for data capture as applied to quality measurement.
For example, in the NQF-endorsed breast cancer measure, using mastectomy as exclusion criteria, we felt that the codification and capture of these exclusion criteria would be problematic as externally performed or historically distant mastectomy would often fail to be captured by the provider in a standardized way for these criteria to electronically operate.
Instead, we included provider entered exclusions as part of the clinical decision support system workflow. During an alert for a specific measure, a provider has the ability to suppress the alert either temporarily or permanently, excluding the patient from the measure. These exclusions must be justified and are auditable and accounted for in the aggregate data. We feel this allows providers to deal with these exclusions at the point of care, for all situations, including those unanticipated or for those which information is known, but has not been or is unable to be codified.
------------------------------------------------------------------------------------------------------
Name: Stephen Persell
Organization: Northwestern University
Date Entered: 1/11/2008 1:44:35 PM
Comments:
I agree that if EHRs data is to be used to accurately measure quality of care for external accountability and provider selection, there is a strong need to ensure that measures obtained from data in one healthcare information system capture the same concepts and speak the same language as those from another. However, the high-priority, standard data elements identified by the panel may not allow for fair comparison of quality across different user groups. The panel is right to prioritize data elements that are readily available from typical work flow. But, the work flows of different kinds of practices differ in important ways that could impact how quality is measured. A multi-specialty practice that includes inpatient and outpatient data within the same information system (e.g., Kaiser Permanente, Veterans Administration) will capture data in a very different way than does a stand-alone single specialty outpatient practice using their own information system. The panel’s recommendations should perhaps consider that even with the most important standard data elements (diagnoses, procedures, medications, allergies, adverse drug effects), there might not be a level playing field for quality comparisons across user groups that are so different from each other. For example procedures and diagnosis codes will more likely be present in the multi-specialty system than the single specialty one and this could have important implications for quality measurement.
------------------------------------------------------------------------------------------------------
Name: Stephen Persell
Organization: Northwestern University
Date Entered: 1/11/2008 1:44:35 PM
Comments:
Cervical cancer screening is a useful example. Procedure codes for pap tests or hysterectomies are more likely to be captured in a large multi-specialty group during standard work flow. A stand alone primary care group does need to track these procedures in order to deliver high quality care. To do this though, it may be preferable to use codes within an EHR that are suited to the quality task at hand. It would not be correct for a practice to insert a procedure code for a specific kind of pap test into their EHR as if the test had happened within that system on a specific date when a patient reports historically that the had the test done elsewhere. There is a need for standard ways to capture data that is relevant to quality measurement but which does not imply a level of certainty that does not exist. In this example, a patient reported pap completed elsewhere on an approximate date is valuable information for quality measurement but users should not be forced to represent this data within their system in an overly specified way in order to not appear to be delivering inadequate care. Flu shots are another good example. If only standard procedure codes are used to measure the quality of flu shot delivery, the care will always appear worse than it is. What is needed is a standard way to capture within the EHR that a patient reports receiving a flu shot this fall or winter. This is a patient reported data element necessary for quality measurement. EHR users should not be forced to represent it as a procedure in order to demonstrate high quality care.
------------------------------------------------------------------------------------------------------
Name: Stephen Persell
Organization: Northwestern University
Date Entered: 1/11/2008 1:44:35 PM
Comments:
There is also a need for standard ways to capture which patients declined a recommended service. This data has a role in performance measurement for accountability or provider selection. Even more importantly, the standardized capture of this form of data will enable EHRs to be used to efficiently identify patients for education outreach at the practice or health plan level.
By establishing standards for documentation of these kinds of exceptions to performance measures, the panel could do a great deal to advance quality measurement so that it could both better serve internal quality improvement and permit more fair comparisons across provider groups working in different settings.
------------------------------------------------------------------------------------------------------
Name: Jeffrey Apfelbaum
Organization: American Society of Anesthesiologists
Date Entered: 1/11/2008 2:39:14 PM
Comments:
The American Society of Anesthesiologists is writing this letter in response to
the National Quality Forum’s Health Information Technology Expert Panel
Report. ASA shares NQF’s goal of improving quality of care, and wishes to
indicate its support of the recommendations contained in this report.
------------------------------------------------------------------------------------------------------
Name: Jeffrey Apfelbaum
Organization: American Society of Anesthesiologists
Date Entered: 1/11/2008 2:39:14 PM
Comments:
Compliance with quality initiatives is currently measured by collecting
healthcare performance data from insurance claims, but these documents
contain limited clinical information. The NQF was commissioned by the
Agency for Healthcare Research and Quality to convene an expert panel that
would accelerate the adoption of standards that allow data from electronic
health records (EHRs) to measure healthcare quality. The report generated by
NQF recommends that common data types be included as part of an EHR.
This would enable the use of specific queries to measure compliance with
clinical initiatives such as timely administration of antibiotics for surgical
infection prophylaxis. ASA agrees that the use of administrative and billing
data limits the information available for later analysis, and agrees with NQF’s
proposed use of a problem list instead of the current diagnostic coding
schema. Fewer than three hundred CPT codes are used to bill for anesthesia
services for all surgical, obstetrical, and diagnostic procedures, meaning that
in some cases, the same anesthesia CPT code is used to describe the
anesthetic technique for more than 100 different surgical procedures. The
implication of very broad and non-specific claims data is that the availability
of more specific and descriptive clinical information from an EHR is especially
critical for quality measurement in perioperative care.
------------------------------------------------------------------------------------------------------
Name: Jeffrey Apfelbaum
Organization: American Society of Anesthesiologists
Date Entered: 1/11/2008 2:39:14 PM
Comments:
ASA is planning a comprehensive clinical outcome registry that will be used
to generate data for clinical research and benchmarking. Information gleaned
from perioperative electronic records can be of value to many stakeholders,
including health care providers, quality organizations such as NQF,
healthcare researchers, and third party payers. NQF’s goals are a subset of
those for ASA’s proposed registry in that NQF’s health information
technology experts propose a system for monitoring compliance with
specific guidelines while ASA is interested additionally in monitoring patient
outcomes and correlating them to anesthesia techniques and practices.
There are, however, several issues that must be addressed in order to
achieve both goals: the appropriate questions must be asked, a common
language must be developed to facilitate data collection and analysis, and
there must be widespread adoption of EHR technology.
------------------------------------------------------------------------------------------------------
Name: Jeffrey Apfelbaum
Organization: American Society of Anesthesiologists
Date Entered: 1/11/2008 2:39:14 PM
Comments:
ASA has long supported the Panel’s recommendation to develop a data
dictionary and standardized terminology through its alliance with the
Anesthesia Patient Safety Foundation (APSF), which formed a Data Dictionary
Task Force in 2001. This early effort led to the formation of the
International Organization for Terminology in Anesthesia (IOTA) that serves
as the official source of anesthesia terminology for both SNOMED and also as
the Special Interest Group for the Generation of Anesthesia Standards for
Health Level Seven (HL7). Several thousand SNOMED terms now constitute a
standardized perioperative terminology. IOTA is directing additional efforts
to use the Clinical Document Architecture framework of HL7 to model the
attributes of the SNOMED/IOTA terms in a fashion that will serve as the
framework for a comprehensive perioperative EHR. ASA believes that this
serves both the AHRQ initiatives as well as the ASA clinical outcomes registry
and related initiatives by other specialties.
In summary, ASA fully supports the recommendations of the NQF Health
Information Technology Expert Panel. The initiatives contained in this report
may have the beneficial effect of encouraging the widespread adoption of
EHRs, a common data dictionary, and adoption of standards for data
interchange. ASA wishes to indicate its interest in working with these
organizations toward the shared goal of improving the quality of healthcare.
------------------------------------------------------------------------------------------------------
Name: Rachel Groman
Organization: American Association of Neurological Surgeons
Date Entered: 1/11/2008 2:49:59 PM
Comments:
Recommendation #1 appears to be yet another attempt to replace the CPT system with ICD-10 or SNOMED. While these two classification systems are more exact, the practicality of asking all U.S. physicians to convert to a substantially more complicated diagnosis/treatment coding system while also converting to EMR will only accomplish a more resolute pushback from the medical community. Hospital- based physicians may find this task easier to accomplish due to better access to resources. However, the majority of the measures for which data collection is desired apply to outpatient primary care physicians. Even if as a fallback hospitals were first required to comply with this transition, this would necessitate the government running two systems (CPT and ICD-10), which makes the transition even more complex and virtually impossible. In the text, the authors assume that data would be coming from those most experienced in EMR. The reality is, as the concomitant push for EMR goes forward, physicians will either not code or low code data sets, or completely bypass the measures by putting a text note in the chart. As experience grows, CMS would find skewed data trends as physicians increasingly code measures correctly. Severity would most likely appear to increase, as would claims, creating suspicion among all parties.
Recommendation #2 simply appears to be a recommendation for a factual document and seems reasonable within its context.
------------------------------------------------------------------------------------------------------
Name: Rachel Groman
Organization: American Association of Neurological Surgeons
Date Entered: 1/11/2008 2:49:59 PM
Comments:
AANS strongly agrees that the issues identified in recommendation #3 need to be addressed and incorporated in all measures.
Recommendation #4 is complicated since labs and other tests would most likely have a "drop down" menu with very basic choices (e.g. “normal,” “not clinically relevant to current service,” “abnormal”), which limits and/or alters decision making. One immediate problem is that physicians will be leery of classifying a radiologist's lengthy report as "not relevant" because of liability concerns. Standardization of laboratory values and ranges may also be an issue.
If we understand recommendation #5 correctly, it sounds like data aggregation by a third party since the prescribing physician with an EMR already has a complete temporal record of medication usage.
In terms of recommendation #6, discharge instructions work well for the emergency room physician, whose job it is to fix one diagnosis. In the office setting, with multiple diagnoses, this is not practical. It is also unclear who would be responsible for constructing these functionalities—medical professional societies?
------------------------------------------------------------------------------------------------------
Name: Rachel Groman
Organization: American Association of Neurological Surgeons
Date Entered: 1/11/2008 2:49:59 PM
Comments:
In terms of recommendation #7, the AANS encourages the NQF to continue to rely on measures developed through the physician-led, consensus-based AMA Physician Consortium for Performance Improvement. A formal process should be developed to determine and compare the costs and benefits of exclusion and inclusion criteria. Unfortunately, CMS will not worry about the cost to acquire more data. It will simply require it and doctors will be left with yet another "unfunded mandate."
------------------------------------------------------------------------------------------------------
Name: Elliot Levine
Organization: E & C Medical Intelligence, Inc.
Date Entered: 1/11/2008 2:52:11 PM
Comments:
Although there are 84 high-priority quality measures listed, only 2 (2.4%) are directly related to pregnancy care, though such care is included in the Institute of Medicine (IOM) list of healthcare areas for which efforts at quality improvement measures should be directed. Given the inordinate liability litigation occurring in the field of obstetrics, and leading to its spiraling healthcare costs, attention should certainly be paid to the measurement of quality in obstetrics, as this may lead to significant error reduction and improved patient safety and potentially decreased liability costs. Further, given the scope of the population impact, the improvability (or preventability of harm) and the inclusiveness of the “pregnancy condition,” the quality measure selection criteria established by the IOM are met, when considering quality measures specifically relating to pregnancy care. With this in mind, consideration should be given to having a greater emphasis for quality measures in this high impact area of obstetrics.
------------------------------------------------------------------------------------------------------
Name: Elliot Levine
Organization: E & C Medical Intelligence, Inc.
Date Entered: 1/11/2008 2:52:11 PM
Comments:
It appears that the High Priority Quality Measure item #74 (Screening for HIV), seen in Appendix B, refers to quality measures in the last trimester of pregnancy care until hospital discharge of mother and newborn. However, the CDC recommends HIV screening in the first trimester of pregnancy, which is also consistent with the recommendations of the American College of Obstetricians and Gynecologists (ACOG). Therefore, consistent with published guidelines and current accepted standards, the following recommendations for pregnant women should be considered with regard to HIV:
- HIV screening should be included in the routine panel of prenatal screening tests for all pregnant women, and typically performed in the first trimester.
- HIV screening is recommended after the patient is notified that testing will be performed unless the patient declines (opt-out screening).
- Separate written consent for HIV testing should not be required; general consent for medical care should be considered sufficient to encompass consent for HIV testing.
- Repeat screening in the third trimester should be considered in certain high prevalence areas and among certain high risk populations.
The last recommendation illustrates the potential vagueness and complexity of the medical issues relating to pregnancy, and therefore, the difficulty inherent in devising quality measures that can be specific enough to effect direct healthcare improvement.
------------------------------------------------------------------------------------------------------
Name: Elliot Levine
Organization: E & C Medical Intelligence, Inc.
Date Entered: 1/11/2008 2:52:11 PM
Comments:
Regarding Quality Measure Item #75 (Anti-D Immune Globulin), this will likely impact less than 10% of pregnant women, as opposed to the greater impact of the following proposal.
As the incidence of Group B Streptococcal (GBS) Neonatal Sepsis has significantly diminished (by 1/3) since the publication of the CDC’s Guidelines for the Prevention of Group B Streptococcal Disease in 2002, it appears that Intrapartum Antibiotic Prophylaxis (IAP) should be appropriately provided when indicated, with quality measures prepared to identify provider compliance with this recommendation. Given that the probable incidence of lower genital tract carriage of GBS is 20% of pregnant women, with the incidence of GBS Neonatal Sepsis as 0.15%, prior to the wide-scale adoption of IAP practices, a large number of patients would be impacted.
Since there are published and accepted algorithms for directing optimal care in this environment, based on a high level of evidence, reducing complex clinical scenarios to reasonably manageable ones, quality measures can be devised that could monitor compliance with these accepted standards. A compelling argument can therefore be made that quality measurement, with regard to intrapartum GBS prophylaxis, can improve ultimate perinatal outcomes.
------------------------------------------------------------------------------------------------------
Name: Elliot Levine
Organization: E & C Medical Intelligence, Inc.
Date Entered: 1/11/2008 2:52:11 PM
Comments:
In terms of the HITEP category types found in Appendix B, one important category type is absent, namely height and weight, which should be considered essential, equal in importance to the other Vital Signs (e.g. BP). Documentation of height and weight are fairly standard measurements that are taken upon a patient’s hospital admission or presentation to an ambulatory clinic. With such measurements, a Body-Mass Index (BMI) can be calculated, which in turn has the potential to be linked to a variety of risk factors, for which specific medically related tasks may be indicated. Body weight has further implications for proper dosing of certain medications, which can potentially relate to multiple separate quality measures, perhaps even relating to medication management. Specifically, with regard to pregnancy care, weight gain in pregnancy has been judged as a risk factor for a number of pregnancy complications (low weight with Intrauterine Growth Restriction, and elevated weights with increased rates of Shoulder Dystocia, for example). As obesity is one of the twenty IOM healthcare areas in which national quality improvement efforts should be focused, it is reasonable to include measures that can precisely quantify this particular health factor, namely weight and height. Recognizing obesity, however it is defined, can further indicate the possible need for nutritional counseling, for example. This is not unlike the Joint Commission’s standards relating to smoking cessation.
------------------------------------------------------------------------------------------------------
Name: Elliot Levine
Organization: E & C Medical Intelligence, Inc.
Date Entered: 1/11/2008 2:52:11 PM
Comments:
It is indeed true that “measuring quality is a first step toward improving American healthcare.”
As an organization devoted to diminishing the incidence of preventable obstetrical error, thus improving the clinical obstetrical care provided and resulting in better perinatal outcomes, E & C Medical Intelligence has developed numerous practice guidelines for this purpose. Its current primary EMR product, the Intelligent Patient Record for Obstetrics (IPROB), among other things, provides an interdisciplinary clinical problem list, not associated with billing codes. It also maintains protocols that keep current with published accepted specialty guidelines (e.g. ACOG) that drive its clinical decision support. It encourages and provides integrated documentation of nurse-physician and physician-physician communication, in the course of care, that is patient-centered. Physician-patient communication is further promoted, for informed patient/physician shared decision-making, and for episodes of unanticipated outcomes. In addition, the printing of patient discharge instructions, specific to the discharge diagnoses and patient circumstance is integral to its EMR function. These described measures perfectly synchronize with the HIT recommendations included in its draft report. We therefore submit that the comments offered are consistent with the perspective and mission of the National Quality Forum, and respectfully ask that they are considered in that light.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/11/2008 3:39:27 PM
Comments:
True interoperability will not occur until data definitions and codes are standardized and incorporated into technical standards. AHIMA recommends including additional text describing how the common data types defined in the report should be aligned with similar data elements in other data sets. This exercise will support movement toward collecting data once so it can be repurposed multiple times for quality, population health reporting, research, and administration.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/11/2008 3:39:27 PM
Comments:
AHIMA suggests specifically calling out the role of standard development organizations (SDOs) and the need align the HITEP common data types with new and existing technical standards. Active engagement of SDOs will aid in bridging the gap between the quality and information technology enterprises.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/11/2008 3:39:27 PM
Comments:
Section III. – Expert Panel Analysis:
Page 15, line 305 – The report discusses "the need to influence 'upstream' processes so that measures being submitted for NQF endorsement meet criteria set to increase the comparative value of quality scores and minimize the effort required to acquire and report out the quality scores in an EHR." Following this statement, Figure 3 is displayed representing the "Organizational Relationships in Quality measurement, Health IT and NQF Influence." Does this figure represent current quality measurement, health IT and NQF relationships? If so, how will the organizational relationships change after the 'upstream' processes have been influenced? It may be useful to include an additional figure depicting the envisioned "future state."
Page 16, Figure 3 – AHIMA recommends incorporating corresponding descriptions for each element of the diagram (similar to the format used in the AHIC Use Cases, with numbers and corresponding detailed descriptions of the entities and processes depicted).
Page 15, line 299 – “…The Expert Panel constructed an organizational chart (Figure 4)…” The figure notation is incorrect. The correct notation should be “Figure 3”.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/11/2008 3:39:27 PM
Comments:
Section IV. – Expert Panel Recommendations:
Page 18, Recommendations 5 and 6 – AHIMA supports recommendations for developing key EHR system functionality that will support automated collection and reporting of quality measurement information, but functionality should be defined and vetted in a coordinated and transparent manner, such as through the development of a Quality Measurement Functional Profile standard based on the HL7 EHR System Functional Model (EHR-S FM). Development of a Quality Measurement Functional Profile will facilitate a collaborative approach toward defining quality related EHR functionality, further demonstrate the business needs for system vendors, and guide the development of CCHIT certification criteria to support quality.
AHIMA suggests that the HITEP enhance recommendations 5 and 6, stating "Quality and information technology stakeholders are encouraged to define EHR functional requirements that support quality measurement through the development of a Quality Measurement Functional Profile. This profile should include functionality to automatically capture the issuance of discharge instructions regarding specific conditions and methods of using data from dispensing pharmacies to automatically determine the duration of medication usage." This type of project will further bridge the gap between the quality and information technology enterprises.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/11/2008 3:39:27 PM
Comments:
Section IV. – Expert Panel Recommendations (cont.):
Page 18, Recommendation 7 – Quality measure specifications were are not designed to leverage clinical data from EHR systems, but with the emerging capability of EHRs to capture clinically-relevant information and support quality measurement reporting, AHIMA supports the HITEP’s recommendation for NQF to “encourage the use of high quality data elements for newly submitted measures and gradually retire endorsed measures that rely on poor quality data elements…”. However; measure developers must provide explicit logic and algorithms when defining quality measure parameters to allow vendors the ability to easily incorporate measure logic into EHR systems. AHIMA recommends that NQF work with both measure developers and SDOs to define technical standards and validated processes for translating quality measure specifications into standardized computable logic.
------------------------------------------------------------------------------------------------------
Name: Carmella Bocchino
Organization: AHIP
Date Entered: 1/11/2008 3:55:56 PM
Comments:
We agree with the report's proposed methodology whereby existing measures are put in a priority order based on the Institute of Medicine’s (IOM) priority conditions and common data types are standardized. This roadmap is essential to automate quality reporting. We also strongly support one of the report’s main recommendations that the NQF should work with measure developers to limit exclusions, where appropriate to make measures easier to automated reporting from electronic health records (EHRs). Measure developers should also be encouraged to utilize common measure specifications that can utilize multiple complementary data sources such as administrative, claims, medical records and EHR.
AHIP strongly supports the goal of report to automate the reporting of quality measures from EHRs. However, as steps are taken to meet this goal, it is important the transition does not lead to two sets of quality measures – one set based on claims and administrative systems and a second set of measures derived from EHRs. It is important that the HITEP recognize that in many provider offices it is premature to move from the current quality reporting based on claims and administrative systems by the health plan on behalf of the provider to a system electronically reported from the provider’s EHR.
------------------------------------------------------------------------------------------------------
Name: Carmella Bocchino
Organization: AHIP
Date Entered: 1/11/2008 3:55:56 PM
Comments:
We recommend the report includes an additional recommendation focusing on the needed strategy and transition plan to implement the proposed changes to the data types used for quality reporting (e.g. movement from billing codes to standard problem lists). This strategy should include a phased-in approach to revising the criteria used by measure development organizations.
------------------------------------------------------------------------------------------------------
Name: Carmella Bocchino
Organization: AHIP
Date Entered: 1/11/2008 3:55:56 PM
Comments:
Recommendation One: Use of Clinical Problem Lists in Place of Billing Codes
We support the use of problem lists as the basis to identify patient conditions, however the report should acknowledge that this transition will require significant administrative costs by physicians and will take time to implement, either concurrently or immediately following the adoption of an EHR by physician or provider-group practice. We recommend that there may need to be a hybrid approach whereby either clinical problem-list diagnoses coded using standard terminologies like the Systematized Nomenclature of Medicine (SNOMED) terminology or billing codes are used concurrently for a short period of time.
Given that the recommended clinical data types will serve as the basis for recognized interoperability specifications by the Healthcare Information Technology Standards Panel (HITSP) and thereby required for implementation in federal healthcare programs, it is important that the recommended code sets for the clinical problem list are consistent with existing HITSP recommended standards. Therefore, we agree with the report’s recommendation that HITSP should consider nationally and internationally accepted standard terminologies like the SNOMED terminology set for coding problem-list diagnoses.
------------------------------------------------------------------------------------------------------
Name: Carmella Bocchino
Organization: AHIP
Date Entered: 1/11/2008 3:55:56 PM
Comments:
Recommendation Five: EHR Vendors Use of Data from Dispensing Pharmacies
We agree with the general premise of this recommendation; we suggest the text of the recommendation should be clarified. It is important that quality reporting systems (with EHRs or administrative) have a mechanism to capture data from pharmacies to determine the duration of medication usage. We recommend the phrase “dispensing pharmacy” is changed to pharmacy or pharmacy network because the data may be more readily available from the network used by the pharmacy as opposed to directly from the pharmacy that dispensed the medication. For example, through agreements with Pharmacy Benefits Managers health plans can determine whether a prescription has been picked up and/or refilled. The Centers for Medicare & Medicaid Services will soon have access to similar data through Medicare Part D.
------------------------------------------------------------------------------------------------------
Name: Carmella Bocchino
Organization: AHIP
Date Entered: 1/11/2008 3:55:56 PM
Comments:
Recommendation Seven: Quality of Data Types Used in Measure Specifications
We agree that NQF should evaluate the quality of data types used in measure specifications as a criterion in the endorsement of new measures, as well as re-assessment of measures for continued endorsement. However, the report should recognize that these measures will need to be transitioned over time and that the system should be able to continue to support measures based on administrative data in the short term.
------------------------------------------------------------------------------------------------------
Name: Debra Ness
Organization: National Partnership for Women & Families
Date Entered: 1/11/2008 4:13:35 PM
Comments:
We applaud HITEP’s work to examine the feasibility of using electronic data to enhance performance measurement and reporting. We, along with a number of other consumer organizations, have long advocated for greater use of EMRs to facilitate performance measurement. Currently, data collection and reporting on measures often require medical record extraction or other burdensome and expensive processes. Greater use of health information technology offers hope for reducing that burden and facilitating the collection and analysis of data. It is critical, however, that there is some uniformity among the various EMR systems if that promise is to be realized. We think the recommendations of the panel are an excellent start.
------------------------------------------------------------------------------------------------------
Name: Debra Ness
Organization: National Partnership for Women & Families
Date Entered: 1/11/2008 4:13:35 PM
Comments:
We suggest making a priority of identifying and including common elements (i.e., vital signs) that could support more robust risk-adjustment.
------------------------------------------------------------------------------------------------------
Name: Debra Ness
Organization: National Partnership for Women & Families
Date Entered: 1/11/2008 4:13:35 PM
Comments:
We urge that the report include a recommendation that the data types used to assess disparities in care (i.e., patient ethnicity/race, language and payment source) be incorporated into EHRs.
------------------------------------------------------------------------------------------------------
Name: Barb Corn
Organization: NAHQ
Date Entered: 1/11/2008 4:26:33 PM
Comments:
Page 18: Future work. If a EHR is certified with the current standards, will outside agencies use the certification at face value and not require audit of the coding?
------------------------------------------------------------------------------------------------------
Name: Joel Slackman
Organization: BlueCross BlueShield Association
Date Entered: 1/11/2008 4:44:06 PM
Comments:
Recommendation #2: The recommendation is somewhat vague as to exactly what the named data dictionary would comprise; BCBSA recommends expanding the dictionary’s description in the report to clarify what is being calling for. Regardless, any data dictionary that is developed should be vetted by industry stakeholders to ensure that it is comprehensive and accurate.
------------------------------------------------------------------------------------------------------
Name: Joel Slackman
Organization: BlueCross BlueShield Association
Date Entered: 1/11/2008 4:44:06 PM
Comments:
Recommendation #5: Data could come from multiple dispensing pharmacies, mail order pharmacies, and/or PBMs. It should be noted that while the data can serve as a valuable guide in estimating duration of medication usage, it only measures the fill rate and is therefore not an exact indicator.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:25:20 PM
Comments:
The American Medical Association (AMA) is pleased to have the opportunity to comment on the National Quality Forum’s (NQF) draft Health IT Expert Panel Report: Recommended Common Data Types and Prioritized Performance Measures for Electronic Healthcare Information Systems. We commend the attention and detail that the NQF has given to this topic and agree that it is imperative to maximize the benefits of electronic health record systems (EHRS) and ease their use for deriving performance measures, which will assist in improving the quality of care delivered. In this letter, we discuss overarching issues, outline our comments on each of the recommendations and provide requests for clarification and recommendations for further refinement for the Panel’s consideration.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:25:20 PM
Comments:
Overarching Suggestions Regarding Data Quality Criteria and Scoring
We strongly support the development of criteria to guide the prioritization of measures and related data elements in the context of EHRS. Because of the importance of such criteria, we recommend that the Panel further describe each criterion and consider carefully the implication or “message” of a low score for each of the criteria.
For example, we suggest that the Panel specify that the criterion “availability in EHRS” not be a rate limiting factor. For example, there may be widespread agreement that a performance measure and related element (eg, ACEI/ARB for HF and ejection fraction) are priorities from a clinical perspective. Yet a data element for ejection fraction is not widely available today in EHRS and would therefore receive a low score on that criterion. If NQF does not place a high priority on this measure, as a result, we may lose the opportunity to advance and maximize EHRS for performance measurement and quality improvement. We believe it is the intent of the Panel to identify necessary data elements and promote the development of those data elements where needed. Therefore, it would be helpful to explicitly articulate this intent in the report. In addition, we suggest that “authoritative” and “accurate” be two separate criteria and more detail be provided regarding the criterion “auditable.”
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:25:20 PM
Comments:
Also toward this point, it may be worthwhile to consider prioritizing measures based on those measures that can be reported on by EHRS today, as well as those critical for fast track resolutions of outstanding issues.
Regarding scoring, we are unsure how the criteria, weighting, “freq%” and “rel%” are used in calculating the overall measure quality scores (see, for example, score for 30-day mortality in AMI and HF tables). Are the scores actually quantitative calculations or were they derived from consensus-based decisions based on available information? We also suggest the Panel clarify why some tables include inpatient and outpatient data elements when the related performance measure focuses on ambulatory care.
Lastly, we have learned through work with the AMA-NCQA Collaborative and the EHRS vendor community that vendors are in various stages of adding the necessary data elements and functionalities to report NQF-endorsed™ measures. Staff at AMA will contact NQF to explore whether the Collaborative could be helpful in soliciting additional information from a broader EHRS vendor community regarding ratings of the data types against the data quality criteria.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:25:20 PM
Comments:
Recommendation 1: Use of problem list to identify patient conditions
We fully support that the medical record (whether paper or electronic) is the gold standard and should be used as the data source to identify patients for performance measurement. We are concerned with the language included in the recommendation that limits identification of patient conditions in the EHRS strictly to the problem list. Because each EHRS may operationalize the concept of “problem list” differently, we would not wish to see the problem list as the only way to identify patients for whom measures are relevant but rather recommend that the language of the recommendation be broadened.
Moreover, we recently worked with Dr. Jeffrey Linder at Brigham and Women’s Hospital to evaluate the feasibility of utilizing an EHRS to report on the PCPI performance measures for outpatient pneumonia and learned that acute conditions are not typically entered on the problem list and, therefore, some exceptions should be noted.
We appreciate the additional recommendation about the accessibility of the ‘problem list’ across care settings, but believe that this is a longer-term goal and somewhat out of scope of this report. There are issues of attribution of both clinical responsibility and delivery of care when using shared problem lists that need considerable discussion by multiple stakeholders before this broad recommendation can be made.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:25:20 PM
Comments:
Recommendation 2: Data dictionary of different data types
While agreeing with the need to produce such a dictionary, we are concerned that pre-existing standards for data types (such as those within HL7 standards for data transmission) are not mentioned in this recommendation. We believe that a recommendation calling for further work to align and harmonize HL7 data types, the HITSP interoperability specification, the data elements identified here, and those identified by the AMA-NCQA Collaborative XML schema would provide greater benefit. Alignment also will be needed with the Quality Reporting Data Architecture (QRDA) work that is currently making progress toward defining output requirements for quality reporting. It would also be useful to look at those data types defined by the Centers for Medicare & Medicaid Services’ Physician Quality Reporting Initiative EHRS pilot for 2008.
The data types identified by the Panel, of necessity, only define those data elements that occur within the 84 measures that were examined. Considering first data elements that are needed to record patient care and then setting standards for the way in which those data elements are represented and made accessible in EHRS would be a better approach. Structured, coded data will then be available for many uses, including direct patient care, clinical decision support, quality measurement, billing, biosurveillance, public health, and research.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:28:19 PM
Comments:
Recommendation 3: Medication allergies and side effects should be distinct from each other and coded
We support this recommendation and would recommend that it be expanded to promote the distinction between allergies, side effects, and contraindications. We believe that, for both quality improvement and performance measurement, these distinctions are critical.
Recommendation 4: Development of standardized codes for summary impressions of diagnostic test results
We agree broadly with this recommendation, but point out that where diagnostic test result data with numerical values are available, these data also should be transmitted with the date the test was performed and with the appropriate code to the ordering physician along with the summarized impression (eg, [date of test] LVEF [coded] = 35% [numerical value], severe LVSD [impression]; the impression might also be coded). There would be educational implications for radiologists, pathologists, and other physicians in ensuring that they understood the need to enter both kinds of data. However, EHRS vendors could assist physicians by producing standard “pick lists” for the test result and the clinical impression.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:28:20 PM
Comments:
Recommendation 7: NQF to evaluate the quality of data types in measures as a criterion in measure endorsement
We concur that NQF should consider the quality of data types in measure specifications; however, as noted above, we suggest that NQF not prioritize measures for data capture within EHRS solely on the state of the technology today. Rather, if a measure and accompanying data elements meet other NQF criteria, NQF is in a strong position to motivate the development of needed codes, data elements, and functionality.
Regarding measure exclusions, the AMA has and continues to work with practice sites and researchers to conduct sensitivity and specificity analyses and would welcome further discussion on this topic with NQF. We respectfully submit that perhaps it also is worthwhile to broaden the discussion of exclusions. We believe it is critically important in patient care for a physician to document the specific reason why a patient may not be receiving a recommended drug or procedure, including both clinical reasons and patient preferences, and it is important for a physician to periodically revisit that decision with the patient. Therefore, we believe we should encourage EHRS vendors to develop ways that physicians can easily capture such information at the time of decision-making and track this information over time.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:28:20 PM
Comments:
Request for Clarification and Recommendations for Further Refinement
Line 19, 118: The statement that “… the majority of readily available electronic health information has been from administrative claims…” is inaccurate. There is extensive health information available in electronic form including laboratory, pharmacy and imaging data. However, these data are often not available in the electronic medical record maintained by a provider. The statement should be restated as “…the majority of health information readily available in electronic health records has been administrative data…”
Line 59: The phrase “standard languages, or code sets” is unclear in this context. The term “language,” when speaking of software, tends to imply a programming language. The phrase “standard languages” should be replaced by terminology, which more accurately represents the concept of codified data.
Line 67: The phrase “…vendors to develop software needed…” should be revised. CCHIT does not have a role in working with EHRS vendors to develop software. CCHIT simply develops functionality criteria that vendors then build. The phrase should be revised to say “…develop functionality criteria…”
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:28:20 PM
Comments:
Line 92: The sentence in line 92 ends with the phrase “…in these national efforts.” It is unclear which national efforts are being referenced here. Clarify whether this sentence references the national effort to create a roadmap or the efforts described in the recommendations (lines 63-71).
Line 206: “These categories and types will be submitted to HITSP who will recommend standardized code sets.”
HITSP has recently issued an “interoperability specification” including data types (see also comments above on Recommendation 2).
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:28:20 PM
Comments:
Line 215: Table 2: The Data Categories “Diagnostic Study” and “Laboratory” could be combined into a single category “Investigations”; the Data Types, “order” and “result” are the same.
We are concerned that there is no indication that each data type should also have an associated date and time component. Although this omission may be a function of the measures studied for this review, when integrating the performance measures into the system knowledge of the date of entry or occurrence is critical to ensure accurate data collection. This information is equally important when analyzing the data from an EHRS in order to determine if the parameters of the metrics have been achieved. Time is an important factor in many measures – investigations/physical exams need to have a time period specified within which they should have been carried out; the order in which data elements occur is important in some measures and there are other examples. “Procedure” includes “past history”; this data type might actually be a data category, and include such data types as “event” (e.g. AMI), “diagnosis” (e.g. CAD), “procedure” (e.g. CABG) – and all of these should have a date attached.
Line 224: We would recommend saying “initial assessment of the availability and quality of a given data type.”
As noted above, we suggest broad input from the EHRS vendor community and will contact NQF to offer assistance through the Collaborative.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:29:15 PM
Comments:
Line 232: Define the attributes of a “comprehensive EHR system.” It is unclear whether a comprehensive EHRS would include any level of interoperability with other external systems (e.g., lab, imaging) or whether the phrase comprehensive refers only to functionality offered by the system itself.
Line 239: “The quality of a measure is a function of the quality of its individual data types.”
We suggest that the quality of a measure should be judged on many factors: the clinical significance of the aspect of care it addresses, whether it is based on evidence, whether it is well-constructed and specified. We do not believe that the quality of individual data types is a sole criterion.
Line 253: The word accurately should be included in this sentence. It should read – “A measure can be calculated accurately only if …”
Lines 309/10: This description of the Collaborative’s work should be revised to say … “focuses on a standard approach of representing measures in a computable format for use by EHRS vendors.”
Line 311: “CCHIT was felt to clearly impact EHRs as well as physicians.”
We are not sure to what this statement refers – the sentence seems out of context.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: American Medical Association
Date Entered: 1/11/2008 5:29:16 PM
Comments:
Line 313: “… guidelines be written so that they are … in EHRs.”
We would suggest that this be amended to say “… guidelines be written with enough precision to enable EHRS vendors to translate recommendations into clinical decision support rules and algorithms, and to enable measure developers to provide detailed measure specifications in agreed formats to EHRS vendors.”
Thank you for the opportunity to comment on this report.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:01 PM
Comments:
The Physician Consortium for Performance Improvement (PCPI) is pleased to have the opportunity to comment on the National Quality Forum’s (NQF) draft Health IT Expert Panel Report: Recommended Common Data Types and Prioritized Performance Measures for Electronic Healthcare Information Systems. We commend the attention and detail that the NQF has given to this topic and agree that it is imperative to maximize the benefits of electronic health record systems (EHRS) and ease their use for driving performance measures, which will assist in improving the quality of care delivered. In this letter, we discuss an overarching suggestion regarding data quality criteria and scoring, outline our comments on some of the recommendations, and provide requests for clarification and recommendations for further refinement for the Panel’s consideration.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:01 PM
Comments:
Overarching Suggestions Regarding Data Quality Criteria and Scoring
The PCPI strongly supports the development of criteria to guide the prioritization of measures and related data elements in the context of EHRS. Because of the importance of such criteria, we recommend that the Panel further describe each criterion and consider carefully the implication or “message” of a low score for each of the criteria.
For example, we suggest that the Panel specify that the criterion “availability in EHRS” not be a rate limiting factor. For example, there may be widespread agreement that a performance measure and related element (eg, ACEI/ARB for HF and ejection fraction) are priorities from a clinical perspective. Yet a data element for ejection fraction is not widely available today in EHRS and would therefore receive a low score on that criterion. If NQF does not place a high priority on this measure, as a result, we may lose the opportunity to advance and maximize EHRS for performance measurement and quality improvement. We believe it is the intent of the Panel to identify necessary data elements and promote the development of those data elements where needed. Therefore, it would be helpful to explicitly articulate this intent in the report. In addition, we suggest that “authoritative” and “accurate” be two separate criteria and more detail be provided regarding the criterion “auditable.”
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:01 PM
Comments:
Also toward this point, it may be worthwhile to consider prioritizing measures based on those measures that can be reported on by EHRS today, as well as those critical for fast track resolutions of outstanding issues.
Regarding scoring, we are unsure of how the criteria, weighting, “freq%” and “rel%” are used in calculating the overall measure quality scores (see, for example, score for 30-day mortality in AMI and HF tables). Are the scores actually quantitative calculations or were they derived from consensus-based decisions based on available information? We also suggest the Panel clarify why some tables include inpatient and outpatient data elements when the related performance measure focuses on ambulatory care.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:01 PM
Comments:
Recommendation 1: Use of problem list to identify patient conditions
The PCPI fully supports that the medical record (whether paper or electronic) is the gold standard and should be used as the data source to identify patients for performance measurement. We are concerned with the language included in the recommendation that limits identification of patient conditions in the EHRS strictly to the problem list. Because each EHRS may operationalize the concept of “problem list” differently, we would not wish to see the problem list as the only way to identify patients for whom measures are relevant but rather recommend that the language of the recommendation be broadened.
We appreciate the additional recommendation about the accessibility of the ‘problem list’ across care settings, but believe that this is a longer-term goal and somewhat out of scope of this report. There are issues of attribution of both clinical responsibility and delivery of care when using shared problem lists that need considerable discussion by multiple stakeholders before this broad recommendation can be made.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:01 PM
Comments:
Recommendation 3: Medication allergies and side effects should be distinct from each other and coded
We support this recommendation and would recommend that it be expanded to promote the distinction between allergies, side effects, and contraindications. During the development and implementation of the PCPI measures, we have found that each of these aspects are critical pieces of information, both for providing care that is high quality and promotes patient safety and for performance measurement purposes.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:38 PM
Comments:
Recommendation 7: NQF to evaluate the quality of data types in measures as a criterion in measure endorsement
We concur that NQF should consider the quality of data types in measure specifications; however, as noted above, we suggest that NQF not prioritize measures for data capture within EHRS solely on the state of the technology today. Rather, if a measure and accompanying data elements meet other NQF criteria, NQF is in a strong position to motivate the development of needed codes, data elements, and functionality.
As you may know, the PCPI Measure Implementation and Evaluation Advisory Committee defines the testing needed for each of the measure components including exclusions and the sensitivity and specificity analyses. We would welcome the opportunity to work with NQF on determining how to best approach this analysis.
We respectfully submit that perhaps it also is worthwhile to broaden the discussion of exclusions. We believe it is critically important in patient care for a physician to document the specific reason why a patient may not be receiving a recommended drug or procedure, including both clinical reasons and patient preferences, and it is important for a physician to periodically revisit that decision with the patient. Therefore, we believe we should encourage EHRS vendors to develop ways that physicians can easily capture such information at the time of decision-making and track this information over time.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 1/11/2008 5:34:38 PM
Comments:
Request for Clarification and Recommendations for Further Refinement
Line 239: “The quality of a measure is a function of the quality of its individual data types.”
We suggest that the quality of a measure should be judged on many factors: the clinical significance of the aspect of care it addresses, whether it is based on evidence, whether it is well-constructed and specified. We do not believe that the quality of individual data types is a sole criterion.
Line 253: The word accurately should be included in this sentence. It should read – “A measure can be calculated accurately only if …”
Line 313: “… guidelines be written so that they are … in EHRs.”
We would suggest that this be amended to say “… guidelines be written with enough precision to enable EHR vendors to translate recommendations into clinical decision support rules and algorithms, and to enable measure developers to provide detailed measure specifications in agreed formats to EHR vendors.”
Thank you for the opportunity to comment on this report.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:47:09 PM
Comments:
General HRSA Comments
HIT Focus on E HR and Tendency Toward Hospital Centric Language and Typology.
Lines 14 to 16
Quality improvement leaders have long recognized that the widespread adoption of Health Information Technology (HIT) will automate and simplify these processes by providing electronic information.3, 4, 5.
Comment
HRSA supports the recommendation to identify common language/codes and to support interoperability and communication among HIT users at all levels of the health system (primary, secondary and tertiary). To maximize patient quality outcomes, we wish to also emphasize the importance of enhanced data sharing capabilities among state and local level health care entities currently involved in patient care and population health management.
Specific Recommendation
• Line 86 to 87 and Table 5. Consider adding key recommendation that EHR vendors should develop functionality to automatically capture population level and chronic disease management program performance measures.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:47:09 PM
Comments:
HIT vs E HR Focus
HRSA recommends that, if the goal is to improve health outcomes across a variety of health care settings, other HIT tools and functions also be considered when developing performance measure quality scores and recommendations to HITSP and CCHIT. This report will be used by vendors and others to create future applications that will influence how health care is practiced and patients are managed. We agree that the electronic health record (EHR) is a fundamental HIT tool for collecting high-quality electronic clinical information, (line 21), especially in hospitals and in the small percentage of ambulatory environments currently equipped with E HRs. There are also other HIT tools that are equally critical for attaining quality outcome improvements resulting from population health management, disease prevention screening, and health provider decision support. A broad variety of stakeholders benefit from these tools, including State Health Departments, State Medicaid/Medicare programs, Federally Qualified Health Centers, QIOs and other Federal Partners.
Specific Recommendation
Data Types made possible by other HIT tools such as dynamic multi-condition ambulatory patient registries, and state immunization registries used by local and state health system partners should be incorporated into the language and data typology (table 2) as well as the scoring matrix in Appendix C.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:47:09 PM
Comments:
Outside the E HR Box
When developing a scoring methodology for performance measures and recommendations to CCHIT and HITSP, it is important to consider data types that have not yet been considered for measurement in the pre HIT era and are currently not incorporated into most E HRs. Specifically, important quality concepts are rapidly emerging in the areas of medical homes (e.g. patient centered care, coordination across multiple settings/providers, bidirectional data exchange) and chronic disease management.
At this time the NCQA has developed standards to assess whether practices are functioning as patient centered medical homes. HRSA recommends that the HIT data typology be revisited as medical home performance standards are developed.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:47:09 PM
Comments:
Outside the EHR Box Continued
- Chronic disease management software programs for care coordination are a different type of HIT. To date few EHRs have incorporated functionalities that allow true population based analysis and management and other HIT has evolved to fill the need in the form of care coordination products and multi-condition registry systems with interoperability to enable data reports. For example, the Chronic Disease Management Program (CDMP) HIT product available from The Department of Defense (DOD) is able to interface with EHRs and has been made available to the public as an “open source” product. Another example of an HIT product capable of better population data analysis and management is the iCare product being used by Indian Health Service (IHS). This is another HIT product that is not an EHR but can work with the IHS EHR product to abstract the population-based information for population-based management of clinical conditions. There are also registry systems such as the Aristos Patient Electronic Care System (PECS) and i2i's Meditracks and others which have functionalities not typically found in most EHRs as most EHRs have been developed for the purpose of managing individual patient's and not to manage the entire population of active patients with any given condition.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:47:09 PM
Comments:
Outside the EHR Box Recommendation:
Specific Recommendations
• Data Types made possible by chronic disease management software programs for care coordination should be considered and incorporated into the language and data typology, as well as the scoring matrix.
• Future data types corresponding to patient-centeredness, care coordination and bidirectional workflow should be revisited as evolving medical home quality standards are translated into practicable performance measures.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:51:54 PM
Comments:
Methodology
General Comments
The IOM priority areas appear broad based and appropriate for a variety of life cycles; however the Methodology used to select Tier One measures used in this report only targeted performance measures already approved by NQF. In setting the Priority Order for Measure Selection, the Panel further limited their field of consideration to AQA and HQA measures endorsed by NQF. This decision appears to result in measure discussions and data types that are hospital-centric and elderly focused. NQF has endorsed a larger scope of quality measures “National Voluntary Consensus Standards for Ambulatory Care” (12/07), which is more inclusive of Pediatric Issues.
HRSA notes that NQF has ambulatory measures for children (e.g. pediatric immunizations) and self-management (e.g. management plan for people with asthma, hypertension plan of care) that aren’t reflected in Appendix B; if used, these may serve to expand the data typology presented (table 2) and include more longitudinal care scenarios that occur in ambulatory settings such as communication for patient education, anticipatory guidance encounters, behavioral and other health risk assessments etc, and planned follow up visits ( not included in the current model).
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:51:54 PM
Comments:
Consideration of Pediatric Populations
In general, Pediatric health issues seem to be missing from the priority areas discussion, and selection of “high priority measures” (Appendix B). In some cases, measures have not yet been developed, in other cases, measures are currently endorsed by NQF but were not included in this project.
One of the IOM priority areas is children with special health care needs. NQF has previously noted the importance of developing newborn screening measures. Every infant in this country (over 4 million) is screened at birth for a variety of congenital disorders. Newborn screening communication processes are of interest to both public health and health care delivery systems.
Specific Recommendations:
• HRSA recommends that the following NQF endorsed ambulatory care measures be added to the high priority list: Childhood Immunization Status (NCQA) and Diagnosis of ADHD (ICSI).
• "HRSA supports the inclusion of more measures for pediatrics which would enhance the generalizability of the proposed approach to all performance measures and it would allow identification of additional data elements and processes unique to high quality care for millions of children".
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:51:54 PM
Comments:
Assessment of Disparities
Specific Recommendations
• Line 209-210. The Panel has identified three data types (not found in the measure set) that will be required to assess disparities. HRSA recommends that age, gender, SES and literacy level be considered as well.
• Line 84 – 87 and Table 5. Consider adding key recommendation that EHR vendors should develop functionality to generate and automatically capture issuance of patient education and self-management materials with built in language and picture options to accommodate non-English speaking and functionally illiterate populations.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:55:08 PM
Comments:
Framework and Pareto Charts:
In general, this is a good start to a comprehensive framework for data types and performance measures for Electronic Health Information Systems. The key recommendation (p. 4) to shift to the clinical problems from the billing code is a giant leap. This recommendation will help make a patient-centered health care system.
Appendix D potentially is useful to decision makers in selecting quality measures because it assigns scores to the various measures. It is not apparent how the section “ identification of high and low yield common data types” on pages 12 to 15 relates to the appendix D measures or to the next section titled “quality domain organizational analysis”.
The Pareto analysis and its relevance to the measures is not self explanatory. Pareto Analysis charts could be more helpful by reinforcing the concepts of frequency and completeness. They could be renamed to highlight frequency and completeness and what the charts show. The discussion of exclusions is interesting, but a question remains how this relates to selection of measures.
The Expert Panel addressed the very complex task of standardizing a means by which data can be collected and utilized primarily by the patient and clinician, as shown in Figure 3 on page 16. However, it is noted that all arrows in Figure 3 are pointed to the patient and clinician. This schematic suggests that the patient and clinician are only acted upon and have no organizational relationship or communications linkages with any of the quality measurement components.
------------------------------------------------------------------------------------------------------
Name: Deborah Willis-Fillinger, MD
Organization: HRSA Center for Quality
Date Entered: 1/11/2008 5:55:08 PM
Comments:
The NQF Health IT Expert Panel (HITEP) Members
Line 39 (Appendix A) were selected to ensure broad representation across quality measurement, health information technology, EHR vendors, health systems, and government organizations.
Comment
HRSA supports seasoned and articulate grantees with considerable experience in EHR implementation who are leaders in quality improvement, including IHI Health Disparities Collaboratives. These experts, familiar with HIT factors that support or confound data and patient management within and across ambulatory settings, and with national experience with multi-condition registries, chronic disease management and care coordination HIT might be of value to the NQF Expert Panel and are available to add the perspective of ambulatory care providers in resource constrained settings.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:01:55 PM
Comments:
Several pediatric experts and organizations have been working together with regard to developing Health Information Technology in support of quality measurement, most notably through the HL7 Pediatric Data Standard SIG. The following comments were submitted by these individuals.
1. The National Quality Forum is to be commended for undertaking this effort.
2. If a semicircle is for numerator analysis, why is it not a semicircle for denominator?
3. We agree with the recommendation for a coded, interdisciplinary clinical problem list that is accessible and used across care settings. This can be addressed by leveraging existing standards such as the Continuity of Care Document/Continuity of Care Record, which is a harmonized standard approved by the ANSI Health Information Technology Standards Panel (HITSP) and roadmapped for the Certification Commission on Health Information Technology’s (CCHIT) 2008 testing criteria. It should also be recognized that, today, clinicians do not routinely use problem lists. It may be unrealistic and create analysis biases if performance measures are driven by fields in which the data are not realistically available. Also, even amongst clinicians that do maintain problem lists, a master problem list may not be as prevalent as a problem list specific to a specialty.
4. In line 85, please clarify whether the intent is to measure whether patients receive the appropriate discharge instructions. Are there specific outcomes to which this measure is linked?
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:01:55 PM
Comments:
5. With regard to line 87, data types in and of themselves may not be sufficient if a description of their meaning/context is not included (for example, does a diagnosis field have a meaning if you don’t have the right reference set/vocabulary, eg, ICD-9?)
6. Rather than inpatient diagnosis/outpatient diagnosis, the measure should state the query criteria to be used, objective findings (age, physical finding), and assessments (eg, diagnosis of diabetes) as the denominator/inclusion criteria. The “plan” would refer to the action to be taken. The numerator is the evidence of plan execution (eg, medication or laboratory plus time). Exclusion criteria should specifically list other patient attributes (eg, pregnancy status). By this, looking at Table 4, is it useful to distinguish between medication (outpatient filled) vs medication order? Why isn't there an opt-out for every measure? Does this rely on a unified system between inpatient and outpatient? Looking at the Figure 1, could this be sorted by frequency?
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:01:55 PM
Comments:
7. With regard to the recommendation that medication allergies and side effects should be distinguished from each other and entered using standardized codes, CCHIT is addressing this issue as follows:
a. For 2008, the draft criteria require that an electronic health record (EHR) be able to specify the type of allergic or adverse reaction, in either free text or discrete data.
b. For 2009, the criteria roadmap proposes to require that an EHR be able to specify the type of allergic or adverse reaction using discrete data.
c. For 2010 and beyond, the criteria roadmap proposes to require that an EHR be able to specific the type of allergic or adverse reaction using standardized codes.
d. This roadmap gives HITSP time to identify appropriate terminology standards for medication allergies and adverse reactions, and also gives EHR vendors time to incorporate those standards into their production systems.
e. Consider existing work effort on Allergy standardization by Consolidated Health Informatics (attached) in defining ontologies and data structure related to allergy documentation.
8. There is no mention in the report that recognized that most of the measures are for adults and therefore child health information technology would be adversely impacted by the lack of comparable guidance. The EHR vendors build what they are told by the market and regulators (such as CCHIT). If they are being told how or encouraged to include pediatric measures, they won't. On page 23, it would be helpful to denote which measures (if any) relate to pediatrics.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:01:55 PM
Comments:
9. It is unfortunate that the committee did not come to a resolution regarding the priority measures.
10. With regard to the recommendation that NQF evaluate the quality of data types used in measure specifications as a criterion in the endorsement of measures, we support this recommendation. We believe that this role should not be limited to NQF, but that all parties involved in the development of clinical guidance and quality measures should be aware of the need for health information systems to accurately and consistently implement clinical guidance and report on quality measures. The AAP has already taken steps to address this need through the establishment of the Partnership for Policy Implementation, which works with clinical content experts to ensure AAP guidance to clinicians is structured to be implementable at the point of care.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:01:55 PM
Comments:
11. HITSP has just released the interoperability specification for the Quality Use case and this should become an important standard for reporting and querying measures. The goal is to be able to define queries to extract the measures from any EHR and to report and aggregate them. Health information exchange and regional health information organizations will add a cross-setting and cross-institutional dimension. Trial National Health Information Network implementations will use this specification. Also to be considered is the Quality Reporting Document Architecture (QRDA), a standard based on the HL7 Clinical Document Architecture, which is currently under development within HL7, supported by the Alliance for Pediatric Quality, and will help meet this need. The proposed QRDA supports use case requirements published by the Office of the National Coordinator for Health Information Technology and is consistent with requirements defined by the professional societies, quality organizations, and vendors. All indications are that QRDA is a critical component required to meet the data export requirements defined by the National Quality Forum (NQF) Health Information Technology Expert Panel (HITEP), The Collaborative for Performance Measure Integration with EHR Systems (The Collaborative), HITSP, and Integrating the Healthcare Enterprise (IHE).
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:06:27 PM
Comments:
11. (Cont'd) In fact, both HITSP and IHE quality work groups have reviewed the Phase I work on QRDA and have expressed a strong desire to adopt it as an integral part of their 2008 development. Phase I discussions indicate that QRDA supports and complements these efforts. Initial work on QRDA indicates that the standard can fit the interoperability requirements for the high-priority measures identified by HITEP.
12. CCHIT will eventually integrate quality reporting into EHR certification. A general purpose solution based on the HITSP spec (if it works) should be an efficient pathway to implementing quality measures.
13. Immunizations, Newborn Hearing, and Blood Spot Screening may become targets for American Health Information Community use cases that will address the ability of EHRs to provide quality measurement and reporting in these limited domains. The work on immunizations ix expected to start in April 2008, with Newborn Screening in 2009.
14. States are getting more involved in interoperability of health information technology and quality measurement (for licensure requirements). A report is expected from the National Governors Association in February on using health information technology and health information exchange to improve quality.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:06:27 PM
Comments:
15. Part of the reason why quality measurement specifications are not designed to leverage EHR systems is the availability of clinical data. Administrative datasets were the most commonly available source for assessing quality. With the emerging capability of EHRs to capture clinically-relevant quality information, measure developers must also have a paradigm shift in developing measures. Therefore, we endorse NQF’s recommendation to “encourage the use of high quality data elements for newly submitted measures.” However, we should also point out that the measure developers must have the ability/capability to provide explicit logic and algorithm for the desired parameters of the specific quality measure (ie, inclusion/exclusion criteria, measure set/category information, etc) so that the logic can be easily adopted by vendors and EHR system developers. The wheel-reinvention among the vendors usually occurs in the interpretation of the measure parameters and how the logic is embedded in the system’s rules engine. Therefore, if there is a well-validated process for standardizing the translation of the measures to computer-speak (i.e., computable logic); then it would make the adoption of the quality functions more readily-acceptable by the vendors. Such an effort is currently being undertaken by the American Academy of Pediatrics through its Partnership for Policy Implementation.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 1/11/2008 6:06:27 PM
Comments:
16. If we envision a standardization of computable quality measurement reporting activity; then a need for an authoritative source/repository of the measures is an important consideration. This way, vendors can target their development (which consumes a lot of resources) based on the authoritative source (updates, clarification, etc) and not from multiple organizations (AQA, Joint Commission, etc). This also necessitates identifying each quality measure uniquely in order to enhance measure interoperability in health information exchange situations, as well as in managing the versioning and updates for the given set of measures.
------------------------------------------------------------------------------------------------------
Name: Bernice Bennett, MPH, CHES
Organization: National Association of Public Hospitals and Health Systems
Date Entered: 1/15/2008 3:06:51 PM
Comments:
It is recommended that the standards for quality measurement include definitions of different patient populations in order to measure disparities, as well as other factors that could influence the measures and make them more useful for quality improvement activities. For race and ethnicity, please consider using the Office of Management and Budget (Census Bureau) list. In addition, efforts should be made to ensure that the selected terminology set can meet the need for identifying unique populations that are not typically captured in billing data, such as “smokers” and “patients at the end of life.” Also, other patients’ unique characteristics, including socioeconomic status, language and learning level/preferences, need to have standardized definitions and be included in HITS. This standardized approach to capturing this information would provide a better opportunity for comparative and collaborative efforts to improve care.
------------------------------------------------------------------------------------------------------
Name: Bernice Bennett, MPH, CHES
Organization: National Association of Public Hospitals and Health Systems
Date Entered: 1/15/2008 3:06:51 PM
Comments:
Consideration should be given to developing a process for discrete data elements to be entered manually to capture results from non-integrated systems. Many public health efforts are addressing services, such as immunizations and cancer screening, which are given outside of their main site. HITS need to allow for this information to be entered for accurate reporting.
------------------------------------------------------------------------------------------------------
Name: Bernice Bennett, MPH, CHES
Organization: National Association of Public Hospitals and Health Systems
Date Entered: 1/15/2008 3:06:51 PM
Comments:
Efforts should be devoted to standardizing the reporting requirements focusing on a format that supports the usability of the data for quality improvement efforts and consumer friendly public reporting. Most importantly, resources should be committed to support the continuous evaluation/ research, refinement of data standards, data collection/transmission and data reporting mechanisms to ensure validity, accuracy and comprehensiveness.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:20:55 PM
Comments:
General Feedback: True interoperability will not occur until data definitions and codes are standardized and incorporated into technical standards. AHIMA recommends including additional text describing how the common data types defined in the report should be aligned with similar data elements in other data sets. This exercise will support the movement toward collecting data once so it can be repurposed multiple times for quality, population health reporting, research, and administration.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:20:55 PM
Comments:
General Feedback: AHIMA suggests specifically calling out the role of standard development organizations (SDOs) and the need align the HITEP common data types with new and existing technical standards. Active engagement of SDOs will aid in bridging the gap between the quality and information technology enterprises.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:20:55 PM
Comments:
Expert Panel Analysis: Page 15, line 299 – “…The Expert Panel constructed an organizational chart (Figure 4)…” The figure notation is incorrect. The correct notation should be “Figure 3”.
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:20:55 PM
Comments:
Expert Panel Analysis: Page 15, line 305 – The report discusses "the need to influence 'upstream' processes so that measures being submitted for NQF endorsement meet criteria set to increase the comparative value of quality scores and minimize the effort required to acquire and report out the quality scores in an EHR." Following this statement, Figure 3 is displayed representing the "Organizational Relationships in Quality measurement, Health IT and NQF Influence." Does this figure represent current quality measurement, health IT and NQF relationships? If so, how will the organizational relationships change after the 'upstream' processes have been influenced? It may be useful to include an additional figure depicting the envisioned "future state."
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:20:55 PM
Comments:
Expert Panel Analysis: Page 16, Figure 3 – AHIMA recommends incorporating corresponding descriptions for each element of the diagram (similar to the format used in the AHIC Use Cases, with numbers and corresponding detailed descriptions of the entities and processes depicted).
------------------------------------------------------------------------------------------------------
Name: Crystal Kallem
Organization: American Health Information Management Association (AHIMA)
Date Entered: 1/15/2008 3:21:30 PM
Comments:
Expert Panel Recommendations: Page 18, Recommendations 5 and 6 – AHIMA supports recommendations for developing key EHR system functionality that will support automated collection and reporting of quality measurement information, but functionality should be defined and vetted in a coordinated and transparent manner, such as through the development of a Quality Measurement Functional Profile standard based on the HL7 EHR System Functional Model (EHR-S FM). Development of a Quality Measurement Functional Profile will facilitate a collaborative approach toward defining quality related EHR functionality, further demonstrate the business needs for system vendors, and guide the development of CCHIT certification criteria to support quality. AHIMA suggests that the HITEP enhance recommendations 5 and 6, stating "Quality and information technology stakeholders are encouraged to define EHR functional requirements that support quality measurement through the development of a Quality Measurement Functional Profile. This profile should include functionality to automatically capture the issuance of discharge