General comments on the document can be reviewed below. Comments are listed in the order they were submitted.
__________________________________________________________
Name: Deborah Mullen
Organization: HealthPartners
Date Entered: 11/21/2007 3:06:18 PM
Comments:
We suggest for increased readability of the report that the tables be incorporated within the text descriptions for tables 1 – 5: Appendix B.
We also suggest that the limitations cited of the current commercial episode groupers be more strongly worded to emphasize the need to create a patient centric view. Ideally this view would account for the multiple co-morbidities of patients, creating a patient centric expected value.
Page 59 we recommend the addition of the following definitions into the “cost of care” bulleted section:
Total cost indices (total cost of care) measure the relative resource use, intensity and price compared to the average total cost for the health plan population. The price indices measure the relative price of services managed by the provider(s) reported as compared to health plan population mean. The efficiency indices measure the relative utilization (volume of services) & intensity of services managed by the provider(s) reported as compared to health plan population mean.
Page 59 line 10: We recommend adding the following text…
units of a resource used) and intensity of services. Measurement…
------------------------------------------------------------------------------------------------------
Name: Deborah Mullen
Organization: HealthPartners
Date Entered: 11/21/2007 3:06:18 PM
Comments:
Page 60 line 17: We recommend adding the following text following the end of the sentence on line 17:
“Ratios may be equivalent for two provider(s), with vastly different quality levels. This occurs when a provider with low cost and low quality has the same ratio as a provider with high cost and high quality. To avoid equally valuing providers with low and high ratios, it is recommended that either a minimum standard for quality or a weighted factor for quality be utilized.”
------------------------------------------------------------------------------------------------------
Name: Christopher Dezii
Organization: Bristol-Myers Squibb
Date Entered: 12/3/2007 10:36:04 AM
Comments:
Please define and/or specifically identify “appropriate medications” as antiplatelet use is at least as important lipid management. (p.52, line 8)
------------------------------------------------------------------------------------------------------
Name: Christopher Dezii
Organization: Bristol-Myers Squibb
Date Entered: 12/3/2007 10:36:04 AM
Comments:
Consider substitution of Acute Myocardial Infarction with the more inclusive Acute Coronary Syndromes which cover ST-segment elevation MI, Unstable Angina and non-ST-segment elevation MI. An alternate could be to define AMI as ST-segment elevation MI, Unstable Angina and non-ST-segment elevation.
------------------------------------------------------------------------------------------------------
Name: Christopher Dezii
Organization: Bristol-Myers Squibb
Date Entered: 12/3/2007 10:36:04 AM
Comments:
Pharmacies and healthcare provider settings such as health clinics position’s do not appear to be clearly articulated in the framework. We suggest specific recognition of these settings.
------------------------------------------------------------------------------------------------------
Name: Christopher Dezii
Organization: Bristol-Myers Squibb
Date Entered: 12/3/2007 10:36:04 AM
Comments:
Clinical guidelines endorsed and developed by the experts (American Heart Association and American College of Cardiology for ACS) need to be equally considered along with performance measures throughout each phase.
------------------------------------------------------------------------------------------------------
Name: Jennifer Eames
Organization: Consumer-Purchaser Disclosure Project
Date Entered: 12/3/2007 2:27:10 PM
Comments:
Thank you for the opportunity to comment on the draft report on the Measurement Framework: Evaluating Efficiency Across Episodes of Care. Overall, we are supportive of the measurement framework outlined in this document. In particular, we want to affirm the following:
• Basing the conceptual framework on health outcomes and total costs over episodes of care. Linking quality with costs over time is important to understanding the true value of health care being provided.
• Emphasizing the urgent need for speedy development of additional outcomes measures. Outcomes measures are more relevant to consumer and purchasers than process or structural measures for understanding if care is effective.
• Using benchmarks to compare performance. Benchmarks that reflect best attainable care, and not just average performance, are important to reaching a higher performing health care system.
------------------------------------------------------------------------------------------------------
Name: Jennifer Eames
Organization: Consumer-Purchaser Disclosure Project
Date Entered: 12/3/2007 2:27:10 PM
Comments:
We strongly support the episode-based approach, and appreciate your providing a fair discussion of both the advantages and limitations of this approach, such as the fact that episode groupers may not capture appropriateness of care. Episode groupers do not, for example, address whether or not an intervention should have been performed in the first place. Including population-based resource use measures, such as those used in the Dartmouth Atlas, to provide more context for efficiency results should be pursued as one means for addressing this concern.
------------------------------------------------------------------------------------------------------
Name: Jennifer Eames
Organization: Consumer-Purchaser Disclosure Project
Date Entered: 12/3/2007 2:27:10 PM
Comments:
Principle 3 should highlight the need for providing information at the physician-level, both to stimulate provider improvement and better inform patient decision-making. Many efforts that have applied to physicians working in groups or teams have delivered sub-optimal results. Strategies that rely on group accountability often do not provide sufficient incentives to change individual physician care patterns. And, most importantly, consumers need and want information about variations in individual physician performance to inform their decisions. Thus, we recommend adding the following text at the end of the description of the principle: “In particular, it is important that results for individual doctors, in addition to higher levels of aggregation, are presented as physicians are a major determinant of health care services and expenditures.”
------------------------------------------------------------------------------------------------------
Name: David Hopkins
Organization: Pacific Business Group on Health
Date Entered: 12/6/2007 12:51:22 PM
Comments:
Thank you for the opportunity to comment on the draft report on the Measurement Framework: Evaluating Efficiency Across Episodes of Care. Overall, we are supportive of the measurement framework outlined in this document. In particular, we want to affirm the following:
• Basing the conceptual framework on health outcomes and total costs over episodes of care. Linking quality with costs over time is important to understanding the true value of health care being provided.
• Emphasizing the urgent need for speedy development of additional outcomes measures. Outcomes measures are more relevant to consumer and purchasers than process or structural measures for understanding if care is effective.
• Using benchmarks to compare performance. Benchmarks that reflect best attainable care, and not just average performance, are important to reaching a higher performing health care system.
------------------------------------------------------------------------------------------------------
Name: David Hopkins
Organization: Pacific Business Group on Health
Date Entered: 12/6/2007 12:51:22 PM
Comments:
We strongly support the episode-based approach, and appreciate your providing a fair discussion of both the advantages and limitations of this approach, such as the fact that episode groupers may not capture appropriateness of care. Episode groupers do not, for example, address whether or not an intervention should have been performed in the first place. We would ask that the Steering Committee consider including the strong recommendation that population-based resource use measures, such as those used in the Dartmouth Atlas and/or perhaps a simple ratio of severity-adjusted number of episodes per capita, should be used along with the episode-based measures to provide more context for efficiency results.
------------------------------------------------------------------------------------------------------
Name: David Hopkins
Organization: Pacific Business Group on Health
Date Entered: 12/6/2007 12:51:22 PM
Comments:
Principle 3 should highlight the need for providing information at the physician-level, both to stimulate provider improvement and better inform patient decision-making. Many efforts that have applied to physicians working in groups or teams have delivered sub-optimal results. Strategies that rely on group accountability often do not provide sufficient incentives to change individual physician care patterns. And, most importantly, consumers need and want information about variations in individual physician performance to inform their decisions. Thus, we recommend adding the following text at the end of the description of the principle: “In particular, it is important that results for individual doctors, in addition to higher levels of aggregation, are presented as physicians are a major determinant of health care services and expenditures.” Such multi-level reporting is consistent both with the IOM report on Performance Measurement and the NQF’s previously articulated principle that individual physician reporting on a measure should occur whenever individual physician performance differences are likely.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:14:41 PM
Comments:
Although I support the framework's explicit recognition of shared accountability among providers and the closely associated need for performance measures and accountable entities that capture the contributions of these providers, the draft report is physician-centric and fails to acknowledge the contribution of the variety of providers who contribute to each episode. Throughout the report, physicians are the sole provider mentioned and medical organizations featured most prominently. A more general term, such as practitioner, is recommended and examples - that are not limited to physicians alone (see pages 2, 15, 17, and 36)- are advisable.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:14:41 PM
Comments:
Page 8, line 12 refers to "medical specialty and subspecialty societies"; however, non-medical professional organizations (e.g., American Nurses Association) have also developed performance measures but have been excluded.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:14:41 PM
Comments:
Page 9, lines 1-14: The following additional bullet is suggested:
* Motivating, inspiring, and incentivizing health care professionals to provide care that is more safe, effective, patient-centered, timely, efficient, and equitable.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:14:41 PM
Comments:
Page 18, lines 1-4. Suggest the following addition:
.....which have been shown to contribute to better patient outcomes "and equitably reward all providers and practitioners who deliver care across the episode."
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:14:41 PM
Comments:
Dr. Mary Naylor has conducted several randomized controlled clinical trials that demonstrate positive effects of a nurse-directed transitional care intervention. These should be cited on page 16.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:27:25 PM
Comments:
Page 34, lines 7-13. Suggest additional bullet:
"How to incentive and reward practitioners equitably based on their contributions to an episode of care."
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:27:25 PM
Comments:
Page 26, figure 1 and Page 48, appendix C.
It is not clear how surveillance and avoidance of complications/comorbid conditions are integrated into this generic model. In this context, performance measures such as failure to rescue, falls, and pressure ulcers are meaningful to consumers in their evaluation of the health care system's performance.
------------------------------------------------------------------------------------------------------
Name: Ellen Kurtzman
Organization: The George Washington University, Department of Nursing Education
Date Entered: 12/6/2007 4:27:25 PM
Comments:
Domain 2 (cost and resource use) and generic episode of care/figure 1.
As an expression of resource use, evidence suggests the usefulness of nursing intensity weights (NIW). Nursing intensity weights (NIWs) are relative values reflecting the quantity and types of nursing services provided to patients in each diagnosis related group (DRG). The addition of NIWs as an example of a measure of resource use is recommended.
See:
Welton, JM, Zone-Smith L, Fischer MH. Adjustment of inpatient care reimbursement for nursing intensity. Policy, Politics, & Nursing Practice. 2006;7 270-280.
Welton, JM. (2007). Rates and inpatient nursing care. Health Aff. 2007;26:900-902.
Welton, JM, Fischer M, DeGrace S, Zone-Smith L. Nursing intensity billing. JONA. 2006; 36:181-188.
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher
Organization: American Nurses Association
Date Entered: 12/6/2007 10:14:26 PM
Comments:
NQF’s effort to achieve consensus on a measurement framework for evaluating the efficiency (quality and costs) of care over extended health care episodes is admirable. However, ANA is concerned to see, in Box 1, that Cost of care is the first definition offered and continues to be a dominant theme. Controlling cost is critical, but ANA suggests that Quality of care be presented first as it is an overarching principle in healthcare, and the evaluation of efficiency. In addition, resources should specify: material, personnel, financial resources.
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher
Organization: American Nurses Association
Date Entered: 12/6/2007 10:14:27 PM
Comments:
Furthermore, it is critical that the “episode” allow for quantification of the cost/value of nursing care/services and that registered nurses (RNs) be involved in the development of shared accountability models.
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher
Organization: American Nurses Association
Date Entered: 12/6/2007 10:14:27 PM
Comments:
ANA is pleased to see the emphasis on patient involvement and inclusion of patient preferences. However, there is no qualifying caveat for health literacy. On page 28 there is mention of an "informed patient;" on page 55 there is a discussion of "decision support aids." None of these will hold any relevance unless the concept of health literacy is addressed (if only as a pre-condition to either being informed or being supported in making decisions).
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher
Organization: American Nurses Association
Date Entered: 12/6/2007 10:14:27 PM
Comments:
Also, there is a glaring absence of any mention of mental illnesses which comprise a very large percentage of U.S. health care expenditures. Specifically, on page 18, depression and schizophrenia (or other illnesses) could be mentioned when various illnesses are listed.
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher
Organization: American Nurses Association
Date Entered: 12/6/2007 10:14:27 PM
Comments:
Finally, as has been noted previously in the association’s comments on a host of NQF projects, ANA does appreciate that physicians are a significant component of the licensed independent practitioners providing care. Nevertheless there are also other clinicians delivering that care. Once again, the association respectfully requests relevant verbiage within NQF documents, including Measurement Framework: Evaluating Efficiency across Episodes of Care, 2007, be changed to be in concert with terminology utilized by other healthcare organizations, including The Joint Commission, reflecting care as being provided by licensed independent practitioners. In particular, ANA wishes to confirm that the care provided by Advanced Practice Registered Nurses (APRNs) is included in "strategies for financing healthcare" (p. 17).
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement SubCommittee)
Organization: American College of Physicians
Date Entered: 12/7/2007 10:37:26 AM
Comments:
The "accountable care entities" are vaguely defined and it is unclear what their scope is. It is difficult to envision measurement without knowing more about the specifics of the accountable care entities.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement SubCommittee)
Organization: American College of Physicians
Date Entered: 12/7/2007 10:37:26 AM
Comments:
Past research tells us that failure to diagnose and misdiagnosis are prevalent and potentially important causes of inefficiency. For example, a patient with GERD symptoms may be properly diagnosed as GERD in one physician's office and never result in an ED visit. Another physician, seeing the same patient may send the patient to the ED to rule out cardiac chest pain based on an inadequate history. The episode framework would classify the patients into 2 different categories even though they presented with the same symptoms (and only the physician who diagnosed GERD got it right). The episode framework cannot detect this sort of inefficiency.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement SubCommittee)
Organization: American College of Physicians
Date Entered: 12/7/2007 10:37:26 AM
Comments:
The Framework claims to represent the patient's perspective, but everything else about it suggests that the key perspectives on efficiency are those of the provider and payer. The choice of disease episode as the organizing framework seems to point to the delivery system as the perspective of interest.
------------------------------------------------------------------------------------------------------
Name: John Bott
Organization: Employer Health Care Alliance Cooperative
Date Entered: 12/8/2007 11:01:16 AM
Comments:
Improvement vs accountability
We suggest that the report better articulate, and more consistently address, whether one of the purposes of efficiency measurement is for accountability (e.g. use in public reporting, use in purchasing). An example of the inconsistency is comparing p. 13 (row 5-11) and p. 32 (row 1-6). Page 13 discusses the purpose of efficiency measurement as improvement and omits any mention of accountability. Meanwhile, principle 6 (p. 32) does acknowledge public reporting of efficiency.
The uncertainty of whether the report is sufficiently addressing efficiency for accountability as well as quality improvement comes up several times. The following provides an example of public reporting and purchasing:
Page 16 (row 13-16): The notion of “shifting the focus away from the individual provider’s actions” is appropriate for quality improvement. However, to provide consumers with information to assist their decision making, people need quality and cost information at an actionable level, such as the physician, ambulatory surgical center and hospital level.
Page 31 (row 7-8): The concept of “the smallest unit of accountability should be measured and reported” is not always the most desirable level for value based purchasing. For example, there is a compelling argument to direct a financial incentive at the group vs the physician level. Thus, for some purposes the smallest unit of analysis is not the most useful/actionable from the purchaser vantage point
------------------------------------------------------------------------------------------------------
Name: John Bott
Organization: Employer Health Care Alliance Cooperative
Date Entered: 12/8/2007 11:01:16 AM
Comments:
Efficiency via measurement beyond episodes
The report appears to limit the measurement of efficiency to employing episodes of care groupers for the quality and cost (e.g. p. 18, row 18 to p. 19 row 3, and p 20, row 14 to 21). The field of measuring efficiency should be open as to all viable means of measurement, which extend beyond episode groupers.
For example, cost of diabetics could potentially be measures via an episode grouper. Meanwhile, the quality component of diabetic care could be measured with a set of nationally recognized diabetic quality measures. Most diabetic quality measures define diabetes in the denominator with a desirable or undesirable event in the numerator. These quality measures are not what we think of as based on episodes of care grouping logic.
We recommend the report be revised to be clear that measurement of efficiency can be performed by all viable methods including, but are not limited to, episodes of care grouping logic.
------------------------------------------------------------------------------------------------------
Name: John Bott
Organization: Employer Health Care Alliance Cooperative
Date Entered: 12/8/2007 11:01:16 AM
Comments:
Efficiency domains
Principle 1 (p. 30), notes efficiency should be measured through three domains. We have comments on each domain:
Patient based preferences and outcomes
It is not clear with constitutes “patient based preferences” in regard to it being a domain. Secondly, “patient based preferences” and “outcomes” seem to be two different domains.
Cost and resource use
Cost and resource use are really two different rubrics. From a purchaser perspective, we are most interested in what McGlynn notes as “productive efficiency”, which is essentially the cost to the purchaser and the quality of the service1.
Process
Although the literature is thin, it suggests purchasers and consumers want to know outcome vs process results. We suggest a revision to note that a desired domain is “outcomes vs. process”. This domain would state that where outcomes results are unavailable, process measures can be of value when it is strongly correlated with outcomes. In this case, a process is a proxy for outcomes. If outcomes can be reported there is then no compelling reason for reporting a proxy to outcomes (process).
One other note in regard to process vs outcomes is in regard to principle 2 (p. 30) where only process is discussed. Given the fact that this principle discusses selecting measures, we strongly recommend that it includes a discussion of choosing outcome measures.
1 McGlynn et al. “Identifying, categorizing, and evaluating healthcare efficiency measures" (2007
------------------------------------------------------------------------------------------------------
Name: Donald Casey
Organization: Atlantic Health
Date Entered: 12/9/2007 2:58:59 PM
Comments:
From Commonwealth Fund, December, 2007
"Assignment of Cost: In a program based on episodes of care, efficiency indexes can be calculated from that portion of costs directly generated by the physician or from total episode costs. Direct costs include office visits to the practitioner, medications prescribed, tests ordered, or hospitalizations under his or her care. Total costs are those services generated by the practitioner in question plus all the costs generated by other providers during the episode. The decision to use directly generated or total costs has important consequences for practitioners’ scores as well as ramifications for their uses. Practitioners reasonably argue that they can only be held accountable for the costs that they themselves engender. However, scoring on direct costs creates an incentive to make other practitioners order the more expensive interventions. In such a system, the practitioner can say, “I know how to appear less expensive—I’ll have you do the work.” This methodological choice has thus resulted in delays in appropriate care because of concerns over who should prescribe chronic medication refills or who should order the MRI, as that might worsen an individual’s efficiency score."
This seems far more aligned with the notion of using standardized cost accounting methods, which is not at all addressed in the document. Cost accounting industry and experts should weigh in on this document by invitation from NQF. Need harmonization!!
------------------------------------------------------------------------------------------------------
Name: Donald Casey
Organization: Atlantic Health
Date Entered: 12/9/2007 2:58:59 PM
Comments:
The NQF document is not aligned with the government’s and the accounting industry’s standard definitions of cost accounting. For example: Cost Accounting Standards
(see http://fast.faa.gov/archive/v1198/pguide/98-30C14.htm#Summary%20of%20The%20Cost%20Accounting%20Standards)
The Three Areas of Cost Accounting (48 CFR 9903.302-1)
Measurement of Cost involves the methods and techniques used in defining the components of cost, determining the basis of cost measurement, and establishing criteria for use of alternative cost measurement techniques. Examples of cost measurement are listed below:
The use of historical cost, market value, or present value;
The use of standard or actual cost; or
The designation of items of cost which must be included or excluded from tangible assets or pension cost.
Assignment of cost to cost accounting period refers to the method used in determining the amount of cost to be assigned to individual cost accounting periods. Examples are the requirements for use of accrual basis or cash basis accounting.
Allocation of cost to cost objectives refers to the method of determining direct and indirect allocation of cost. Examples of allocation issues are listed below:
The accumulation of costs; The determination of whether to charge costs direct or indirect; or The determination of the composition of cost pools and their allocation bases.
Are we suggesting the creation a "parallel universe" that is neither aligned or harmonized with industy??
------------------------------------------------------------------------------------------------------
Name: Donald Casey
Organization: Atlantic Health
Date Entered: 12/9/2007 2:58:59 PM
Comments:
Several members of the NQF Board have talked about having the same approach to health care value measurement as that promulgated by the Financial Accounting Standards Board (FASB) through the well established Generally Accepted Accounting Principles ("GAAP"). Some of the characteristics of GAAP includes:
1. Relevance: “the capacity of the information to make a difference in a decision”
2. Timeliness: As the future becomes the present, past data becomes increasingly irrelevant (Auditing of data that is old does not compensate for improved data quality)
3. Reliability: verifiability, representational faithfulness, neutrality: has care been measured accurately and is an adequate reflection of care delivered
4. Neutrality: FASB should consider only the relevance and reliability of the data, but not the economic impact
5. Consistency & Comparability are pervasive problems
6. Materiality: It makes a difference in valuation of the firm
7. Qualitative characteristics of information are as important as quantitative interpretations
In addition, FASB has established a specific hierarchy of accounting qualities (see http://fasb.org/pdf/con2.pdf) that frame the process of the development GAAP. The members of the NQF Steering Committee should review this document in detail and take an important lesson as an enhancement to a very good start on a very difficult subject. See also
http://www.fasb.org/facts/ for facts about FASB.
------------------------------------------------------------------------------------------------------
Name: Donald Casey
Organization: Atlantic Health
Date Entered: 12/9/2007 2:58:59 PM
Comments:
In the end, the document is a good start, but it is very “wonkish” and devoid of expertise from the Financial Accounting and Management professions. I think that the next step should be to test this approach with these experts in order to align and harmonize the proposed theoretical construct with the day-to-day harsh realities of cost estimation and accounting. This discipline is currently lacking per se within health care in my opinion. Why not set a new tone for a more rigorous disciplined approach to this very complex topic?
It also seems to me that the document’s perspective is very much from a third-party payor standpoint and not a health system or provider standpoint. Hence, payors derive their own costs from provider price information, which is already highly regulated and (out of providers’ economic prudence and necessity) not transparent.
------------------------------------------------------------------------------------------------------
Name: Donald Casey
Organization: Atlantic Health
Date Entered: 12/9/2007 2:58:59 PM
Comments:
Note also that the accounting industry has developed XBRL (see : http://www.xbrl.org/Home/ and http://www.xbrl.org/Specification/XBRL-RECOMMENDATION-2003-12-31+Corrected-Errata-2005-04-25.htm) which is described below:
XBRL is the specification for the eXtensible Business Reporting Language. XBRL allows software vendors, programmers, intermediaries in the preparation and distribution process and end users who adopt it as a specification to enhance the creation, exchange, and comparison of business reporting information. Business reporting includes, but is not limited to, financial statements, financial information, non-financial information, general ledger transactions and regulatory filings, such as annual and quarterly reports.
I think that a strategic document such as this one put forth by NQF should necessarily include some vision of achieving a similar type of process to develop similar standards for the measurement of efficiency for healthcare organizations and practitioners so that in the “electronic age” we can all get on the same page (eventually).
Suppose your organization were to be audited by Ernst & Young or PwC to validate your “cost of care” methods. Do you think their experts would agree with and understand the NQF framework well enough to do their work? Hence there is a very compelling need to further validate this document and its tenants with experts within the cost accounting industry.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 12/10/2007 10:19:45 AM
Comments:
Thank you for the opportunity to comment on the draft document, Measurement Framework: Evaluating Efficiency Across Episodes of Care. I am pleased to offer the following comments on behalf of the National Association of Children’s Hospitals and Related Institutions (NACHRI).
This is an important and effective document in furthering the understanding of key concepts. Minor comments include the following:
• Page 7/ line 4: Is variation in spending ever positively related to the quality of care?
• Page 10/ lines 15 and 16: Although the selection of AMI and low back pain are reasonable for the purposes of this initial work, the framework may need additional refinement to be generalizable to other populations, such as children. For example, the length of an episode of care may be far longer and the likelihood of co-existing conditions and morbidities more likely in children’s health care. The assessment of efficiency of care in this population may require a much longer time frame.
• Page 13/lines 6 and 7: Given the discussion of the importance of shared decision making elsewhere in the documents, might it make sense to include patients and caregivers in the list of “component parts?”
• Page 16/line 7: A minor point, but is the one-year in parentheses meant to be an example? If so, it should be “e.g.” not “i.e.”
• Page 18/line 13: It might be helpful to give an example of a “tracer” condition.
• Page 21/lines 1 and 2: There appears to be something missing from this sentence.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 12/10/2007 10:19:45 AM
Comments:
• Page 24/footnote 28: It might be helpful to mention the National Institutes of Health PROMIS effort in this footnote.
• Page 24/line 17: A definition of standardized prices would be helpful.
• Page 25/lines 17 through 22: The placement of patient experience with care under the domain of processes of care is potentially confusing. Whether or not the patient’s experience with care was measured or whether or not they were involved in decision making do seem to be processes of care. But how the patient experienced the care (e.g., presence of pain, satisfaction with the interaction) seem more like intermediate outcomes. This area might warrant more discussion.
• Page 28/lines 15-17: This sentence seems unclear. By “another population” do you mean the broader population (e.g., adults without heart disease)?
• Page 30/lines 11-12: A minor comment, but is it efficiency that is multi-dimensional or efficiency measurement (or both)?
• Page 35: It might be helpful to discuss value exchanges and RHIO’s in this section.
• Page 35/lines 16 and 17: Recommend including the word valid (i.e., “where the data are reliable, valid and useful”).
• The FACCT framework is introduced for the first time in Appendix C. It might be useful to describe it in the main document given its widespread use and usefulness in explaining the trajectory of health and health care.
------------------------------------------------------------------------------------------------------
Name: Cheri DiGiovanni
Organization: Ingenix
Date Entered: 12/10/2007 11:42:34 AM
Comments:
How does geography and region fit into the model and outcomes discussion? Is there consideration for these factors here?
------------------------------------------------------------------------------------------------------
Name: Cheri DiGiovanni
Organization: Ingenix
Date Entered: 12/10/2007 11:42:34 AM
Comments:
While in theory agreement on an effective measurement framework across all components of a healthcare system is the goal, the NQF does not provide ideas as to how to gain accountability in today’s fragmented healthcare environment. We view this to be a challenge for this project.
------------------------------------------------------------------------------------------------------
Name: Cheri DiGiovanni
Organization: Ingenix
Date Entered: 12/10/2007 11:42:34 AM
Comments:
Page 3 – Lines 3-5 - Measuring outcomes varies across entities today. Will the NQF propose uniform methods for measuring this for this project?
Page 3 - Line 16 – Measuring productivity varies across entities today. Will the NQF propose uniform methods for measuring this for this project?
------------------------------------------------------------------------------------------------------
Name: Cheri DiGiovanni
Organization: Ingenix
Date Entered: 12/10/2007 11:42:34 AM
Comments:
Page 8 – Barriers to Transformation – Perhaps some consideration should be made around several current national committees creating measures for the same conditions that do not agree. Currently, there is no single resource viewed to build condition measures that are considered to be uniform and are agreed upon by all interested parties.
------------------------------------------------------------------------------------------------------
Name: Cheri DiGiovanni
Organization: Ingenix
Date Entered: 12/10/2007 11:42:34 AM
Comments:
Page 18 – Limitations of an episode of care approach – It is important to note that current commercial episode groupers are designed to group claims into episodes of care so that post processing methods can occur that allow for such activities as addressing appropriateness of care and performing comparisons among organizations. While we agree that it is crucial to include quality measurement logistics into determining a provider’s efficiency, we also think that a point should be made about how different post-processing methodologies such as provider attribution, for example, can have an impact as to how a provider is measured in different environments.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement SubCommittee)
Organization: American College of Physicians
Date Entered: 12/10/2007 11:47:42 AM
Comments:
The American College of Physicians has a paper that supports the linking of cost of care with the quality of care delivered.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement SubCommittee)
Organization: American College of Physicians
Date Entered: 12/10/2007 11:47:42 AM
Comments:
The Performance Measurement SubCommittee suggests that the NQF consider the use of the government's and accounting industry's cost accounting standards.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:00:29 PM
Comments:
The American Medical Association (AMA) is pleased to have the opportunity to comment on the National Quality Forum’s (NQF) draft Measurement Framework: Evaluating Efficiency across Episodes of Care Report. This report advances the current work in the area of efficiency and we generally support the overall framework and principles. In this letter, we outline our support for several key components/concepts, provide requests for clarification and recommendations for further refinement, and discuss several overarching issues for the Committee’s consideration.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:00:29 PM
Comments:
Support of Key Components/Concepts:
This report does a laudable job in fleshing out some key concepts in the discussion of healthcare efficiency to advance mutual understanding and shared meaning in stakeholder dialogue and efforts in this important area. We are very supportive of the concept of shared accountability rather than holding an individual physician, a hospital, or other type of provider solely responsible, which is critical since it is how care is delivered – by more than one individual or entity as discussed in Principle 3. In particular, this concept is necessary when discussing a longitudinal, episode-based approach as discussed on page 16. In addition, the issue of attribution and actionability must also be discussed and our specific comments related to that issue are outlined below.
The need to focus on processes of care that have a strong link to outcomes is essential and we commend the Committee for emphasizing this need.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:00:29 PM
Comments:
Request for Clarification and Recommendations for Further Refinement:
The term “accountable care entity” is used on page 5, line 3 and page 35, line 7. The true meaning of this term is unclear and we request further discussion and elaboration on this term.
We are very supportive of Principle 6 on page 32 and would ask that the recent work undertaken by the New York Attorney General to have insurers comply with guidelines for physician profiling activities should be included as a template or example of how to implement this principle. The guidelines agreed to include the following:
• Physician rankings should not be based solely on cost, and the degree to which any ranking is based on cost should be disclosed
• Established national standards to measure quality and cost efficiency should be used, including measures endorsed by the National Quality Forum (NQF) and other generally accepted national standards
• Several measures should be used to foster more accurate physician comparisons, including risk adjustment and valid sampling
• How the program is designed and how physicians are ranked should be disclosed to consumers, and a process for consumers to register complaints about the system should be provided
• How physicians rankings are designed should be disclosed, and a process to appeal incorrect rankings should be provided
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:00:29 PM
Comments:
While we strongly agree with Principle 7 that inappropriate care should not be considered efficient care, we request that this section also emphasize that the determination of what is or is not appropriate care involve multiple stakeholder groups. This assessment of what constitutes inappropriate care is highly complex and should not be determined by one individual or stakeholder group.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:00:29 PM
Comments:
Overarching Comment and Specific Recommendations:
A significant contribution of this report is the identification of a meaningful unit of observation – the episode of care upon which to begin to think about how measurements can be made along variables that pertain to efficiency (eg, outcomes, cost-of-care). The episode of care has the advantage of being an intuitive and thus interpretable unit of observation that would be meaningful to patients, consumers, and purchasers.
The section “Advantages of an episode of care approach” notes the third advantage is that “episodes could foster and enable new strategies for financing healthcare that could eliminate current incentives to overuse of certain services…and underuse of others…” There is no way to assess the possible success of any new financing strategies without a detailed description of that strategy. Financing strategies, no matter how well intended, have unintended incentives, and resulting consequences, that need to be fleshed out. In particular, disincentives to care for complex, difficult or noncompliant patients must be avoided. Grouping and attributing services via episodes of care may not adequately address fair reimbursement for services.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:02:35 PM
Comments:
In addition, we are not certain if the discussion of key concepts and the proposal of a standard unit of observation is the full intent of the NQF document. If the intent of this framework is to outline the conceptual components, their interrelationships, and additional methodological requirements necessary for constructing measures of efficiency or value, then we would suggest additional conceptual and methodological detail, which we believe would further enhance this document. However, it is our understanding that the NQF is developing a second document on implementation that may address these methodological concerns and limitations of the current document. We have outlined our recommendations for your consideration below.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:02:35 PM
Comments:
Measuring Value:
We applaud the NQF’s initiative to propose measurement of consumer value of healthcare. The notion of value as distinct from cost-containment is at the forefront of discussions about U.S. healthcare expenditures. While outcomes, cost-of-care and process are key components that feed into measures of efficiency and ultimately, value, a critical component that is necessary is patient valuation of outcomes.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:02:35 PM
Comments:
The report also acknowledges the need to account for patient preferences. It is unclear, however, how patient preferences will be operationalized, measured, and integrated into the framework. Currently patient preferences are subsumed under the same domain as outcomes. We question whether outcomes are confused with patient preferences for those outcomes. Outcomes can be measured in terms of mortality or survival. If measured by survival, we agree that survival time should take into account quality of life. This involves some measurement of health status or health state, which can be accomplished using the Health-Related Quality of Life (HRQoL) instrument. HRQoL, however, tells us nothing about preferences for improving health status, or what value patients place on that improvement. To arrive at value, NQF would have to go beyond quantifying outcomes, to quantifying the value to patients of those outcomes. For example, a patient who is diagnosed with prostate cancer may value a shorter lifespan with improved quality of life (eg, no impotence) and this patient preference must be measured and factored in this evaluation of care.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:02:36 PM
Comments:
Limitations of Adopting Consumer Perspective:
The Committee has explicitly adopted the consumer’s perspective in its approach to measuring efficiency (page 13: ”the Committee concluded that its work should proceed primarily from the patient’s perspective”). Yet, it appears that there is interest in having a set of measures that could be used by multiple stakeholders (for example, see the examples and discussion in Appendix B). Measures designed from the perspective of one stakeholder – in this case, the patient – may be highly informative and optimized for their use. However, it cannot be assumed that the same measures will be informative and optimized for the purpose and use of other stakeholders. Measures that are designed to be maximally informative and helpful in guiding patient choices of provider may not be informative and helpful in guiding healthcare improvement (with respect to quality and/or efficiency) or in guiding societal choices about healthcare allocations. In other words, we caution whether one measurement framework can suit all stakeholders equally without the generation of unintended consequences for one or more groups – one size may not fit all.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:02:36 PM
Comments:
The Domains:
We find the approach to identify domains of constructs for assessing efficiency and value to be a very helpful approach. However, this approach could be made even more effective by re-organizing the domains to first reflect those pertaining to the “input” side of healthcare production, followed by those pertaining to the “output” side of healthcare production. For example, Processes of Care and Cost and Resource Use might be re-numbered as Domains 1 and 2; and Patient-level Health Outcomes could be re-numbered as Domain 3. We further suggest adding a fourth domain: Patient Preferences. The importance of incorporating patient preferences in evaluations of healthcare is discussed, but it is not entirely clear how preferences will be measured and assimilated with measures along all the other domains to derive an inference about efficiency. It currently appears that preferences are included under the Domain of Patient-level Health Outcomes. We caution that this may lead to a tendency to confuse outcomes with preferences for those outcomes. Thus, we suggest separating out patient preferences as a fourth domain.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:03:43 PM
Comments:
Actionability:
A key characteristic of an informative and actionable measure will be how much sense it makes to patients. Using episodes as the unit of observation makes intuitive sense. Patients can understand what an episode means, and they can use that information to guide decision-making. However, episodes as a whole may be less helpful to physicians attempting to improve quality and/or efficiency.
A physician must be able to trace the component(s) or element(s) of care that can be improved and that are actionable to her. This actionability/accountability is referenced in Principle 3 on page 31. If this explicit detail is not provided, the episode will become a black box, and a physician wishing to improve efficiency or quality per episode will not know how to focus her efforts.
This possible loss of detail that occurs in episode grouping is an additional limitation that should be acknowledged. It will be an even more pronounced problem when a longitudinal or “prolonged” episode is the unit of analysis.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:03:44 PM
Comments:
Attribution:
If the Committee intends measures of efficiency to be actionable for physicians as well, then using an episode-of-care framework poses an additional problem – namely, that of attributing care to one physician, when providing care is a collaborative effort. NQF recognizes the need for shared accountability, but remains silent on offering practical methods for achieving shared accountability. It will be necessary to specify how care should be attributed to different physicians involved in delivering care within an episode. We recommend that a discussion on attribution be included in this report, and that full explication occur in the implementation report, which is planned.
------------------------------------------------------------------------------------------------------
Name: Nancy Nielsen
Organization: AMA
Date Entered: 12/10/2007 3:03:44 PM
Comments:
Interrelationships:
In this document, NQF has taken an instrumental first step in laying out some of the key components of efficiency and value: processes of care, costs of care, and outcomes (we suggest in preceding comments that patient preferences are a separate component). However, it is difficult to move beyond this collection of constructs to some measure (or method for measuring) efficiency on the basis of what is provided in this document alone. It would be helpful to explicitly lay out how measures of processes of care, measures of costs of care, measures of outcomes, and measures of patient preferences for those outcomes might be combined either in a single metric, or in a reporting strategy, that would yield coherent information about “efficiency” or “value.” What is lacking in this document are empirical definitions of cost-of-care, quality, outcomes, value, and efficiency and we recommend that these concepts be defined in this report.
We welcome the opportunity to comment on this report and look forward to reviewing additional reports from this project, including the forthcoming detailed implementation strategy.
------------------------------------------------------------------------------------------------------
Name: Madeleine Smith
Organization: AdvaMed
Date Entered: 12/10/2007 3:05:06 PM
Comments:
We applaud the Committee’s report Measurement Framework: Evaluating Efficiency Across Episodes of Care. It provides a comprehensive, balanced, and well-written document on a difficult topic. Most importantly in our view, it strongly urges consideration of both quality and costs when analyzing efficiency and value.
------------------------------------------------------------------------------------------------------
Name: Madeleine Smith
Organization: AdvaMed
Date Entered: 12/10/2007 3:05:06 PM
Comments:
We have several suggestions that we urge the Committee to consider:
• We do not believe that a provider should be considered efficient if he or she does not deliver high quality care, or, at a minimum, an acceptable level of quality. Similarly, we believe that a provider that delivers a substantially higher level of quality should not be considered inefficient if his or her costs are higher than a provider who delivers a substantially lower quality of care. Perhaps these observations could be added to the text on page 31, after line 23, in the description of Principle 5. On the other hand, the Committee may believe that these observations illustrate our “individual preferences” and could be added as illustrations in the discussion on pages 42 and 43.
• Page 10, line 22 states “most efficient, high quality healthcare possible.” We recommend changing this to “highest quality, most efficient healthcare possible.”
• Page 60, lines 18 – 19, state that efficiency is directly proportional to quality, and inversely proportional to cost. We would agree that: 1) efficiency is a function of both quality and cost; and 2) efficiency is positively related to quality and negatively related to cost. We are not sure that this translates into “proportionality.” Does “proportionality” imply a static relationship between quality and cost?
• Pages 18-19 discuss limitations of existing commercial episode groupers. We believe that this discussion should include difficulties in attribution.
------------------------------------------------------------------------------------------------------
Name: Madeleine Smith
Organization: AdvaMed
Date Entered: 12/10/2007 3:05:06 PM
Comments:
Some minor observations:
• We believe that page 22, line 14 should eliminate the word “unnecessary” as a modifier to “waste.” What is “necessary” waste?
• We do not understand what is meant by “quality waste” on page 61, line 15.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:07:31 PM
Comments:
The Physician Consortium for Performance Improvement (Consortium) is pleased to have the opportunity to comment on the National Quality Forum’s (NQF) draft Measurement Framework: Evaluating Efficiency across Episodes of Care Report. This report advances the current work in the area of efficiency and we write in support of the overall framework and principles. In this letter, we outline our support for several key components/concepts, provide requests for clarification and recommendations for further refinement, and discuss several overarching issues for the Committee’s consideration.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:07:31 PM
Comments:
Support of Key Components/Concepts:
This report does a laudable job in fleshing out some key concepts in the discussion of healthcare efficiency to advance mutual understanding and shared meaning in stakeholder dialogue and efforts in this important area. We are very supportive of the concept of shared accountability rather than holding an individual physician, a hospital, or other type of provider solely responsible, which is critical since it is how care is delivered – by more than one individual or entity as discussed in Principle 3. In particular, this concept is necessary when discussing a longitudinal, episode-based approach as discussed on page 16. In addition, the issue of attribution and actionability must also be discussed and our specific comments related to that issue are outlined below.
The emphasis on the strong link between processes of care and outcomes is essential and we commend the Committee for emphasizing this need. The Consortium is continually striving to document this link when developing physician-level measures.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:07:31 PM
Comments:
Request for Clarification and Recommendations for Further Refinement:
We note that the definition of Quality of Care included in the AQA Principles of Efficiency Measures on the AQA web site differs from the definition used in Box 1 (page 1) in this report. The phrase, “…a measurement construct of pure benefit,” cannot be found in the AQA principles document. It would be helpful to further explain its meaning and align the definitions if possible.
The term “accountable care entity” is used on page 5, line 3 and page 35, line 7. The true meaning of this term is unclear and we request further discussion and elaboration on this term.
While we strongly agree with Principle 7 that inappropriate care should not be considered efficient care, we request that this section also emphasize that the determination of what is or is not appropriate care involve multiple stakeholder groups. This assessment of what constitutes inappropriate care is highly complex and must not be determined by one individual or stakeholder group.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:07:31 PM
Comments:
Overarching Comment and Specific Recommendations:
A significant contribution of this report is the identification of a meaningful unit of observation – the episode of care upon which to begin to think about how measurements can be made along variables that pertain to efficiency (eg, outcomes, cost-of-care). The episode of care has the advantage of being an intuitive and thus interpretable unit of observation that would be meaningful to patients, consumers, and purchasers.
The section “Advantages of an episode of care approach” notes that third advantage is that “episodes could foster and enable new strategies for financing healthcare that could eliminate current incentives to overuse of certain services…and underuse of others…” There is no way to assess the possible success of any new financing strategies without a detailed description of that strategy. Most strategies, no matter how well intended, have unintended incentives, and resulting consequences, that need to be fleshed out.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:07:31 PM
Comments:
In addition, we are not certain if the discussion of key concepts and the proposal of a standard unit of observation is the full intent of this report. If the intent of this framework is to outline the conceptual components, their interrelationships, and additional methodological requirements necessary for constructing measures of efficiency or value, then we would suggest additional conceptual and methodological detail, which we believe would further enhance this document. However, it is our understanding that the NQF is developing a second document on implementation that may address these methodological concerns and limitations of the current document. We have outlined our recommendations for your consideration below.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:08:55 PM
Comments:
Measuring Value:
We applaud the NQF’s initiative to propose measurement of consumer value of healthcare. The notion of value as distinct to cost-containment is at the forefront of discussions about U.S. healthcare expenditures. While outcomes, cost-of-care and process are key components that feed into measures of efficiency and ultimately, value, a critical component that is necessary, is patient valuation of outcomes.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:08:55 PM
Comments:
The report also acknowledges the need to account for patient preferences. It is unclear, however, how patient preferences will be operationalized, measured, and integrated into the framework. Currently patient preferences are subsumed under the same domain as outcomes. We question whether outcomes are confused with patient preferences for those outcomes. Outcomes can be measured in terms of mortality or survival. If measured by survival, we agree that survival time should take into account quality of life. This involves some measurement of health status or health state, which can be accomplished using the Health-Related Quality of Life (HRQoL) instrument. HRQoL, however, tells us nothing about preferences for improving health status, or what value patients place on that improvement. To arrive at value, NQF would have to go beyond quantifying outcomes, to quantifying the value to patients of those outcomes. For example, a patient who is diagnosed with prostate cancer may value a shorter lifespan with improved quality of life (eg, no impotence) and this patient preference must be measured and factored in this evaluation of care.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:08:55 PM
Comments:
Limitations of Adopting Consumer Perspective:
The Committee has explicitly adopted the consumer’s perspective in its approach to measuring efficiency (page 13: ”the Committee concluded that its work should proceed primarily from the patient’s perspective”). Yet, it appears that there is interest in having a set of measures that could be used by multiple stakeholders (for example, see the examples and discussion in Appendix B). Measures designed from the perspective of one stakeholder – in this case, the patient – may be highly informative and optimized for their use. However, it cannot be assumed that they will be informative and optimized for the purpose and use of other stakeholders. Measures that are designed to be maximally informative and helpful in guiding patient choices of provider may not be informative and helpful in guiding healthcare improvement (with respect to quality and/or efficiency) or in guiding societal choices about healthcare allocations. In other words, we caution whether one measurement framework can suit all stakeholders equally without the generation of unintended consequences for one or more groups – one size may not fit all.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:08:55 PM
Comments:
The Domains:
We find the approach to identify domains of constructs for assessing efficiency and value to be a very helpful approach. However, this approach could be made even more effective by re-organizing the domains to first reflect those pertaining to the “input” side of healthcare production, followed by those pertaining to the “output” side of healthcare production. For example, Processes of Care and Cost and Resource Use might be re-numbered as Domains 1 and 2; and Patient-level Health Outcomes could be re-numbered as Domain 3. We further suggest adding a fourth domain: Patient Preferences. The importance of incorporating patient preferences in evaluations of healthcare is discussed, but it is not entirely clear how preferences will be measured and assimilated with measures along all the other domains to derive an inference about efficiency. It currently appears that preferences are included under the Domain of Patient-level Health Outcomes. We caution that this may lead to a tendency to confuse outcomes with preferences for those outcomes. Thus, we suggest separating out patient preferences as a fourth domain.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:08:55 PM
Comments:
Actionability:
A key characteristic of an informative and actionable measure will be how much sense it makes to patients. Using episodes as the unit of observation makes intuitive sense. Patients can understand what an episode means, and they can use that information to guide decision-making. However, episodes as a whole may be less helpful to physicians attempting to improve quality and/or efficiency.
A physician must be able to trace the component(s) or element(s) of care that can be improved and that are actionable to him/her. This actionability/accountability is referenced in Principle 3 on page 31. If this explicit detail is not provided, the episode will become a black box, and a physician wishing to improve efficiency or quality per episode will not know how to focus his/her efforts.
This possible loss of detail that occurs in episode grouping is an additional limitation to the use of an episode of care as the unit of observation that should be acknowledged. It will be an even more pronounced problem when a longitudinal or “prolonged” episode is the unit of analysis.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:09:33 PM
Comments:
Attribution:
If the Committee intends measures of efficiency to be actionable for physicians as well, then using an episode-of-care framework poses an additional problem – namely, that of attributing care to one physician, when providing care is a collaborative effort. NQF recognizes the need for shared accountability, but remains silent on offering practical methods for achieving shared accountability. It will be necessary to specify how care should be attributed to different physicians involved in delivering care within an episode. We recommend that a discussion on attribution be included in this report.
------------------------------------------------------------------------------------------------------
Name: Bernard Rosof
Organization: Physician Consortium for Performance Improvement
Date Entered: 12/10/2007 3:09:33 PM
Comments:
Interrelationships:
In this document, NQF has taken an instrumental first step in laying out some of the key components of efficiency and value: processes of care, costs of care, and outcomes (we suggest in preceding comments that patient preferences are a separate component). However, it is difficult to move beyond this collection of constructs to some measure (or method for measuring) efficiency on the basis of what is provided in this document alone. It would be helpful to explicitly lay out how measures of processes of care, measures of costs of care, measures of outcomes, and measures of patient preferences for those outcomes can be combined either in a single metric, or in a reporting strategy, that would yield coherent information about “efficiency” or “value.” What is lacking in this document are the metrics (empirical definitions) of cost-of-care, quality, outcomes, value, and efficiency and we recommend that these concepts be defined in this report.
We welcome the opportunity to comment on this report.
------------------------------------------------------------------------------------------------------
Name: Deborah Fritz
Organization: GlaxoSmithKline
Date Entered: 12/10/2007 3:59:42 PM
Comments:
GlaxoSmithKline (GSK) appreciates this opportunity to comment on National Quality Forum: Measurement Framework: Evaluating Efficiency Across Episodes of Care, 2007. GSK is a world leading research-based pharmaceutical company with a mission to improve the quality of human life by enabling people to do more, feel better and live longer.
We commend the National Quality Forum’s efforts to develop a framework for efficiency measures using episodes of care. We feel that it is especially appropriate that the measures will be patient-centered and will consider quality, health outcomes, cost and processes of care.
We support the Committee’s plans to develop the next report, which will focus on approaches to implementation of the measurement framework and will further address methodological issues and data coordination between multiple settings.
------------------------------------------------------------------------------------------------------
Name: Janet Leiker (on behalf of the AAFP Commission on Quality)
Organization: American Academy of Family Physicians
Date Entered: 12/10/2007 4:55:37 PM
Comments:
There should be a tie-in to the effects of EHR's on the three domains. While there is a dearth of information on this effect in practices that use the full complement of features in the newest EHR's, this should also drive development of EHR's to support the NQF's long term goals.
------------------------------------------------------------------------------------------------------
Name: Janet Leiker (on behalf of the AAFP Commission on Quality)
Organization: American Academy of Family Physicians
Date Entered: 12/10/2007 4:55:37 PM
Comments:
The best way to capture the data is with networks of providers, laboratories, radiology facilities, ambulatory surgery facilities and hospitals. That way, the NQF's "episode of care" can actually be tracked and the cost of care, quality of care and efficiency of care might be measured with some degree of data quality.
------------------------------------------------------------------------------------------------------
Name: Dave Adler
Organization: Am. Society for Therapeutic Radiology and Oncology
Date Entered: 12/10/2007 5:20:17 PM
Comments:
ASTRO appreciates the opportunity to provide comments on the draft NQF Measurement Framework: Evaluating Efficiency Across Episodes of Care. We are grateful for the significant time, energy and thought that went into the preparation of this report by the NQF committee and staff.
ASTRO believes that the framework generally includes the essential quality and cost elements for evaluating efficiency across episodes. However, we are concerned that the report lays out a vision that seems very far from being achievable and doesn’t adequately recognize the realities of the way medical care is currently organized.
------------------------------------------------------------------------------------------------------
Name: Dave Adler
Organization: Am. Society for Therapeutic Radiology and Oncology
Date Entered: 12/10/2007 5:20:17 PM
Comments:
The report relies on a far more robust set of measures for quality and cost than currently exists--or is likely to emerge in the near-future--across the spectrum of care and disease. This problem is particularly acute in fields such as radiation oncology, where measurement has only just scratched the surface (i.e. measures are still claims-based) and the evidence base is constantly and rapidly evolving. In addition, while the framework appropriately includes a strong focus on patient outcomes, this is a highly complex area for cancer patients as effective, efficient care can still result in poor short- or long-term patient outcomes. Furthermore, when considering the application of this the framework to cancer, it’s very difficult to determine when the episode begins: Is it at initial symptom, first screening, diagnosis, or earlier? Likewise, when does the episode end? At local control? At ascertainment of metastasis or at some arbitrary point in time following treatment? These issues are particularly difficult to resolve for patients with metastasis. If patients live for different times following diagnosis, is measuring cost of care over a year an appropriate amount of time? Also, such an assessment of cancer providers would have to include the cost of treatment of recurrence as well as cost of management of side effects and other toxicities, all of which are difficult to capture.
------------------------------------------------------------------------------------------------------
Name: Dave Adler
Organization: Am. Society for Therapeutic Radiology and Oncology
Date Entered: 12/10/2007 5:20:17 PM
Comments:
Another concern is the focus on “shared accountability.” While the concept is certainly laudable, it is hard to see it being implemented fairly as physicians that deliver highly efficient care could still be penalized by their peers who don’t achieve the same standards, or worse, are negligent or motivated by financial benefit. Furthermore, the report often fails to recognize the patient’s role in shared accountability, such as following treatment regimens.
We appreciate that the report acknowledges many of the barriers that will delay and limit the framework’s application into real-life. For instance, the report notes the lack of adequate risk adjustment for case/severity mix of patients and the complexity of being able to accurately attribute care across settings. The report appropriately recognizes the difficulty in collecting necessary data to allow for a robust evaluation of performance without creating undue burden on providers. Another acknowledged area of limitation is the inability of existing episode groupers to address appropriateness of care, risk-adjust, deal with patients with multiple chronic conditions and compare organizations, which are all key components of the report’s framework. It’s unclear when, if ever, these significant barriers will be resolved.
We look forward to the committee’s next report on implementing this framework to learn about the potential to overcome these many complex obstacles.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:27:32 PM
Comments:
We applaud the deliberate choice to conceptualize efficiency from the
patient perspective.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:27:32 PM
Comments:
The episode-based approach is clearly the basis for formulating a meaningful measure of efficiency. Episode-based analyses have been advocated for over 20 years in the Family Medicine literature, and some health systems (e.g., the Netherlands) have long since implemented them
successfully. We might benefit by looking at their experiences, caveat the marked differences between their relatively integrated primary-care-based system and our fragmented non-system.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:27:32 PM
Comments:
The paper identifies the major barrier to episode-based measures almostcorrectly. The fundamental issue is not quite that commercial grouping algorithms fall short on the four points they identify, but that post-hoc algorithmic construction of episodes from claims data will never be sufficiently accurate to allow meaningful cross-system comparisons. Over 30 years after Weed made it clear that we need problem-oriented medical records, we still lack them. Perhaps the advanced medical home concept, should it take root, will help.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:27:32 PM
Comments:
At the beginning of the document there is some confusion over the definition of cost of care. The box on page 1 gives a definition that is essentially “direct costs.” On page 3, under the domain of cost and resource use, some indirect costs are included—“costs to patient and lost productivity.” The discussion of this domain in the body of the paper doesn’t mention the indirect costs. The preference would be for measures related to overall patient costs but such data are difficult to obtain. Direct costs are all we can hope for in the foreseeable future. Even then, those costs will be measured by whoever is paying the bill as the cost to them—hospital, MCO, employer, CMS, etc.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:27:32 PM
Comments:
In Domain 2, the inclusion of opportunity costs is an important step forward.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:30:53 PM
Comments:
The concept of “accountable care entities” is mentioned on p. 5 and in the body of the document on p 5. In addition, there is a discussion of “shared accountability” and a rationale for not measuring at the individual provider level on p. 16. This is an interesting concept that NQF has not used on the quality measure side. The document cites the IOM report on performance measurement as the source for the shared accountability concept. This concept rings true and recognizes the reality of how healthcare is currently provided. It should probably be applied to quality measures as well. The Episode of Care concept that presented along with a diagrammatic representation is interesting and probably useful. Phase 1 (Population at Risk) is pretty much ignored at this point because of practical considerations. It would be useful, though, to have a metric of the costs of preventive interventions taken to avoid one “episode of care” at some presumed average cost. One could then calculate an ROI for prevention once enough episodes have been accumulated in a database to have a meaningful average.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:30:53 PM
Comments:
It would appear that, although this is presented as a framework for efficiency measures, it should be a construct for quality measures as well. If efficiency is a derivative of cost and quality, shouldn’t both variables be measured over the same time periods and at the same provider levels?
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:30:53 PM
Comments:
A nod is given to the fact that patients tend to have episodes for multiple conditions going on concurrently, but this is not followed up.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:30:53 PM
Comments:
The inclusion of HRQoL as a real outcome to be measured and reported in Domain 1 is long overdue, and NQF's work on this document will be worthwhile if nothing more comes of it than that.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:30:53 PM
Comments:
In explaining Phase I of their model, the committee makes the assertion that primary prevention is the most efficient approach. While the sentiment is laudable, the assertion is not uniformly true. Primary prevention is efficient for some problems, but not for all. The efficiency of primary prevention for any given health problem is an empirical question, not an axiom.
------------------------------------------------------------------------------------------------------
Name: Elizabeth Hoy
Organization: American College of Surgeons
Date Entered: 12/10/2007 5:31:07 PM
Comments:
This document goes way beyond any commonly agreed to construct on how to measure efficiency (e.g., quality/cost) by imposing value-laden constructs on what is included in the measurement of efficiency. For example, folding the degree to which shared decision-making takes place during an episode of care into the definition of quality presupposes that all consumers want to engage in shared decision-making and that all consumers value shared decision-making to the same degree – which we know from extensive consumer research is not true. It’s not so much the denominator – though we do think there remain many challenges to creating accurate and reliable measures of cost per episode of care – as what’s getting loaded into the numerator that we have significant concerns about. In this document, quality includes not only clinical outcome measures but service quality and patient preferences. This seems to go far beyond what has commonly been thought of as efficiency into the realm of trying to measure value, which we believe is unsupportable both politically and methodologically at this time.
------------------------------------------------------------------------------------------------------
Name: Elizabeth Hoy
Organization: American College of Surgeons
Date Entered: 12/10/2007 5:31:07 PM
Comments:
We would strongly oppose any approach to efficiency that attempts to impose weights on the various measurable dimensions of quality – clinical outcomes, quality of life, and patient experience of care – and/or create a composite measure of efficiency at this time. Such an approach is insupportable with currently available measurement tools and methodologies. We also strongly oppose attempts to impose idealized norms for patient preferences onto efficiency measures to try to create measures of “value” as suggested on page 20 in the first paragraph. We are not aware of any valid, reliable, and widely accepted measures of patient preferences that could accurately address the wide range of patients. Should health policy experts infringe on patient autonomy by presupposing to reflect patient values and preferences at the time of decision making?
------------------------------------------------------------------------------------------------------
Name: Elizabeth Hoy
Organization: American College of Surgeons
Date Entered: 12/10/2007 5:31:07 PM
Comments:
Rather than overreaching, the NQF should focus on those aspects of care for which we have valid, reliable nationally accepted measurement tools. Instead of trying to include patient preferences and shared decision-making in efficiency measures – since there are no reliable measures of either and Judy Hibbard’s research (among others) has shown that patient preferences are unstable – NQF could include CAHPS® patient experience of care measures which are the NQF-endorsed national standard and extensively tested. The American College of Surgeons, as the umbrella organization for the Surgical Quality Alliance (SQA) is currently sponsoring the development of a CAHPS Surgical Questionnaire, focusing on those aspects of the quality of surgical care that are important to surgical patients and for which the patients are the most reliable source of reporting.
------------------------------------------------------------------------------------------------------
Name: Elizabeth Hoy
Organization: American College of Surgeons
Date Entered: 12/10/2007 5:31:07 PM
Comments:
The example shown in Appendix B beginning on page 43 -- arraying cost data, with clinical quality indicators, with patient experience scores at the same unit of analysis (e.g., physician, hospital, etc) – appears far more realistic and implementable than the expansive theoretical framework elaborated in the beginning of the paper. Arraying measures of quality (including patient experience of care) and cost per episode and allowing users to apply their own weights and values is much more feasible and ethically appropriate if we respect patient autonomy. Of course, this assumes transparent, reliable, valid ways to determine cost and link cost to episodes -- which we don't yet have -- and good clinical outcome measures – of which we have a few -- and good, quality-focused patient experience measures -- which are available.
------------------------------------------------------------------------------------------------------
Name: Belinda Ireland
Organization: BJC HealthCare
Date Entered: 12/10/2007 5:32:23 PM
Comments:
BJC agrees with NQF’s assessment of the limitations of existing episode treatment groupers. We would like to see more detail describing the issues, including but not limited to:
• Use of administrative data vs. the medical record
• The difficulty of acquiring and combining electronic information from many different administrative systems which were not designed for this purpose
• Difficulty in physician/provider attribution
• Potentially incomplete pharmacy data – Pharmacy data can be skewed if patients do not fill prescriptions; if patients receive prescription samples; go out of network; or use mail order
• Small n - Few episodes of care can skew cost efficiency ratios
• Pooling data to increase power can then make it difficult to evaluate the separate impact of various healthcare delivery providers and systems
• Reconciling the differences between costs and charges across multiple systems: The definition of cost of care on page 1 may be intentionally broad at this point, but specific criteria for what is included are vital to any evaluation which includes an economic analysis.
• Among individuals with multiple medical conditions, allocating medical services to a particular episode of care for a particular disease, when the service might have been relevant to several conditions
------------------------------------------------------------------------------------------------------
Name: Belinda Ireland
Organization: BJC HealthCare
Date Entered: 12/10/2007 5:32:23 PM
Comments:
While BJC ultimately agrees with NQF’s definition of ‘cost-of-care’, we suggest NQF consider using clearer references. “Cost” references in the document appear to be defined using the payor’s perspective (unit price multiplied by utilization), but by using the term ‘resources used’ within the definition, it can also be considered by the reader to be the provider’s actual costs. NQF may want to consider using the terminology from a patient-centered approach, including out-of-pocket costs. Using NQF’s definition of “cost of care”, we strongly recommend that NQF include appropriate adjustments to the rates for teaching hospitals that account for medical education and bio-medical research, as well as adjusting for disproportionate-share providers that care for the ‘sickest of the sick’, provide safety net obligations, and provide regional stand-by trauma services. Currently the commercial population funds these additional services provided by an academic medical center. Any measurement needs to account for the pubic financing of health care system in the U.S. today.
------------------------------------------------------------------------------------------------------
Name: Belinda Ireland
Organization: BJC HealthCare
Date Entered: 12/10/2007 5:32:23 PM
Comments:
BJC supports the National Quality Forum (NQF) development of efficiency measurement across episodes of care, and applauds NQF for including quality of care within the definition of efficiency. We also support the inclusion of risk-adjustment methodologies for the ‘cost of care’ component as well as for clinical quality outcomes. We agree that any efficiency measurement program must be accurate and understandable to consumers to allow them to easily make informed judgments. We therefore recommend that NQF adopt provisions that ensure the full transparency of how programs are designed, how providers are ranked, and include guidelines on how information is presented/displayed to enable consumers to make more educated decisions.
------------------------------------------------------------------------------------------------------
Name: Belinda Ireland
Organization: BJC HealthCare
Date Entered: 12/10/2007 5:32:23 PM
Comments:
This document would be strengthened by a more complete description of the cost analysis method the authors propose to use, and a more thorough analysis of alternative methodologies (with reasons for not recommending) including but not limited to:
Cost-of-illness analysis: a determination of the economic impact of an illness or condition (typically on a given population, region, or country) e.g., of smoking, arthritis or bedsores, including associated treatment costs
Cost-minimization analysis: a determination of the least costly among alternative interventions that are assumed to produce equivalent outcomes
Cost-effectiveness analysis (CEA): a comparison of costs in monetary units with outcomes in quantitative non-monetary units, e.g., reduced mortality or morbidity
Cost-utility analysis (CUA): a form of cost-effectiveness analysis that compares costs in monetary units with outcomes in terms of their utility, usually to the patient, measured, e.g., in QALYs
Cost-consequence analysis: a form of cost-effectiveness analysis that presents costs and outcomes in discrete categories, without aggregating or weighting them
Cost-benefit analysis (CBA): compares costs and benefits, both of which are quantified in common monetary units
If the authors do plan to endorse multiple methods of cost analysis, they need to specify conditions for use of those approaches.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:33:41 PM
Comments:
The scope of the document is a bit narrow. Though presented as a framework for measuring efficiency for chronic conditions, the examples and their conceptualization focus upon acute and subacute conditions (e.g., AMI, low back pain) and time horizons that are short by primary care standards. Actual chronic conditions, e.g., hypertension, are not addressed.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:33:41 PM
Comments:
The 9 principles are conceptually sound, and particular attention in implementing them should be paid to #8: unintended consequences. It is so important, and it is so difficult to discontinue a measure that is doing things it wasn't intended to do once it's become entrenched, that it wise to set specific expiration dates on measures to force their timely reconsideration. Given the complexity of what is proposed to be measured, unintended consequences are not a risk but a certainty. We should expect the first round of measures to have to be revised substantially based on experience.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:33:41 PM
Comments:
The description of Phase 2 in the AMI example is a bit confusing. It starts out saying that Phase 2 begins at the onset of symptoms and is meant to capture things like emergency response. Later it talks about measuring from arrival in the ED. Any metric of cost or quality at the physician level should start at presentation to the ED.
------------------------------------------------------------------------------------------------------
Name: Joseph Drozda
Organization: American College of Cardiology
Date Entered: 12/10/2007 5:33:41 PM
Comments:
Phase 4 (Secondary Prevention) in the AMI section extends until 1 year after hospital discharge meaning that it varies with hospital length of stay. It is not certain whether or not this is intended. For consistency, the timeframe should be calculated with the same reference point as the other phases measuring either from onset of symptoms or presentation to the ED.
------------------------------------------------------------------------------------------------------
Name: Rachel Groman
Organization: American Association of Neurological Surgeons
Date Entered: 12/10/2007 5:43:38 PM
Comments:
Domains of Assessment:
The AANS believe that the framework generally includes the appropriate quality and cost elements for evaluating efficiency across episodes. We are especially pleased that outcomes are recognized as an essential