NQF Member comments due June 9, 2008 by 6:00 PM ET
Public comments due May 29, 2008 by 6:00 PM ET
Comments are listed in the order they were submitted.
General Comments
__________________________________________________________
Name: Sean Currigan
Organization: ACOG
Date Entered: 5/27/2008 6:22:02 PM
Comments:
1) How will NQF enforce the disallowing of "modification of NQF endorsed(TM) measures by subsequent users?
------------------------------------------------------------------------------------------------------
Name: Sean Currigan
Organization: ACOG
Date Entered: 5/27/2008 6:22:02 PM
Comments:
Does NQF endorsement of a proprietary measure based on models/equations from a database/registry imply endorsement of the database itself? Otherwise, shouldn't the public domain transparency of the measure include the clinical and demographic variables AND the risk adjustment algorithms such that another entity could 1) collect the appropriate data elements and variables and 2) use the risk adjustment algorithms to produce the same resulting measure. It is understood that the associated software that does the calculation or sets up the database is fully proprietary. If NQF is going to endorse proprietary measures, additional data/materials demonstrating why/how the particular database/registry is better than another should be reviewed by the steering committee. Is NQF expecting to review databases (auditability, integrity of the data, etc).
------------------------------------------------------------------------------------------------------
Name: Sean Currigan
Organization: ACOG
Date Entered: 5/27/2008 6:22:02 PM
Comments:
The pricing structure should be disclosed.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:25:10 AM
Comments:
The NQF 2003 policy required that all “entities have equal access to endorsed measures’ specifications”. This policy required that any NQF endorsed measure be available to all entities (e.g., providers, payers other vendors, etc.). The new principles allow the developer of the measure to protect its intellectual property before providing access by other entities to the measure. However, the requirement that “entities have equal access to endorsed measures’ specifications” is not listed as one of the NQF core principles. NQF should clarify whether this requirement is one of its core principles.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:25:10 AM
Comments:
Limiting full disclosure of the full specifications of measures to NQF review panels does not achieve transparency for the entities that will be evaluated by the application of the measures. Indeed, from the perspective of the end user, the measure remains essentially a “black box”. This will severely limit the acceptance of the measure and the extent to which the measure can be used to improve quality of care. There is a middle ground between the proposed “NQF review panels only” proposal and full public disclosure. There should be full disclosure of the specifications of the NQF endorsed measures to any entity that agrees to legal protection and enforcement of the developer’s intellectual property rights (i.e., core principle 3). Such open access would better achieve the level transparency that the health care community expects.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:27:43 AM
Comments:
The NQF 2003 policy required that all “entities have equal access to endorsed measures’ specifications”. This policy required that any NQF endorsed measure be available to all entities (e.g., providers, payers other vendors, etc.). The new principles allow the developer of the measure to protect its intellectual property before providing access by other entities to the measure. However, the requirement that “entities have equal access to endorsed measures’ specifications” is not listed as one of the NQF core principles. NQF should clarify whether this requirement is one of its core principles.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:27:43 AM
Comments:
Limiting full disclosure of the full specifications of measures to NQF review panels does not achieve transparency for the entities that will be evaluated by the application of the measures. Indeed, from the perspective of the end user, the measure remains essentially a “black box”. This will severely limit the acceptance of the measure and the extent to which the measure can be used to improve quality of care. There is a middle ground between the proposed “NQF review panels only” proposal and full public disclosure. There should be full disclosure of the specifications of the NQF endorsed measures to any entity that agrees to legal protection and enforcement of the developer’s intellectual property rights (i.e., core principle 3). Such open access would better achieve the level transparency that the health care community expects.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:29:25 AM
Comments:
The NQF 2003 policy required that all “entities have equal access to endorsed measures’ specifications”. This policy required that any NQF endorsed measure be available to all entities (e.g., providers, payers other vendors, etc.). The new principles allow the developer of the measure to protect its intellectual property before providing access by other entities to the measure. However, the requirement that “entities have equal access to endorsed measures’ specifications” is not listed as one of the NQF core principles. NQF should clarify whether this requirement is one of its core principles.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:29:25 AM
Comments:
Limiting full disclosure of the full specifications of measures to NQF review panels does not achieve transparency for the entities that will be evaluated by the application of the measures. Indeed, from the perspective of the end user, the measure remains essentially a “black box”. This will severely limit the acceptance of the measure and the extent to which the measure can be used to improve quality of care. There is a middle ground between the proposed “NQF review panels only” proposal and full public disclosure. There should be full disclosure of the specifications of the NQF endorsed measures to any entity that agrees to legal protection and enforcement of the developer’s intellectual property rights (i.e., core principle 3). Such open access would better achieve the level transparency that the health care community expects.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:43:24 AM
Comments:
“Under all options, publicly disclosed information would include a full measure description, including all clinical and demographic variables, and an individual patient calculator, as necessary.”
A patient calculator should be required to be available for all NQF endorsed measures. The calculator should be available on the internet and accessible for no fee to any entity that agrees to accept the legal protection and enforcement of the developer’s intellectual property rights. While a calculator does not provide full disclosure of the measure specifications, it does provide a practical means for entities being evaluated by the measure to better understand the measure without the need to interpret the actual specifications. The calculator makes transparency a practical reality.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:43:24 AM
Comments:
“The new policy will take effect for all previously endorsed consensus standards when reconsidered during the routine measure maintenance process.”
In addition to the requirement that all previously endorsed measures adhere to the revised intellectual property agreement, any measure that was not submitted to NQF because of the restrictions in the previous intellectual property agreement (e.g., categorical measures for length of stay and readmissions) should have the opportunity to be submitted under the revised intellectual property agreement.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems, Inc.
Date Entered: 5/29/2008 10:43:24 AM
Comments:
“Although the current policy does encourage competition among vendors to process provider data - and may thus lead to moderation of the fees charged by stewards to generate a profit above and beyond what they have paid out for licensing fees - it does not address the underlying issue of the potential monopoly power of the steward where actual competition may not truly exist.”
Although this policy specifically refers to the processing of provider data, the general policy of encouraging competition and avoiding a monopoly being given to any one vendor is an important issue. Relative to any specific measure (e.g., readmissions), NQF should endorse all such measures that meet its standards, thereby creating competition among vendors. To the extent possible, the endorsement of a specific measure from only a single vendor should be avoided.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems
Date Entered: 5/29/2008 1:22:33 PM
Comments:
“It is given that the steward’s pricing structure would be subject to review and approval by the NQF expert review panel.”
Clarification should be provided that the NQF panel’s “review and approval” authority relates to NQF endorsement only (i.e., to ensure that an endorsed measure(s) is not generally cost prohibitive for users) rather than authority to dictate the specific price(s) at which the measure(s) must be licensed. If NQF implements such a policy, NQF should focus its review on an agreed-upon cross-section of examples involving typical users (e.g., 250 bed hospital licensing the measure for use on a PC; a payer licensing the measure for use on a mainframe to process records for a million covered lives, etc.) because pricing structures often involve very complex issues arising from different computer platforms, volume discounts, group purchasing arrangements, direct/indirect distribution arrangements, etc. This approach allows NQF to exercise its oversight responsibilities by focusing on common situations that are both representative and manageable, while avoiding involvement in the complexities of pricing variations that occur in the ordinary course of business. Any time pricing constrains are imposed on the market place there are potential antitrust implications. Antitrust law is very complex and NQF should consider seeking legal consultation before finalizing its policies relating to pricing.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems
Date Entered: 5/29/2008 1:22:33 PM
Comments:
“At issue is whether there should also be a requirement that the steward’s pricing structure be publicly disclosed. Member input regarding this issue is being sought.”
NQF should be able to perform its oversight responsibilities without public disclosure. Interested parties can always voice their opinions concerning any vendor’s pricing structure to NQF. Further, public disclosure could raise issues relating to potential competitive pricing collusion increasing the likelihood that there could be antitrust ramifications of the NQF pricing policies. If such disclosure is determined to be necessary and legal, it should be limited to the cross-section of examples involving typical users as described in a prior comment.
------------------------------------------------------------------------------------------------------
Name: Richard Averill
Organization: 3M Health Information Systems
Date Entered: 5/29/2008 1:22:33 PM
Comments:
“Should a measure steward deviate from a pricing structure agreed upon during the endorsement process, NQF would reserve the right to rescind endorsement.”
NQF will always have the right to withhold or rescind its endorsement. Pricing structures are temporal by nature, and vendors need to reserve the right to implement reasonable price increases which reflect their increased cost of doing business over time. The word “deviate” could be replaced with the phrase “increase its previously reviewed prices by more than five percent (5%) in any one year period”. This would give vendors a level of certainty concerning pricing flexibility while establishing a clear benchmark beyond which NQF reserves its right to rescind its endorsement. As noted in a previous comment, the imposition of any constraints on the market place relating to price is subject to both legal and fairness standards which can have antitrust implications and therefore, must be carefully evaluated before being implemented.
------------------------------------------------------------------------------------------------------
Name: Edward Garcia
Organization: CMS
Date Entered: 6/3/2008 4:29:32 PM
Comments:
The Centers for Medicare and Medicaid Services (CMS) believes that the National Quality Forum (NQF) would be best served to limit its consideration of modifications to the existing NQF Intellectual Property Policy to the issue of full public disclosure of database elements versus full disclosure of such elements to the NQF review panel only. As NQF has noted, entering into the field of defining the reasonableness of database element pricing is beyond NQF’s realm of authority and as such should not be a consideration in NQF's endorsement process. Rather, CMS suggests that NQF first allow evidence to be brought forth on the potential prohibitive nature of charges to database elements on measure use before further consideration of permitting cost assessment by NQF review panels as part of the consensus development process.
------------------------------------------------------------------------------------------------------
Name: Janet Leiker (on behalf of the Commission on Quality)
Organization: American Academy of Family Physicians
Date Entered: 6/3/2008 5:14:42 PM
Comments:
The American Academy of Family Physicians has reviewed the draft report for the Intellectual Property Agreement Task Force and does not have any comments at this time.
------------------------------------------------------------------------------------------------------
Name: John Kusske
Organization: AANS
Date Entered: 6/4/2008 3:48:05 PM
Comments:
The AANS recognizes that a knowledge and understanding of risk adjustment models that may accompany a standard are beyond the grasp of many providers. We also recognize that without sophisticated database and statistical support, many of these standards are not meaningful. Furthermore, transparency is critical to ensure the models are accurate and that they result in reliable information.
Despite our understanding of NQF's reasons for updating its policy on this subject, the AANS would like to express a couple of concerns. One area of particular concern to the AANS is the possiblility that vendors who maintain certain databases and who provide statistical support might consider selling their data, collected for NQF purposes, to any willing buyer in the market. Although this issue is addressed in the final recommendations of the Task Force, we would like to reinforce the fact that such materials should never be available for use in the commercial market.
We are also concerned by the suggestion that all the information that is collected should be made public. This is a policy we do not agree with unless the validity of the measure (including all relevant risk and case mix adjustments) has been assured.
The AANS appreciates the opportunity to share its comments.
------------------------------------------------------------------------------------------------------
Name: Cary Sennett
Organization: American Board of Internal Medicine
Date Entered: 6/4/2008 5:59:40 PM
Comments:
The report is a thoughtful analysis of a complex set of issues; in general, we support the Task Force’s recommendations, though appreciate that there are number of areas where further conversation may be warranted.
------------------------------------------------------------------------------------------------------
Name: Cary Sennett
Organization: American Board of Internal Medicine
Date Entered: 6/4/2008 5:59:40 PM
Comments:
There is one issue that does not receive much attention in the report, and where the Task Force may inadvertently have created the possibility for unintended consequence. On page 13, the Task Force recommends “that data stewards not be permitted to compel providers to allow the resale or use of their data for purposes other than the required public disclosure;” that may be a stronger proscription than the Task Force intends. That language could preclude the use of such data for research; both research directed towards improving measures and research directed toward improving health care. While it may be possible to configure a mechanism through which providers of data are able, voluntarily, to permit data they provide to be used for such research, that mechanism may be administratively unwieldy, and opt-in (or out) provisions could lead to reductions in the value of such data for research. We believe that the public has a legitimate interest in the use of such data for research, and that that use does not create inappropriate rewards for data stewards. We think it reasonable that de-identified data (both with respect to patient and provider) be routinely used in such a way. While we are anxious about language that suggests that providers would be “compelled” to allow such data to be used for research, we believe that the Task Force can find language that would allow for the routine use of data—appropriately de-identified—for research that serves the public’s interest.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 6/4/2008 6:01:24 PM
Comments:
Thank you for the opportunity to review this document. It is a timely reconsideration of an important issue and, in general, appears to be an improvement over the current policy. Of course, the "devil is in the details." We are in general agreement with the recommendation of Policy Option 2. However, it would seem that the expert advisory panel would require special expertise to review complex risk adjustment systems. Does the NQF plan to convene a special panel for this purpose? Ensure that each project technical adviosory panel includes this expertise? Additionally, the report is a bit inconsistent in its discussion of pricing structure. On the one hand, the report states (page 12) that "establishing guidelines for what constitutes a 'reasonable charge' is beyond NQF's competence and authority," while later on the same and next page, it states that the "Task force agreed that charges and pricing structures, including any revisions must be subject to evaluation by NQF's review panel" and on page 14 states "it is a given tht the steward's pricing strucutre should be subject to review and appproval by the NQF expert review panel." Absent guidelines, how will the NQF fulfill this expectation? This issue seems to require more consideration and, hopefully, the NQF will receive thoughtful comments to guide that consideration.
------------------------------------------------------------------------------------------------------
Name: Ellen Schwalenstocker
Organization: NACHRI
Date Entered: 6/4/2008 6:01:24 PM
Comments:
On its face, it would seem fair that the pricing structure be publicly disclosed when member organizations are voting on specific measures. Cost would be an important consideration for feasibility
------------------------------------------------------------------------------------------------------
Name: Barbara Corn
Organization: NAHQ
Date Entered: 6/6/2008 11:25:22 AM
Comments:
In general NAHQ prefers Option 2A which would require full public access to the measures if used and or required for public reporting. In your summary the task force is asking for input on the stewards pricing structure and if it should be publicly disclosed? We would recommend that the pricing structure be disclosed and look at ways you can make the fees affordable for example for rural hospitals- if they are a critical access hospital can we have band pricing for this?
------------------------------------------------------------------------------------------------------
Name: Rita Munley Gallagher, PhD, RN
Organization: American Nurses Association
Date Entered: 6/6/2008 2:10:34 PM
Comments:
The American Nurses Association (ANA) concurs that the current NQF Intellectual Property policy fails to provide clear guidelines to those who develop and maintain measures and cannot accommodate the increasingly complex measures and risk-adjustment models that have emerged as the field has evolved.
ANA believes disclosure of a pricing schedule to NQF should be for information purposes only; NQF should not take any action that would affect a party's pricing decisions, nor should NQF reject an indicator based upon a submitter's pricing schedule.
Given the options presented, ANA is supportive of:
OPTION 2: FULL DISCLOSURE ONLY TO NQF (Beta values, standard errors, episode groupers, and algorithms will be disclosed to NQF’s review panel only)
OPTION 2C: No restriction on charges for data processing to produce public reports.
OPTION 2B.2: PRICING DISCLOSURE TO NQF ONLY (Pricing framework will be disclosed to NQF’s review panel only.
------------------------------------------------------------------------------------------------------
Name: David Hopkins
Organization: Pacific Business Group on Health
Date Entered: 6/6/2008 8:52:27 PM
Comments:
The purchaser community commends the Intellectual Property Task Force for revising this policy. We need to have the measures that were developed by proprietary vendors and are used by health plans and others subjected to the rigorous NQF endorsement process. Yet, few of these vendors have been willing to submit their measures under the prior policy.
PBGH believes that the Task Force has struck the right balance between the need for full disclosure of measure specifications and acknowledging that measure developers may need to secure revenue to recapture their investment and/or maintain their measures. Therefore, we strongly support the Task Force recommendations.
In addition, we would suggest that NQF seriously consider applying the principle that measure developers should disclose the cost of implementing their measures -- whether that be through payment to a vendor or effort and resources needed to collect the measure by the delivery system -- to all measures that are submitted for endorsement.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement Subcommittee, PMSC)
Organization: American College of Physicians
Date Entered: 6/9/2008 2:03:51 PM
Comments:
Regarding Principle 1 – The PMSC suggests that there should be a way to allow for verification of the endorsed measure specifications by an independent auditor with appropriate expertise.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement Subcommittee, PMSC)
Organization: American College of Physicians
Date Entered: 6/9/2008 2:03:51 PM
Comments:
Since there may be situations where performance measures are required for P4P (essentially mandatory), the developer would hold monopoly power. This situation could be viewed as analogous to a public utility model (water, gas, power) where monopoly power is allowed but there is tight regulation and rate setting. The way to achieve this is in Option 2B.1: public disclosure of pricing framework.
------------------------------------------------------------------------------------------------------
Name: Lea Anne Gardner RN, PhD (on behalf of the Performance Measurement Subcommittee, PMSC)
Organization: American College of Physicians
Date Entered: 6/9/2008 2:03:51 PM
Comments:
The PMSC agrees that the expert panel that reviews the undisclosed models should have input on the pricing, particularly revisions.
------------------------------------------------------------------------------------------------------
Name: Rebecca Zimmermann
Organization: AHIP
Date Entered: 6/9/2008 5:54:44 PM
Comments:
AHIP recommends that NQF adopt Option 2C. Option 2B permits “’reasonable’ charges, but subject to review by NQF’s expert panel.” Option 2C has “[n]o restriction on charges for data processing to produce public reports.” (Draft p. 10).
The Task Force’s recommendation to support Option 2B raises two issues. First, the setting or approval of downstream fees by an entity such as NQF could raise antitrust issues.
• An entity such as NQF may appropriately set its own fees (e.g., for endorsement), but generally may not set fees for transactions to which it is not a party (e.g., transactions between developers of NQF-endorsed measures and purchasers of data).
• Price is the most sensitive area of antitrust law and “price setting” raises antitrust issues.
• Such antitrust issues exist whether NQF actually sets prices or only approves them.
Second, the setting or approval of downstream fees by an entity such as NQF is likely to create harm by distorting appropriate market outcomes. While the desire to protect against “unreasonable” pricing has face value appeal, there is no reason to believe that NQF’s experts will reach better decisions about prices than those set by the market. Instead, such interference with market pricing can lead to harmful effects. Here, the likely harmful effect could be that fewer measurement developers would bring their measures to NQF for endorsement, depriving consumers of the valuable information that such an endorsement process provides.
------------------------------------------------------------------------------------------------------
Name: Rebecca Zimmermann
Organization: AHIP
Date Entered: 6/9/2008 5:54:44 PM
Comments:
If developers “have to” bring their measures to NQF (e.g., if NQF endorsement is necessary to be used by certain purchasers), then the harmful effect of NQF price supervision will have a negative effect on entities that develop measures. AHIP recommends against unreasonable pricing by vendors who control various methodologies in a limited market; however we do not recommend that NQF determine such fees.
Lastly, AHIP recommends that any measure, including measure specifications, endorsed by NQF be transparent to all users. Such transparency is essential for performance measurement entities to produce comparable data in a given market.
------------------------------------------------------------------------------------------------------
Name: Rebecca Zimmermann
Organization: AHIP
Date Entered: 6/9/2008 5:54:44 PM
Comments:
Specific Comments Related to the Draft
AHIP recommends that the following revisions be made to the draft document:
After considering all of the options presented above, the Task Force recommends that there be (1) disclosure of database elements to NQF’s review panel only, with (2) no restriction on charges for data processing to produce public reports. (Option 2C). The combination of this option with the five Core Intellectual Property Principles set forth above would become NQF’s new Intellectual Property Policy. As noted above, this option would require public disclosure of measure descriptions and all clinical and demographic variables, but would limit disclosure of database elements (i.e., beta values, standard errors, episode groupers, and algorithms) to NQF’s review panel during the endorsement and maintenance review processes. This option would protect developers’ intellectual property rights while providing for a thorough assessment of candidate standards and verification of the integrity of the data calculation process for measure users.
The selection of Option 2C reflects NQF’s understanding that defining or approving “reasonable” fees for measure use and data processing is not an appropriate role for NQF. Such fee setting activities by NQF would raise legal issues and also would have the counterproductive effect of harming consumers of health care services by making developers of beneficial measures less willing to seek NQF endorsement.
------------------------------------------------------------------------------------------------------
Name: Rebecca Zimmermann
Organization: AHIP
Date Entered: 6/9/2008 5:54:44 PM
Comments:
In addition, while the endorsements offered by NQF reflect a significant indication of the validity of measures, the endorsements are not designed to, and do not, make a particular measure the exclusive approach to evaluating particular quality attributes. Thus, Option 2C reflects the optimal approach of adding value through NQF evaluation and, as appropriate, endorsement of measures, while letting such measures compete, and be valued, based on their validity, usefulness, and a range of other factors, including, but not limited to, NQF endorsement.
------------------------------------------------------------------------------------------------------