Quality and Performance Evaluation

Quality Measurement and the Medicare Star-Rating System

The SNP Alliance believes that any measurement system for Medicare-Medicaid complex care beneficiaries and the health plans that serve them should be based on the following principles:

  1. Meaningful & Relevant – Measures and measure specifications reflect the dual and complex care population’s needs and characteristics.
  2. Equal Access – Dually eligible, high social risk, and complex care beneficiaries have equal access to participate in the quality measurement process as individuals who do not have these complexity issues.
  3. SDOH & Risk Factors Recognized – Social determinants of health risk factors are fully considered when analyzing outcomes.
  4. Standards for Developers – Minimum standards are set by CMS for measure developers and stewards for testing and adjusting measures for social risk factors.
  5. Balance – There is a balance of process and outcome measures, but any measure used is objective, valid, and reliable, where results can be replicated and can be used to enhance quality improvement.
  6. Peer Grouping – Plans serving similar populations are compared with each other, with cut points, ratings, and bonus payments adjusted appropriately.
  7. Attribution – Actions by the entity responsible for the measurement result are attributed to that entity, but characteristics impacting the result independent of action of the entity are discerned and considered.
  8. Utility of Reporting – Quality reporting displays results by plan peer group cohort (e.g. all plans divided into three or more groups based on percentage of their enrolled population that is low-income, disabled, or dually eligible) so that consumers and plans have the information they need to make relevant comparisons.
  9. Alignment – There is attention to measure alignment–so that plans serving beneficiaries with specific conditions and care management needs can maintain a consistent and effective approach–across settings, over time, and across states.
  10. Do No Harm – Providers and plans disproportionately serving high-risk, high-need dually eligible and complex care beneficiaries are not harmed by the quality measurement and reporting system because of the underlying characteristics of the people they serve.
  11. Fairness/Equity – Fairness and attention to measurement burden is demonstrated.

In working with special needs health plans and analyzing the current quality measurement system under Medicare, we identified three priority issues and are working to educate and advocate for ways to address and remedy these issues:


– Measures are “one size fits all”
– These quality measures have not been adequately tested in the diverse, complex, and high SDOH groups and there is evidence they are not well-matched to these groups. The resulting scores (such as the PCS and MCS generated from the Health Outcomes Survey) may not provide useful information.

– Limited Methods & Adjustments – Methods to collect data and conduct adjustments have limitations which are more likely to negatively impact high risk complex special needs populations.

 – Scoring & Reporting Limitations – Because all MA plans are combined in a single group, plans with high enrollment of high risk and complex individuals are treated the same as plans with very few complex individuals. Given that complexity and SDOH risk factors have been shown by multiple independent expert and research panels and committees to be one of the largest predictors of poor health outcomes, independent of provider or plan action—this is a serious limitation in the scoring and reporting under Medicare Stars.

Another substantial limitation is that reporting is at the organizational, not geographic level—so that it is difficult to discern if the results reported are in fact accurate for the location where a person would receive care.  When non-contiguous regions and plans of different size enrollment are combined, the findings have limited utility for the consumer, provider, or plan—as it does not provide information needed to guide plan selection or quality improvement.

Social Determinants of Health & Condition Complexity

The complexity of conditions and social risk factors are part of what make special needs populations “special.” Complexities and risk factors have long been recognized by special needs health plans and the SNP Alliance as fundamentally important to care and support. Many SNPs have been working within and with their communities (prior to becoming a special needs health plan and after doing so)— focusing on low-income, elderly, and at-risk individuals.

The Special Needs Plan Alliance and its members have emphasized the importance of recognizing social determinant of health risk factors and continue to advocate for better policy, payment, and measurement to focus on SDOH, address underlying issues, and improve understanding of effective approaches.

Over the last five years, the Special Needs Plan Alliance and its members have emphasized the importance of recognizing social determinant of health risk factors. We have listened and learned, provided education,  participated in policy and research analysis, and conducted surveys, facilitated discussions, served on expert panels, and prepared reports to disseminate and share what we’ve found.

Plans have described in facilitated discussions, key informant interviews, and surveys how their health risk assessment, outreach, care models, tailored care management, calling centers, database systems, risk stratification models/processes, and quality measurement and improvement efforts have taken both SDOH and care complexity factors into account.

The SNP Alliance has surveyed these plans on SDOH risk identification, data sources, methods, aggregation, and challenges. We have collected information on health plan/provider/community partnerships. We have written Issue Briefs, Reports, and comment letters to ASPE, GAO, NASEM, and CMS. We have conducted key informant interviews on SDOH challenges and heard about successful case stories.

Health Outcomes Survey

The Medicare Health Outcomes Survey (HOS) is a self-report survey designed for the 65+ population to capture information on their health status, chronic conditions, functional abilities, and other characteristics.

The HOS is used by researchers to determine patterns, trends, and compare population characteristics and by data analysts and regulatory agencies. The HOS was created from a survey instrument developed in the late 1980’s and tested using a veteran population of primarily older white males. Items from this tool, the Veterans RAND 12-Item Health Survey, are used today in Medicare quality measurement.

The HOS is currently used by CMS as the data source for generating several measures in the Medicare quality measurement and reporting system for Medicare Advantage health plans, based on a random sample of plan members who voluntarily agree to participate. Measures collect data on specific questions and then ask that same person two years later the same questions. Responses from the later timeframe are compared to the earlier timeframe to determine if the person has improved, declined, or stayed the same.

HOS measures and the format of questions in HOS (e.g., wording, sequence, length, response scale) have not been validated in diverse populations, for example non-English/non-Spanish speaking individuals, those with low literacy, ethnically diverse, those with significant chronicity or frailty impacting endurance, those with serious and persistent mental health or cognitive issues, people with progressive physical or intellectual disabilities, and those with housing insecurity or other social determinant of health risk factors which complicate administration (e.g., no address, no phone).

The SNP Alliance strongly supports individuals self-reporting on their experiences of care, outcomes, and how well plans and providers have addressed their needs. This is critical information to guide evaluation and improvement.

Unfortunately, the HOS instrument does not provide the necessary information to do this–in its present form and given how measures from the HOS data have been crafted, the disconnect with the characteristics of the special populations that SNPs serve, and the long time delay from first data collection to being able to access the measurement results (3+ years). Because of these issues, the SNP Alliance supports refinement and retesting of HOS with changes to the instrument, case-mix and predictive model, methods of surveying, and analysis of results.

HOS Sample Measure Items

In general, would you say your health is: Excellent, Very Good, Good, Fair, Poor

Does your health now limit you in these activities? If so, how much? (Scale: limited a lot, limited a little, not limited at all) :

  • Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf?
  • Climbing several flights of stairs?

During the past 4 weeks, have you had any of the following problems with your work or other regular daily activities as a result of your physical health? (Scale: none of the time, a little of the time, some of the time, most of the time, all of the time)

  • Accomplished less than you would like.
  • Were limited in the kind of work or other activities.

During the past 4 weeks, how much did pain interfere with your normal work (including work outside the home and housework)? (Scale: not at all, a little bit, moderately, quite a bit, extremely)

During the past 4 weeks, have you had any of the following problems with your work or other regular daily activities as a result of any emotional problems (such as feeling depressed or anxious)? (Scale: none of the time, a little of the time, some of the time, most of the time, all of the time)

  • Accomplished less than you would like
  • Didn’t do work or other activities as carefully as usual

How much of the time during the past 4 weeks: (Scale: all of the time, most of the time, a good bit of the time, some of the time, a little of the time, none of the time)

  • Have you felt calm and peaceful?
  • Did you have a lot of energy?
  • Have you felt downhearted and blue?

During the past 4 weeks, how much of the time has your physical health or emotional problems interfered with your social activities (like visiting friends, relatives, etc.)? (Scale: not at all, a little bit, moderately, quite a bit, extremely)

Model of Care

Special needs health plans have several requirements that general Medicare Advantage health plans do not have to meet.

One of these is that SNPS must submit a written Model of Care (MOC) to CMS for external independent review by the National Committee for Quality Assurance (NCQA) based on specific scoring criteria.  The plan’s MOC must address four domains (Population, Care Coordination, Provider Network, and Quality). Each domain has defined elements and factors. These factors have required components which are scored. The factor scores are combined to create an overall score for the Model of Care as a whole. Based on the overall score, the health plan is assigned a 1, 2, or 3-year cycle for the next review. The plan follows this Model of Care in their care management, coordination with providers, and quality improvement actions.

SNP Alliance Recommendations for Model of Care: Based on plan surveys, discussions and analyses, the SNP Alliance has offered several recommendations for modifications/edits in the Model of Care guidelines and processes—providing these recommendations to CMS and to NCQA:

  • Eliminate areas of duplication/redundancy and simplify the elements and factors listed
  • Substantially modify the Provider Network Training requirement under Model of Care to re-focus this toward ensuring providers can effectively and easily access the enhanced care coordination services and other beneficiary supports offered by SNPs.
  • Streamline annual submission to be required of C-SNPs on Model of Care by asking plans for redline/substantive changes only (CMS)
  • Improve training of Medicare Advantage auditors around Model of Care so that the audit review is in sync with the NCQA reviewer approach and uses consistent guidelines
  • Address issues with off-cycle review “window closed” timeframe which restricts necessary care management, operational improvement, and other infrastructure or process changes that need to be made by the health plan