Executive Summary
Community-level interventions have the advantage of facilitating system-wide delivery and payment reform because they shift the focus of reform efforts away from particular institutions to the broader community where patients receive their care and manage their conditions, thus facilitating greater care coordination across settings. Community-based initiatives, however, have created a new set of methodological challenges around how best to determine whether and to what extent these initiatives have had an impact on cost and quality and, where they have, determining what worked and what could have worked better.
The challenge of evaluation derives from the nature of community-based innovation. Initiative design and primary objectives may vary in different communities, given different priorities and the types of available resources; typically lack randomization and control in an environment where patients may elect to participate in innovative programs non-randomly based upon characteristics that affect measured health outcomes; proceed in a dynamic environment, where many things may be changing simultaneously; and they may have differential, and sometimes completely opposite effects, on subsets of the population, in a setting where there is very limited opportunity to evaluate the impact on some of those subsets. The risk in that setting is that apparent improvements in an outcome, such as cost, in one population (e.g., Medicare enrollees) may be the consequence of cost-shifting to an unobserved population (e.g., commercial enrollees) rather than the result of genuine improvements in the efficiency with which care is delivered.
To begin to address these challenges, the Engelberg Center for Health Care Reform at Brookings has worked closely with three communities, the Primary Care Information Project (PCIP) in New York City, Vermont’s Blueprint for Health (Blueprint) and the Wisconsin Health Information Organization (WHIO) , and analysts from the Dartmouth Institute, New York University, Onpoint Health Data and OptumInsight (formerly Ingenix) to develop a multi-payer template for evaluating their ongoing and future community-based health care reform initiatives. The objective was to better equip these communities to conduct rigorous evaluation of the impact of their interventions on the cost, quality, and utilization of health care and to provide a template that would enable other communities implementing accountability-based multi-payer payment and delivery system reform initiatives to understand how they might do so.
