Can outcomes-based funding support evidence-based college success programs?

Policymakers and education leaders have multiple evidence-based options for substantially improving college success for low-income and first-generation college students. Prominent examples include the City University of New York’s (CUNY’s) Accelerated Study in Associate Programs (ASAP) and Bottom Line.

Despite their track record of success, there are several challenges to bringing these programs to scale. Some evidence-based college success programs are quite expensive to operate, often costing thousands or even tens of thousands of dollars per student served. Many programs are built around a model of in-person support between students and advisers, making it difficult to expand their services beyond the current geographic markets in which they operate. 

These challenges to sustainability and scale notwithstanding, there is strong interest from a broad range of stakeholders in scaling evidence-based interventions to serve more students. How, then, can we develop innovative financial models to sustainably fund and scale high-impact but resource-intensive college success programs? This question is particularly relevant now, given several recent studies that find that low-cost informational and nudge interventions that had impacts at the local level did not sustain effects when scaled statewide or nationally.

To help investigate this question, we assembled a team of higher education finance experts to explore whether and how colleges could leverage outcomes-based funding (OBF) models to defray the cost of evidence-based college success programs. We encourage you to read the full report that outlines the promises and challenges of leveraging OBF models to help pay for high-impact success programs, including perspectives from OBF, state policy, philanthropic, and college success leaders. We summarize the key insights in this column.

Outcomes-Based Funding Models

OBF is an increasingly popular financing strategy that allocates a portion of higher education funding based on colleges’ and universities’ achievement on specific student outcomes. A core premise behind OBF models is that, by linking how much higher education institutions receive in state funding to institutional performance, institutions will be incentivized to invest in policies and programs that generate improvements in the outcome measures that the model prioritizes. Especially in states that set aside a large share of higher education funding for the OBF model, colleges that improve student outcomes and subsequently receive increased state funding might have the motive and means to help pay for evidence-based programs. While we provide a brief overview of OBF models in the full report, for a more thorough review, refer to the Lumina Foundation’s series of papers.

Despite the growing adoption of OBF models by states across the country, higher education leaders and researchers have expressed reservations about OBF models’ ability to effectively incentivize higher education institutions as intended. Two primary concerns are that: (1) the total funding amount in a given year is often fixed at the state level, so that one institution’s greater revenue through improved performance comes at the expense of other institutions’ funding; and (2) state higher education funding fluctuates from year to year. Colleges may be reluctant to make investments in long-term, high-impact programs given the uncertainty of how OBF models will be funded in years further out.

Forecasting OBF Revenue Available to Support College Success Programs

As part of this project, HCM Strategists developed an OBF financial forecasting tool that allows four-year public universities in Ohio to estimate improvements in student outcomes that they can achieve through different college success investments, and subsequently observe dynamically how these improvements would translate into revenue returns from the Ohio OBF model. The tool also allows a given university to make assumptions about rates of improvement at their peer institutions to see how these concurrent changes to student outcomes at peer institutions affect the institution’s OBF revenue returns.

To better illustrate how this OBF forecasting tool can be applied, we developed a use case estimating the revenue a four-year public university in Ohio could receive by investing in Bottom Line, a college access and success advising program for which we have randomized controlled trial (RCT) estimates of impacts on student persistence.

We estimate Bottom Line’s impact on two key metrics used in the OBF formula, course completion and graduation, in order to observe how improvements in student outcomes generated by Bottom Line translate into revenue returns from the Ohio OBF model. We outline the entire set of assumptions we use to populate the use case in the full report.

In addition to modeling OBF revenue using the impact estimates from the ongoing Bottom Line RCT, we make projections under scenarios that Bottom Line would have a larger or smaller impact in Ohio. We find that a given four-year university in Ohio could expect between $62,000 and $220,000 in OBF returns over a period of six years. This range of estimated OBF returns that the Ohio university could expect could cover approximately 10% to 34% of Bottom Line’s costs.

This analysis suggests that a university’s investment in evidence-based but expensive college success interventions like Bottom Line may result in modest OBF returns that cover only a share of the costs necessary to implement the program.

There are noteworthy challenges to this financial forecasting approach. Revenue projections for any one institution will be imprecise due to the fixed “pot” of money available. Even if an institution improves on outcome measures like degree attainment, they may not realize funding increases if other institutions have improved at a similar or faster rate.

Additionally, not all college success programs have rigorous evaluations of their impact on student outcomes. Lack of evidence-based estimates hinders institutions’ ability to reasonably estimate improvements in student outcomes that they might expect from investment in a college success program, and, as a result, how much OBF revenue they might receive.

Closing the Funding Gap

As the preceding analyses indicate, a university in Ohio could expect OBF revenue to defray only a share of the total costs of an evidence-based, high-impact college success program—even under the most generous predictions. We are pursuing several additional strategies to complement the OBF revenue and as part of our broader effort to develop innovative financial models to sustain and scale high-impact but resource-intensive college success programs. We describe two of these below, and explore additional strategies in the full report.

• Integrate OBF in a pay-for-success (PFS) framework: PFS models transfer financial risk of investing in a social program that may or may not produce improvements in key outcomes from a college or university to a private investor (e.g., philanthropic foundations). With some estimated revenue generated through OBF and an upfront investment from philanthropies, colleges and universities might be more willing to invest in effective but expensive college success programs. Because of the small share of program costs that OBF revenue covers, universities might not be willing to enter into a PFS arrangement where they were responsible for the full cost of the program. But even in a scenario where a philanthropic foundation doesn’t receive 100% of its initial investment, receiving at least some portion of the initial investment may be more appealing for foundations than the traditional model of providing philanthropic support and receiving 0% in return.
• Engage a more diverse set of funders for evidence-based programs: Some interventions, particularly those that provide more holistic support to students, could conceivably improve not only academic outcomes like persistence or graduation, but also non-academic outcomes like financial health and physical well-being. Beneficiaries of such improvements include not only students, but also a wide range of private and public entities like financial institutions or health care providers. Andrew Barr and Ben Castleman, in collaboration with NORC at the University of Chicago, are currently conducting a survey to investigate whether Bottom Line directly improves non-academic outcomes for its students. If survey results indicate positive impacts on a more holistic set of outcomes, it might be possible to engage a more diverse set of funders (e.g., banks or health insurers) to help pay for the program.

Supporting our higher education system to produce more college graduates is critically important. Yet public colleges universities face ongoing state funding cuts and philanthropic foundations’ finite resources can’t fill the financial gap. If we want to both increase college access for disadvantaged students and help them succeed when they get there, we must continue to investigate innovative approaches to sustainably fund evidence-based but resource-intensive college success programs.