Overcoming the obstacles to quality cost data

UNESCO estimates that every year $5 trillion is spent on education around the globe, with less than 1% of that spent in low-income countries. Billions are needed to tackle child malnutrition, which leaves 20% of children worldwide stunted and over 5% suffering from wasting, or acute malnutrition. The COVID-19 pandemic has undone progress across nearly every sector impacting children and youth around the world, necessitating an increase in global spending to meet the Sustainable Development Goals (SDGs) and protect the most vulnerable. Funders, organizations, and governments hope that their spending on programs, interventions, and reforms improve outcomes for children and youth in an equitable way, and that it ultimately leads to more educated, thriving, and productive citizens. Yet for all the spending, data on costs generally remains elusive. Without high-quality cost data, decisionmakers face difficult and uninformed choices in implementing, changing, reducing, or scaling programs. While cost data are not the only type of data needed to make these decisions, for example impact data are necessary for cost-effectiveness analysis, they are unfortunately often neglected because of challenges faced in their collection and analysis.

The cost data challenge

There are significant barriers to quality cost data for programs focusing on children and youth, which negatively impact the decisionmaking abilities of policymakers, implementers, and governments. While only some of these barriers may be present in each scenario, in every case, they work to reduce the likelihood that cost data inform decisionmaking processes.

On the one hand, the supply of cost data to undertake such analyses is often insufficient or unavailable. Underlying data may be inadequate for thorough analysis, such as in the case of aggregated budget data versus actual expenditure data, or substandard. Key stakeholders may find the task of collecting and analyzing cost data too burdensome to overcome, especially in consideration of the paucity of costing tools. Further difficulties arise from the methodological complexities inherent in quality costing analysis. Without ample and accurate cost data, any cost analysis will come up short.

Furthermore, there are often obstacles to quality costing emanating from the demand side. This includes a lack of prioritization of compiling complete cost data and conducting comprehensive cost analyses. Aversion to transparency can also cripple costing efforts from the start. Such reluctance to analyze costs may come from a concern that doing so would reveal inefficiencies, misspending, or worse. There may also be hesitance to engage in an exercise that uses salary information that may be considered private. These demand-side issues can hamper costing work from even being initiated. The combination of these factors, even if not all are present in each and every context, inhibits the use of quality cost data to benefit children and youth.

While we have indeed seen a rise in evidence-based policymaking, this has primarily been based on effectiveness data without cost data. In fact, studies have shown that even at the highest levels, most evaluations of programs and initiatives do not include cost data analysis.  Without cost data, policymakers and implementers are essentially reaching conclusions with one hand tied behind their back.

History of our work

Brookings has been working to understand and reduce the barriers to costing on both the supply and demand sides of the equation for nearly a decade. In 2014, the Center for Universal Education launched a multi-pillared work program on early childhood development (ECD) and education, including a pillar focused on the global lack of cost data. Early research focused on the importance of costing in ECD for advocacy, budgeting, planning, and scaling. Research also identified the absence of a standardized template or tool that could be used to cost a range of ECD activities while providing micro-level data that could be extrapolated for macro-level budgeting.

Building off this early work, in 2017, the Center for Universal Education launched the Standardized Early Childhood Development Costing Tool (SECT), a global public good that aimed to alleviate the costing issues identified. SECT was piloted in five countries in a variety of ECD programs including parenting education for caregivers of children ages 0-3, preprimary programs for children 3-6, and nutrition and stimulation programs for children under 5 in partnership with the Strategic Impact Evaluation Fund at the World Bank. SECT was a major step forward in reducing the barriers to costing within the ECD community, but it also illuminated further challenges and future improvements that became the crux of more recent work. This included the need for a costing tool that could be expanded to more services serving children and youth as well as the need for a global database for costs to allow for transparency and learning.

The Childhood Cost Calculator

Now, Brookings is launching the Childhood Cost Calculator (C3) which replaces but builds upon the strengths of SECT while moving to a more user-friendly online application format based on the Tangerine platform from Research Triangle Institute (RTI International). The tool has also been expanded to be applicable to interventions across a wide range of sectors related to young people including education from early childhood to tertiary levels, health, nutrition, water and sanitation, social protection, and governance. It can be used by stakeholders across the spectrum including policymakers, funders, implementers, and researchers to answer a multitude of costing questions including:

Questions for planning

• What resources are needed to deliver an intervention?
• Is the project feasible within a given budget?
• How do the costs of intervention A compare to those of intervention B?
• What are the cost drivers of this intervention?
• What is the cost per beneficiary (unit cost) of an intervention or program?

Questions for adaptation and changes

• What are the cost implications of a programmatic change, such as in dosage?
• What would be the cost of scaling up a program or intervention?

Questions about cost distribution

• How are costs distributed across cost categories for this intervention or program?
• How are the costs distributed across resource types for this intervention or program?
• How are the costs distributed between investment costs and recurring costs?

C3 can also contribute to answering the following questions provided additional data:

• What did this intervention cost per outcome delivered?
• How does that compare to other interventions that produce this outcome?
• How did the costs of this intervention compare to the monetary value of all the benefits by this intervention?

C3 was engineered with the user in mind and includes built-in functionalities including currency conversion and amortization. Users enter costs into a questionnaire, which then generates data visualizations that can be filtered by cost category, resource type, cost year, investment/recurrent costs, and intervention sector and type. Disaggregated data can be downloaded in CSV format. Through pilot testing we found that the tool was easy to use and successful across a variety of interventions around the world.

What we learned from piloting C3

C3 was piloted in Cambodia, Ghana, and Honduras with interventions in either early childhood or general education sectors. Pilot partners were chosen with several factors in mind. First, and perhaps most importantly, the pilot partner implementing team needed to be intrinsically motivated to conduct a costing exercise for the benefit of their own projects. An integral consideration in developing C3 was the ability of local teams to cost their own projects without outsiders or experts doing it for them. Thus, finding three teams that were internally driven to own the costing process was critical. Additional factors in choosing pilot partners included both geographic and intervention diversity, as well as variation in the type of actors involved in each program. Owing to this diversity, each partner began the costing exercise with different costing questions they aimed to answer through their use of the C3 tool.

The diversity of the pilot partner projects allowed us to see all aspects of C3 at work and provided opportunities to refine the tool using real-time data and feedback from our pilot partners. Feedback from the piloting process, for instance, suggested a wider array of intervention types was needed. The final presentation of data was also adjusted to provide more useful information up front with the inclusion of both average cost per year and cost per year calculations for programs. The piloting experience also revealed small technical glitches that could only be found through active use of the tool, allowing the research team and the application designers to tease out the underlying mechanism and make corrections. The pilot process also allowed for learnings about the costing process itself, beyond the tool. Coherent costing teams with a desire to engage and an interest in the cost data were critical to the success of the exercise. We also learned that the tool served as a catalyst for critical programming and policy conversations across a mix of stakeholders. This was enhanced by bringing in those stakeholders including—for instance, ministries, universities, and other civil society organizations—at the start of the costing process and incorporating their views on costing needs. Piloting proved to be an invaluable chance to make certain that C3 would meet the goals behind its development: a user-friendly tool that does not require technical expertise to employ and is widely applicable to interventions across the child- and youth-centered domains.

Conclusion

With the launch of C3 as a freely accessible global good, significant barriers in the costing of interventions and programs for children and youth have been reduced. There is now a tool that can be used across agencies and organizations, providing consistent analyses in a user-friendly platform. At the same time, with the option to include aggregated data in the Cost Data Explorer, a database of childhood intervention and program costs will begin to advance transparency by making costing data widely and publicly available. It is our hope that this pushes forward the drive to make cost data about and for all a priority.