The first question typically asked about a development intervention is: “Does it work?” Frustratingly, the answer is usually “sometimes” and “it depends.”
The impact of transparency on school-level accountability, unfortunately, is no exception. In a new report, we found that only a select number of initiatives in low- and middle-income countries have reduced corruption; improved managerial, parental, and teacher effort; and led to more efficient targeting of reforms and resources. These limited successes, too, appear to be context-specific and difficult to replicate.
Former Research Analyst - Center for Universal Education, Brookings Institution
Tamar Manuelyan Atinc
Nonresident Senior Fellow - Global Economy and Development, Center for Universal Education
The assumed theory of change behind these types of information-based interventions, termed “social,” “citizen-led,” or “demand-side” accountability, leads from Information shared with citizens is generally of two types: (1) based on rights, entitlements, and roles in attaining services; (2) concerning the quality and performance of service providers, either in terms of inputs (i.e., teacher attendance, textbooks) or outputs (test scores, pass rates). Dissemination strategies vary in intensity, from more passive media or information campaigns to more active interventions such as social audits or participatory budgeting.
Similarly, open government and open data initiatives—the “younger relatives” of social accountability—typically achieve their objective by presenting new or previously hidden information, or by repackaging existing information to make it more usable. The underlying assumption, however, is the same—that transparent information will increase civic participation and provide incentives for providers to offer better services.
To provide more insight into our general findings, we undertook a systematic review of recent impact evaluations in the education sector to generate lessons on the component parts of information-based initiatives. This exercise makes it possible to identify what makes particular initiatives fail or succeed and what strategies can mitigate constraints. The idea here is not just to help clarify what works but why. To do so, we distinguished among a range of intensities of information interventions based on whether information is collected (at its most passive), disseminated, or made actionable (i.e., through the joint-creation of school improvement plans). We then link these interventions to their associated change agents—the actors that are presumed to change behaviors in response to the information being collected and shared—to see if actions line up with expectations.
There are some lessons to be learned here:
Clearly, the design is important. Perhaps not surprisingly, we find that information-based interventions are far more likely to succeed when they are implemented in combination with direct avenues for action, either through facilitated meetings, the co-design of school-improvement plans, or through training on how to improve a child’s learning. Of 13 unique interventions that were made “actionable,” eight show positive impact on either student learning or intermediate variables (i.e., teacher attendance; parental engagement). This is telling when compared to more passive information campaigns, including open data platforms and school report cards, which demonstrate positive effects in only 6 out of 17 interventions. This implies that a lack of information is only one constraint among many in enhancing school-level accountability.
So, too, the audience matters. Our findings suggest that interventions are more likely to succeed when they are directed at the management level, either exclusively or in tandem with teachers or parents. Of the five “actionable” interventions with null effects, four did not attempt to engage school administrators or district managers in the process. This makes sense since the management level has the most discretion over the use of funds as well as the ability to adjust actions that are more consequential for student learning.
In thinking through the engagement strategy, it is important to understand which types of information are likely to elicit a response from the targeted audience. Parents, it seems, are more likely to respond to data on inputs or information clarifying their roles and responsibilities to schooling rather than information on school outputs. For instance, an intervention that gave parents information on their oversight roles in schools and education services in three Indian states led to improved learning outcomes and reduced teacher absenteeism, driven by increased participation of parents in school committee meetings. Conversely, another study found that providing parents with information about their children’s performance on literacy and numeracy tests led to no impact on parental behavior or engagement, even when combined with materials about how to be more involved in improving their child’s learning.
These lessons make it clear that information interventions need to consider carefully the audience, design, and presumed causal pathway to improved service delivery. It is not enough to put information in the public domain and hope that it enhances accountability, especially since marginalized parents and communities have the least amount of time, resources, or influence to take up the reins of structural change.