Scaling education innovations for impact in low- and middle-income countries during COVID

Contents

• Introduction
• What is ROSIE?
• Who are the ROSIE collaborators?
• Emerging themes from ROSIE’s design-based research
• ROSIE next steps

Introduction

Interest in scaling promising innovations to effect systemic change in education around the world has grown over the last decade. Scaling has become fashionable because the modern landscape of educational improvement is littered with short-term projects that temporarily succeeded only to later dissipate, isolated pursuits that never crossed into broad adoption, or specialized policy programs that floundered. Moving beyond 20th-century technical-rational implementation and acknowledging the mixed history of global development in low-and middle-income countries, newer iterations of scaling have sought to collaboratively embed promising education ideas and technologies into whole systems. Increased recognition of the interconnectedness of culture, governments, global development architecture, and the learning sciences has reframed education scaling as a holistic process of mutual adaptation and collective transformation. Lasting impact has replaced size or scope as the goal. As a result, this past decade of scaling and research has offered hope and possibility—even as it has also underscored the sometimes maddening complexity of this work. Reckoning with COVID-19

And yet, as challenging as scaling is under ordinary 21st-century circumstances, it has become downright arduous over the last two years due to the global COVID-19 pandemic.

The work of scaling education innovations (up, out, down, and into deep and lasting impact) has always required nimble attention to the unpredictability of implementation. Much has been written about the need for adaptive mindsets, planning for scale at the beginning, enlisting allies and champions, and embedding continuous improvement loops into the whole process (Al-Ubaydli, List, & Suskind, 2019; Gargani & McLean, 2019; Perlman Robinson, Curtiss, & Hannahan, 2020).

Yet COVID-19 has brought all that into the starkest of relief. Not only have the last 21 months been devastating across the planet, but to say that COVID-19 has placed unprecedented stress on the work of scaling education innovations is a laughable understatement. And yet the work must continue. Practitioners, policymakers, researchers, and donors have persevered, motivated by an urgent need to “build back better;” address learning loss and the emotional tumult suffered by countless children and adults; and continue improving education quality, access, and equity. Adaptability proved to be the rule. Scaling and scaling research went on, just differently.

This first annual brief reflects on scaling insights from different scaling teams across many low- and middle-income countries to jointly learn and share best practices related to scaling in education. Effective scaling is not just about designing and delivering promising innovations for use but also embedding them in thoughtful, mutifaceted ways to ensure that they grow, deepen, and have lasting impact. This brief discusses how several teams went about this work during the difficult last year.

What is ROSIE?

In 2020, the Center for Universal Education (CUE) at the Brookings Institution joined the Global Partnership for Education’s (GPE) Knowledge and Innovation Exchange (KIX), a joint partnership between the Global Partnership for Education and the International Development Research Centre (IDRC), to facilitate a cross-national, multiteam, design-based research and professional support initiative called Research on Scaling the Impact of Innovations in Education (ROSIE). The intention of ROSIE is to bring together researchers and practitioners to study the process of scaling education initiatives.

In early 2021, after several KIX scaling teams from the larger population of KIX global grantees applied[1], six teams were selected to join ROSIE. This first cohort of ROSIE scaling teams worked alongside the CUE team to think about, study, and deepen the impact of their work. In September 2021, after another round of applications, a second cohort of scaling teams was selected to join—this time nine teams from the KIX regional grantees. In total, these 15 ROSIE collaboration teams currently work in 29 countries. Additionally, the CUE team is conducting complementary qualitative research on how governments engage in the work of identifying, supporting, and adopting education innovations to scale.

During the last year, the CUE ROSIE team collected information from the initial six collaboration teams as well as hosted several capacity-strengthening activities, online meetings, and team calls, which recently included the second cohort, too.

Who are the ROSIE collaborators?

Emerging themes from ROSIE’s design-based research

Five common learning questions were developed by the CUE team and the first cohort of collaborators together.

Working from the common learning questions, the CUE team developed a set of qualitative instruments for collaborators to use to gather existing information and generate focused reflections on their current scaling and research work, giving them the chance to choose a single common learning question and gather information accordingly. Collaboration teams submitted the information to the CUE team, which then coded it and discussed emerging findings and lessons during online calls with each team.

This brief reports only on the first round of information gathering and therefore only captures initial experiences of the first cohort of ROSIE collaborators. As a result, these reflections are partial and tentative. Because the study design includes subsequent rounds of information-gathering and collaborative learning over the remaining two years, findings will likely deepen and change over time—particularly with the addition of the second cohort. Given both the tentative nature of these findings and ethical considerations, this brief does not attribute names of researchers, collaboration teams, or even countries.

1. COVID-19 set in motion some favorable scaling changes and knock-on effects

While the costs and challenges of the COVID-19 pandemic remain enormous, some positive adaptations emerged. Because in many countries the government closed schools (or, even if they did not, significant numbers of families kept children at home), many scalers adapted their school-based scaling initiatives. Sometimes this meant printing hard copies of online learning materials and supporting teachers to make weekly home visits to deliver learning bundles and work with students and siblings. This not only kept the projects running when no one could access computers in schools, but fortuitously encouraged teachers and families to get to know each other, teachers to see students in their home environment, and school-family engagement to strengthen. Other times this meant that, in lieu of schools as locations of collective learning, the community centers, churches, and sometimes even homes became places where children congregated to learn. The willingness of these locations to allow teachers and other educators to use them for teaching became a welcome addition to community involvement. A typical scaling goal is to increase local ownership of an initiative and, the pandemic advanced this goal in several ROSIE initiatives. Additionally, because many initiative leaders mandated grouping students in community locations in very small numbers to minimize transmission of the virus, small learning groups were serendipitously created and teachers could experiment with more active instruction.

These unpredicted changes carry value. As a result, scaling and research teams might try to more systematically test, monitor, document, and learn from these natural and spontaneous experiments in order to advance the principles of their innovations (see CUE’s adaptations tracking tool). In many ways, it is the aggregate of adaptations that becomes the scaling journey.

This underscores that ed tech should always be in service of meaningful learning—not a replacement for it—and should be understood as a tool that opens learning up rather than narrowing it.

Because of disruptions in education schedules and tech offerings, some collaborators found that participating educators experimented with new pedagogies and found their own “aha!” moments around learning, literacy, and the power of student-centered instruction within the initiatives being scaled. Without access to the digital learning platforms, sometimes teachers turned their focus away from the machines and back onto student learning. This underscores that ed tech should always be in service of meaningful learning—not a replacement for it—and should be understood as a tool that opens learning up rather than narrowing it (for example, see Save our Futures white paper).

Similarly, it was impossible for teachers in many countries to continue their online professional development courses when they could not enter schools to access computers. Scaling teams whose initiatives include teacher professional development (TPD) knew that not all participating teachers had personal digital devices, so some ROSIE projects encouraged groups of teachers to share one person’s phone or tablet for the online courses. The small teacher groups would meet in a central village, sometimes outside under a tree for additional air safety, and engage in the online coursework together. As a result, teacher collegiality was strengthened, and teacher learning became collaborative—always a goal for teacher development specialists. When education routines are disrupted, committed and knowledgeable educators will often seek out new solutions, frequently finding innovative pedagogies in the process. This should be anticipated and productively exploited by education scaling teams.

ROSIE researchers reported benefits of information and communications technology (ICT) for coordinating their own work and connecting with local researchers across whole countries. Shifting meetings and work onto ICT reduced collaboration costs, facilitated multilanguage conversation with greater ease, and offered access to ready-made recordings and transcriptions. Moving inter- and intra-national work online offers new uses of 21st-century technology that can catalyze transformation in areas like team communications, local research and community project support; and meetings, governance structures, and policymaking dynamics, respectively.

Not all knock-on effects can be harnessed and not all of them are beneficial, but alert researchers can capture them and their possibilities so that scalers can consider how to harness the positive implications and mitigate the negative ones.

These and other direct adaptations that collaborator-led initiatives made in the face of COVID-19 disruptions set in motion additional second- and third-order changes (known as “knock-on effects”) and, as a recent CUE blog reported, whole systems were altered. Researchers will benefit from study designs flexible enough to capture not only direct effects of adaptations but also the indirect, sometimes subtle, subsequent ones. Not all knock-on effects can be harnessed and not all of them are beneficial, but alert researchers can capture them and their possibilities so that scalers can consider how to amplify the positive implications and mitigate the negative ones.

2. Increased reliance on technology includes some capacity concerns

Though a known issue before the pandemic, during COVID-19 it became even more pronounced that many regions—especially rural areas—have insufficient internet access. A 2020 UNICEF report found that two-thirds of the world’s children (1.3 billion children aged 3-17) have no internet connectivity at home (UN News, 2020a). When schools shut due to COVID-19, there was often no way for these children to learn remotely. As a result, 463 million students (1 in 3 children globally) are missing out on remote education (UN News, 2020b).

A significant contributor to this digital divide is the cost of home internet. Therefore, some scaling teams negotiated bundle deals with service providers to offer internet to whole communities at reduced rates. Other times, teams had to adjust their tech offerings—making asynchronous materials available for download and later use or developing radio or print solutions so schoolteachers or community volunteers could continue the projects’ work.

Scalers and researchers [should] build sufficient training and equipping of local personnel into their initial planning to understand and maintain the hardware needed for tech-learning innovations.

Some ROSIE collaborators shared that teachers in some initiatives were so focused on the logistics of using the hardware and software that they neglected the actual learning and teaching the platforms were designed to facilitate. In other cases, the opposite emerged: teachers neglecting the machines altogether to focus on student learning. Both circumstances reinforce the need for scalers and researchers to build sufficient training and equipping of local personnel into their initial planning to understand and maintain the hardware needed for tech-learning innovations.

With the increased popularity of digital technology as an education solution, there are other considerations (Ganamian, Vegas, & Hess, 2020). In some instances, more experienced head teachers or teacher leaders—who might be years older than new teachers and less likely to be “digital natives” comfortable with new technology—struggled with the skills needed for online instruction or participating in online TPD. As a result, there was concern that head teachers may have difficulty leading their faculty into the future of teaching and learning. This highlights the need for consideration on how best to support whole faculties to share their own collective mixes of digital expertise, pedagogical experience, and new instructional practices as new technologies are introduced at scale.

A second consideration is that a few teams worried that the embrace of ed tech might currently be exciting for local educators and national governments because it is new but, over time, educators and governments will become less enamored of ed-tech solutions. While governments may appreciate the cutting-edge symbolism and political expediency of technology, there are concerns that they will not support the costs and hardware maintenance longer term. And there are worries that, when the novelty of ed tech wears off for local educators, motivation for the additional learning it requires will fade.

And finally, the global movement toward a reliance on digital technology—pushed exponentially forward by the pandemic—is often underwritten by for-profit companies that collect personal information in exchange for use or sacrifice privacy for efficiency. As a result of these and other deeper implications, the move to increased reliance on ed tech should be an object of critical study even as it is simultaneously a mark of progress.

3. COVID-19 highlighted that “allies and champions” are not only donors, governments, and development partners but also volunteers, community groups, and families

Enlisting support and buy-in from allies and champions has become axiomatic in scaling. The logic holds that an innovation stands a better chance of being adopted and supported by a jurisdiction or government if there is broad support among a diverse array of influential stakeholders, and that one good way to build that support is to bring in key individuals and groups whose assistance will reap benefits. Often, the allies and champions are those with material or symbolic capital in a location: high-level government personnel, leaders of community organizations or teacher associations, or external agencies with funding or influence. That was certainly accepted as true here, and most ROSIE collaboration teams are thoughtfully pursuing partnerships with allies. While important, it is also clear from collaborators that this is not always easy. A few collaborators lamented the absence of pre-existing communication structures that scaling teams could use to access appropriate government ministers and other national-level policymakers. It would be good for governments to establish semi-permanent, readily available pipelines so that education scaling leaders and ministry officials can establish and maintain communication.

It would be good for governments to establish semi-permanent, readily available pipelines so that education scaling leaders and ministry officials can establish and maintain communication.

ROSIE collaborators reported a trade-off in their collaborations: Government officials with the authority to influence decisions are not always the ones with the proactive mindset or skills and, conversely, those who are motivated and skilled in effecting productive change are not always the ones with the authority. It is important to attend to this paradox and work with the right personnel or, even better, make efforts to work with both outward-facing appointees and inward-facing civil servants. This point underscores the importance of mid-level technocrats: a tier of government officials who are typically knowledgeable about the innovations, stay in their positions longer, and focus on reform particulars. A recent CUE team blog discussed the need for innovators and government decisionmakers to work together.

However, a massive disruption like COVID-19 also teaches that power is diffuse and variable. It is not only those in high-level government positions or donor agencies who matter. Three important groups emerged that should not be overlooked:

• Local volunteers who operate after-school programs, find locations for children’s learning when the schools close, or coordinate the initiatives or research when national or international personnel cannot.
• Community groups such as the church or local associations that stepped up to offer space, recruit and vet volunteers, and support children.
• Families who understood that public schools are not always the sole providers of their children’s education and thus supported the initiatives and encouraged other families to support them, too.

These three stakeholder groups became essential to the advancement of several initiatives that ROSIE collaborators are working to scale. This often-unheralded reliance on community volunteers and committed families should not go unrecognized, especially by governments. Leveraging these three populations carries the opportunity to reach more participants and potentially sustain local engagement in ways that last beyond changes in political leadership. Yet, as discussed next, there are challenges around sustaining high-quality community support.

Because COVID-19 constrained formal and informal economies in many countries and imposed financial hardships on people, it was not uncommon for community volunteers to terminate their work on initiatives in search of paid jobs. When community volunteers exit the projects for which they are working, they not only take their efforts and talents with them but their training and project knowledge, too. It is recommended that initiatives, nongovernmental organizations, and governments go out of their way to recognize, reward, and show appreciation for these unpaid, fundamental parts of education systems (Winthrop et al, 2021).

4. Building trust with partners is time-consuming but essential

Several ROSIE collaborators emphasized the value of trust. Engaging government support and building partnerships with allies, champions, and local stakeholders are essential to scaling an education innovation, and there is no shortcut to building productive relationships. Learning to view implementation complexities through the eyes of others, knowing when to be patient and when to push, and understanding what kinds of evidence people need to feel comfortable with an innovation are necessary components of healthy partnerships. And trust appears to be the glue that holds it all together.

Knowing when to press and when to wait is key.

Some governments have high rates of turnover among officials, so establishing long-standing personal relationships can be difficult. Yet, when a trusting relationship is present, scalers can intuit when to press their government partners to adopt an initiative or when to pause. When a ministry leadership is relatively secure, there may be less pressure on mid-level officials, which can be a moment to press for change. Conversely, when the ministry leadership feels threatened or insecure, the pressure on middle management is high and risk-aversion, defense of the status quo, and pressure for centralized decisionmaking increases. Knowing when to press and when to wait is key. One collaboration team pointed out that to get a good read on their partners in government, one has to physically be in the same room with government officials. Unless you are reading the body language, annoyance on the part of the government official (meaning it is not time to push) cannot always be distinguished from tiredness (which does not preclude applying a gentle nudge). This has been a casualty of COVID-19: When you cannot meet people in person, you cannot easily discern between the different emotional states of your government partners.

Likewise, trust is a key ingredient in building alliances with parents and other caregivers. Tschannen-Moran (2014) found that trust between teachers, principals, parents, and students accounted for 78 percent of the variance in achievement on standardized reading and math assessments. As Winthrop, et al (2021) pointed out, teachers’ trust in students and parents is one of the two strongest contributors to this variance. Families need to believe that the innovation has the best interests of their children in mind, and scaling implementers need to view families as a fundamental pillar in the education of children. Without mutual trust, support will collapse—and trust cannot be feigned.

5. Teachers matter—and all teachers are not alike

Michael Fullan (1991) once wrote that, “Educational change depends on what teachers think and do. It’s as simple and as complex as that.” Building pedagogical capacity among teachers and authentically including them in the planning and scaling of an innovation are often central to scaling. Especially during times of change or uncertainty in education, teachers must be supported to remain motivated and creative. ROSIE collaborators reported on teachers who, when they realized that the electricity would likely fail in the afternoons, retooled their curricula to present the online learning in the morning. Collaborators reported on teachers who downloaded and printed learning materials at the beginning of the week to use as hard copies days later when the internet inevitably collapsed or teachers who had to develop their own teaching materials out of household items.

Scalers should do all they can to include teachers in the design, adaptation, and dissemination of initiatives.

It is difficult to encourage pedagogical commitment and creativity in teachers who feel controlled, diminished, or overworked (Olsen & Sexton, 2009). As a result, scalers should do all they can to include teachers in the design, adaptation, and dissemination of initiatives. Because teachers are reluctant to undertake sustained additional work or learning when it becomes overly burdensome, education systems leaders should find ways to compensate teachers for extra work, offer advanced education credits, or provide release time for professional development.

Some ROSIE collaborators found that teachers became fatigued when asked to teach in new ways or use online platforms without sufficient training. Other collaboration teams reported difficulties with teacher migration: When a teacher left for another school, the training moved with them (this became even more worrisome when the departing teacher was trained to teach the innovation to other teachers). Teachers had difficulty traveling from their home community to nearby professional development sessions—and were therefore unlikely to attend TPD without a travel allowance. One collaboration team reported that program mentors conducted teacher observations with teachers who lived closer to them more frequently than those who lived farther away. And, finally, many collaborators shared that teachers in some countries had difficulty accessing the online portals and TPD courses when the sign-in required email addresses. Sometimes it is the simple details that hamper a complex effort. Learning from other scaling teams and collaboratively reflecting on past experiences with the small but essential details of program implementation can help scalers to anticipate and address logistical barriers.

For those innovations that include public schools, the system level of teachers, teacher support, and teaching conditions is central to scaling. Teachers mediating the innovations (intentionally or automatically) according to their own pedagogical purposes or understandings of student learning will likely alter, improve, or subvert the innovations as they are translated into practice in classrooms. While it may not be feasible to account for every teacher’s unique profile, it is good to identify whole-faculty strengths and knowledge gaps in particular schools, locations, or populations around things such as experience level, educational philosophy, and instructional habits. If strong policy and financial commitments from government to support quality teacher development are met with an equally strong learning commitment from teachers and school leadership, there is increased chance of effective, high-quality local ownership of the initiative in classrooms.

And, finally, a note about studying the effects of TPD. Decades of research on efforts to change teacher practices have revealed that attendance at TPD or mere use of the materials provided does not mean that teachers were actually changed by the TPD experience (Anderson & Olsen, 2006). Researchers interested in studying whether a TPD approach improves instructional practice may wish to go beyond collecting self-reported teacher survey data and teacher attendance rates. While those might be a good starting point, collecting and analyzing data on if and how teachers’ practices and professional beliefs change (and remain changed over time), what their teaching and learning look like months after the professional development finishes, and exactly which teacher development interventions lead to what kind of student learning outcomes are even better practices. Admittedly, this is more difficult and costlier, especially when studying TPD at scale or across geographical distance, but if TPD is going to fundamentally improve teaching, this is likely necessary.

ROSIE next steps

In the two years ahead, ROSIE’s collaborative empirical work will produce broader, deeper, and more detailed insights. Additionally, as mentioned previously, ROSIE’s complementary research is exploring how national education decisionmakers approach scaling—including how they identify innovations to scale, how innovations can be integrated into and sustained within existing government systems, and what public sector decisionmakers see as enablers and barriers to scaling education innovations (see Olsen, Hannahan, and Arcia, 2021 and Olsen and Arcia, 2021).

Importantly, the work of building a professional learning community among ROSIE collaborators and the larger KIX and other global scaling networks will continue, too. Scaling education innovations in low-and middle-income countries is difficult and sometimes isolating. ROSIE hopes not only to continue gathering, studying, and exchanging what is learned, but also share best practices and support others’ scaling journeys both inside and outside the KIX ROSIE community. Creating a neutral, supportive space where researchers and scalers join together across contexts and initiatives to pause, reflect, and consider what they are learning by doing and how to adjust their work accordingly is rare but, ROSIE believes, worthwhile if efforts to improve education globally are to advance.

Download the policy brief»