Drawing from improvement science to bridge education research and practice

A three-legged stool can be tough to balance if the legs are uneven. We’ve been thinking a lot about how to do just that with the launch of the Millions Learning Real-time Scaling Labs starting in Brazil, Jordan, Tanzania, Côte d’Ivoire, and the U.S. city of Philadelphia. How do we balance and give equal weight to our three primary objectives: learn from, document, and support education interventions in the process of scaling? We think that drawing from principles behind adaptive and iterative learning methodologies, such as improvement science, can help.

Our starting point with the scaling labs is to learn more about how effective education interventions expand and deepen their impact—what we refer to as scale—especially among some of the world’s most marginalized communities. The global education community is learning a lot about how best to scale, some of which is seen in our 2016 report Millions Learning: Scaling Up Quality Education in Developing Countries. But there is certainly a lot more to learn, particularly around the core drivers behind adapting an evidenced-based initiative to a new context as well as about the enabling conditions in the environment that allow for good ideas to take root and spread. Through the lab process, we hope to learn even more about this scaling as it unfolds in real-time.

Secondly, we hope to document this process so that we gain deeper insight into the scaling journey, not only to understand what outcomes are achieved but, just as importantly, to understand how they are achieved, and to be able to “tell the story of what happened” so that other decisionmakers can learn from it. We are thinking a lot about how best to capture and share this insight in an actionable way to help inform future policy, financing, and programmatic decisions around scaling in education.

And the final leg of the stool is helping to support scaling efforts at the center of the Real-time Scaling Labs. Rather than merely learning from and documenting what is happening as external, silent observers, instead we are co-creating a peer-to-peer, iterative learning approach with partner institutions around the world to actively support the scaling process. During the three to four-year long lab process, we will routinely convene a diversity of stakeholders involved in the initiative(s) to identify scaling goals, develop and/or refine scaling action plans, and engage in a participatory and iterative process of implementing the scaling plan, reflecting on progress, and making course corrections along the way. The intention is to help strengthen capacity to scale among decision-makers through this participatory and applied research approach.

In designing these Real-time Scaling Labs, we certainly didn’t start from scratch. We spent the past year researching a wide range of adaptive learning and collective impact approaches and considering how these methodologies might be applied to challenges around scaling in global education. One of the most useful approaches that we have drawn from is improvement science, a user-centered, problem-driven, iterative approach designed to accelerate learning-by-doing. Originally pioneered in the healthcare field by the Institute for Healthcare Improvement, improvement science uses iterative cycles to rapidly test change ideas, and adjust based on data in a systemic rigorous way. I had the opportunity to attend the Carnegie Foundation for the Advancement of Teaching’s annual Summit on Improvement in Education earlier this month in San Francisco. At the conference, I heard about three key components of improvement science that really resonated with what we are trying to create with the scaling labs.

1. Possibly incorrect and definitely incomplete

Improvement science begins with the notion that the ideas we wish to test about how to improve education are possibly incorrect and definitely incomplete, and so will need to be reconsidered, adapted, and possibly even abandoned as we learn more. This notion is also at the core of the Real-time Scaling Labs, recognizing that scaling is not a linear process but is, in fact, cyclical, with ongoing challenges to overcome and opportunities to seize. We recognize that the first plan developed to scale an education intervention will most certainly not be the final one; it must change and be adapted throughout the process of scaling. Therefore, if we want to scale effectively and sustainably, we must continue to assess and adapt our strategies in response to ongoing changes that occur within any complex system, reacting to real-time learning rather than strictly adhering to our first assumptions. This underscores the importance of taking a flexible approach to designing, delivering, and financing interventions for scale. Part of this is also recognizing that setbacks will be an unavoidable and important part of the journey as they are critical moments to learn.

2. Every system is perfectly designed to get the results that it gets

Improvement science recognizes that we don’t exist in silos and vacuums. If we want to affect large-scale change, we must start by focusing on the broader system and understanding what in the system is producing the outcomes we currently see. Similarly, the scaling labs begin by convening a diverse group of stakeholders in-person to discuss and assess the nature of the problem the intervention(s) seeks to address and the system that produces it. By taking a systems approach, we are able to understand and address the processes and mechanisms within the system that are hindering the type of improvement we seek and to tackle the underlying causes behind the problems we are trying to solve. Additionally, by deliberately engaging a diverse composition of participants in the labs, we recognize the importance of getting multiple perspectives, as one person can only see part of the system.

Source: The Baseline Company www.theblindelephant.com

3. Control urges for solutionitis

Before jumping to any conclusions about a potential solution, improvement science requires asking deeper questions about the data to truly understand the actual problem to be solved. An excellent speaker in one of the sessions I attended gave the example of the Lincoln and Jefferson Memorials, which were decaying at an increasingly rapid rate. Evidence suggested it was a result of the cleaning process. Rather than rushing to make changes, improvement science led to asking why the monuments were cleaned so frequently and with such harsh chemicals. They found the frequent cleaning was to remove the large amount of bird droppings on the memorials. Why were so many birds flying over? Because of a large spider population on the memorials attracting the pigeons as food sources. And why so many spiders? Because of gnats swarming the monuments at dusk. And why so many gnats? They were drawn by the lighting of the monuments. This series of deep questions led to the decision to turn on the memorial lights one hour later, after sunset, eliminating the root cause of the problem (gnats). This will also be a core tenant of the scaling labs, as we will not be satisfied with identifying scaling challenges and opportunities to leverage at a surface level and jumping immediately to finding solutions, but rather will spend time to collectively understand the root causes and ensure our scaling plans seek to take those into account.

In designing the Real-time Scaling Labs, we are fortunate that improvement science has a rich history for us to learn from and experts around the world who are using this approach to tackle some of the most complex social challenges facing our societies, thus helping us to keep our three-legged stool in balance.