Can we assess if school and classroom practices align with national educational goals?

Fact: Young people need a broad set of competences to thrive in adult life.
Fact: Many schools continue to prioritize a narrow set of learning outcomes.

Now more than ever, countries are orienting their policies toward equipping children and youth with a broad range of skills to succeed in the 21st century. Given this widespread endorsement at the policy level, why don’t we see it happening in more schools? Could it be that schools lack a means of examining how to assess and teach these skills?

An important step in this process is examining whether school and classroom practices align with the national educational goals, so that different levels of the education system are working together to provide quality learning opportunities to develop breadth of skills in students. The focus tends to be on assessments of learning outcomes, but if no opportunities are available to learn the skills, how can we expect students to perform adequately? What if, in addition to evaluating an education system on the learning outcomes demonstrated by students, we also looked at the opportunities students have to learn a broad range of skills?

The BOLO (Breadth of Learning Opportunities) initiative was designed to fill this information gap. Convened by the Center for Universal Education (CUE) at Brookings and Education International (EI), the BOLO initiative seeks to develop an alternative and complementary approach to existing evaluation of education quality by providing tools to measure the breadth of learning opportunities to which children and youth are exposed in an education system at the national, school, and classroom levels. The tools can be used to document 1) whether opportunities are provided for learning across a diverse group of domains, and 2) how key components of an education system (curriculum, assessments, teacher supports, monitoring, and school resources) align across the different levels of the system to support delivery of breadth of learning opportunities. The initiative has its origins in the Learning Metrics Task Force, which sought to offer recommendations for measuring learning globally that would not result in narrowing of curriculum and instruction.

The BOLO toolkit, published today, was developed by a consortium of researchers, policymakers, and practitioners. CUE and EI convened an international working group to develop the framework and early drafts of the tools, and then worked with governments and partner organizations in nine countries to pilot the tools. EI conducted intensive workshops with teachers in Kenya and Zambia to develop drafts of the tools and pilot them with teachers, and CUE recruited consultants in eight countries, including Argentina, El Salvador, Kenya, Pakistan, Palestine, Rwanda, Tunisia, and Zambia, to pilot the policy-level tools. FHI 360 was engaged to design the school-level tools, create enumerator guides, data entry interfaces, and codebooks for all three tools, and support data cleaning and basic descriptive analysis after tool piloting. Full pilots of all three tools were conducted in 2017 by ziziAfrique in Kenya and Centro de Cooperación Regional para la Educación de Adultos en América Latina y El Caribe (CREFAL) in Mexico.

How the tools work

Here is an example of how the tools work in practice. In Michoacán, Mexico, a group of teachers and the school principal sit around a table looking at a series of shaded boxes. The lighter colors represent less emphasis by teachers, schools, and policymakers on a given academic subject, and the darker colors represent more emphasis. The differences in opinion across these groups are highlighted in respect to seven learning domains. After a brief explanation of this “heat map,” the teachers’ eyes light up.

One teacher says, “Aha! So this is saying that while the curriculum does not include critical thinking and collaboration as an important component of learning Natural Sciences, schools and teachers are incorporating these skills in their teaching of that subject?” The teachers, who participated in the surveys underlying the heat maps, studied them closely. “And I see here that the state policy is to provide professional development on all subjects, but for physical education and arts education, very few teachers say they receive that training,” says another teacher. These are just a few examples of the types of information the teachers explored during the feedback meetings in Mexico and Kenya for the BOLO pilots.

This toolkit represents a first step in measuring the breadth of learning opportunities an education system offers, but more piloting and research needs to be done before using these tools for making decisions about education reform. We hope that as governments approach education reform, they can use this toolkit as data points that inform policy.