The World Bank recently launched Teach, an open-source classroom observation tool designed to capture what goes on in classrooms in low- and middle-income countries. Teach captures time spent on learning, but it also is intended to capture the quality of teacher practices that support children’s cognitive and social-emotional skills, in an effort to monitor and improve teaching practices.
When it comes to learning, it is not about how much time is spent in class but how that time is spent. Studies show that high quality learning experiences, such as a supportive learning environment, positive teacher-student interactions, consistent routines, and effective instructional interactions that focus on constructive feedback and real-world applications, are central for student learning and development. Despite the research, teacher effectiveness and high quality education are typically measured using proxies, such as test scores and number of years of teaching experience, which 1) do not identify what the teacher is doing in the classroom to help students learn, and 2) are not useful for helping teachers to improve their teaching practices.
So, does Teach sounds too good to be true? Maybe or maybe not—only time will tell, but a Brookings report published last year reminds us that whether assessment is able to improve learning is related to the quality of the data and how well the information is used. Since its launch, there have been some questions raised about whether the Teach tool should be used (see here and here), but two additional concerns come to mind in light of the Brookings report. One is about validity, and the other is about purpose.
Validity: Is Teach capturing teacher practice or something else?
A tool that has reliability, meaning that it consistently produces the same results, is, of course, important. But, just because a tool has reliability does not mean it has validity. Validity refers to whether the tool really measures what it claims to measure.
Teach includes three teacher practice components—Classroom Culture, Instruction, and Socioemotional Skills—that are observed in a series of two 15-minute lesson observations and scored using a 5-point scale. Although observational measures can capture the complexity of the interactions and behaviors within a classroom setting, the measures are also affected by what is happening in the environment, which may call into question the validity of most, if not all, classroom observations. In a recently published article, my colleagues and I examined an observation tool that assesses children’s interactions in the classroom, not unlike how Teach is designed to capture teacher behaviors and interactions. What we found was that children’s behaviors were variable from one observation to the next, meaning that their interactions in the classroom depended on the time of day (morning or afternoon), who was observing, what activity they were doing (e.g., whole class versus small group activities), and the number of adults in the room. In other words, the tool was capturing more of the contextual factors (things that children could not control) rather than children’s actual behaviors (what they could control). All this to say, there may be more beneath the surface that needs to be unpacked—that the score on the tool may be reflecting factors beyond what the teacher does in the classroom. If this is the case, the question to ask is whether what is being captured is an accurate reflection of quality teacher practice or is it more about context—and what are the implications if it is the latter?
Purpose: How will the information from Teach actually be used?
The question should not be whether Teach should be used or not, but rather, how data from Teach will be used in practice—and whether that will translate into improved teaching and higher quality classrooms. The Brookings report mentioned above identified key principles relating to use of data from assessments, two of which are particularly applicable here. The first is that data should be collected with a clear purpose in mind, rather than just collecting for the sake of collecting.
The second is that what the information is used for matters. For instance, indicators of supportive learning environments may be collected, but if that information is used solely to rank teachers, their teaching practices—and student learning in turn—will most likely not improve. The fact that Teach is open source affords access for anyone to use—a definite plus, but it does make it that much more difficult to determine whether the information collected will be used as intended, which could mean unintended consequences for schools, teachers, and ultimately, the students.
An arguably more pressing question is where does Teach fit within the education system? Teach is a tool that any stakeholder can use in any country—again, a plus. But, to support student learning, a new Brookings report indicates that alignment across the components of the education system is a must. Quality teaching practices need to be the focus, not only at the classroom level, but at the national level as well. That means investing in teacher professional development, having learning goals that are in line with these practices, and providing the opportunities for teachers to demonstrate quality teaching practices.
These are just some of the issues that need to be considered moving forward. Nevertheless, the Teach observation tool provides a strong foundation and spotlights the importance of focusing on what is actually happening in the classroom. I look forward to seeing how this tool is implemented in different countries and how the data from the tool are used to improve teaching practices and student learning.