New school accountability measures must be careful not to reinforce student poverty

The newly enacted Every Student Succeeds Act offers reprieve to states that felt encumbered by No Child Left Behind’s onerous school accountability measures. Not only will the ambitious 100 percent proficiency standard no longer loom over the shoulders of state superintendents, but under the new law, states are also to adopt new school accountability measures that include an indicator of performance not directly tied to testing or graduation.

States have considerable discretion about what indicator to choose. They could feasibly measure and hold schools accountable for student or teacher engagement, attendance, school climate or most any other measure states can imagine, so long as it can be disaggregated to student subgroups. Though the revised school accountability measures are still intended to place much of their weight on traditional academic outcomes, the new inclusion of nonacademic ones could be considered a small victory by those that felt too much focus was placed on testing under the old law.

Before going headlong into the development of new accountability measures, however, policymakers would be wise to take stock of what we know about the old accountability measures to inform future actions. To begin, it’s worth keeping in mind that the current school accountability measures in many states have already evolved some distance from the original mandates of the prior law. By late 2005, just a few years after No Child Left Behind’s enactment, the U.S. Department of Education began allowing states to apply for waivers specifically from the law’s school accountability provisions.

At the time, states demanded waivers because the original school measures were criticized for focusing primarily on proficiency rates, which implicitly penalized schools serving high-poverty students. This penalty was due to the high degree of correlation between students’ backgrounds and their achievement levels on tests. The 2005 waiver opportunity allowed states to begin experimenting with measures of student growth to complement proficiency rates in their accountability measures. Student growth measures are demonstrably better measures of the input of schools, as they control for students’ starting points and thus remove much of poverty’s influence on the measure.

Now, the introduction of new performance indicators in school accountability measures will reinvigorate that same levels-versus-growth debate in a slightly new form. With these new measures, policymakers should be cautious not to choose a nonacademic indicator that could simply be a stand-in for student poverty. Measures that may seem free from such a bias may still implicitly carry it; for example, both student and teacher absences are well correlated with poverty levels. Observational ratings of teachers’ instruction, which are intended to focus just on teachers’ practices and not students, also tend to be dragged down in high-poverty contexts; it’s reasonable to speculate school climate measures, when brought to scale, could reflect a similar relationship.

Entirely removing this poverty link may not be feasible without administering a well-developed, fully vetted psychometric test. Though adding new tests to get at this nonacademic indicator – intended in part to dilute the pressure of standardized tests – will not likely be a popular strategy in most states. And we don’t necessarily need the correlation with poverty to be zero in order for the measure to be informative, but it should be distinct enough to provide a new signal of performance. As policymakers consider potential indicators for this purpose, disadvantaged students and the educators working with them will be best served by choosing indicators that mitigate this bias to the extent possible.

The underlying principle for wanting to remove the influence of poverty is that school accountability measures are most useful to the extent that they actually identify what the school contributes to students, not what students are contributing to the school. This is an issue that has been grappled with many times in the context of using test scores to evaluate schools and teachers, and there is a long trail of research backing this up. When using the right models, we find there are many excellent schools and teachers serving high-poverty students, just as we’ve suspected all along – though you wouldn’t know it based on proficiency rates only. Moving on to novel indicators, however, requires that we go back and review the paper trail from the beginning, using the lens of the new nonacademic outcomes.

My coauthor Ben Backes and I recently conducted a study in which we attempted to validate new measures of teacher performance based on nontested outcomes commonly found in states’ administrative data. Though we focused on teacher evaluations, the same principle of removing the influence of student background applies to schools, and our lessons are useful in this context. We found that several of the nontest measures we analyzed did pass muster, though not all did; and importantly, we didn’t find any nonacademic measures that satisfied the bias tests in high school grades. In other words, simply applying the new measures to the old models will likely not work for all students and all schools.

Ultimately, the mandate for states to incorporate new nonacademic indicators presents an opportunity for states to break school accountability’s close association with standardized testing. I also urge states not to squander this opportunity by inadvertently reinforcing its prior association with student poverty.