Probing charter schools’ differential success in urban and nonurban settings

With the nation’s first state charter school law, from Minnesota, approaching its 25th birthday, we will soon hear more about the past, present, and future of U.S. charter schools.  Charter schools are publicly funded, autonomous schools that typically operate alongside district public schools as tuition-free alternatives for families.  Few topics in education have attracted as much attention or controversy as charter schools over the last quarter-century, with much of the discussion focused on the effects of these schools on American students and public education. 

This post takes a closer look at what we know about these effects.  In particular, it examines the emerging finding that charter school performance varies considerably by school location and student demographics.  Urban charter schools that serve disadvantaged students are tending to produce more positive effects on student achievement than other charters, at least as measured by state test scores.  This distinction sometimes gets lost in policy debates that treat charter schools as a monolithic entity and strive to define charter school effectiveness in a single word.  But it is a distinction that reveals a great deal about today’s charter schools and how they serve their students.

Some of the differences between charter and district schools that make it important to compare their effects on students also make it difficult to do so.  Most challenging is the difference in how students enroll: by virtue of where they live for district schools and via some type of application for charter schools.  If there is a gap in the performance of charter and district students, even within the same city, should we attribute that gap to differences between schools or differences between the families that enroll in charter and district schools?  It can be hard to tell.   

Researchers have taken a few approaches to isolating the effects of charter schools.  Some have attempted to control for differences between who attends charter and district schools.  Others have looked at charter schools that receive more applications than they can accommodate, comparing the outcomes of students who were randomly selected for a seat to students who were not.  Still others have focused on students who switched between charter and district schools, examining how they performed in each setting. 

Each of these strategies has strengths and weaknesses, and there is some notable variation across studies in their results.  Perhaps the most interesting variation, however, exists within these studies.  Many of them examine both urban and nonurban charter schools that serve varying proportions of minority and disadvantaged students.  Consistently, urban charter school students, and charter school students from low-income families, outperform their district school comparison groups on state tests.  Nonurban charter school students, and higher-income students, do not.

This pattern is evident in the work of the Center for Research on Education Outcomes (CREDO), which has assembled the most comprehensive data on charter school students across the country.  CREDO compares charter students’ scores to the scores of matched students in nearby districts.  Their nationwide analyses show similar test score gains for charter school students and their matched district school peers.  But urban charter school students, they find, show substantially more test score growth than their peers (differences that CREDO equates to 40 additional learning days per year in math and 28 per year in reading).  

The pattern is also evident in studies comparing charter school lottery winners and losers.  A group of MIT researchers studied Massachusetts schools, finding test score gains for urban charter schools, particularly among minority, low-income, and previously low-scoring students.  However, nonurban charter schools generated no such gains and in some cases reduced student achievement.  Researchers from Mathematica Policy Research and the U.S. Department of Education saw similar results in a lottery-based study of 33 charter middle schools across 13 states.  They observed generally positive effects for urban charter schools and low-income students, and generally negative effects for nonurban charter schools and higher-income students.

But why?  If charter schools’ defining features—parent choice, and increased autonomy for accountability—are common across urban and nonurban areas, then why might the results differ so sharply between them?  Answers to this question remain speculative, but a couple of possibilities stand out.

• First, it could have to do with the district school comparison groups for urban and nonurban students.  In these studies, urban charter schools are generally compared to district schools with much lower baseline scores than nonurban charter schools.  For example, CREDO found relatively large positive effects for Detroit charter schools.  The 2015 NAEP reports 4 percent of Detroit’s 8th graders scoring Proficient or better in math and 7 percent scoring Proficient or better in reading.  Of course, many of the challenges confronting Detroit Public Schools students also confront Detroit charter school students.  But to the extent that the lowest-scoring district schools have the most room for improvement, nearby charter schools are well positioned to look successful by comparison.

• Second, and related, urban and nonurban charter schools might pursue different goals.  Perhaps most urban charter schools strive to be better than their local district schools, while nonurban charter schools strive to be different.  Charter schools need to actively attract families to enroll.  A school competing with low-scoring district alternatives might find unmet demand for schools that effectively teach core academic subjects.  On the other hand, a school competing with higher-scoring district alternatives might find unmet demand for schools with unconventional missions, themes, pedagogies, or disciplinary philosophies.  While urban charter schools may have incentives to adopt “No Excuses” practices likely to improve test scores—and more flexibility than district schools to do so—nonurban schools may have incentives to do the opposite.

Of course, this list of explanations is far from exhaustive.  For example, there are differences in who creates, manages, staffs, and supports urban and nonurban charter schools, and urban charters might benefit from resources unavailable to districts and nonurban charters.

Regardless of its explanations, charter school heterogeneity has implications for education policy and research.  Charter schools are not monolithic, and assessing what charter schools do—and how well they do it—requires nuance and understanding of context.  Moreover, there are consequences to being reductive and narrow in the measures we use to evaluate performance.  Overemphasizing test score gains could be inappropriate and debilitating for charter schools that can fill valuable educational niches only by limiting their focus on state tests.  At the same time, overemphasizing test score gains could give undue credit, and create unhealthy incentives, to charter schools that pursue those gains at the expense of other valuable outcomes.

These are just a few of the issues sure to be discussed as the 25th anniversary of America’s first charter school law draws near.