Memo: A shift in the federal role is needed to promote school improvement

Changes in federal education law under the Every Student Succeeds Act (ESSA) and emerging knowledge on more effective approaches to education reform have cleared the way for a potentially significant shift in thinking about strategies for improving educational systems across the United States. The shift has already begun in a handful of states and districts that are trading in the usual flurry of disconnected initiatives and faith in “magic bullets” for the systematic application of improvement science to get better at the work of teaching and learning. Federal policies can and should do more to support the development and spread of these practices while being less intrusive than they have been in recent years.

Download a PDF of this memo.

THE SITUATION

Untested initiatives, often based on little more than hunches, abound today across education as the field seeks to alter the aims of instruction from rote learning to learning skills, a curiosity for knowledge, and critical thinking. Some of the initiatives originate from new, yet incomplete, research evidence, while others have little evidence base at all. Either way, these initiatives often become a panoply of top-down mandates thrust upon teachers, schools, and districts without regard to local contexts and conditions, existing initiatives, and the preparation required to implement them at the scales envisioned in the short time periods allotted. This is an all too common recipe for failure. 

The variability among states, districts, and schools poses another daunting national problem: how to assure reliable, quality educational outcomes, day in and day out, for different subgroups of students across the diverse backdrops of schooling in America?

Additional resources may be necessary, but we also know from past experiences that simply adding more money, more materials, more technology, or even more people doesn’t guarantee improvement. Neither does adding new interventions that may have been shown to work in one situation but haven’t been tested to determine if they’re likely to succeed or be adaptable in other contexts. Put simply, what works in some places often doesn’t work in many others.

The lack of generalizability raises a core concern: how can organizations get better at what they do while confronting a continuing stream of new demands posed by rapidly changing external environments such as new policy requirements or shifting economic conditions?

A new context for improvement

This memo addresses the question of how federal and state policy can help build the infrastructure to support a systematic and continuous improvement approach to find solutions for many of the major educational problems facing the country. In order to substantially increase achievement, college readiness, and graduation for all students, especially low-income, African American, and Hispanic students, U.S. classrooms, schools, districts, and state offices must become continuous improvement organizations.

Passage of the ESSA law provides part of the answer. ESSA does two things that support the improvement effort. First, it reduces the intensity of top-down pressure and cedes a great deal more responsibility to the states and districts to address their problems of improvement. Second, it continues efforts of the federal and state governments over the past decade to create high-quality data systems throughout the nation and to train people to use them.

The evolution of improvement science itself holds new answers. After 60 years of being honed and refined by industry, improvement science has increasingly been adopted by social services, with dramatic efforts in health care around effectiveness, efficiency, and quality of patient experience. Improvement science promotes this adoption through its disciplined focus on processes:

• Seeing the complexity of the system that creates unsatisfactory outcomes in order to identify workable solutions
• Conducting rapid, small-scale testing of proposed interventions and embracing failures as learning opportunities
• Continuously testing outcomes with data, then revising, redesigning, and iterating to achieve quality outcomes reliably across varied local conditions

These ideas have been part of the education literature under many names and in many forms. In recent years, there has been a substantial upturn in the number of schools and districts applying the basic principles, methods, and tools of improvement science to implement change with notable successes in student achievement and motivation and professional collaboration. Districts have also improved efficiency and effectiveness of Human Resources, IT, and other operational departments that directly or indirectly support the efforts of teachers, principals, students, and their families.

The major challenge now is to take these strategies to scale—to reach more education systems and to implement them more deeply to benefit both the children and adults in those systems.

Effective continuous improvement within educational organizations

We begin with two quite different examples of systems of schools that have applied improvement strategies to increase their effectiveness and efficiency.

A turnaround in suspensions: The School District of Menomonee Falls, Wisconsin, had a higher suspension rate than nearby districts in the 2010-11 school year. The district used improvement science strategies to implement restorative practice and positive behavior interventions and support projects.

Understanding a situation from the users’ perspectives is a key principle in improvement science in education, just as it is in successful businesses all over the world. The school district focused on identifying situations where inappropriate behavior most often happens and worked to prevent it by directly engaging students in the problem-solving process. District teams documented changes and continuously refined and adapted strategies based on a rapid series of iterative tests referred to as the Plan, Do, Study, Act (PDSA) cycles. Every 45 days the principals reported the results to the School Board, analyzing which tools were making the greatest impact. As a result, students are no longer removed from their schools and the learning culture in the schools has shown marked improvements as documented through periodic student, staff, and parent survey data. In the five years since Menomonee Falls introduced these measures, the district’s suspension rate has fallen by 63 percent.

Bringing the college promise to all: High Tech High, a renowned system of 13 public charter schools in San Diego County, California, serving a racially and socioeconomically diverse student population, saw significant gaps in college readiness and college-going rates between students of color and low-income students compared to their wealthier and white classmates.

Using improvement science methods and tools, High Tech High identified key processes that were impeding student success and iteratively experimented and measured changes to determine the most effective improvements. In three years, High Tech High reduced the gap in honors course completion between students of color and white students from 18 percent to 2 percent; lowered the failure rate among young men of color from 7 percent to 1 percent; increased overall four-year college attendance from 67 percent to 73 percent, cutting in half the gap for low-income students; and cut chronic absenteeism from 11 percent to 2 percent. High Tech High now uses the improvement science approach as part of a continuous process to review how well its school system is doing at making education better for all students.

Other notable examples of continuous improvement district approaches are taking place across the country and around the world. Public schools in Fresno, Garden Grove, and Long Beach, California; Austin, Texas; and Montgomery County, Maryland; and colleges, such as Georgia State, have improved student achievement and graduation rates, increased the number of students taking college preparatory courses, and improved the quality of district support systems, including the human resource departments in their districts. Efforts to bring improvement science into education are now underway in such numerous and varied countries as New Zealand, Australia, Singapore, Chile, and Sweden.

Using networks to accelerate improvements at scale

Educational problems such as those illustrated above are widespread across the nation’s schools. Improvement efforts, when they occur, have been based almost entirely on the initiative of local education officials. Typically, each teacher, school, or district tries to solve these problems on its own as if no one else shares the same problem, and without access to (or building on) the progress that others have already made. This is a very slow mechanism for learning to improve and inhibits realized gains from spreading easily.

In the last 10 plus years, we have begun to see growth of a second approach for achieving greater improvement success at scale—networked improvement communities (NICs). Their strength is drawn from district leaders, teachers, and researchers working together and sharing the knowledge of their expertise to define problems, determine what caused them, and design and refine changes to address them. This collaborative process allows NICs to more easily innovate, accelerate progress on complex problems, and more rapidly diffuse results. Coordinated small tests of change may occur across diverse sites and, based on the results, the network revises, refines, and tries again and again until it gets positive outcomes to occur reliably across diverse conditions. Following are two of the successful examples of this approach.

The Strategic Education Research Partnership (SERP) collaborated with the Minority Student Achievement Network, a national coalition of suburban and small urban districts, to address the longstanding achievement gap in Algebra 1 classes in different schools. Researchers and teachers from eight districts joined together to explore what created the gap and how various districts tried to remedy the problem. This information gave them evidence to design and conduct rapid testing cycles on a curricular plug-in called AlgebraByExample that could work in a variety of classrooms.

A subsequent field test of AlgebraByExample in the networked schools found that students who used the program had higher gains on state assessments than peers with the same teachers who did not use the program. Additionally, students at the lower end of the performance distribution had the greatest increases. As a result, this work has now spread to numerous districts well beyond the Minority Student Achievement Network.

In 2011, the Carnegie Foundation for the Advancement of Teaching began to work with a diverse group of academic researchers and college teachers to develop two alternative pathways to success for college students in remedial courses called Statway and Quantway. Designed to use continuous improvement principles to target low passage and matriculation rates among students placed in non-credit bearing remedial courses, these pathways show considerable promise and are now implemented in more than 50 colleges. Based on one of the author’s analyses, students in these pathways are estimated to be three times as likely to pass developmental math and earn college math credit within one year as compared to what a matched sample of traditionally-remediated students achieve in two years. Moreover, improvements have been documented in almost all colleges, with most achieving success rates in excess of 60 percent. [Full disclosure: Two of the authors work for Carnegie, which is a nonprofit corporation that offers Statway and Quantway on a fee-for-service basis.]

RECOMMENDATIONS

The goal of new federal policies should be to nudge and support state and local systems in the direction of continuous improvement and expand the infrastructure of research-practice improvement networks. This goal leads to two major recommendations.

1. Create an improvement infrastructure that supports schools, districts, and states.

A compliance orientation around implementing programs and policies pushed down from above has become normal in schools. A major change is needed. To solve the problems ahead, local educators must become active agents in improving their own work. The federal government needs to enable and support them to make their schools work better.

ESSA resources can be used to develop more effective and improvement-oriented state agencies along with their regional and county offices. Likewise, these resources can be used to deepen the improvement capabilities of current school district staff. The U.S. Department of Education should make clear in guidance to states and districts that in almost all instances it is appropriate to use ESSA funds for these purposes.

The new administration might also consider small incentive grants to colleges and universities to include specific preparation in improvement research in all teacher, principal, and superintendent training programs. This proposal could be accomplished in the Higher Education Act, which is due for reauthorization. It fits with the capacity-building function of the federal government and with other recommendations from the project (see this Friday’s memo from Doug Harris about the federal role in research).

Complementing a direct focus on strengthening state and local capacity, the federal government needs to catalyze stronger engagement of educational researchers in practical problem-solving. Federal efforts for more than a decade, principally through the Institute of Education Sciences (IES), have created incentives for researchers to undertake rigorous studies of programs and policy impact. A similar initiative should now be undertaken to bring relevant academic expertise into more active engagement with school improvement. This can take many different forms, including: a) expanding the scope of the Regional Labs; b) increasing the funding for IES and other agencies, such as the National Science Foundation, to support research-practice partnerships; and c) providing direct support for Networked Improvement Communities focused on solving high-leverage problems as illustrated above. There are a variety of other sources for funds that support a better understanding of how to implement improvement efforts including, for example, the evaluation provision of ESSA, Section 8042.

2. Creating cultures of improvement and new forms of accountability at state and district levels.

For 22 years we have lived with an accountability system that relies primarily on one source of data—student test scores—collected once per year. Schools and districts were held accountable for substantially improving these outcomes, often without the necessary support and capabilities to have a realistic chance of actually achieving these goals.

In a culture of improvement, everyone is expected to continually audit themselves to ensure that their work and their organization is as effective and efficient as possible. Everyone has a stake in the quality of the organization and, therefore, everyone is obligated to participate in continuous improvement. With the end of NCLB, the states and districts are in a position to develop new forms of accountability specifically designed to promote school improvement.

Such accountabilities require broader and more diverse types of data to determine the quality and effectiveness of internal policies and practices of schools and districts, in addition to student achievement and attainment. Many states, such as California and New Hampshire, are already exploring new accountability systems that make their environments friendlier to a culture of improvement. The Department of Education should encourage such innovation.

Information produced by inspection systems could be a significant contributor in this regard. Other countries and some states use an inspection system to review the quality and effectiveness of schools and districts. The systems use formal rubrics or protocols to produce consistent measures of quality, which inspire improvement at the local site. In New Zealand and the Netherlands, teams of reviewers make periodic visits to each school and write public reports highlighting strengths and weaknesses. When they find evidence of shortcomings, they require the school to develop and initiate a plan to address the problems. The school is then reviewed again after some time to assess the progress toward improvement. Although some inspection systems have been clearly summative and judgmental, others are continually evolving. New Zealand’s system, for example, is specifically intended to be more formative, with the inspection process itself driving and supporting school improvement. New York state is currently carrying out a trial inspectorate system in a variety of districts.

Federal financial support to states and districts for efforts such as these is important because these improvement activities will require additional resources to get off the ground and may entail ongoing expenses beyond traditional test-based accountability reports.

CONCLUSION

We live in an era in which schools are under extraordinary pressure. The idea that each school or district is left to its own devices to improve yields a weak mechanism, one that guarantees continued great variability in performance. Such variability typically shortchanges those who are already most disadvantaged. The federal government can help overcome this disparity by supporting policies that build new infrastructures for improvement.

Support for this project was generously provided by the Spencer Foundation. Brookings recognizes that the value it provides is in its absolute commitment to quality, independence, and impact. Activities supported by its donors and by outside contributors reflect this commitment. The authors did not report receipt of financial support from any firm or person for this memorandum or from any firm or person with a financial or political interest in this article that creates a conflict of interest.

Suggested citation:

Bryk, A., Ladd, H. F., O’Day, J., & Smith, M. S. (2016). A shift in the federal role needed to promote school improvement. In M. Hansen & J. Valant (Eds.), Memos to the President on the Future of U.S. Education Policy. Washington, DC: Brookings Institution.