Policymakers shouldn’t underestimate the value of high-quality education data

Over the past 10 years, many states—including Tennessee, North Carolina, Delaware, and others—have transformed their education data systems by improving both quality and depth. In part, this has fueled optimism among education reformers about the role of education data in implementing effective reforms. The tremendous progress of some states, however, exists alongside the struggles that others have faced in building State Longitudinal Data Systems (SLDS) with basic features.

State databases are valuable resources for monitoring educational outcomes throughout the country. Policymakers in federal and state governments should prioritize the continuous improvement of these systems, which utilize data to inform efforts to improve efficiency and equity across the nation. In addition, much of the empirical research done in education over the last decade or so has directly benefited from SLDS investments by states and the Department of Education. This high-caliber data provides more clarity and precision about which policies affect which students, challenging previously held beliefs about teachers and schools. Improving SLDS will pay dividends for schools in terms of more targeted and actionable research.

The value of longitudinal databases

SLDS are databases that include individual-level records (students or teachers). The Education Department defines an SLDS as “a unit-level data system designed for collection, management, analysis, and reporting of statewide education data over time and across programs.” The most advanced SLDS link data from pre-kindergarten to labor market. The typical SLDS includes student-level records that are linked across one or more years.

The capacities of SLDS are thus directly linked to the quality of educational reforms like teacher evaluation. Estimating accurate teacher value added scores, a rigorous measure of teacher quality, requires longitudinal, student-level achievement data. For example, Lockwood and McCaffrey (2014) find that controlling for multiple prior test scores can help to ameliorate test measurement error.  Myriad current education reforms—including teacher evaluation systems, early warning systems for high school dropout, and continuous improvement efforts in districts—are possible only because of increased capacity to collect and analyze data in schools in recent years. SLDS allow researchers and practitioners to extend the time horizon of policy research beyond the K-12 setting, into college and potentially the labor market. In turn, research suggests that early warning systems may decrease student absences and decrease the risk of dropout. Furthermore, state education officials use these data to generate reports that guide high-level policy decisions about school administration.

The data are trapped in silos

A defining feature of SLDS is the connection between different state data systems across grades. However, linking data across time has proven to be a formidable challenge. State departments of education have found that in many cases, linking individual records has proven impossible. Many states that received federal grants were unable to overcome this basic technical hurdle and had to engineer entirely new systems.

There is considerable variation in the number of links between state data systems across the country. Only 16 states created a full P-20 pipeline (early childhood to the workforce development). Thirteen states have not connected any databases (early childhood, primary/secondary, and post-secondary). The inability of numerous states to link student records across time or systems implies a potential roadblock that may prevent states from effectively implementing policies that require such data. For example, states may not be able to track the effectiveness of their early childhood investments later into elementary and middle school, or whether high school reforms aimed at improving college readiness pay off.

Challenges for creating the pipeline

A primary challenge for improving the capabilities of SLDS is funding. The federal government started providing grants to states to develop their state data systems since 2005. The last round of grants was awarded in 2015 and will end in 2018. The federal grant program was intended to supplement state efforts rather than funding it entirely out of Congress’s pocket.

But, not all states committed the necessary funding towards this task. For example, Rhode Island has yet to appropriate dedicated funds for the growth and maintenance of the SLDS after winning a federal grant. Furthermore, federal SLDS grants were intended to subsidize the creation of new systems, not to provide for long-term maintenance. Without annual appropriations, staff that have technical and institutional knowledge may leave, which may weaken the continued effectiveness of past investments in SLDS. Proposed cuts to the Department of Education under the Trump budget, if implemented, may mean that states divert funds from SLDS development and maintenance to other programs in which federal funding shrank or was eliminated.

How to maximize data’s potential?

Given the current state of SLDS capacities, reformers eager to use the power of data to transform education systems should pay close attention to state (and federal) investments in SLDS databases. Access to high-quality longitudinal data is central to evaluating current policies, identifying potential policy solutions, and assessing the efficacy of reforms.

Several states recognize the potential value of a fully operational SLDS. Maine’s draft ESSA plan describes how the state intends to use the SLDS “to store and analyze crucial teacher, school, and student improvement data.” Other states recognize the challenge posed by inadequate data systems. For example, New Jersey cites in a draft of its ESSA plan the “[l]ack of access to data on educator preparation program quality” as a “root cause” of inequitable access of students to highly qualified teachers. State leaders clearly recognize the important role that fully operational SLDS can play in efforts to provide high-quality public education to all students.

The proposed Trump education budget does not suggest major changes in funds allocated for statewide data systems. While imminent cuts do not seem likely, neither does an influx of federal support to improve SLDS databases. In this scenario, concerted state-level investments to improve these systems may be key to implementing effective reform efforts. This investment may be a tall order for many states facing budget shortfalls, leaving open the question of which states will be able to develop and capitalize on longitudinal data systems.

Despite the clear benefits that SLDS offer, their value lies in the long-run payoffs that rigorous data analysis can provide. If states are forced to choose between spending funds on building high-quality data systems (spending to serve an abstract, long-term goal) or providing after school care to low-income youth (spending to meet observable, immediate needs), SLDS funding may dry up quickly—perhaps in states where investments in data-driven policy decisions are needed the most.