Lessons in using data to improve education: An Australian example

When it comes to data, there is a tendency to assume that more is always better; but the reality is rarely this simple. Data policies need to consider questions around design, implementation, and use.

To offer an illustrative example, in 2010 the Australian Federal government launched the online tool My School to collect and publish data about the nearly 10,000 schools across the country. My School provides a “report card” for Australian schools, by collating and presenting data on each school’s financial resources, the socio-economic background of students, and academic performance.

My School is a unique achievement. Before its creation, public information was limited to what schools chose to make available, and even State and Federal governments (let alone the general public) had trouble accessing comparable information about school performance and resources. This information is now publically available for all schools across the country and updated every year on the My School website.

A new case study report, drafted by graduate student researchers at Columbia University, attempts to understand the role of My School data in Australia. The report explores who uses these data; decisions or compromises made to roll out the system; and lessons for transparent data systems globally.

In 2013, approximately 790,000 unique users visited the My School website. Parents use the site to make decisions for their children about changing schools, starting school, or moving between elementary school to high school. My School has also influenced policymaking, aiding in the development of major school funding reforms by enabling the comparison of school funding across states and sectors. Many education researchers, too, use My School data to compare performance of schools over time.

My School was able to overcome early opposition largely due to strong support from influential policy leaders. Julia Gillard, who transitioned from education minister and deputy prime minister to prime minister of Australia during My School’s implementation, is widely acknowledged as the driving force behind its success. In Gillard’s own words: “I fought a ferocious battle… to create My School and to get each of us, all of us, more information than we have ever had before on the education of our children.”

Further, a unique period of alignment between the Federal government, which sought to collate the data, and the state and territory governments that controlled the data helped bolster My School. At the time when My School was negotiated and implemented, the Centre-Left Federal Labor government enjoyed the support of Labor governments in seven out of eight states and territories. This provided a unique window for initiatives requiring close cooperation between levels of government. The federal government was also able to use financial levers to encourage state and territory participation in My School, by tying the ongoing allocation of federal funds to collection and publication of school data.

Not all stakeholders welcomed My School, however. Teachers unions were particularly vocal critics. The unions argued that publication of school performance data would unfairly stigmatize disadvantaged schools and create a high-stakes learning environment, akin to similar backlash against the No Child Left Behind Act in the United States. There were also concerns that that My School would make classrooms less inclusive and less able to cope with a diversity of students and performance levels. Supporters of My School, however, have helped make schools and teachers comfortable with the tool through frequent communications and trainings.

Debates also arose about protecting My School data. On one hand, critics were concerned that news media and other parties would use the tool to create league tables, unfairly comparing schools without providing enough contextual information. These critics called for strict technical and legal protections against such use. On the other hand, members of the open data community criticized such protections as being unduly restrictive and argued that controls over use of the data would contravene My School’s stated objective of opening up school data to the general community.

These debates shaped the design and content of the tool. To guard against league tables, creators added technical controls to prevent data scraping, legal controls prohibiting the unauthorized use of content, and bureaucratic controls limiting access to raw data. They also only allowed schools to be compared if they served similar populations, which is evaluated  through the Index of Community Socio-Educational Advantage—a measure of student factors (such as parents’ occupation and education), as well as contex­tual factors (such as the school’s geographical location, the proportion of indigenous students, and the proportion of students with language backgrounds other than English).

Importantly, My School is a living tool. Officials regularly make improvements to the functionality of the site and update the breadth of data. An important lesson shared by the architects of My School is that you do not need to wait until you have a perfect tool to launch. Launching the site provided valuable feedback on what data users what and the format that they want to see it published, enabling rapid iteration.

My School is a strong model for data publication as a tool to improve transparency and accountability in education. For other countries looking to share education data publically, My School also demonstrates the need to consider how we collect, present, and protect education data. When it comes to data, more is not always better.