The Government Reform Series: Performance, Evidence and the Missing Link

Editor’s Note: Government failure is something everyone complains about, but does little to address. Over the next two weeks, FixGov will review work on government reform: identifying problems in the federal government and offering solutions to get government back in working order. In this post, Elaine Kamarck reviews 
The Performance Stat Potential
, 
by Harvard Professor Robert D. Behn and 
Show Me the Evidence: Obama’s Fight for Rigor and Results in Social Policy
, 
by Brookings scholars Ron Haskins and Greg Margolis. 

In recent years, those seeking an end to the seemingly never ending battle over the size and shape of the U.S. Federal government have sought to measure and quantify government performance.  Beginning in 1993 with the passage of the Government Performance and Results Act (GPRA), governments all around the country adopted measurement systems designed to track and, ostensibly, improve, government performance.  The Obama Administration continued the movement with amendments to GPRA in 2010 and with the formal introduction into legislation of evidence based policy making.

Two recent books chronicle the state of these efforts today.  In The Performance Stat Potential, Harvard Professor Robert D. Behn, tracks the “CompStat Craze.”   Compstat refers to the New York Police Department’s revolutionary use of crime data.  The success of this program in turning New York City from one of the most dangerous cities in the country to one of the safest in the world, created a veritable tsunami of copycat programs throughout the country and the world.  The Compstat craze swept not only police departments but city governments with the introduction in Baltimore of “Citistat” to “BlightSTAT” in New Orleans.  Suddenly governments everywhere were collecting data and having meetings modeled on those at the NYPD.

A second book Show Me the Evidence: Obama’s Fight for Rigor and Results in Social Policy, by Brookings scholars Ron Haskins and Greg Margolis details how the Obama Administration has sought to inject evidence into six government programs.  The heart of evidence-based policy making is the inclusion of random-assignment designs and a process for evaluation into the policy architecture.  The goal is to see whether or not the program actually makes a difference in the well-being of those included in it compared to a control group of similar people.  The removal of selection bias is key to trying to determine whether the programs work.

Both movements, the performance movement and the evidence-based policy movement, are extremely valuable.  They are popular among academics and technocrats. However, the real question is whether they will succeed in improving outcomes.  As all the authors admit, these are simply tools – tools that can help or hinder progress depending on the twin intangibles of leadership and politics.

Behn points out that “The Data Never Speak for Themselves.”  They only speak “…through a framework, a theory, a cause and effect concept.”  Because data qua data is simply that, when performance data is successful it is because it has been used to transform an organization into a learning organization.  Compstat in New York City worked because it helped “…to accelerate ‘the process of learning what does and does not work.’”  Finally, Behn points out that “a performance target is a political target.  The public executive who establishes an explicit, public, performance target is painting a bull’s-eye on his or her back.”  Thus, performance targets only really work when leaders commit to them, and for many of them it is a “dangerous commitment.”  It is much easier to assert that forces outside of the organization are responsible for outcomes.

Haskins and Margolis’ caveats about evidence-based policy making center upon the intrusive nature of politics.  The first caveat is that rigorous evaluation may often show only small positive changes from a given program or policy.  This is likely to disappoint supporters of those programs at the local and national level and to provide fuel for those who oppose them.  A second caveat is the risk that a concentration on evidence will push out the need for innovation.  And a final problem is illustrated by the conflict that surrounded the “Maternal, Infant and Early Childhood Home Visiting Initiative.”  For many reasons politicians tend to have their favorite programs, and they want to support them regardless of what the data might say.

The desire to be rigorous about government performance and policy outcomes is important.  But in both instances, the rational world of numbers runs into the irrational world of politics and perception.  Responsiveness to data comes in two forms: internal, administrative and operational improvements and legislative action to kill bad programs and/or expand good ones.  It is not surprising that performance management has been most powerful at the city level where most goals are simple and operational (pot holes filled, for example) and where operations can be improved without running into as much political debate.  And it is not surprising that at the legislative level, politicians routinely ignore social science data in order to sustain a program that they helped shape or that their constituents love.

With few exceptions, the missing link to the performance movement is the politicians themselves.  In an ideal world, they would utilize objective measurement rather than parochial concerns, but if we are realistic about electoral incentives we see how tough this is.  Nonetheless, modern governments are rapidly accumulating data and evidence, and the emergence of big data will only increase this body of knowledge.  Unlike the best program administrators who use data in their day-to-day operations, legislators are probably not going to start using data in every budget cycle.  Nevertheless, there come times in American politics, “punctuated equilibriums” as the political scientists Frank Baumgartner and Bryan Jones write, when the paradigm changes and then the political system opens up to new ways of doing things.  If these moments can be caused, or at least shaped by, the kinds of data and evidence these authors write about we will be better off.