This article originally appeared on The Hill on February 22, 2017.
Andrew R. Feldman
Former Brookings Expert
Institute Fellow - Urban Institute
President Trump wants to bring a business-like approach to government. That will require much more information from rigorous program evaluation to determine what is working, what is not, and how programs can be improved.
What stands in the way of doing more program evaluation in the federal government? Lack of funds? Not always. Partisanship? Nope. Lack of data? Sometimes.
The barrier most frequently cited by government experts, based on our experience, is an obscure law called the Paperwork Reduction Act (PRA) that hampers government’s ability to improve programs, scale up effective approaches, and stop doing what isn’t working.
What is the PRA? The law stipulates that the White House Office of Management and Budget (OMB) must approve any information collection requests — in other words, forms, required reports, surveys — of 10 or more people, conducted by, or for, a federal agency. The request must first go through two public comment periods (30 days and then 60 days), then OMB review, and if approved, receive an OMB control number, which must be clearly displayed on the item before it can be used. In other words, the process takes a lot of bureaucratic steps and time.
The PRA is well intentioned. It ensures that when federal agencies collect information from businesses, the public, or organizations, it does not overburden those responding. For example, we do not want agencies sending out surveys willy-nilly to thousands or millions of Americans. (Not that there is any evidence they do that.) Requiring agencies to obtain PRA clearance for national surveys for statistical or census purposes makes sense.
Minimizing the burden on small businesses and the general public was the initial goal of the PRA. Program evaluation, on the other hand, which is essential for understanding how well programs work, was not the intended focus of the law. But because the legislative language was broad and referred to surveys, over time OMB has expanded the application of the PRA to program evaluations — even those without surveys or forms. Studies are caught in the PRA’s web like a dolphin in a fishing net.
The problem is that getting through the PRA clearance process can take a year or more, in part because there are so many requests for OMB staff to review. That time delay can have serious consequences, including sometimes making meaningful evaluations impossible. For example, let’s say Congress funds a new job training program for veterans. An important question for decision makers would be: Is it effective? The same question could be applied to existing programs, too.
To answer the question, the agency may decide (or be required by Congress) to conduct a rigorous program evaluation. Because of the PRA, the evaluation may not be able to start for a year or more after the program begins. At that point, it is impossible to collect baseline data on veterans going into the program because the program has already begun. But baseline data are essential to accurately measure the change the program makes. The result is often either doing an evaluation that is not as strong as it could be or the agency decides to forgo an evaluation altogether, if it is not required by Congress.
The costs of PRA are not just borne by federal agencies wanting to learn what works. They are ultimately borne by Americans who use public services that are not as effective as they would have been with better evidence, and by taxpayers who receive less value for their tax dollars.
The Trump administration should work with Congress to implement two reforms to the PRA related to program evaluation. These reforms are in the same spirit as those recommended by the Administrative Conference of the United States, which called on Congress to reform the PRA to refocus it on its original purpose.
First, the PRA trigger level should be raised to a sample size of 1,000 for rigorous program evaluations, up from 10 today. No federal agency should need permission from OMB to undertake continuous improvement activities that involve small numbers and no significant burden to the American public.
Second, program evaluations should be exempt from the PRA if an agency has strong standards and quality controls to ensure that evaluations are independent, rigorous, go through peer review, use existing high-quality data wherever possible, protect privacy, and are transparent by releasing reports publicly regardless of the findings. (A set of explicit criteria could be created to determine exemptions.) Today, that standard is met in evaluation offices of several agencies, such as the Departments of Education and Labor, plus the Administration for Children and Families in the Department of Health and Human Services. Any agency with those standards does not need OMB micromanaging and duplicating activities.
These two actions would ameliorate the roadblock that the PRA creates for evidence-based decision-making and reduce bureaucracy, but still uphold the purpose of the law. In doing so, the new administration could ensure that the Paperwork Reduction Act actually reduces paperwork, rather than increasing bureaucracy and reducing the ability to learn what works in government.