Sections

Research

Making Measurement Matter: The Challenge and Promise of Building a Performance-Focused Environmental Protection System

Shelley Metzenbaum
SM
Shelley Metzenbaum Nonresident Senior Fellow - Volcker Alliance

October 1, 1998

Executive Summary

Governments across the country and around the world are rapidly moving to adopt
performance-focused approaches to management and decision-making. Entire national governments,
including the United States, Australia, and New Zealand, have adopted laws requiring agencies to
adopt performance measurement and management as a way to strengthen performance and
accountability, inform public decision-making, and reinvigorate citizen confidence in government.
State and local governments in the United States, Canada, and elsewhere have been experimenting
individually and collectively for over a decade on the best ways to use performance measurement to improve management and governance.

Environmental protection is one area where interest in strengthening the emphasis on performance information has been especially high. Numerous experiments are currently underway to explore the best ways to realize the potential of a performance-focused environmental protection system. A few, such as the ambient air quality standards of the Clean Air Act, are relatively mature.

Many others are more recent. In 1995, for example, President Clinton announced Project XL, which offers regulated entities increased flexibility in return for improved environmental performance. Two months later leaders of the U.S. Environmental Protection Agency and state environmental agencies launched the National Environmental Performance Partnership System (NEPPS). NEPPS replaces a process-focused framework that prescribed allowable state activities funded with federal dollars with a performance-focused framework. This system is designed to encourage attention to the highest priority environmental issues and to invite collaboration between each state and EPA in addressing those priorities. Heightened interest in environmental performance measurement is occurring not only in the U.S. but abroad and not only in government but also in business. The Dutch government, for example, recently established performance reporting requirements for certain companies, and many businesses are working collaboratively through projects such as the Global Reporting Initiative to develop a common set of business environmental performance metrics.

Factors Motivating Change. Rising enthusiasm for performance measurement and management as a means for dealing with public problems and, more specifically, for enhancing environmental protection is driven both by a compelling need to improve the existing system and by the promise of performance measurement as a powerful tool for bringing about that improvement. Several factors underlie the recent surge of interest in performance-focused environmental protection approaches: a downward trend in the levels of public support for government over the past quarter century; frustration with the inability of current approaches to environmental protection to deal with emerging and some existing environmental problems even as public commitment to the environment has grown; and frustration with organizational rigidities in environmental agencies. In addition, performance measurement holds promise as a means to respond to the increasing inclination to devolve government implementation responsibilities to states and localities in order to tap their growing management skills and to avoid service delivery problems associated with “bigness” while still protecting the national standards that are needed to deal with cross-boundary issues, local political pressures, and basic environmental and public health.

Increased interest in performance-focused, information-rich systems has been further facilitated by dramatic changes in technology that make it far more affordable and technically feasible to measure performance and to transmit, aggregate, analyze, and disseminate performance information.

The Promise of a Performance-Focused System. Strong interest in performance measurement should not be surprising. Indeed, in some ways, what is more surprising is the paucity of performance measurement in government practice. How can managers and oversight agencies run programs without information about how well the programs are performing? How can people in the workforce exercise sound judgment without better information to inform their daily decisions and allow them to learn not only from their own experiences but from that of others in comparable situations? In some ways, a performance-focused system seems almost an inevitability.

An effective performance focused system can improve the way we address public problems in several complementary ways — by boosting outcomes, strengthening accountability, and enhancing the transparency of processes and decisions that affect the public’s well-being. Simply by creating increased awareness of problems and sharpening organizational focus, performance measurement can advance program outcomes. Communities cannot organize to fix a problem unless they know it exists and individual offices in an agency are more likely to focus resources on an agency’s priorities if they clearly understand what the priorities are and know that their offices’ performance in meeting those priorities will be measured. By focusing on results and not processes, performance measurement allows adaptation and encourages innovation. By linking performance measurement to appropriate incentives, performance measures motivate performance improvements. By analyzing the connection between different intervention strategies and results, performance measurement contributes to organizational learning which in turn enhances performance..

Performance measures also strengthen accountability by providing a common language to clarify expectations between two parties about the level of performance promised or expected. By agreeing on performance measures, they establish a common metric for reporting on and determining the status of agreed-upon or expected deliverables, whether between appointed officials and elected officials, elected officials and the electorate, the government workforce and their managers, one governmental body with another, contractors and the government, businesses and the public, or among government workers.

An information-rich, performance-focused system can also boost the transparency of the system if performance information is shared with the public. This, in turn, strengthens accountability, enhances efficiency, and improves the quality of public decision-making. The value of performance measurement and management has been widely demonstrated in the private sector. The framework for public financial corporate performance reporting established by the Securities and Exchange Commission serves as the cornerstone of a thriving American economy. Corporate leaders use internal and comparative performance measurement to guide daily management decisions. The availability and consistent use of performance measures is a healthy and constant pressure to improve the quality and price of private sector services and products, and hence enriches the quality of life for those using those services and products. Performance measures also help many people in their daily decision-making, so much so that some performance measures are taken for granted. Performance measurement of consumer products collected and disseminated by magazines such as Consumer Reports and PC World regularly inform purchasing choices throughout the country.

Public sector users of performance measures face a different situation than that faced by those who use performance measures for private decisions because governments tend not to function in a competitive environment. The challenge is to translate the dynamic mechanisms that make private sector use of performance measures so powerful to the public sector without introducing such significant political problems for those being measured that it overwhelms their ability to function effectively.

The performance-focused system that is most likely to realize its full dynamic potential is one that simultaneously employs performance measures in three distinct ways: as targets; for comparative and benchmark purposes; and to facilitate experiential learning. Probably the most familiar sort of performance measurement system is one that uses performance measures to set targets which performers are expected to meet, linking rewards and penalties to different performance levels as an incentive or accountability mechanism. This is the approach used in the Clean Air Act, the Government Performance and Results Act, and the Australian performance system.

The dynamic capacity of performance measurement is unleashed when it is used to compare the performance of one performer to another because it creates a mechanism that automatically updates performance expectations whenever new measurements are taken, thereby motivating continual performance improvement without necessitating a complicated and often lengthy decisionmaking apparatus to update targets. This is an approach that nearly a hundred localities, working with the International City/County Management Association, are testing.

Performance measurement has even greater value when it is incorporated into analyses that allow organizations to learn from theirs and their peers’ experience. Simple comparative analysis facilitates learning by identifying top performers to “benchmark.” More sophisticated analyses that probe the links among inputs, outputs, and performance results allow an organization to learn by identifying the strategies that have yielded the best historical results. The Agricultural Extension Service is perhaps the pre-eminent example of a system that has effectively used analysis of performance measures to enhance outcomes.

Common to all three approaches, and critical to their effectiveness, is the use of performance measures to motivate improved performance and inform management and resource allocation decisions, along with broad dissemination of performance information. What distinguishes them is the way the performance measures are used. A well-functioning performance-focused environmental protection system will not employ just one of these three approaches but rather use them all, applying each as appropriate to performers, users of performance measures, and specific uses. Performance measures used as targets are, for example, appropriate when a unique organization is being measured—such as an EPA program office setting a national goal—or when two parties—such as two states that share a watershed—need to cooperate. When customers or investors seek information about environmentally strong products or companies, comparative performance measures can function as an enormously powerful tool. Comparative measures can also be a powerful tool when similar government agencies seek to identify strong programs to use for benchmarking or to spotlight weak programs requiring attention. Once strong programs have been identified, performance measures can be used together with other information to analyze the factors contributing to program effectiveness. Used together, the three approaches generate the dynamic capacity to improve performance, accountability, and transparency.

The Challenge of Implementing a Performance-Focused System. Enthusiasm and technical breakthroughs, even when bolstered by legislative mandates, will not transform the existing system to realize the full potential of a performance-focused system. Significant implementation challenges must be surmounted. These include the technical challenge of building a system that can gather and deliver performance information that is reliable and useful; the organizational challenge of getting managers, the workforce, and oversight authorities not only to use the measurements but also to use them sensibly and sensitively to motivate improvement; the political challenge of getting elected officials, the press, and the public to use performance information constructively and mindful of the danger of using them sensationally; and the human challenge of taking collective action. For a performance-focused system to work, attention needs to be directed to tackling all of these concerns.

Implementing an effective performance-focused system requires the effective execution of many discrete activities. These include the selection, measurement, reporting, verification and standardization, collection and storage, analysis, presentation, and dissemination of performance measures. Each discrete activity must work relatively well for the system as a whole to realize its full potential.

This creation of this capacity will not occur by law or fiat. Instead, it will require the understanding, effort, commitment, and experimentation of large numbers of people and organizations who are already part of the system so that they can and will work individually and collectively to establish and maintain the various activities. That understanding and commitment is not yet widespread. Moreover, implementation of the system is likely to encounter active resistance or indifference from a few managers, workers, and even oversight parties who distrust change. Thus, a key challenge in implementing a performance-focused system must be helping managers, workers, elected officials, candidates, shareholders, customers, the press, and the public appreciate the utility of performance measures and the potential power of performance measurement for achieving enhanced results.

Perhaps the greatest challenge to the whole effort to implement a performance-focused environmental protection system is political. Efforts to create performance-focused environmental protection programs are being attempted in a highly charged political atmosphere. This raises the possibility that opponents of existing levels of environmental protection will try to use the reform effort to reopen questions about levels of performance standards. Any attempts to use performance-focused systems to justify relaxation of standards will threaten the viability of this approach and should be resisted broadly and vociferously by all proponents who appreciate the potential for gain that a performance-focused, information-driven system promises. It also raises the possibility that as performance-focused experiments proliferate throughout large bureaucracies, the flexibility message will resonate more loudly than the message of improved results, resulting in gains in flexibility without counterbalancing gains in environmental outcomes, accountability, and transparency. This danger is real and warrants serious attention and discussion by management to assure that staff understand the need to link flexibility with effective accountability mechanisms and information reliability and dissemination requirements.

Performance-focus as an Option. A performance-focused approach may not work for every organization. Some companies and even some small government entities may be unwilling or unable to assume the risk of operating in a performance-focused world. Smaller organizations, and even many larger ones, may not be able or interested in assuming the costs of experimenting with different strategies to achieve environmental performance targets. Many would and have preferred for the government to prescribe the processes they need to follow to be in compliance with the law. Forcing these firms (or governmental organizations) to take on the risks associated with a performancefocused system may not add greatly to the public benefit; instead, it makes sense to establish a twotrack system with a prescriptive track for risk-wary entities.

For other organizations that can tolerate greater risk associated with their environmental activities, a performance track may be attractive because of the flexibility it offers. This may be especially true for companies required to seek frequent permits or permit revisions and government agencies seeking to innovate who find themselves caught in extensive debates with EPA about process changes. For these organizations, and for those who recognize that performance measurement will help them manage more effectively and responsibly, or who seek the publicity value of strong environmental performance, a performance-track should be created.

Not everyone should be eligible to pursue the performance track if it also offers less oversight and case-specific review. In these cases, participation should be limited to, for example, facilities with strong historic compliance records that prepare environmental reports that provide state-of-the-art information using standardized metrics whenever possible and whose historic environmental emissions levels have been better than that which would have been required at minimum compliance levels. Specific qualifying criteria have yet to be developed for such a performance track, precluding the possibility of initiating a full-fledged performance-track approach in the near future. The initiation of such an approach could be greatly accelerated if those interested in pursuing the performance track, whether businesses or governmental bodies, would develop concrete proposals defining performance thresholds.

Realizing the Possibility. There is no right way to build a performance-focused, information-rich environmental protection system. It will inevitably be a trial-and-error endeavor. Yet the tremendous potential of performance measurement and management makes that endeavor worthwhile. We have begun to create a system that is in its infancy with enormous possibilities and significant gaps. The foundation is being put into place that can support a thriving system that continuously drives improved performance while affording far more flexibility than the current system. At the same time, gaps in the foundation need to be filled. It is time to engage each other in a discussion about what those gaps are and how to fill them so that a dynamic, healthy, thriving performance-focused environmental protection system will be the dominant mode of operation in the next decade.