Finding meaningful IT metrics in government

Government spending on information technology (IT) has intensified over recent years in terms of dollar amount and strategic impact. Yet, we lack good measures to evaluate performance of our IT projects and to track outcomes associated with these IT investments.

There are metrics that the federal government uses through such programs as PortfolioStat, TechStat, and the IT Dashboard. However, each of these projects has inefficiencies that somewhat function as “check-the-box” performance monitoring without being completely effective on a continuous basis. There are also new legislative and administrative reform efforts that aim to restructure federal IT to improve performance such as the Federal Information Technology Acquisition Reform Act, or FITARA.

A Chief Information Officer in every agency

The most significant IT reform in decades, FITARA, was passed in 2015 to improve IT management by appointing an agency-level Chief Information Officer (CIO) with more authority and responsibility for producing projects that are on time and on budget. This new position will mitigate past issues of authority bottlenecks (i.e. HealthCare.gov development). FITARA strengthens some of the weaknesses of the 1996 Clinger-Cohen Act. Both acts were designed to have a CIO oversee IT investments that would be smaller, simpler, and clearly connected to the agency’s overall operations. However, CIO performance was often stifled through the Clinger-Cohen Act due to bureaucratic sluggishness.

Federal CIOs under FITARA will oversee such functions as hiring IT personnel, approving IT budgets, and signing IT contracts—functions that were previously spread across various organizational departments as legal, human resources, and acquisition. FITARA will also require metrics for agency CIOs to evaluate and meet. However, like most new initiatives in government, their implementation takes time. With FITARA, full implementation will not happen until late Fiscal Year 2016.

FITARA is a significant opportunity for improvement for the IT community, but more needs to happen on metrics. The good news is that the federal government understands this issue. Programs such as 18F Digital Services—a team of designers and developers housed in the General Services Administration who provide consultation for government projects—has called for a focus on metrics, especially when it comes to measuring impact, outcomes, and the contribution of IT to innovation.

Specifically, 18F encourages government agencies to develop metrics to evaluate performance from start to finish. They also emphasize the importance of performance metrics (e.g. response time, uptime, system maintenance) in defining contracts to set right expectations for performance early on. Finally, 18F implores agencies to use data to guide decisions. In short, for agencies to succeed with their IT programs, they must deploy metrics early, thoroughly, decisively, and consistently; they must also use the outputs to inform decisions on improvements.

In Creating a Balanced Portfolio of Information Technology Metrics, I report on the state of IT metrics in government from the perspective of federal, state, and local CIOs. I interviewed 27 CIOs who discussed their successes, failures, ideas, challenges, and frustrations with establishing and utilizing IT metrics. Below are key challenges that the CIOs identified:

• Many IT metrics will depend on other organizational units meeting their performance targets
• IT metrics require a focus on long-term planning
• CIOs will need to coordinate efforts to integrate metrics and develop standards for comparison
• IT departments must have the ability to collect real-time data on IT operations

The challenges mentioned above are simplified versions of the complexities CIOs deal with when attempting to strengthen IT across their agency. Flawed organizational processes that reward short-term thinking often win out over long-term planning, which places CIOs in a position of neglecting metrics for less important matters. In the report, I offer 14 recommendations for CIOs of all levels to begin to tackle metric design and establishment within their organizations. The recommendations are organized in order of metric design, implementation, and evaluation.

Benefits of public sector IT metrics

Developing performance metrics for IT projects in the public sector has the potential to transform the way we manage IT investments and deploy our IT capabilities in several ways. First, it will allow CIOs to share key performance data with internal and external stakeholders. Second, designing client-centric metrics allows CIOs to trace the impact of IT on business operations, service delivery, and goal attainment (see my article in FCW for more details). Third, metrics allow us to move beyond “one-off-incidents” and see the broader picture of IT performance. Put differently, if a system is up and running 99.9% of the time, we should not get too hung up on minor inconveniences during its 0.01% downtime. Too often, the stories of the 0.01% dominate the conversation.

Finally, IT metrics ensure that tax-dollars are being used effectively. This is a non-trivial issue given that we expect all levels of government to increase their investments in IT. Additionally, our agencies are more dependent on IT to achieve their mission objectives than ever before. Innovative uses of IT are also critical to tackling some of our most complex social challenges.  Whether it is developing communities, reducing recidivism, bettering health and educational programs, or bolstering national security, we will be able to apply IT systems to outcomes that we care about through judicious use of IT metrics.