Artificial intelligence (AI) agents—technology that plans and executes tasks autonomously—have accelerated sharply over the past two years in both capability and projected market size. In 2025, agents improved their accuracy in computer tasks from 12% to 66.3%, lagging the average human baseline accuracy by just 6 points. In terms of market size, Precedence Research estimates that the market will grow from $8 billion in 2025 to $199 billion by 2034.
Such rapid acceleration implies that existing governance models for AI more generally might not be sufficient. The shift from static model oversight to dynamic system governance raises new challenges around control, speed, traceability, and human oversight. That distinction matters. The central governance question is no longer only what the model said. Increasingly, it is what the system did, why it was allowed to do it, and who is accountable. How, then, can governance mechanisms be built to seize the opportunities of agentic AI and mitigate its new and complex risks?
These issues were at the center of the May 2026 World Economic Forum panel at the AI+Expo in Washington, D.C. A key theme was that risk-based approaches, life-cycle governance, accountability, safety, and human oversight remain relevant but must become far more operational as systems act faster and with less direct human intervention.
Applications of agents
AI agents are proving to support productivity, public services, education, health care, translation, accessibility, growth of small- and medium-sized enterprises, and development goals across both the Global North and the Global South. Already, they are compressing research timelines for pharmaceutical companies, optimizing industrial goods supply chains, and reducing costs of customer service.
In manufacturing, agents have moved from simply flagging a defect to analyzing the systems, finding the probable cause, and removing the defective unit. These agents do not require large models or the newest chips, which makes them more accessible to lower-income countries than full digital transformations are.
In the public sector, a World Economic Forum study of 70 core government functions found public services among the highest-readiness targets for agents but cautioned that information-driven tasks should come before judgment-based ones such as eligibility assessment, where agents would affect sensitive decisions about individuals. Early returns are promising even from simpler deployments: The U.S. Centers for Disease Control and Prevention estimates a 527% return on its staffwide AI chatbot. With good governance of AI agents, agencies can leverage their strengths to improve productivity, complete complex workflow automation, and respond to crises with rapid needs assessments. Yet documented evidence available today on deployed value is largely pre-agentic, and the governance question is arriving ahead of the deployments that will test it.
So far, adoption remains dominated by knowledge workers in higher-GDP countries, and enterprise uptake trails consumer use as compliance, budgets, and internal rules slow deployment. A recent Microsoft analysis of 100,000 Copilot chats found that almost half of the requests were for the agent to support cognitive work, while the remaining half comprised working with people, finding information, and producing work. These usage patterns are an early indication of where value is concentrating.
What changes when AI can act autonomously?
Agents combine autonomy and the ability to operate independently with the authority, permissions, and access rights that allow them to act within defined boundaries. Autonomy, adaptability, and complexity are the cornerstones of agentic systems and also the characteristics that make effective governance difficult.
AI agents use complex machine-learning algorithms to analyze probabilities and determine actions in a way that is not necessarily possible for humans to understand and therefore audit.
Existing governance structures are based on discrete steps and stages, moving from model development to procurement, deployment, and output review. Yet AI agents make those boundaries less meaningful due to the combination of models with memory, orchestration, tools, and access to external systems which leads to a completely different risk profile. For example, traditional AI chatbots can be reviewed by humans before action is taken, but AI agents are structured specifically to invoke another system or action before a human intervenes. Once agents are connected to tools and granted authority, they become consequential in a way that conventional software controls are not equipped to manage, leading to risks like goal misalignment, behavioral drift, tool misuse, and coordination failure. They thus start to resemble delegated decisionmakers, requiring much stronger oversight to deal with the expanded capabilities.
Governance challenges
A governance gap often opens when organizations know what an agent do, but do not necessarily know what they should allow it to do. Even when organizations create the strongest systems, they remain far from uniformly reliable, as even a properly authorized agent can fail, whether its misunderstanding a task, failing to recover after an unexpected event, or increased vulnerabilities after integrating a tool. A more complete governance model should assess at least five interacting factors, including capability, authority, architecture, context, and consequence.
Models can differ in risk levels based on different applications and stages. For example, a model could be low risk during actions like retrieving information, but it could turn higher risk when modifying records, communicating, or initiating transactions. A helpful tool to deal with this reality is the Agent Capability and Authorization Profile developed by the World Economic Forum. This tool suggests that models, systems, and agent cards document what an agent can do, while a profile records what it has been authorized to do. This makes the scope, operating context, permissions, controls, evaluation evidence, monitoring, and ownership explicit for a specific deployment within a specific workflow. Each agent likely carries several profiles, so these records should complement, not substitute, technical evaluation of the entire underlying system.
Governance structures for traditional AI are typically focused on controlling how humans interact with AI through measures such as content filtering, data protection, and output review. Agents complicate this approach as governance now has to focus on how agents interact with systems and with one another. Stage-based models that are tied to design, procurement, or deployment will not be sufficient in the face of decentralized deployment and rising autonomy. As every step in an agent’s chain of action accumulates new risk, governance must act continuously and at speed.
Privacy and cybersecurity risks are critical, as attacks that manipulate the underlying data could lead to the agent making wrong and potentially dangerous decisions. The data needed to power the agent, such as personal, enterprise, or system data, could also leak, exposing sensitive information. For example, prompt injection is the most common attack technique where one injection in a model can interfere with the entire system largely due to the architecture of agents where all inputs are treated as a unified token sequence, which makes it difficult to place boundaries or identify which part was interfered with.
This leads to a lack of traceability and limited verification, which both point to the general challenge of the lack of human oversight that is possible. Within enterprises, new protocols like the model context protocol and agent-to-agent protocol are making agents more capable of interacting with one another across platforms without needing to integrate them. While the increased coordination is promising in terms of productivity and capability, it requires more intentional governance as security attacks would become even more damaging with greater access to connected systems.
Underlying all of this are accountability gaps between model developers, deployers, users, and affected communities. Where communities are absent from the design of governance itself, harm becomes more likely whether it is intended or not.
The opportunity is large but uneven
As the Washington, D.C., panel highlighted, smaller, multilingual, open, or reusable models can support applications such as translation and accessibility with substantially less compute, which extends the opportunity well beyond the countries and companies able to deploy the largest systems.
Regulatory expertise, testing capacity, infrastructure, and bargaining power are unevenly distributed across countries and organizations, meaning global agent governance should not assume every deployer can negotiate risk allocation with major technology providers or access resources such as evaluation teams or extensive compute. Unequal capacity globally thus poses significant risks, like that of traditional AI, where bias in the type and source of data they are built on can lead to agents that do not serve the interests or needs of certain communities. In countries where there is limited regulatory capacity, risks related to AI agents specifically are amplified. In authoritarian contexts, agentic AI could be used with malicious intent to track opposition or push disinformation. Within and across countries, reskilling and labor force disruptions are amplified with agentic AI, especially for those in coding or customer service who are facing rapid automation by this technology. Governance needs to be robust enough to protect users and practical enough to work in lower-capacity environments.
Pathways forward
To ensure governance frameworks are up to the challenge to effectively regulate AI agents, they should keep existing principles that are working well but make them practical, measurable, and enforceable. For example, the Hiroshima AI Process—the G7-led initiative for international guiding principles—and other risk-based, life-cycle governance approaches still apply, but the main gap is the implementation: turning principles into operational controls. Rather than relying on a “regulate and forget” mindset, adaptation and learning will be the foundation. Specifically, the public and private sector alike should:
1. Define specific permissions and access for agents
Structured authorization models should name the agent’s owner and grant only necessary permissions. Scope limits and the actions requiring human intervention need to be set in advance, so that high-risk actions are determined preemptively. Two ways to do this are to implement guardrails and constraints. Guardrails stop actions, through input validation, content filtering, and scope enforcement. Constraints bound them through data boundaries, output limits, cost and time ceilings, and action restrictions. Continuous, traceable evidence is equally important for mapping agents to their scope and actions. Already, the EU AI Act, the National Institute of Standards and Technology (NIST)’s AI Risk Management Framework, and the OpenID Foundation’s October 2025 white paper on agentic identity all point toward requiring traceable evidence, which must continue to be enforced and updated as agents evolve.
2. Create stronger feedback loops between deployers and developers
End-to-end tracing, centralized gateways, and shared dashboards are important tools to improve visibility of agent errors, fallback triggers, and policy blocks for both deployers and developers. Runtime governance, which inspects agent actions across multiple inspection points, including continuous monitoring, logging, escalation, auditability, and redress, will be necessary to keep pace with the autonomous decisions happening in seconds. The EU AI Act places its heaviest obligations on the highest-risk categories, including health and employment, by requiring automatic logging and human oversight and backing those requirements with fines. However, these requirements are not yet in force. The Digital Omnibus on AI, which entered into force in July, defers high-risk obligations to December 2027 for stand-alone systems and to August 2028 for AI embedded in regulated products. Transparency and AI literacy duties were not postponed, but for roughly the next 16 months the controls governing agents that act in high-stakes domains will be organizational rather than statutory.
Meanwhile the NIST AI RMF 1.0, a co-created, voluntary U.S. framework, is based on four functions focused on cross-cutting policies and accountability, system-level risk context, quantitative and qualitative risk assessment, and risk treatment and incident response, noting that often the third function drives the most implementation challenges as a lack of data or infrastructure to do so causes failure that must be prioritized. Making the framework mandatory could accelerate its adoption implementation.
3. Develop better benchmarks and evaluations for agentic systems
Across the development and deployment cycles, better benchmarks and evaluations should be integrated, including trajectory tracking by observing decision chains and their interactions instead of only the output; factual grounding by using automated probes or rubric-based judges to see how and if actions are based on trusted sources; and adversarial red teaming by stress-testing models with realistic threats. Task success rate is a helpful benchmark which can be strengthened with more specific tracking under different scenarios and constraints. Customizable evaluation that is built into design will be key along with predeployment impact assessments.
4. Incentivize testing, assurance, and bias evaluation rather than discouraging them
Sandboxes allow agent decisionmaking to be tested before deployment, and public-private partnerships, infrastructure investment, and capacity building can all reward up-front testing. ISO/IEC 42001 is the first international management system standard for AI that is certifiable and supports readiness for the EU AI Act obligations without conferring presumption of conformity. Singapore’s open-source AI Verify Toolkit offers an accessible way to test for fairness and robustness, and in January 2026, the country’s regulator released the Model AI Governance Framework for Agentic AI, a pioneering framework built specifically for agentic systems.
5. Promote international interoperability while allowing local adaptation
Where security risks can be managed, open standards such as Model Context Protocol support global interoperability. Establishing core safety and human-rights alignment targets can also help streamline cross-border operations so local systems meet the core standards, while leaving room for regional authorities to adjust their enforcement systems to local needs. For example, Singapore’s AI Governance Framework acts as a hub for Southeast Asia and focuses on transparency and accountability while Africa’s AI Regional Hub Initiative collaborates with universities and focuses on ethics.
Conclusion
As agents continue to become more complex and widespread, the path forward for effective governance becomes more vital and necessary. We need practical and multistakeholder processes that promote human safety and security. These types of systems will require continuous monitoring, real-world evaluation, stronger feedback loops, escalation mechanisms, better benchmarks, and coordination among industry, governments, civil society, affected communities, and international frameworks. A defined shared responsibility across developers, deployers, and users is critical, along with a push to involve affected communities, civil society, regulators, academia, and industry in a co-creation process.
Related Content
Authors
-
Acknowledgements and disclosures
Microsoft is a general, unrestricted donor to the Brookings Institution. The findings, interpretations, and conclusions posted in this piece are solely those of the authors and are not influenced by any donation.
The Brookings Institution is committed to quality, independence, and impact.
We are supported by a diverse array of funders. In line with our values and policies, each Brookings publication represents the sole views of its author(s).
Commentary
Shaping global governance of AI agents
October 12, 2026