When Agents Talk to Agents, Who Governs the Conversation

In 2025 a team at UC Berkeley ran seven open-source multi-agent frameworks on coding, maths and general agent tasks. ChatDev, MetaGPT and Magentic-One were among them. The team annotated 1,642 of the runs. Depending on the framework, between 41 and 86.7 percent of tasks failed, according to the MAST study. The study sorted the failures by how the system broke, not by which model erred. About 44 percent were problems of system design: an agent ignoring its task or its role, repeating steps, or not knowing when to stop. About 32 percent were agents falling out of step with each other. About 24 percent were checks that were skipped, ended early or passed the wrong answer.

Those were test runs. In production, the same failures act on customers and on money. My argument is that you cannot govern a multi-agent system by governing each agent. The risk sits in the handoffs, so the handoff is what an executive has to approve, log and be able to stop.

A multi-agent system is a set of AI agents that pass work, context or instructions to one another, so one agent's output becomes the next one's input. Its signature risk is the cascade: an error accepted at one handoff is acted on, and often amplified, by every agent downstream.

This reference is for the COO, CRO or CIO asked to sign off a multi-agent deployment. For the basics, start with what an agent is and how it differs from a chatbot. For how much autonomy a single agent should get, see the five levels of autonomy. For who answers when an agent gets it wrong, see the piece on accountability.

Why Is the Handoff Where the Risk Lives?

In a single-agent system an error produces a bad output, and someone may catch the error. In a multi-agent system that output is the next agent's input. The receiving agent has no reason to doubt it. It acts, and passes its own result along. OWASP's Top 10 for Agentic Applications, published on 9 December 2025, gives this its own entry, ASI08 Cascading Failures, which "focuses on the propagation and amplification of faults rather than their origin." Two neighbouring entries name the channels a cascade travels through: ASI07, Insecure Inter-Agent Communication, and ASI06, Memory and Context Poisoning. Shared memory is why a cascade can outlast its cause. Fix the agent that started the cascade and the others are still reading what that agent wrote.

The quieter failure is drift. When a person delegates to Agent A, Agent A works from its reading of the request. Agent B gets Agent A's reading, not the request, and so on down the chain. As arithmetic only: if each handoff kept 90 percent of what was meant, five handoffs would keep 59 percent and ten would keep 35. I know of no published measurement of the real figure for a production system, which is itself the problem. MAST's second category is this drift seen in the lab: agents that don't ask for clarification, wander off the task, or act against their own stated reasoning.

The protocols make handoffs easier, not safer. Google launched the Agent2Agent protocol on 9 April 2025 with more than 50 technology partners, so that agents from different vendors can find and call each other. Writing in O'Reilly Radar in February 2026, Shreshta Shyamsundar named the side effect: A2A and the Agent Communication Protocol "removed the natural 'brakes' that used to force governance conversations."

Four Architectures and Their Governance Bills

Multi-agent systems come in a few shapes, and the shape sets the governance bill. Microsoft's architecture guidance names five orchestration patterns: sequential, concurrent, group chat, handoff and magentic. Singapore's agentic AI framework names three: sequential, supervisor and swarm. For a governance decision I group them by who holds control, which gives four.

1. Hierarchical: a supervisor directs workers

One agent owns the goal, hands parts of it to specialist agents and assembles the result. Accountability has an address: the supervisor and the team that owns it. This is the easiest shape to log and audit. Its weakness is the supervisor, a single point of failure. Still, the evidence is on its side. In Google's scaling study, which tested 260 configurations on six benchmarks, independent agents amplified trace-level errors 17.2 times compared with a single agent. Centralised coordination held this to 4.4 times "by enforcing validation bottlenecks that intercept errors before aggregation."

2. Peer-to-peer: agents negotiate directly

Agents call each other without a coordinator, which protocols such as A2A now make routine. There is no single point of failure, and no single point of record either. When three agents settle a decision between them, nobody can say who approved it unless every exchange was logged. In the same scaling study, decentralised systems amplified errors 7.8 times.

3. Market-based: agents bid for work

Agents advertise what they can do and bid for tasks, and a manager awards the task to the best bid. The scheme predates the current wave by decades. Reid Smith described it in 1980 as the Contract Net Protocol, in which contractors "submit bids" and managers "award contracts to the nodes they determine to be most appropriate." The governance cost is incentive design. My judgment is that an agent rewarded for winning work can learn to win it rather than do it well, and that this is hard to see from outside.

4. Swarm: many agents, behaviour that emerges

Many agents act at once and hand off to each other as they see fit, and the useful behaviour comes from the whole. No agent holds the goal, so no agent can be held to it. Only monitoring of the whole system works, and emergent behaviour is the hardest to test in advance. Singapore's framework warns that when several non-deterministic agents work together, they can produce "emergent behaviours that cannot be predicted from testing each agent individually."

Side by side, the four shapes differ most in who answers for a decision and where the chain breaks.

My advice is to start hierarchical. Microsoft's guidance goes further: "Use the lowest level of complexity that reliably meets your requirements." It calls a single agent with tools "often the right default for enterprise use cases." I would move to another shape only when every handoff in it can be logged.

Seven Principles for the Cascade

I draw these seven from the failure modes above and from older cases of parts that worked alone and failed together. Every one of them breaks the chain at a different point.

1. Test the system, not the components

Agents that pass their own tests one by one can still fail together. A 2025 Gradient Institute report, supported by Australia's industry department, puts the point in one line: "a collection of safe agents does not imply a safe collection of agents." Before go-live, run the whole chain against bad inputs. Feed the first agent subtly wrong data and watch how far the error travels. Give two agents goals that conflict and see which one wins.

2. Make the checker independent of what it checks

In MAST, about one failure in four was in verification. A checker agent built on the same model, fed the same context and scored on the same goals as the agents it reviews will tend to miss what they miss. The Therac-25 radiation machine is the old lesson: between June 1985 and January 1987 the machine was involved in six accidents with massive overdoses, and in Leveson and Turner's investigation, "software checks were substituted for many traditional hardware interlocks." Give the checker a different model or different data, different goals and a different owner.

3. Label what an agent inferred, and test it for proxies

Agents fill gaps. Say an intake agent infers a missing value. Unless the data says so, the next agent cannot tell that guess from a fact the customer gave. Inferred and derived variables are also where proxies for protected traits hide. In July 2025 the Massachusetts Attorney General settled with the student lender Earnest for $2.5 million. Her office alleged that a "Cohort Default Rate" variable, the average default rate at a borrower's school, produced a disparate impact on Black and Hispanic applicants. Earnest agreed to stop using the variable. Mark every inferred field as inferred, carry the flag through every handoff, and test derived features against protected characteristics before launch.

4. Measure the harm you don't want next to the output you do

Agents optimise what they are measured on. A dashboard of throughput, cost per decision and error rate can keep improving while the system harms a group nobody is counting. The same settlement alleged that Earnest failed "to test its models for disparate impact." Put the harm measures that matter for the use, such as approval rates by group, complaints and reversals on appeal, on the same screen as the performance numbers, with a named person who acts on them.

5. Keep one audit trail across every agent

Agents that log separately leave no record of how a decision was made. At every handoff, record what the agent was asked, what it took the request to mean and what it did, with the agent's identity and the authority it acted under. A reviewer can then see where meaning drifted, and a regulator can follow a decision from the output back to the person who asked.

6. Set the blast radius and circuit breakers before go-live

Give every agent scope limits, not only task limits: which customers, which amounts, which systems it may touch. Then set thresholds that halt the chain automatically and fall back, first to fewer agents, then to people. Knight Capital shows what speed does without them. On 1 August 2012 its order router sent millions of orders into the market over 45 minutes, and the firm "lost over $460 million" from the positions, according to the SEC order. Knight paid a $12 million penalty under a rule that requires brokers to keep risk controls on market access. A threshold set in advance is the only kind that works at machine speed, and choosing one is the subject of when to stop an AI deployment.

7. Match autonomy to readiness

A multi-agent system multiplies whatever readiness gaps a single agent has. Grant no agent more autonomy than your weakest governance capability can hold. The A7 readiness assessment scores that, and the five levels stage it. Name a person who signs off any action that cannot be undone. Singapore's framework makes the same demand in its second dimension: "Make humans meaningfully accountable."

For a review meeting, the table below gives the point where every principle breaks the cascade and the sign to look for.

What the Rulebooks Cover Today

No rulebook yet governs the handoff itself. OWASP's list names the threats. Singapore's Model AI Governance Framework for Agentic AI, launched on 22 January 2026, gives guidance; its update of 20 May 2026 added a section on systemic and multi-agent risks. In the United States, NIST's Center for AI Standards and Innovation launched an AI Agent Standards Initiative on 17 February 2026, to support industry-led agent standards and research on agent security and identity. The EU AI Act regulates AI systems by their use. A chain of agents that scores credit falls under the high-risk rules for Annex III systems, which apply from 2 December 2027 after the Digital Omnibus moved the date. On my reading, the Act says nothing specific about the handoffs inside such a chain. Until something does, the handoff rules are yours to write.

Five Questions Before You Approve a Multi-Agent Deployment

If I were asked to sign one off, I would want a yes to all five of these.

• Can we trace any decision from the output back through every agent to the person who asked?
• What may each agent touch, and what halts the chain automatically?
• Is the checker independent of the agents it checks, in model, data and ownership?
• Which fields were inferred rather than supplied, and have they been tested for proxies?
• Who signs off actions that cannot be undone, and can we revoke one agent's permissions without stopping the rest?

Sources and Changes

• Mert Cemri and others, UC Berkeley, "Why Do Multi-Agent LLM Systems Fail?", arXiv 2503.13657, version 3, 26 October 2025.
• Yubin Kim and others, Google Research, Google DeepMind and MIT, "Towards a Science of Scaling Agent Systems", arXiv 2512.08296, version 3, 8 April 2026.
• OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026, 9 December 2025.
• Google Developers Blog, "Announcing the Agent2Agent Protocol (A2A)", 9 April 2025.
• Shreshta Shyamsundar, "AI, A2A, and the Governance Gap", O'Reilly Radar, 17 February 2026.
• Microsoft, "AI agent orchestration patterns", Azure Architecture Center, read 6 October 2026.
• Infocomm Media Development Authority, Singapore, Model AI Governance Framework for Agentic AI, launched 22 January 2026; version 1.5, 20 May 2026, updated 5 June 2026.
• Reid G. Smith, "The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver", IEEE Transactions on Computers, December 1980.
• Gradient Institute, "Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems", 29 July 2025.
• Nancy Leveson and Clark S. Turner, "An Investigation of the Therac-25 Accidents", IEEE Computer, July 1993.
• Massachusetts Attorney General, "AG Campbell Announces $2.5 Million Settlement With Student Loan Lender", 10 July 2025.
• US Securities and Exchange Commission, In the Matter of Knight Capital Americas LLC, Release 34-70694, and press release 2013-222, 16 October 2013.
• NIST Center for AI Standards and Innovation, AI Agent Standards Initiative, 17 February 2026.
• Regulation (EU) 2024/1689 (the AI Act), Article 113, and Regulation (EU) 2026/1744 (the Digital Omnibus), Official Journal texts, read 6 October 2026.

Changes on 6 October 2026. Cut from about 5,100 words, with a new headline. The MAST figures are now described as test runs on benchmark tasks, not production traces, and the failure shares follow the study's third version. This piece absorbed the seven lessons of "The Cascade", whose URL now redirects here; that article's fictional company and its figures were not carried over. Removed for want of a primary source: a 1,445% rise in inquiries, market-size and agent-count forecasts, adoption survey figures, and the claim that six OWASP threats grow "exponentially" in multi-agent systems. A composed delivery-app scenario, the reading-path block and four custom diagrams went too.