How to Measure Agentic AI ROI: The Work-Unit Ledger

In brief

An AI agent that completes tasks with nobody checking each step is closer to a hire than to software, and it should be measured like one. The usual business case prices a tool: buy it, spread the cost over seats, wait for payback. That model treats supervision and rework as minor costs, which is fine for a tool and wrong for an agent, where those two lines decide the return. The Agentic Work-Unit Ledger prices an agent in five lines: the unit of finished work, the loaded cost of each unit including human supervision, the rework rate, whether the absorbed work meant redeploying people or reducing roles, and the named person who carries what is left. Klarna’s customer service assistant shows what public figures answer and what they leave out. The board question that follows: what does each unit of the agent’s work cost, how often is it redone, and what happened to the person whose work it took?

Key Takeaways

→Judge an agent that completes tasks unaided as you would a hire: by the loaded cost of each finished unit, not by cost per seat.
→The Agentic Work-Unit Ledger has five lines: three you count (work unit, loaded cost per unit, rework rate) and two you answer in words (redeploy or reduce, and the named human).
→Supervision and rework are what tool business cases treat as footnotes. For an agent they decide whether it pays.
→Close the rework partition to 100 percent. An agent that gets a customer to give up has hidden the work, and the deflection bucket is where vendors keep it.
→Used cynically, the ledger is a firing spreadsheet. Used honestly, it makes the redeploy-or-reduce choice a decision with a name on it, said where the affected people can see it.

An agent that finishes work on its own is labour, and it should be priced like a hire. The Agentic Work-Unit Ledger does that in five lines: the unit of work, the loaded cost of each unit, the share that comes back, whether the absorbed work meant redeploying people or cutting roles, and the name of the person who carries what is left.

I wrote this for the CFO or COO about to approve agents that resolve tickets, reconcile accounts or review contracts with no person checking each step. Those approvals are being judged on the wrong model. Most AI business cases price a tool: buy it, spread the cost over seats, wait for payback. That suits software helping people who still own the output. Once the software produces the output itself, the costs that decide the return are supervision and rework, and a tool case files both as footnotes.

The companies doing best already separate the two: in Deloitte’s October 2025 survey of 1,854 executives, 85 percent of its AI ROI leaders (the top fifth) said they explicitly use different frameworks or timeframes for generative and agentic AI. Deloitte names focus areas for each, not a method. This is mine.

The category error

A tool business case answers one question well: was the money spent on this capability worth what it returned? Once the software starts doing the work, that question is aimed at the wrong object.

Workers raise other questions. What does one finished unit of their work cost, fully loaded? How often does it come back wrong? Who used to do this, and where are they now? Those are labour questions, and the honest answers move week to week in a way a purchase price never does.

To be precise: my own checklist in The Six-Month Review Can’t See Which AI Projects Will Pay lists the costs a review tends to miss, and a diligent analyst could file supervision and rework there. For a tool that runs on rails, that is the right place. For an agent acting on its own judgement, they can be the biggest lines on the page. The Ledger moves them to the top.

The Ledger, line by line

Five lines, each with an operating rule finance teams can run on Monday.

Line 1, the work unit. Name the finished output, such as resolved tickets or reconciled accounts. If you can’t name it, stop until you can. Then hold the definition still. As an agent proves itself, it gets handed harder cases, so its cost per unit rises. That is success. When scope shifts like that, start a new ledger. Comparing this quarter’s hard cases with last quarter’s easy ones is how a working agent gets cancelled.

Line 2, loaded cost per unit. Tokens, orchestration, tooling and human supervision, divided by finished units. Tokens are the part everyone quotes. Supervision is the line software never carried, because nobody had to watch it in case it improvised.

Report it as a curve: the trailing average and its worst month, side by side, since supervision can spike after the first silent failure, when everyone goes back to checking everything. And sample it instead of waiting for a timesheet code. For a week or two one reviewer logs supervision minutes per unit; you scale up the ratio and sample again after any change of model or scope. The objection I hear most is that line 2 can’t be measured. It can. Most teams never sample it.

Capability numbers belong here too, read as costs. In May 2026 METR measured an early version of Anthropic’s Claude Mythos Preview succeeding half the time on software tasks that take a human expert about 17 hours, a figure METR warns its tasks can’t measure reliably. Ask for 80 percent success and the horizon drops to about three hours. Read as a cost: on clean benchmark tasks that long, half the attempts fail, and someone has to catch them.

Line 3, rework rate. The share of units escalated, corrected, redone or bounced back. This line separates working agents from expensive demos, and vendors are best at hiding it.

Close the partition. Every unit lands in exactly one bucket, and the buckets add up to 100 percent: resolved clean, reworked, escalated, or deflected and abandoned. Deflected units count against the agent unless independently confirmed, and someone audits a sample of the clean ones. Getting a customer to give up is hiding the work. If a vendor reports rework but no deflection rate, look in the bucket they left off the slide.

Line 4, redeploy or reduce. You don’t compute this one. You answer it in words, with evidence, in the operating review: when the agent absorbed this work, did you move the person who did it, or reduce the role? Say which. Booking a headcount cut as “reinvestment yield” or “freed capacity” is laundering. I used to reach for the freed-capacity framing myself. I stopped. It is the sentence that lets an organisation avoid looking at what it did.

Line 5, the named human. Name who carries the rework tax, a real person in a real role; the worker whose output the agent absorbed, retrained, redeployed or reduced; and the executive who decided. Named, and kept out of any yield number.

I'll say the uncomfortable part before someone says it for me. Used cynically, this ledger is a firing spreadsheet. Line 1 names the work, line 2 prices the human out of it, and line 5 tells you which humans. So here is the guardrail that separates the two uses. An honest ledger states its workforce disposition out loud, in the operating review, where the board and the affected function can both see it, and it treats "reduce" as a decision with a name on it, never a rounding error inside a productivity metric. A firing spreadsheet hides the line the Ledger requires. Nobody can stop a hostile reading of a framework. You can make that reading require dishonesty to work.

Klarna, read through the Ledger

A well-documented public case of an agent taking over a body of work is Klarna’s customer service assistant. In February 2024 Klarna said it had handled 2.3 million conversations in its first month, two-thirds of its customer service chats, doing “the equivalent work of 700 full-time agents”. It also reported satisfaction on par with human agents, a 25 percent drop in repeat inquiries and an estimated $40 million profit improvement for 2024. Here is what those numbers answer.

Line	What Klarna published	What the Ledger asks next
1. Work unit	2.3 million conversations in a month	How many ended in a resolved errand?
2. Loaded cost	An estimated $40 million profit gain	Cost per resolved errand, supervision included, by month
3. Rework	Repeat inquiries down 25 percent	The full split, including customers who gave up
4. Redeploy or reduce	Not replacing staff who leave (2025)	A policy on leavers; silent on the 700 agents’ work
5. Named human	Not published	Who carries the cases the assistant gets wrong?

Repeat inquiries are a real rework measure, and a stated attrition policy is half an answer to line 4. What the public numbers never showed was cost and quality per resolved errand over time. In May 2025 Sebastian Siemiatkowski, the chief executive, told Bloomberg that “cost unfortunately seems to have been a too predominant evaluation factor” and that the result was “lower quality”, as Fortune reported. Klarna began recruiting human agents again, while a spokesperson said it was “very much still AI-first”. Lines 2 and 3, kept monthly, are built to show that trade-off before a chief executive has to. The full case is in Most AI Projects Are Failed by the Organisation, Not the Model.

Where this belongs, and where it doesn't

Not all AI spend is agentic. When AI helps someone who still owns the output, such as a draft they will rewrite, keep the tool model. The Ledger is for work the agent finishes itself, and that spend is the hardest to see.

KPMG’s Global AI Pulse for the second quarter of 2026 surveyed 2,145 senior leaders in 20 markets. Only 7 percent reported established ROI, 42 percent had only partial visibility into AI spending, and a third named a limited understanding of usage costs as a key challenge in deploying agents. Leaders with strong cost visibility were five times as likely to report established ROI, 15 percent against 3. Pricing is moving toward the labour model too. In AI coding agents, Gartner reports that vendors are shifting from seat-based subscriptions to usage-based pricing. Salesforce already sells agents as “digital labor”; buyers should hold them to it.

I’ll concede the lineage. This is activity-based costing, as Robin Cooper and Robert Kaplan set it out in Harvard Business Review in 1988, pointed at a worker that did not exist then. Rework is an old quality measure that software business cases leave out. What is new are the redeploy-or-reduce disclosure and the named human.

The one board question

Stop asking what the agent cost. Ask what a unit of its work costs, fully loaded. Ask how often that work is redone. And ask what happened to the person whose work it took. If your agents sit on a payback curve built for software, you are measuring a hire like a purchase, and the money and the ethics both go missing in the gap.

Sources and Changes

Deloitte, "AI ROI: The paradox of rising investment and elusive returns", October 2025.
METR, "Task-Completion Time Horizons of Frontier AI Models", updated 8 May 2026.
Klarna, "Klarna AI assistant handles two-thirds of customer service chats in its first month", 27 February 2024.
Fortune, "As Klarna flips from AI-first to hiring people again", 9 May 2025.
KPMG, Global AI Pulse Q2 2026, press release, 24 June 2026.
Gartner, "Leading in Enterprise AI Coding Agents Requires More Than Product Momentum", 2026.
Salesforce, "Salesforce Introduces New Flexible Agentforce Pricing to Accelerate the Digital Labor Revolution", 15 May 2025.
Robin Cooper and Robert S. Kaplan, "Measure Costs Right: Make the Right Decisions", Harvard Business Review, September 1988.

All sources checked on 7 October 2026.

Changes on 7 October 2026. New headline, summary and hero art. Line 5 now names the executive who decided. The opening no longer describes my ROI article as it stood before its rewrite. METR’s figure now carries its 80 percent counterpart and caveat. KPMG’s 23 percent measured struggle with usage-based costs, not blindness to them, and is cut. Gartner’s pricing shift is about coding agents, and the text says so. An unsupported Anthropic line is cut. Klarna’s published figures replace an invented example, and the closing call to action is gone.