Fourth in a series on building agents that survive regulatory review. Previously: delete your approval rule, a checkpoint is not evidence, interrupt() is a pause, not an approval.
We build systems; your counsel interprets the regulation.
An inventory of what your guards actually watch
Write them out and the gap becomes obvious.
| Guard | Catches |
|---|---|
| Budget / step limit | too expensive, too long |
| Retry and circuit breaker | the provider fell over |
| Loop detection | repeating itself |
| Policy gate | not permitted to do that |
| Cancellation | no longer wanted |
| Schema validation | output has the wrong shape |
| Timeout | hung |
Every row is about process. Not one is about whether the right question was answered.
So consider a run where the agent misreads the request, works competently on its own reading, and produces a well-structured answer to a question nobody asked. Budget: fine. No repetition. Every tool call succeeded. The output validates. Cancellation never fired.
Green all the way down. And wrong in the only way that matters.
Why this is the expensive one
Other failures are loud. A crashed run gets retried. An over-budget run gets flagged. A schema violation throws.
This one is silent by construction, and silence has a particular cost in regulated work.
A confident, fluent, well-sourced answer to the adjacent question is more dangerous than a bad answer, because it survives review by a busy person. It reads correctly. The citations are real. The numbers come from a document that exists. It just answers “what were Q3 redemptions” when the question was “what were Q3 net flows”, and by the time anyone notices, it’s in a board pack.
The failure mode of finance-grade document Q&A is almost never hallucination in the popular sense. It’s retrieval from the adjacent document — a related policy, the previous year’s filing, the version superseded in March — with faithful, accurate quotation from the wrong source.
Two cheap calls
The fix is smaller than people expect: one model call before the work, one after. Both configurable, both roughly a rounding error against the run they protect.
Before: restate the task
Not “make a plan”. Plans go stale the moment the first document is read, which is why explicit plan-and-execute was largely abandoned — the plan is written from guesses, and by step three either the agent ignores it or it walks into a wall.
This is about the input contract, which doesn’t go stale. One call producing four things:
A restatement — the task in the agent’s own words, specific enough that someone could check whether it was done.
Assumptions — readings chosen without being told. “’Income’ means monthly rent, not service charges.” If the assumption is wrong, that’s visible now rather than in the output.
Acceptance criteria — what a complete answer must contain. Concrete and checkable, not “be accurate”.
Ambiguities — genuine forks. And here’s the whole design problem.
The anti-theatre rule
A model asked “what’s ambiguous here?” will invent ambiguities. It’s agreeable, it costs nothing, and it produces a list that looks thoughtful. Ship that and you’ve built a bot that interrupts a compliance officer twice a day about nothing, and they’ll turn it off in a week.
So: an ambiguity is only acted on if each reading states what different answer it produces.
{"id": "let", "leads_to": "building-7 leads on let income"},
{"id": "lost", "leads_to": "building-9 leads on shortfall"}If the model can’t say what changes, nothing changes, and nobody is interrupted. One required field, and it’s most of what separates this mechanism from decoration.
On our test data the effect is measurable: a genuine fork — two readings of a status that name different answers — is surfaced at step zero, before a single tool has run. Half a cent spent, and the wrong answer never gets built.
After: check the answer against the request
Not “did you follow the plan”. “Does this answer the question that was asked.”
Three verdicts, and the third is the point:
completeincomplete— right question, part unansweredwrong_question— it’s about something else
Most implementations collapse the last two into “needs improvement”. Keep them apart, because the repairs differ. An incomplete answer gets the specific gaps. A misread gets the original request restated, because re-reading is the thing that has to happen.
Bounded retries, and on exhaustion the run finishes anyway — with the shortfall written to the log as an error, not dissolved. A system that silently gives up looks identical to one that succeeded.
Where the judge sits
One structural point that decides whether the check is worth anything.
The acceptance check must not be a self-assessment turn in the same conversation. An agent that has just produced an answer, asked whether the answer is good, will say yes. It has the reasoning that produced it, and the reasoning is persuasive to itself.
Give the check only two things: the original request, and the final answer. No conversation history, no tool results, no plan. It didn’t do the work and has no stake in it. That’s the entire source of its usefulness.
It’s the same discipline as delegating to a subagent with a context firewall, and for the same reason: a fresh reader catches what the author can’t.
Cost, and the honest accounting
Two calls per run. On a twenty-step run that’s ten percent. On a three-step run it’s forty. Worth it where being wrong is expensive; not worth it on a high-volume classifier where you’d use sampling instead.
Off for sub-agents. A subagent’s task was written by another machine, is already precise, and usually carries a return contract. Clarifying it again is pure cost. Ours default to skipping both phases when a run has a parent.
Neither is free of the thing it’s protecting against. The clarification is a model call and can misread. The acceptance check is a model call and can approve a bad answer. This reduces the failure rate; it doesn’t eliminate it. Anybody claiming otherwise is selling.
And a structured restatement always looks convincing. Which is the trap, and it’s serious enough to be the next section.
You cannot evaluate this by reading transcripts
Here’s the part I’d want to know if I were reading this.
Every mechanism above produces output that reads well. A restatement of the task is articulate. A list of assumptions looks diligent. A complete verdict with reasoning is satisfying. Read fifty transcripts and you will conclude it works.
That conclusion is worth nothing, because you’d have concluded the same from a version that added no accuracy at all.
Which means these two phases are exactly the kind of feature you must not ship on intuition. You need a labelled set, a baseline without the phases, and a comparison. Not because the idea is doubtful — because the evidence available by inspection is systematically misleading here in a way it isn’t for, say, a retry policy.
We’ve built the harness. We do not yet have a large enough real-world labelled set to report a number, and I’d rather say that than quote one from generated data. Which brings us to the next post.
The exercise
Take ten outputs your system produced last month. For each, without looking at the transcript:
- Write down what was actually asked.
- Write down what the answer is about.
- Are they the same question?
Then count how many of the mismatches your monitoring flagged.
The expected answer is zero, because nothing in a standard stack watches for this. If you find mismatches and your dashboards were green throughout, you’ve measured the gap.
Where this fails in our own work
No real-world numbers. As above. The mechanism is tested — a misread answer is sent back, the specific gaps reach the agent, the retry budget is bounded and the shortfall is logged. Whether it reduces wrong answers on your documents is a measurement neither of us has yet.
The clarification can invent assumptions too. The leads_to rule constrains ambiguities. Assumptions have no equivalent guard, so an agent can state a confident assumption that is simply wrong, and it goes into the brief looking authoritative.
Acceptance criteria are only as good as the restatement. If the task was misread at step zero, the criteria encode the misreading, and the final check confirms the answer meets them. Two mechanisms agreeing because they share an error. The one thing that catches this is the criteria being visible to a human before the work starts — which is available, and which nobody does.
Changelog, for the month: both phases were off in our own demos for two weeks without anyone noticing, because the test suite covered them and the demos didn’t. Fixed by putting the demos into the build.