Skip to main content
Covey
SecurityNov 4, 20265 min readBy Matt Hogan

When an AI agent gets it wrong: the review steps that catch it first

When an AI agent makes a mistake, a review step should catch it before it reaches a customer or your books. We build those steps before the agent does its first piece of work, and we only remove one when the agent's record has earned it.

At CG Landscape, a large commercial landscaping company, agents read each incoming check, including handwritten ones, match it to the right invoice, and enter it. Anything an agent is unsure about goes to a person instead of being guessed. The first time it clicked for them was watching an agent accurately enter 100 handwritten checks. The obvious next question is what happens on check 101.

It will go wrong sometimes. A new hire misreads a smudged invoice number too. Each mistake has to get past several layers of review before it does damage. Here they are, from the narrowest layer outward.

Each agent has one job

An agent that reads checks does not also draft emails or build forecasts. We keep scope narrow because it keeps the work reliable and makes mistakes small and easy to find. When the check entry agent gets something wrong, there is one place to look and one set of instructions to fix. An agent with ten jobs gives you ten places to look and no idea which one broke.

A second agent reviews the first

We don't ask an agent to grade its own work.

On finance work, an auditor agent reviews every financial deliverable before it goes out. At CG Landscape, that auditor checks the contract renewals. On marketing, an editor-in-chief agent reviews every piece against your brand standards before anything is published.

The reviewing agent has a different job and different instructions. It is looking for what's wrong in the work, and nothing else. Companies already work this way with people: whoever wrote the invoice is not the only one who checks it.

A person approves what matters

Agent review catches the routine errors. People keep the final say on anything that touches your money, your customers or your prices. At CG Landscape, the team approves every contract renewal before it goes out. That is what keeping a human in the loop means in practice.

Sensitive changes, like budgets or pricing, need approval from two teams: yours and ours. Your team knows whether a number makes sense for your business. Ours knows how agents fail and what a change will touch downstream. If either side says no, the change doesn't happen.

What a caught mistake looks like

Take a hypothetical check with a coffee ring over the invoice number. If the agent can't read the number with confidence, it sends the check to a person instead of guessing. If it reads the number confidently and gets it wrong, a person catches it. In month one, a person reviews every entry, sees that the name on the check doesn't match the customer on the invoice, and sends it back. Either way, in month one a person sees it before your books do.

Every rejection also tells us why the agent misread the check. We change its instructions and test the change before the agent goes back to work, so the same coffee ring doesn't cause the same error next week.

Month one is a review, and the thresholds move

For the first month, nothing an agent produces goes out without a review. At the end of it we sit down with you and look at the record: what the agents got right the first time, what reviewers rejected, and why.

Agents take on more as they earn your team's trust, the same way a new employee does. Where an agent has a clean record, the thresholds loosen. It handles the routine cases on its own and only the unusual ones come to a person. You decide how far that goes. Budgets and pricing keep their two-team approval however good the record looks.

Access and data are separate questions with their own articles. Read-only first covers how an agent starts with look-but-don't-touch access, and Are AI agents safe for your business? covers how information is kept apart. This piece covers the work itself.

If a provider tells you their agents don't make mistakes, ask where the reviews sit and who sees a rejected piece of work. You should get a specific answer. Real work with our agents starts within 2 to 4 weeks, and the review steps are in place before it does.

The CG Landscape story shows the full result. If you want to see where the reviews would sit for the work you'd hand over first, book a call.

Questions

What happens when an AI agent makes a mistake?
A review step should catch it before it reaches a customer or your books. Each agent has one job, a second agent such as an auditor or an editor-in-chief reviews its work, and a person approves anything that touches money, customers or prices. In month one, every piece of work is reviewed.

What does human in the loop mean for AI agents?
A person approves the agent's work at set points before it takes effect. At CG Landscape, the team approves every contract renewal an agent prepares, and any check an agent can't read with confidence goes to a person. As an agent builds a record, the approval points can narrow. Sensitive changes like budgets and pricing keep approval from both your team and ours.

Can an AI agent check its own work?
We don't rely on it. A separate agent with a different job does the checking: an auditor agent for financial work, an editor-in-chief agent for marketing.

Ready when you are

See where the reviews would sit for your work.

A free 30-minute planning session. We will map the review steps to the first piece of work you would hand over.

Newsletter

Get the next piece when it lands.

Short notes for owner-operators putting AI to work. No hype, no listicles.