WRITING · AI AGENTS
How I make AI agents reliable
Better AI agents are not mainly about better prompts. They are about better sources of truth, clear procedural instructions, validation, provenance, and correction loops. I treat accuracy as an operating problem, not a model problem.
Where agents actually fail
Most agents fail in three ways: a business concept maps to several plausible sources, definitions go stale, or the right information exists but the agent never finds it. The operating layer I build around every serious agent is designed against exactly those failures.
The operating layer
- Canonical source of truth for definitions, statuses, routing rules, and escalation conditions.
- Reference docs written for agents: short and structured, with when to use a source and when not to.
- A procedural playbook: what to check first, which source wins, when to stop, when to escalate.
- Evals built from realistic examples and past human corrections.
- A provenance footer on every important output: where it came from, freshness, confidence, whether it was reviewed.
- Correction harvesting, so every "wrong source" or "too generic" becomes a doc update or a new eval.
- A maintenance trigger: when a schema, label, or definition changes, the matching doc changes in the same edit.
What that means in practice
I would not just connect Claude to a workflow and trust it to act. I define the source of truth, add approval and escalation rules, test against real examples, and show provenance so users know what a recommendation was based on.