What makes an AI agent production-ready
A demo proves the model can do the task. Production needs scoped tools, guardrails in code, approvals, an audit log, evals from real cases, monitoring and a cost ceiling.
- KKK Manoj KumarCo-founder
4 min read
Most agent demos work. That's the problem. A demo on five hand-picked inputs tells you the model can do the task, and almost nothing about whether you can let it do the task on a Tuesday afternoon with nobody watching.
Over six years and 170+ automation projects, the gap between "it works" and "it runs" has rarely been the model. It's everything around it: what it can touch, what stops it, who signs off, what it leaves behind, and how you know it's still doing the job next month.
Here is what we check before an agent goes live.
Scope the tools, not just the prompt
An agent is only as dangerous as the tools you hand it. A prompt that says "never issue refunds over $500" is a request. A refund tool that rejects amounts over $500 is a rule.
Give each agent the smallest set of tools that covers its job, and make each tool as narrow as you can:
- Read access by default. Write access only where the workflow needs it, and only to the records it owns.
- One tool per action.
createPaymentHoldis easier to reason about, test and audit than a generalupdateInvoice. - Credentials per agent, not a shared service account, so you can revoke one without breaking five.
If you can't list every system an agent can change on one page, the scope is too wide.
Put guardrails in code
Guardrails belong where the model can't talk its way past them: in the tool layer and the runtime, not the system prompt. Hard limits on amounts, recipients and steps per run. Checks that stop a run outright when something looks wrong, like a supplier's bank details changing the day an invoice arrives.
Here is roughly what that looks like as configuration. The names are illustrative; the shape is what matters.
export const invoiceAgent = defineAgent({
name: "supplier-invoice-matching",
tools: {
readInvoice: { system: "erp", access: "read" },
readPurchaseOrder: { system: "erp", access: "read" },
createPaymentHold: { system: "erp", access: "write", approval: "required" },
},
guardrails: {
maxAmountWithoutApproval: 5_000,
allowedCurrencies: ["USD", "EUR", "INR"],
stopIf: ["supplier_bank_details_changed", "duplicate_invoice_number"],
},
approvals: {
approver: "accounts-payable-lead",
expiresAfter: "24h",
onExpiry: "escalate",
},
limits: { maxStepsPerRun: 20, maxCostPerRunUsd: 0.5 },
});Every value there can be read, argued about and changed without touching a prompt.
Approvals where the risk is
Human approval is not a sign that the agent isn't good enough. It's how you decide where judgment sits. The two common mistakes are approvals everywhere, which trains people to click "approve" without reading, and approvals nowhere, which means you hear about the first bad action from a customer.
Put approvals on actions that are expensive to undo: money moving, messages leaving the company, records being deleted. Make each request self-contained: what the agent wants to do, why, and the evidence it used, without opening three other systems.
Then decide what happens when nobody answers. An approval that waits forever is an outage with extra steps.
An audit log someone can read
When something goes wrong, the first question is "what did the agent do, and why?" You need the answer in minutes, not days.
A useful audit log records each run as a sequence: the input that started it, every tool call with its arguments and result, every approval with who gave it and when, and the outcome. The runtime writes it, not the agent, so nothing is left out, and the process owner can read it, not only the engineer who built it.
Evals from real cases
A public benchmark tells you little about your invoices. Build the evaluation set from your own work: real past cases, with the outcome a person actually chose. Include the awkward ones on purpose: the partial shipment, the duplicate, the customer who replied in two languages.
Run the set before every change to the prompt, the tools or the model. When the agent gets a production case wrong, that case goes into the set. After a few months, it's the most accurate description of the workflow you have.
Monitoring that matches the job
Uptime is not the metric. An agent can be up all day and quietly wrong. Watch the numbers that describe the work:
- How many runs finished, failed or stopped for approval.
- How often people reject or edit what the agent proposed.
- How long runs take compared with the manual baseline.
- How many inputs look unlike anything in the eval set.
Alert on changes, not only on errors. A jump in rejected approvals is often the first sign that something upstream changed.
Know the cost of one run
Every step costs money: model tokens, tool calls, retries. Know what one run costs, set a ceiling per run and per day, and stop the run when it hits the ceiling. A loop that retries a failing tool all night is the most expensive bug you'll ship. Compare cost per run with the cost of doing the work by hand, and you have the number the business cares about.
The checklist
Before an agent handles real work, we want a yes to each of these:
- Every tool is listed, scoped to the minimum and has its own credentials.
- Hard limits live in code, not in the prompt.
- Irreversible actions need approval, and unanswered approvals escalate.
- Each run leaves a complete audit trail the process owner can read.
- An eval set built from real cases passes before every change.
- Monitoring tracks outcomes, rejections and unfamiliar inputs, not just uptime.
- There is a cost ceiling per run and per day.
- One named person owns the workflow and reads the reports.
None of this is exotic. It's the discipline you'd apply to any system that acts on your business. The difference is that an agent makes decisions, so the boundaries around them have to be explicit before day one, not after the first incident.