It ran green for three weeks and did nothing
The production checklist every agent workflow should ship with, drawn from what buyers find after freelance and agency handovers. Outcome checks, idempotency, cost caps, owned credentials, a runbook.
- KKK Manoj KumarCo-founder
4 min read
The stories builders tell in automation forums are rarely about crashes. One describes a dedupe rule that stopped firing and ran green for weeks (r/n8n). Another describes an agent that closed tasks whenever a client replied "thanks", which took three weeks and an angry client to notice (r/n8n). A third describes one bad IF condition that kicked off 47,000 runs in six hours, about $2,500 in direct cost and three days of database cleanup (r/automation). Then come the handovers: credentials in the freelancer's personal accounts, alerts still going to the builder's inbox after the contract ended, backups nobody ever tried to restore (r/n8n).
None of these are model problems. The model did roughly what it was asked. What was missing is the production work around it, and that's the work burned buyers now ask about by name. This is the checklist we ship every agent workflow with.
Job status is the wrong signal
A green run means the code didn't throw. It says nothing about whether invoices became bills or leads reached the CRM.
Every workflow needs a count of what it was supposed to produce, compared with what it did produce. Invoices received against bills created. Orders emailed against orders in the ERP. Run that comparison daily, alert when it drifts, and alert loudest when the output count is zero on a working day, because zero is what a silent failure looks like. Canary records help too: a known test item that should come out the other end every day, with an alert when it doesn't. For anything that writes to a system of record, snapshot what the agent is about to change and diff it afterwards, so "updated the wrong one of two near-identical contacts" shows up in a report before it shows up in a complaint.
Idempotency is the other half. The CRM times out after creating the record, the workflow retries, and now there are two leads. Every write needs a key derived from the source event, such as the email's message ID plus a hash of the line, and the write checks for that key before creating anything. Retries become safe, and you can replay a bad day without making it worse. A downstream duplicate check that matches on amount, date and vendor alias as well as the reference number catches what the key misses.
Limits that hold at 3am
Cost caps belong in code: a ceiling per run, per day and on runs per hour, with a circuit breaker that pauses the workflow when one is hit. Alert at half the daily budget and stop at the full amount. The 47,000-run night is what a missing runs-per-hour limit looks like, and adding one before go-live takes an afternoon.
Retries need a ceiling as well. A few attempts with backoff, then the item moves to an exception queue that a person works through. A tool that fails at midnight and gets retried until morning costs more than most bugs you'll ever ship, and it hides the real error under thousands of identical log lines.
A handover that survives the builder leaving
Most complaints after freelance and agency projects come from this part, and it's the easiest part to check before you pay the final invoice.
Credentials live in the client's accounts. API keys, service accounts, the model provider, the hosting: created by the client or transferred to them, with the builder's access removed on a date written into the handover. Alerts go to the client's channel or on-call rotation from day one, never to the builder's email.
Run a restore test. Take last night's backup, restore it into a scratch environment, run the workflow against it and write down the date it worked. Until that has happened, the backup is a hope.
Name a reviewer. One person on the client side reads the weekly report, owns the exception queue and receives the alerts. If nobody can be named, the workflow isn't ready to run unattended.
Write a runbook that someone who didn't build the workflow can follow. What it does, how to pause it, how to replay a day, where the logs are, what the common failures look like and what to do about each one.
Then hand over properly. The code goes into the client's repo, a walkthrough gets recorded with their engineer, and there's a week where the builder is unavailable on purpose and the client runs it alone. Whatever breaks that week goes into the runbook.
The checklist
Before an agent workflow goes live, we want a yes to each of these:
- Daily outcome counts, with alerts on drift and on zero.
- An idempotency key on every write, plus a downstream duplicate check.
- Cost caps per run, per day and per hour, enforced in code.
- Credentials and alerts in the client's own accounts.
- A restore test, with the date it passed.
- A named reviewer on the client side.
- A runbook someone other than the builder has followed.
- A handover week with the builder hands-off.
If you're buying one of these, ask to see each item demonstrated on the running workflow before the final payment. Each one takes a few minutes to show, and anything that takes much longer than that probably isn't there. The same list is part of how we work.