Deterministic pipelines: the load-bearing rule for production AI agents

I get asked most weeks what the single hardest thing about building AI agents is. The honest answer is neither the model, the prompt, nor the tools. It is drawing the right line between what the model gets to decide and what code has to own.

Get that line right and the agent runs for years. Get it wrong and the agent works fine for two weeks, then breaks at 11pm on a Friday in a way that costs someone a customer.

The concrete failure

The clearest way to see the rule is to watch it get broken.

A team builds an agent that intakes new jobs from a chat channel. The agent talks to the person, works out what kind of job it is, and creates the job in the operational system. So far so good.

Somewhere in the prompt is an instruction along the lines of: when the job is confirmed, allocate the next job number in the sequence and enter it into the record. Sequence rule, the agent is told, is prefix by year, then a hyphen, then a four digit counter. The agent looks at the last few jobs, sees the pattern, and does the sensible thing.

For a while, this is fine. Then, quietly, three things start to go wrong. The agent occasionally skips a number, because it saw the previous job as 26-0117 when actually the last committed job was 26-0122. The agent occasionally duplicates a number, because two intake conversations happen in the same window and the model has no visibility of the other one. And on one memorable Tuesday, the model decides that the natural next number after 26-0999 should be 26-1000 in one conversation and 26-0100 in another, because the training corpus has more of the latter shape.

The finance team notices later. Or the auditor. Or the operations manager trying to explain a missing job in a client meeting.

The rule

The model reasons. Code owns the truth.

Anywhere in an agentic workflow where the answer has to be exactly right (an id, a number in a sequence, an arithmetic result, a lookup, a state transition), a deterministic pipeline component owns the answer. The model can propose that the action happens. Code executes it and writes the value.

Anywhere the answer needs judgement (was this enquiry a Type A job or a Type B job, is this a duplicate or a genuine new lead, does the description warrant the shorter template or the longer one), the model reasons and returns a proposal. A human, or a downstream code component with clear rules, confirms.

The line is not always obvious. The mistake most teams make in 2026 is putting it too far to the model side.

Three places I see this get missed

Identifiers and sequences. The example above. Never allocate a job number, invoice number, customer id, or any other sequence value from the model. Use an atomic counter in a database, or a canonical sequence generator in the operational system. The model can say "please create the next job." It never names the number.

Money arithmetic. Fees, totals, splits, discounts, tax. All of these need to be computed by code, from typed inputs, with clean rounding and audit trails. Every fee-letter engine I have ever seen that let the model do the arithmetic ended up with two invoices that were $17 apart because the model rounded internally in a way the total did not. Do the arithmetic in code, in integer minor units, and return the number to the model to be rendered in the response.

Identifier lookup. "Please pull the record for the customer named John Smith." There are eleven John Smiths in the database. The model, faced with an ambiguity, picks one. It picks confidently. It happens to pick the wrong one. The next action writes to the wrong record.

Lookups that could be ambiguous should be treated as tool calls that return a disambiguation candidate list back to the agent, not a single value. If the candidate list has one entry, proceed. If more, either ask the user or refuse. Never let the model quietly pick.

What the pattern looks like in code

The mental model is a two-track system.

The reasoning track is where the LLM lives. It reads context, thinks about intent, drafts responses, classifies inputs, decides what to do next. It never touches state that matters.

The commit track is deterministic. It runs code, calls typed APIs, updates databases, sends emails, writes to storage. It only fires when a specific proposal from the reasoning track has been confirmed, either by a person (approve-first) or by a downstream rule check.

The interface between the two tracks is a tool schema. Tools have typed inputs and typed outputs. The model asks a tool to do something. The tool decides whether it can and how. The tool logs what happened. Nothing about the resulting state depends on the model's phrasing or reasoning after the tool is invoked.

This looks obvious when it is written down. It is the opposite of what a lot of 2025-era agent frameworks make easy.

Why "just tell the model to be careful" is not the answer

The tempting shortcut when you see a failure mode is to add another instruction to the prompt. "Always double-check the job number." "Never round the total." "Verify the customer id."

This works for a while. It fails at scale for two reasons.

First, prompt instructions are advisory to the model, not enforceable. The model complies most of the time. When it does not comply, you find out on Tuesday.

Second, prompts get long, and long prompts hide their own contradictions. The tenth "please be careful" is at war with the third "be concise" and the fifth "act decisively." The model is doing its best to interpret a document written by committee.

The load-bearing invariants of your system belong in code, where they cannot be overridden by a stochastic process. The nice-to-haves and the tonal preferences belong in the prompt.

The design consequence

When you commit to the deterministic-pipeline rule, a lot of downstream decisions get easier.

The audit trail becomes precise, because every state change is a code event with a timestamp and an actor.

The rollback story becomes clean, because every code action can be inspected and reversed by code.

The onboarding of new agents becomes cheaper, because the reasoning layer is model-portable but the commit layer stays the same. You can swap Claude for something else next year and the deterministic core does not change.

The safety story with the auditor becomes real, not aspirational, because you can point at where the model influenced anything commercially serious and show that the answer came from code, not from stochastic completion.

One rule, most often ignored

If I had to give a single piece of design advice to any team building an AI agent in 2026, this is it. Draw the line early. Draw it firmly. Anywhere it matters exactly, code owns the answer. Anywhere it needs judgement, the model reasons and code confirms.

Most agents that fail in production fail because this line was drawn too far to the model side. Most agents that survive their first year drew it early and held it.

---

This piece is a companion to the AI Agentic Development pillar page. If you are thinking about where deterministic pipelines should sit in your own agent build, get in touch.