The Model Is the Easy Part

In the AI systems I have worked on, the model invocation is small enough to understand in an afternoon. Most of the engineering work lives around it: permissions, policy, evals, observability, retries, and the boundary around which parts of a workflow the model may touch.

A capable model makes the first demonstration arrive quickly. Production adds partial permissions, stale records, malformed tool arguments, user corrections, and provider timeouts. Improving the model may help some of those cases, but it does not decide who can authorize a payment or how the system recovers after a tool fails halfway through a workflow.

The following questions now come before model selection in my reviews.

Product surface

A model proposal still has to cross the product’s boundary

A conceptual product map. Choose a valid proposal, a policy rejection, or a provider timeout and follow the resulting workflow.

Scenario

Valid proposalPolicy blocks itProvider timeout

This example assumes delivery is confirmed after authorization; future changes are evaluated separately.

Sequence

1. Capture the inputCurrent
2. Bound the contextUpcoming
3. Request a proposalUpcoming
4. Enforce the boundaryUpcoming
5. Deliver into the workflowUpcoming
6. Keep the traceUpcoming
7. Evaluate a future changeUpcoming

Capture the input

Accept the input in the channel where work happens and preserve its provenance.

Read the full explanation

Valid proposal

This example assumes delivery is confirmed after authorization; future changes are evaluated separately.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Enforce the boundary. Code checks the proposed action against schemas, allowed actions, and required approvals before delivery.
5. Deliver into the workflow. Write the authorized result back to the workflow and retain evidence of the side effect.
6. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.
7. Evaluate a future change. Run retained cases against a candidate model, prompt, or retrieval change before release. Offline evals do not replace the per-run policy gate.

Policy blocks it

A plausible output can still be outside the product’s authority.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Reject the action. The proposed action violates the configured runtime boundary even if the model’s language sounds confident.
5. Hold the side effect. Do not deliver the rejected action. Preserve the proposal and reason so a person can resolve it or the workflow can stop.
6. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.

Provider timeout

The surrounding system still owes the workflow a clear outcome.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Handle missing output. A provider timeout leaves no usable proposal. The configured failure path holds delivery and hands off or stops the run.
5. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.

Valid proposal

This example assumes delivery is confirmed after authorization; future changes are evaluated separately.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Enforce the boundary. Code checks the proposed action against schemas, allowed actions, and required approvals before delivery.
5. Deliver into the workflow. Write the authorized result back to the workflow and retain evidence of the side effect.
6. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.
7. Evaluate a future change. Run retained cases against a candidate model, prompt, or retrieval change before release. Offline evals do not replace the per-run policy gate.

Policy blocks it

A plausible output can still be outside the product’s authority.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Reject the action. The proposed action violates the configured runtime boundary even if the model’s language sounds confident.
5. Hold the side effect. Do not deliver the rejected action. Preserve the proposal and reason so a person can resolve it or the workflow can stop.
6. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.

Provider timeout

The surrounding system still owes the workflow a clear outcome.

1. Capture the input. Accept the input in the channel where work happens and preserve its provenance.
2. Bound the context. Fetch permitted records, history, and rules before asking the model for a proposal.
3. Request a proposal. The model classifies, extracts, or drafts the narrow output it is assigned. A proposal does not authorize a side effect.
4. Handle missing output. A provider timeout leaves no usable proposal. The configured failure path holds delivery and hands off or stops the run.
5. Keep the trace. Preserve context, proposal, outcome, latency, and cost as available. Use failures and corrections to inform future tests and system changes.

Try it yourself

Inject a bad proposal into the product boundary

Compare the same sample input with independent runtime checks and with prompt-only instructions. This local simulation performs no tool calls or downstream actions.

Model or provider result

Valid proposalMalformed argumentsDisallowed actionProvider timeout

Runtime boundary

Code enforces checksPrompt instructions only

Evidence retained

Full traceFinal answer only

Downstream result if the tool is invoked

Effect confirmedRejected, no effectOutcome unknown

1. 01
   Intake and context

   Keep source, workflow identity, and permissions attached.

   Continues
2. 02
   Model proposal

   Output is a proposal; authority remains outside the model.

   Continues
3. 03
   Schema check

   Tool arguments match the contract.

   Continues
4. 04
   Authority check

   The proposed action exceeds the allowed boundary.

   Stopped / failed

blocked. The proposal is rejected before any side effect. Preserve the attempted action and the violated rule.

After this run · offline evaluation

Prompt, context, model version, tool arguments, policy verdict, and side-effect outcome can inform diagnosis and a regression case.

Offline evals inform the next release. They cannot retroactively block this run.

Inspect the simulation’s limits

The sample schema and authority gates are deterministic. The downstream result is a separate observation, used only when the tool is invoked. Passing validation does not prove delivery. Real products also need permissions, idempotency, validation at each boundary, and operation-specific recovery. A confirmed result with prompt-only instructions demonstrates one successful input; it does not establish a safe product boundary.

The part no demo shows

A production path needs an answer for retries, stale records, partial permissions, user corrections, provider timeouts, and malformed tool arguments. It also needs an owner when the system cannot decide safely.

If I am reviewing an AI feature now, I ask for these before I ask which model it uses:

Teams often plan to add those controls after launch. By then, production traffic is the only dataset and each fix happens under customer pressure. Tracing, evals, and recovery paths let the team learn from that traffic without treating every new case as an incident.

Start from the user's day, not the capability

Projects lose time when a team begins with a capability and searches for somewhere to install it. I start by walking through the workflow: where work accumulates, what gets dropped, and which judgments depend on an experienced operator. That walkthrough may lead to an LLM, or it may lead to a queue and a form.

Decide what the model is not allowed to decide

The most important architectural call in any system I've shipped was drawing the line between what the model proposes and what the code enforces. Anything involving money, compliance, or an irreversible action goes on the code side of the line. A prompt is a suggestion to a statistical process; it can drift with the next model version. Behavior that must never happen should be impossible, not unlikely. And "impossible" is spelled with an if statement, not with capital letters in the system prompt.

Don't fight the model longer than the evidence supports

You can't out-prompt a wrong architecture. I've tried, and I wrote up how that went on a voice product. There's a version of persistence that's really just attachment to your first design. When production keeps producing the same class of failure and your fixes keep moving the needle less each time, the system is telling you something. Low ego is operationally useful here. Being comfortable deleting your own work is a competitive advantage in a field where the first version is usually wrong in some way you couldn't have predicted.

The outer loop is the durable advantage

Model quality is a rented advantage. Whatever you have, your competitor can call the same API next quarter. What compounds is the loop around it: evals that tell you whether a change helped, observability that turns a vague complaint into a traceable failure, retrieval and context that improve with every correction, feedback from real usage flowing back into the system. Two teams with identical models will diverge completely on the strength of their outer loops. I've never seen prompt cleverness compound. I've watched eval suites do it.

The loop has to be outside the model to be useful. If the same prompt that generates the answer is also expected to enforce policy, validate tool calls, decide when to escalate, and explain whether it did a good job, the system has no independent instrument. The model can be part of judgment when the question is semantic. It should not be the only witness.

The models are capable and continue to improve. The engineering question remains: what does the application do when a model response is wrong, incomplete, or unsafe to execute? Permissions, policy code, traces, and evals determine whether the team catches that response before a user experiences the consequence.