An agent demo is a short walk down a happy path. Production is a long walk down every other path, and the gap between the two is where most agent projects quietly die. The failures are rarely about model quality. They are structural, and they repeat across every codebase we are called into.
Error compounding is multiplicative, not additive
A single tool call at 95% reliability feels fine. Chain ten of them and you are at 0.95^10 — roughly 60%. Two out of five runs fail somewhere. Teams respond by upgrading the model, which moves per-step reliability to maybe 97% and overall to 74%, then wonder why the remaining failures feel unfixable.
The fix is not a better model. It is fewer steps in the critical chain, checkpoints that let a run resume rather than restart, and treating any step below 99% as a step that needs deterministic code around it.
Context windows are a budget, not a container
Long-running agents accumulate history: tool outputs, retrieved documents, intermediate reasoning. Teams treat the window as somewhere to put things, until behavior degrades halfway through a session for no visible reason.
What actually happens:
- Instructions at the top get buried under thousands of tokens of tool output and lose their pull on behavior.
- Retrieved chunks contradict each other, and nothing in the prompt says which to trust.
- Cost scales with the square of session length, because every turn resends the whole history.
Treat the window as a budget with a hard cap. Summarize aggressively, keep the system instructions adjacent to the current task rather than at the far top, and drop tool output that has already been acted on.
Tools fail in ways the model narrates instead of surfacing
Give a model a tool that returns {"error": "rate limited"} and it will frequently apologize, invent a plausible result, and continue. The run completes. The output is wrong. Nothing in your logs looks like an error.
Tool errors need to be structural, not textual — a failed call should halt or retry in your orchestration layer, not arrive as a string the model gets to interpret. Anything the model can rationalize, it eventually will.
There is no evaluation, so there is no feedback loop
Most teams ship agents on vibes: someone runs a few prompts, output looks good, it goes out. Then a prompt tweak three weeks later breaks a case nobody remembers testing.
A minimum viable eval is smaller than people expect — 30 to 50 recorded real inputs with known-good outcomes, run on every prompt change, scored on task completion rather than text similarity. That is a day of work and it converts prompt engineering from guesswork into measurement.
Non-determinism gets pushed onto users
The last failure is a design one. Teams wrap a stochastic system in an interface that implies certainty: no way to see what the agent did, no way to correct it mid-run, no way to retry a single step.
Show the plan before execution on anything expensive or irreversible. Make each step inspectable. Let people intervene without starting over. Users forgive a system that is visibly uncertain far more readily than one that is confidently wrong.
The pattern underneath
Every failure above comes from the same mistake: treating a probabilistic component as a deterministic one. The model is a component with a known error rate, and the engineering around it — retries, checkpoints, evals, guardrails, human handoffs — is what turns that component into a system.
That work is unglamorous and it is most of the job. Teams that accept this ship agents that survive contact with real users. Teams that keep waiting for a model good enough to skip it are still waiting.