Your Agent Works. It's Still Not a System.
A prototype that shines in a demo and a system that holds up day after day in production are not the same thing. Most "AI systems" in enterprise environments are prompt wrappers with a pleasant UI — one large model, one oversized prompt, unrestricted tool access, and hope. Here's what replaces that.
Why the giant prompt falls apart
The instinct in every AI project is the same: one model, one prompt, every tool wired in. It works in the demo. Then reality arrives. Behaviour becomes unpredictable, because one prompt is trying to do everything. Nobody can explain afterwards why the system did what it did. Every user implicitly holds every permission the system has. And each capability you add makes all the previous ones slightly worse, because they compete for the same context.
Enterprise-grade agent systems are not configured. They are designed — for the specific use case, with the right layers, guards, and quality assurance.
The anatomy of an engineered system
An orchestrator, kept separate
The orchestrator is the traffic controller: it reads what the user is after, hands the request to the right agent, and decides which tools that agent may use. Keeping this job separate is what makes behaviour predictable — and what lets you see afterwards why the system did what it did. Bigger tasks get decomposed into subtasks, routed to sub-agents in parallel, and reasoned back together.
Specialist agents, narrow by design
Separate agents for research, origination support, marketing, internal helpdesk. Each stays narrow. Narrowness is what makes agents testable — and it means each agent only ever gets the access it genuinely needs. The research agent has no path to the invoicing system, not because a prompt forbids it, but because the architecture never gave it one.
One door for tools: the MCP gateway
Every time an agent reaches for a tool or data source, it goes through a single authenticated gateway — OAuth2-based, with per-agent allowlists. Agents are created for a purpose and see a curated selection of tools, not a universe of options. That saves token cost, cuts latency, and measurably improves quality: agents get worse when overwhelmed by unnecessary choices. In a dev system, an agent calls whatever it likes. In production, the gateway is where access is configured, granted, logged, and audited. It is probably the single most important security decision in the design.
One door for models: the LLM gateway
The same idea, applied to models: authentication, per-request routing, safe credential exchange. Run on a managed platform today, bring in a private model later if a real reason appears — without losing your grip on cost or usage. The model becomes a routing decision, not an architectural commitment.
Delegation: the agent acts as the user
Data does not equal data — some is public, some internal, some confidential. An agent should work in delegation of a user: it carries a delegation token derived from the user's access token, and downstream systems grant exactly what that user may see. CRM, trading, and finance data stay inside the wall; market data, news, and web search sit outside it. This is painful to retrofit, so it belongs in the design from day one.
The tool chain is an attack surface
| Threat | Architectural answer |
|---|---|
| Poisoned tools — manipulated outputs steer agent behaviour | Allowlists, provenance checks, internal/external separation at the gateway |
| Instruction injection — instructions smuggled in via tool descriptions or retrieved content | Treat all tool output as untrusted input; guards at the orchestrator, not in the prompt |
| Silent exfiltration — data leaves via an over-permissioned external tool | Per-agent scoping, delegation tokens, full egress logging |
| Supply-chain weak links — a tool dependency changes underneath you | Curated registry with review gates — the gateway as a genuine boundary, not a label on a box |
Governance is what lets you say yes
Every conversation, decision, tool call — and the cost of each — recorded and reviewable. For a regulated business this isn't a nice-to-have: it's what lets you stand behind the system in front of a regulator or a SOC 2 auditor. And it pays twice: usage data becomes the input for systematic improvement, and you gain full visibility of how data flows — from whom, to where, and why.
For any answer your system gave last Tuesday, can you say which agent produced it, which tools it called, what data it saw, whose permissions it ran under, and what it cost? If yes, you have a system. If no, you have a prompt wrapper — technical debt waiting for approval.
Go-live is the easy part. Staying reliable, monitored, compliant, and used — month after month — is the real job. The systems that fail rarely fail technically first; they fail organisationally, when nobody owns the running system. Engineering is what you do so that ownership is even possible.