Secure AI Agent Architecture for Enterprise Applications

The tools you give an agent define its blast radius. The trust boundaries you draw define your attack surface. Both are architecture decisions — and most teams make them without thinking about security.

An agent with read-only tools in a sandboxed environment can be prompt-injected and still cause minimal harm. An agent with email-sending tools, database write access, and API keys for external services in an unsandboxed environment can be injected and cause catastrophic harm. The architecture determines the damage ceiling, not the model's safety features.

This article presents a complete reference architecture for secure enterprise AI agents — not a theoretical framework, but the specific components, their interactions, and the security properties each must provide.

1. Why Architecture Decisions Are Security Decisions

The canonical AI agent security failure is prompt injection: malicious content in the agent's input — a web page it retrieved, a document it read, a tool response it received — that causes it to take actions the user did not authorize. The severity of that failure depends entirely on what tools the agent has and what trust boundaries exist around them.

Three architectural decisions have the most security impact. First, the tool grant surface — what tools the agent can call, and whether high-privilege tools like email and HTTP are available at all. Second, the trust boundary between untrusted external content and privileged operations — whether the agent treats a retrieved web page with the same trust as an internal database. Third, the enforcement layer position — where policy is checked relative to actions with side effects.

2. The Nine Core Components

A production-grade secure AI agent architecture has nine distinct components, each with specific security responsibilities:

• Agent Runtime: The orchestration layer — LangGraph, CrewAI, AutoGen, or custom. Coordinates multi-step reasoning, maintains conversation state, and routes between tools.
• MCP Servers: Tool backends — filesystem, database, web, email, code execution. The attack surface. These are where side effects happen.
• LLM Provider: OpenAI, Anthropic, local Ollama, or vLLM. The model makes decisions; the provider choice affects data residency and latency.
• SDK Enforcement Layer: In-process interception of every LLM call and tool call. Policy evaluation, PHI redaction, trifecta detection, audit event emission. The architectural center.
• Policy Engine: The control plane — stores and distributes policy bundles. The SDK caches the bundle locally and re-fetches on a configurable TTL (default 5 minutes). Policy changes propagate within one TTL.
• Audit Store: Structured event log with every inference call, tool call, policy decision, cost record, and enforcement action. The forensic record.
• Human Approval Service: Out-of-band approval workflow for high-risk tool calls. Blocks the tool call until a human approves or denies it.
• DLP / Redaction Layer: In-process PHI and PII detection and redaction, running inside the enforcement layer before the LLM call. Ensures regulated data never leaves the process unredacted.
• Runtime Security: Process isolation, secret management, network egress controls. Operates at the infrastructure layer independently of the enforcement SDK.

3. Full Reference Architecture

4. The Enforcement Layer

The enforcement layer is the architectural keystone — where security guarantees are actually provided. It has two interception surfaces with deliberately separate re-entrancy guards.

LLM surface — wraps BaseChatModel._generate_with_cache, _agenerate_with_cache, stream, and astream. A single context-local _active flag prevents double-counting when streaming falls back to invoke internally. Every LLM call goes through _enforce() (pre-call: model access, content rules, egress, PHI redaction) and _record() (post-call: token count, cost, audit event).

Tool surface — wraps BaseTool.run/arun and mcp.ClientSession.call_tool. A separate _active_tool flag deduplicates tool-on-tool nesting. The tool guard is deliberately separate from the LLM guard: a tool that calls an LLM internally must still have that LLM call governed. Two guards, two purposes, neither suppresses the other.

The pre-call enforcement sequence for an LLM call:

1. Resolve provider and destination endpoint from the live client object
2. Evaluate model access policy: is this model allowed for this customer and agent?
3. Evaluate content rules: does the prompt match any blocked or flagged patterns?
4. Evaluate egress policy: is the resolved endpoint approved for this provider?
5. Check budget: is there daily spend headroom? (Cloud call, only when a budget cap is configured)
6. Run PHI redaction: strip identifiers from the prompt before sending to the LLM
7. Emit pre-call audit event (fire-and-forget, non-blocking)

If any step returns a block verdict, PolicyViolation is raised and the LLM call never happens. The prompt never leaves the process.

5. Tool Trust Hierarchy

Not all tools carry equal risk. The trust hierarchy classifies tools by the privilege level of their actions, which determines what controls apply before they execute.

The trifecta guard is the enforcement layer's most powerful tool-level control. When a session has touched both private data (data classified as sensitive by tool classification or content rules) and untrusted content (web pages, user documents, external API responses), any call to an exfil-capable tool (email sender, external HTTP, clipboard) is blocked before it executes. This triad is the exact precondition for a successful prompt injection attack that achieves data exfiltration.

6. The Session Model: Why (customer_id, thread_id) Is a Correctness Requirement

All per-session state in the enforcement layer — trifecta capability taint, trace ID, step counter, and span links — is keyed by (customer_id, thread_id), never thread_id alone. This is a correctness invariant, not an optimization.

Here is the specific bug that occurs if you key sessions on thread_id only, in a multi-tenant SaaS agent serving multiple customers from one process:

• Customer A's session processes a medical record — private_data=true is set on the session keyed to thread_id="session-1"
• Customer B coincidentally also uses thread_id="session-1" (application-assigned IDs like this collide far more than UUID4s)
• Customer B's session retrieves a web page — untrusted_content=true is now set on the same session key
• Customer B's next tool call to an email sender triggers a trifecta block — a false positive caused entirely by Customer A's private data taint bleeding into Customer B's session

The (customer_id, thread_id) compound key ensures session state is always customer-scoped. These taints never cross tenant boundaries because the key space never overlaps. When no thread_id is provided, the SDK mints one scoped to the current async context via context variables — concurrent requests in the same process never collapse into a shared session. Session maps are also LRU-bounded by INFERENCEFORT_THREAD_MAX (default 10,000) to prevent memory leaks in long-running processes with many distinct conversations.

7. Multi-Tenant Architecture

A single agent process safely serving thousands of customers requires three properties to hold simultaneously.

Context isolation. Each request carries its own (customer_id, thread_id) propagated through async call chains via Python's contextvars or TypeScript's AsyncLocalStorage. The enforcement layer reads these variables at evaluation time. No request can read another request's context variables.

Policy isolation. Each customer's effective policy bundle may differ. The control plane issues customer-specific bundles when INFERENCEFORT_CUSTOMER_ID is set. The enforcement layer selects the right bundle for each call using the request's customer_id — no shared policy state between tenants.

Credential isolation. The vault credential feature injects API keys from the policy bundle into the process environment at bundle apply time, scoped per customer. Customer A's OPENAI_API_KEY is isolated from Customer B's agent calls.

8. Egress and Data Residency Architecture

The egress control system enforces that each AI provider's calls go only to pre-approved destinations. This is the technical implementation of data residency policies — ensuring EU patient data only reaches EU-region model endpoints, or that on-premises data never leaves the corporate network.

The mechanism is a tenant-level provider-to-endpoint map stored in the policy bundle (egress_endpoints). Before every LLM call, the enforcement layer resolves the provider (from the client class and model name) and the destination endpoint (from the client's openai_api_base, azure_endpoint, or region_name attribute). It matches the resolved destination against approved endpoint patterns for that provider.

Endpoint pattern matching supports four forms: exact URLs, host suffix wildcards (*.openai.azure.com), region prefixes (region:eu-*), and wildcard (*). A call to a non-matching endpoint is blocked before any network traffic is sent. The Go backend mirrors this pattern matching exactly (policy.EvaluateEgress / MatchesEndpoint) for the async ingest backstop.

9. Human Approval Integration

The approval gate is implemented as a tool_action: "require_approval" verdict from policy evaluation. The enforcement layer holds the tool call and emits an approval request event. The approval service — your webhook receiver, Slack bot, or web UI — receives the event, notifies a human, and signals the verdict back. Approved calls proceed with an approved=true flag in the audit record; denied calls raise PolicyViolation with the denial reason and approver identity.

10. DLP Integration

PHI and PII redaction runs inside the enforcement layer's pre-call sequence, after content rule evaluation and before the LLM call. The ordering is deliberate and matters for compliance:

• Content rules run on the original text — a rule blocking SSN patterns detects and flags the SSN even if it will also be redacted
• The LLM sees only the redacted text — the model provider never receives the original identifiers
• The audit record logs what was detected (identifier types and count) not the original values — forensics without re-creating the PHI exposure

Redaction uses Microsoft Presidio under the hood: entity recognition via spaCy plus anonymization patterns. Installing presidio-analyzer and presidio-anonymizer activates it; if not installed, redaction is a no-op with a logged warning and never raises into user code.

11. Audit Architecture

Every enforcement event generates a structured audit record emitted fire-and-forget to the ingest API. The record contains the full execution context needed for forensic investigation and compliance evidence:

• Identity: user, agent_id, customer_id, thread_id
• Model routing: model, provider, resolved_endpoint
• Token economics: input_tokens, output_tokens, cost_usd
• Policy decision: verdict (allow/block), rule_id, local_decision
• Tool context: tool_name, tool_server, mcp_capabilities, mcp_trifecta
• Trace linking: trace_id, span_id, parent_span_id, step

The Go backend stores events in Postgres (partitioned by tenant and date), re-evaluates each as an authoritative backstop, and can forward to Splunk HEC, Azure Sentinel, or Elastic. The dashboard renders them in real time with filtering by user, agent, model, verdict, and trifecta state.

12. Reference Deployment

services:
  postgres:
    image: postgres:16-alpine
    volumes: [pgdata:/var/lib/postgresql/data]
    environment: {POSTGRES_PASSWORD: "${DB_PASS}"}

  inferencefort-backend:
    build: ./backend
    environment:
      DATABASE_URL: "postgres://..."
      JWT_SECRET: "${JWT_SECRET}"
    depends_on: [postgres]

  # Your agent process
  agent:
    build: ./agent
    environment:
      INFERENCEFORT_KEY: "${IF_KEY}"
      INFERENCEFORT_API_URL: "http://inferencefort-backend:8080"
      INFERENCEFORT_FAIL_MODE: "fail_closed"  # block on error
      INFERENCEFORT_CUSTOMER_ID: "${CUSTOMER_ID}"
    depends_on: [inferencefort-backend]