Building Complete Audit Trails for AI Agents

When your AI agent takes a wrong action, the first question is: what did it see, what did it decide, and why? If you can't answer that from a log, you don't have an audit trail — you have a black box with a history of outputs.

Most teams logging AI agent activity capture too little: the final output, maybe the model used, maybe an error if one occurred. What they're missing is the full execution context that compliance frameworks require and incident response depends on — every inference call, every tool call, every policy decision, linked into a distributed trace with cost accounting included.

This article describes what a complete AI agent audit record looks like, what each field provides, and how to build the infrastructure to capture, store, and query it.

1. Why AI Agents Need Audit Trails

Autonomous actions with side effects. A traditional application takes actions because a human clicked a button. The human made the decision; the application executed it. An AI agent takes actions because the model decided to — based on its interpretation of the system prompt, conversation history, tool descriptions, and current input. When an agent sends an email you didn't intend, or queries a database you didn't expect, the only way to understand what happened is a complete record of what the agent was told, what it decided, and what it did.

Non-determinism makes testing insufficient. The same input to an AI agent can produce different outputs and different tool call sequences on different runs. You cannot enumerate all possible behaviors in a test suite. Audit logs are the runtime evidence that the system is behaving as intended, and the forensic record when it doesn't.

Compliance requirements are explicit and enforceable. HIPAA §164.312(b), SOC 2 CC6, GDPR Article 30, and EU AI Act Article 12 all require specific logging of AI system activity. An audit trail that can't answer an auditor's specific question is as bad as no audit trail — and may constitute a compliance failure in its own right.

2. The Anatomy of an AI Audit Record

A complete audit record for an AI agent LLM call contains these fields, each serving a specific forensic or compliance purpose:

{
  // Event identity
  "event_id": "evt_01j9xk2mq3nf8vab7cdp4r5s6t",
  "kind": "llm_call",             // "llm_call" | "tool_call"
  "timestamp": "2026-07-14T09:23:41.882Z",

  // Principal identity
  "user": "alice@acme.com",
  "agent_id": "support-bot-v2",
  "customer_id": "cust_acme_prod",
  "thread_id": "conv_xk29abcd",

  // Model and routing
  "model": "gpt-4o-2024-11-20",
  "provider": "openai",
  "resolved_endpoint": "https://api.openai.com/v1",

  // Token economics
  "input_tokens": 1842,
  "output_tokens": 347,
  "cost_usd": 0.00624,

  // Policy decision
  "verdict": "allow",             // "allow" | "block"
  "rule_id": null,               // rule that fired if blocked
  "local_decision": true,        // decided by cached bundle
  "approval": false,

  // Content findings (PHI, content rules)
  "findings": [
    {"kind": "phi", "entity": "US_SSN", "action": "redacted"},
    {"kind": "content_rule", "rule_id": "rule_ssn_detect", "action": "flag"}
  ],

  // Trace linking
  "trace_id": "trc_7f2a9b1c4e",
  "span_id": "spn_4c8d2f1a",
  "parent_span_id": "spn_3b7c1e9f",
  "step": 3,

  // Tool-call specific fields (kind="tool_call" only)
  "tool_name": null,
  "tool_server": null,
  "mcp_capabilities": null,
  "mcp_trifecta": null
}

Three fields deserve specific attention. resolved_endpoint is the compliance receipt for egress controls — the actual URL the call went to, read from the client object at call time, not just the provider name. local_decision distinguishes in-process enforcement (from the cached policy bundle, sub-millisecond) from cloud decisions (budget check, ~20ms). findings records what was detected and what action was taken — without storing the original sensitive values that would create a secondary PHI store.

3. User Prompts in the Audit Trail

What to log from user prompts is the most complex design decision in audit trail architecture. Four competing pressures exist simultaneously:

• Forensic need — understanding what the user said is essential for incident investigation
• GDPR data minimization — storing full prompt text creates a secondary PII store with its own retention and deletion obligations
• HIPAA — if PHI appeared in the prompt, the audit log must not become an uncontrolled PHI repository
• Storage cost — full prompt text for millions of calls at 1,000–2,000 tokens average is significant at scale

The recommended approach is metadata-rich, content-minimal: log what was detected (entity types, rule matches), log a SHA-256 hash of the original text for integrity verification, but not the prompt text itself. If forensic need requires the actual content for a specific incident, that should live in a separate, access-controlled store with explicit retention policies — not in the general audit log where it's accessible to anyone with log access.

For HIPAA specifically: if PHI was redacted before the LLM call, the audit record should contain detected entity types (phi_entities: ["US_SSN", "PERSON"]) and the redaction action — not the original values. The record proves PHI was handled correctly without creating a secondary exposure.

4. Tool Call Audit Records

Tool call events use the same schema with kind="tool_call" and additional tool-specific fields. These are the most security-critical events in the audit trail — the record of what the agent actually did in the world, not just what it decided.

{
  "event_id": "evt_01j9xk3pr4qg9wbc8def5s7u8v",
  "kind": "tool_call",
  "timestamp": "2026-07-14T09:23:44.211Z",

  // Same identity fields as llm_call
  "user": "alice@acme.com",
  "agent_id": "support-bot-v2",
  "customer_id": "cust_acme_prod",
  "thread_id": "conv_xk29abcd",

  // Tool identification
  "tool_name": "search_knowledge_base",
  "tool_server": "https://kb.internal.acme.com",

  // Policy decision for this tool call
  "verdict": "allow",
  "local_decision": true,
  "approval": false,

  // MCP / trifecta state at the time of this call
  "mcp_capabilities": {
    "private_data": false,
    "untrusted_content": false,
    "exfil_capable": false
  },
  "mcp_trifecta": false,

  // Trace linking — connects to parent llm_call span
  "trace_id": "trc_7f2a9b1c4e",
  "span_id": "spn_5d9e3g2b",
  "parent_span_id": "spn_4c8d2f1a",
  "step": 4,

  // No token economics for tool calls
  "cost_usd": 0
}

The tool_server field is the Destination column in the dashboard — it tells you which external service the agent called. The mcp_capabilities snapshot records the trifecta state at the time of the call, which is essential for forensic reconstruction of why a trifecta block did or did not fire at that step. The dashboard renders a red trifecta banner on events where mcp_trifecta: true.

5. Policy Decision Audit

Every enforcement decision — allow or block — needs to be recorded with enough context to explain it to both a compliance auditor and an engineer debugging an unexpected block. For block verdicts, the audit record must include:

• verdict: "block" with the timestamp of the block decision
• rule_id: the specific rule that matched, such as rule_ssn_pii
• rule_name: a human-readable name for dashboard display and compliance reports
• local_decision: true: the decision was made from the cached policy bundle, not by the cloud control plane

For allow verdicts that required human approval, the record additionally includes the approver's identity and the approval timestamp — creating an evidence trail that a human reviewed and explicitly approved the specific action, satisfying the EU AI Act's Art. 14 human oversight requirement.

6. Model Response Audit

Model outputs carry their own DLP risk. A response that includes customer data the agent retrieved from a database should not be logged in full to a third-party observability platform. The same PHI detection logic that runs on inputs should run on outputs before they are logged anywhere external.

For outputs, the audit approach matches inputs: log detected entity types and counts, not the full response text. If the response triggered a content rule (an output content scanner detected PII in the model's response), record the rule match and what action was taken — flag, block, or redact before the response reaches the user.

7. Cost Tracking as an Audit Trail Concern

Cost accounting is a first-class audit trail concern, not just an operational metric. The cost_usd field on every LLM call enables four capabilities that go beyond billing:

• Budget enforcement: Daily spend aggregated by customer, agent, and user is the basis for the budget gate (POST /v1/budget). The audit trail feeds the budget, not the other way around.
• Anomaly detection: A sudden spike in token consumption by a specific agent is a security signal — possibly a prompt injection that caused the agent to generate unusually long outputs, or a misconfigured agent in an infinite retry loop.
• Chargeback and attribution: In multi-tenant systems, cost attribution to specific customers and business units requires per-call cost records linked to customer_id and agent_id. Aggregate billing data is not sufficient for per-customer cost reporting.
• Compliance documentation: Internal audit requirements and financial controls may require evidence of AI spend and its business attribution — the audit trail is that evidence.

Pricing is calculated from a configurable rate table keyed by (provider, model). The rate table is part of the policy bundle, so pricing changes propagate through the same TTL refresh mechanism as policy changes — no SDK updates required when a model's pricing changes.

8. Compliance Logging Requirements

9. Distributed Trace Integration

An AI agent that completes a multi-step task generates many audit events. Without trace linking, these events are disconnected records — you can see each individual call but not how they relate to each other or what task they were executing.

Four fields link events into a coherent trace:

• trace_id: stable identifier for the entire agent run, from user request to final response
• span_id: unique ID for this specific event
• parent_span_id: ID of the event that triggered this one, enabling LLM call → tool call → LLM call linking
• step: monotonically increasing counter within the trace, enabling correct ordering without relying on timestamp precision across distributed systems

With these four fields, you can reconstruct the exact execution sequence of an agent run: which LLM call decided to invoke which tool, what the tool returned, which subsequent LLM call processed that result, and so on. The step counter is critical for multi-agent systems where events from different sub-agents may arrive at the audit store out of timestamp order.

The trace fields are compatible with OpenTelemetry's trace and span model, allowing InferenceFort spans to be correlated with broader application traces in Jaeger, Zipkin, or a cloud-native tracing backend.

10. SIEM Integration

Production AI agent deployments require SIEM integration for security monitoring, alert correlation, and long-term log retention. The structured JSON event schema makes integration straightforward — every event has consistent field names across all event kinds.

For Splunk, the structured JSON events map cleanly to indexed fields. Useful starting queries:

• All policy blocks by rule: index=ai_agents verdict=block | stats count by agent_id, rule_id
• Trifecta events by customer: index=ai_agents mcp_trifecta=true | timechart count by customer_id
• PHI detections by agent: index=ai_agents findings{}.kind=phi | stats count by agent_id, findings{}.entity
• Budget anomaly — cost spikes: index=ai_agents | timechart sum(cost_usd) by agent_id | where sum(cost_usd) > 50

For Azure Sentinel, events flow through Log Analytics as a custom table, enabling KQL queries and alert rules. A Sentinel alert that fires when an agent makes more than 50 tool calls in a single session, or when trifecta events are detected for a specific customer, can correlate with network events or identity events from other sources to build a complete incident picture.

11. Audit Trail Architecture Diagram

12. Audit Implementation Checklist

• Emit a structured audit event for every LLM call including model, provider, resolved_endpoint, token counts, cost_usd, verdict, and findings — capture both pre-call (enforcement decision) and post-call (token accounting) data
• Emit a separate tool_call event for every tool invocation, with tool_name, tool_server, mcp_capabilities snapshot at call time, mcp_trifecta verdict, and parent_span_id linking it to the LLM call that triggered it
• Include (customer_id, thread_id, user, agent_id) on every event — these four fields enable every cross-cutting query that compliance auditors and incident responders need
• Use a compound (trace_id, span_id, parent_span_id, step) linking scheme so agent execution sequences can be reconstructed in correct order even when events arrive out of timestamp sequence
• Record cost_usd per call using an updateable rate table from the policy bundle — not hardcoded rates — so model pricing changes don't require SDK updates or create billing gaps
• Never log raw PHI or PII values in audit records; log entity types (US_SSN, PERSON) and action taken (redacted, flagged) — the record must prove correct handling without creating secondary exposure
• Implement WAL fallback for the audit event queue so events are durably buffered if the ingest API is temporarily unreachable — no audit gap on brief network interruptions
• Partition audit storage by (tenant_id, date) so retention-based deletion and per-customer data exports are partition-level operations, not full-table scans across billions of rows
• Build GDPR Art. 17 per-customer log deletion before you receive the first erasure request — building it under deadline pressure is how data gets incorrectly deleted or kept
• Forward events to SIEM in real time; configure alerts for trifecta events, budget breach events, and anomalous tool call volumes using the specific queries in section 10
• Protect audit log integrity: ensure records cannot be modified or deleted without a separate access-controlled operation that itself generates an audit entry in an independent store
• Test audit completeness quarterly: run a known policy violation in a staging environment, then verify the event appears in the audit store within your defined SLA with all required fields populated and queryable