AI Compliance Checklist for Healthcare, Finance, and Enterprise AI Agents

Regulators don't care that your agent made a decision autonomously. They care that you can prove what it decided, on what data, and who was accountable. When an AI agent reads a patient record to draft a care summary, it is a HIPAA business associate. When it answers a European customer's support query, GDPR applies to the prompt text itself. The compliance obligations don't change because the actor is software.

What makes AI agents genuinely harder to govern than traditional software is that they are autonomous, non-deterministic, and multi-step. A traditional application has a defined set of database queries and API calls auditors can enumerate. An agent has open-ended tool access, dynamically generated payloads, and behavior that emerges from the interaction between the model, the system prompt, and runtime inputs.

This guide covers six frameworks in depth: HIPAA, SOC 2, GDPR, PCI DSS, ISO 27001, and the EU AI Act. For each, we answer what specifically applies to AI agents, what you need to implement, and what the penalty is for getting it wrong.

1. Why AI Agents Create New Compliance Challenges

Autonomous decisions with side effects. An agent that can send emails, execute database writes, or trigger external API calls can cause compliance violations without any human instruction at the moment of violation. Compliance frameworks assume a human decision-maker — or at minimum a deterministic system — triggered every regulated action. Agents break that assumption, which means technical enforcement has to fill the gap where human process reviews used to be sufficient.

Prompt text is regulated data. When a user tells a support agent "my account number is 4111..." or a healthcare assistant "I need to refill my Lipitor prescription," that prompt text is regulated data. It flows through your inference infrastructure, gets logged by the LLM provider, and may be retained for model improvement. Every layer of that pipeline is in scope — not just the application database.

2. HIPAA and Healthcare AI

HIPAA's Privacy and Security Rules apply whenever an AI agent processes Protected Health Information (PHI). PHI is any individually identifiable health information — clinical, demographic, or financial — if it could be used to identify a specific patient. Penalties for willful neglect range from $50,000 to $1.9 million per violation category per year.

The 18 HIPAA Safe Harbor Identifiers

Safe Harbor de-identification under 45 CFR §164.514(b) requires removing all 18 identifiers. When a patient tells a healthcare AI assistant "I'm Jane Smith, DOB 03/12/1971, and I need to know if my insurance covers the procedure my cardiologist Dr. Patel scheduled for next Tuesday," that single sentence contains at least six identifiers. Your enforcement layer needs to detect all of them before the prompt reaches any model provider.

• Names, geographic data smaller than a state, dates (other than year) related to an individual
• Phone numbers, fax numbers, email addresses, Social Security Numbers, medical record numbers
• Health plan beneficiary numbers, account numbers, certificate and license numbers
• VINs and serial numbers, device identifiers, URLs, IP addresses
• Biometric identifiers, full-face photographs, and any unique identifying number or code

BAA Requirements

Business Associate Agreements are required with every service that processes PHI on your behalf — including your LLM provider. OpenAI offers a BAA through their Enterprise tier; Anthropic has HIPAA-compliant options. But a BAA only covers data sent to those APIs. If your agent logs prompts to a third-party observability platform or caches responses in a vector store, those services need BAAs too.

The most defensible posture for healthcare AI is in-process PHI redaction before the LLM call. If PHI is stripped before the prompt leaves your process, your LLM provider never sees it, and the BAA question for that provider becomes moot.

Audit Logging Under HIPAA

The HIPAA Security Rule (§164.312(b)) requires audit controls that record and examine activity in systems containing or using PHI. For AI agents, this means retaining records for six years that include:

• Every prompt that contained PHI, including which identifiers were detected and redacted
• The tool calls the agent made, and what data was passed to those tools
• The model and provider used for each inference call
• The user and agent identity for every interaction
• All policy enforcement actions — blocks, redactions, and flags

3. SOC 2 for AI Systems

SOC 2 (AICPA Trust Service Criteria) is the dominant compliance framework for SaaS companies. When AI agents process customer data, they expand your SOC 2 scope in ways most companies haven't fully addressed. The AICPA released AI-specific guidance in 2024; auditors are now actively testing for AI controls.

Trust Service Criteria and AI

Security (CC6) — logical access controls. The tools you grant an agent are equivalent to the permissions you grant a service account. An agent that can query a customer database should have its access logged, reviewed, and bounded by least privilege — the same governance you apply to any service identity.

Availability (A1) — new dependency chain. Your AI agents introduce LLM providers, external tool APIs, and your policy enforcement layer as availability dependencies for business processes. Your SOC 2 controls need to document what happens when these are unavailable, including explicit fail-open versus fail-closed policy choices.

Confidentiality (C1) — prompts are confidential data. Customer data in prompts must be treated as confidential. Every third-party observability or logging tool you've added to your inference pipeline needs to be in scope for your confidentiality controls — including the ones added for debugging convenience.

Processing Integrity (PI1) — output validation. SOC 2 auditors are now asking what checks exist on agent outputs before they affect business data. Models are probabilistic; testing alone is not sufficient evidence of processing integrity for autonomous agents.

What Auditors Are Now Testing

• Evidence that model access is controlled — which models the agent can call, approved by whom, enforced at runtime
• Evidence of prompt data handling — how customer data in prompts is processed, logged, and protected
• Evidence of tool call governance — documented tool grants, approval workflows for high-risk tools
• AI-specific incident response procedures covering prompt injection, data leakage via tools, and model provider outages

4. GDPR and AI Agents

GDPR applies whenever an AI agent processes personal data of EU residents — regardless of where the agent or controller is located. The maximum fine is €20 million or 4% of global annual turnover, whichever is higher. For AI agents, several concrete requirements differ from traditional software.

Lawful Basis for Processing

Every processing activity needs a lawful basis under Article 6. For enterprise AI agents, the most common applicable bases are:

• Contract (Art. 6(1)(b)): The agent processes data to fulfill a contract with the individual — common for customer-facing AI assistants.
• Legitimate interests (Art. 6(1)(f)): Internal agents processing employee or business data. Requires a Legitimate Interests Assessment (LIA).
• Consent (Art. 6(1)(a)): Rarely appropriate for enterprise agents — consent must be specific, informed, and freely given for each processing purpose.

Special category data (Art. 9) — health data, biometrics, racial origin, political opinions — requires explicit consent or a specific regulatory basis. Healthcare AI agents processing EU patient data need both Art. 6 and Art. 9 bases established before any data touches the agent.

Right to Explanation and Automated Decisions

Article 22 GDPR restricts solely automated decisions that produce legal or similarly significant effects. If an AI agent's output directly determines a credit decision, an insurance premium, or an employment outcome, you need either human review in the loop or explicit consent from the data subject. Users also have a right under Articles 13 and 14 to know that an AI is involved in decisions affecting them and how that AI works.

Data Minimization in Prompts

Article 5(1)(c) requires data minimization — only processing personal data adequate, relevant, and limited to what is necessary. System prompts that pull entire customer records "just in case" violate this principle. Your prompt construction should be as specific as your database queries — request only the fields the agent needs for the specific task at hand.

Third-Country Transfers: The OpenAI and Anthropic Question

Sending personal data to OpenAI or Anthropic APIs may constitute a transfer to a third country. Both providers are enrolled in the EU-US Data Privacy Framework (effective 2023), which provides a transfer mechanism. You still need to:

• Confirm the provider is DPF-certified for the specific data types you're sending
• Execute a Data Processing Agreement (DPA) with the provider
• Document the transfer in your Record of Processing Activities (RoPA)
• Perform a Transfer Impact Assessment (TIA) if sending special category data

5. PCI DSS

PCI DSS v4.0 governs cardholder data (CHD) — primary account numbers (PANs), cardholder names, expiration dates, and service codes. Penalties for non-compliance range from $5,000 to $100,000 per month, and card brand fines can reach $500,000 per incident. The question for AI agents is whether they can encounter CHD in prompts, tool responses, or retrieval context.

When Cardholder Data Appears in Prompts

The scenarios are more common than teams expect:

• A customer service agent where users paste card numbers to report billing disputes
• A finance agent with tool access to transaction records that include full PANs
• A document processing agent that reads payment receipts or invoices
• A support agent with a tool that retrieves account payment methods including card details

If any of these apply, the agent's inference infrastructure is in PCI DSS scope. The LLM provider, the SDK, and the audit log system all need to meet PCI DSS requirements — or you need to remove CHD before it reaches any of those systems.

Scope Reduction Strategies

The least painful path is scope reduction — prevent CHD from ever appearing in prompts. Three practical approaches:

• Token substitution at the source: Replace PANs with vault tokens before data reaches the agent. The agent sees "TOKEN_4321" not "4111 1111 1111 1111".
• In-process PAN detection and redaction: Detect Luhn-valid card number patterns in prompts before the LLM call. Redact to the last four digits.
• Tool response filtering: Apply the same content scanning to tool call responses — strip PANs from database query results before the agent processes them.

PCI DSS Requirement 3.4 requires PANs to be rendered unreadable anywhere they are stored. If your audit log captures full prompt text containing PANs, those logs are in scope. In-process redaction before logging is the only way to keep audit logs out of PCI DSS scope.

6. ISO 27001

ISO 27001:2022 added Annex A controls directly relevant to AI systems. The framework is risk-based — you identify risks to information assets and select appropriate controls. AI agents introduce new risks that need to be explicitly addressed in your Statement of Applicability (SoA). Certification failure means losing the certification and rebuilding trust with enterprise customers who require it.

Relevant Annex A Controls for AI

• A.5.23 (Information security for use of cloud services): LLM providers are cloud services. Your cloud security policy needs to cover model API providers, including data handling requirements and incident notification SLAs.
• A.5.30 (ICT readiness for business continuity): AI agent availability now depends on LLM provider uptime — a new continuity dependency to assess and document.
• A.8.4 (Access to source code): Agent system prompts are functionally source code — they define the agent's behavior. Access controls to system prompts should match your access controls for application source.
• A.8.8 (Management of technical vulnerabilities): Prompt injection is a technical vulnerability. Your vulnerability management process needs to include AI-specific attack surfaces and mitigations.
• A.8.16 (Monitoring activities): Monitoring must cover AI agent behavior, tool calls, and policy enforcement events — not just network and system logs.

Risk Assessment for Autonomous Systems

ISO 27001's risk assessment (Clause 6.1.2) requires identifying risks to information assets. For AI agents, the novel risks that need entries in your risk register include: prompt injection leading to unauthorized actions, data exfiltration via tool calls, model inversion attacks exposing training data, and supplier risk from model providers with unilateral data handling policy changes. Each needs an associated control and documented residual risk acceptance.

7. EU AI Act

The EU AI Act is fully applicable from August 2026 and creates obligations based on risk classification. Penalties for non-compliance with high-risk system requirements reach €30 million or 6% of global annual turnover. For enterprise AI agents, the key question is whether the agent falls into a high-risk category under Annex III.

High-Risk AI System Classification

Several Annex III categories apply directly to common enterprise agents:

• Employment and workers management (Annex III, point 4): AI used to recruit, evaluate, promote, or manage employment relationships.
• Access to essential services (Annex III, point 5): AI used in credit scoring, insurance risk assessment, or emergency dispatch routing.
• Law enforcement (Annex III, point 6): AI used for risk assessments in criminal justice contexts.
• Education (Annex III, point 3): AI used to determine access to educational institutions or to evaluate students.

A loan underwriting agent, an HR candidate screening agent, and a fraud detection agent are all clearly high-risk under these definitions. If you have agents that influence individual decisions in these domains, assume high-risk classification until legal review says otherwise.

High-Risk System Requirements

High-risk AI systems require a set of technical and governance controls before deployment:

• A risk management system maintained throughout the lifecycle (Art. 9)
• Training, validation, and testing data governance documentation (Art. 10)
• Technical documentation covering architecture, capabilities, and limitations (Art. 11)
• Automatic logging of events sufficient for post-hoc accountability (Art. 12) — minimum 6-month retention
• Transparency and information provision to deployers and affected individuals (Art. 13)
• Human oversight measures — a technical mechanism to monitor, override, and halt the system (Art. 14)
• Accuracy, robustness, and cybersecurity controls including adversarial testing (Art. 15)
• Conformity assessment before deployment, including third-party assessment for certain categories (Art. 43)

Article 12's automatic logging requirement is specific: logs must capture the operating period of each use, the reference model version, the input data or a reference to it, and the outputs. For AI agents this maps to: session timestamps, which model and policy bundle version was active, a hash or reference to the prompt, and the agent's outputs and tool calls.

Prohibited Practices

Article 5 prohibits AI systems that manipulate individuals through subliminal techniques, exploit vulnerabilities of specific groups, or conduct social scoring. Enterprise agents generally don't fall into these categories, but "social scoring" warrants review — any agent that generates assessments of individuals' trustworthiness or behavior based on data unrelated to the immediate task needs legal sign-off before deployment.

8. Audit Logging Requirements by Framework

9. Compliance Architecture

10. Implementation Checklist

These 20 items are the minimum viable AI compliance posture for an enterprise deploying agents on regulated data. Each item is specific enough to verify — not a policy statement, but a technical control you can test.

HIPAA (Healthcare)

• Execute BAAs with every service in your inference pipeline that may process PHI — LLM provider, vector store, observability platform, logging service — before any PHI reaches those services
• Deploy in-process PHI detection covering all 18 Safe Harbor identifiers before prompt text leaves your network — not pattern matching on a subset, all 18
• Implement pre-LLM redaction using Presidio or equivalent so the model provider never receives identifiable PHI — redaction confirmed by audit log showing entity types detected and replaced
• Log every inference event with user identity, agent identity, detected PHI entity types, and enforcement actions taken — retain for 6 years in tamper-evident storage
• Implement minimum necessary controls in prompt construction: system prompts should request only the specific PHI fields the agent needs for the task, not entire patient records

SOC 2

• Define a model access policy stating which AI models can be used, approved by whom, and enforce it at runtime — not just in a document that auditors read
• Govern tool access as you govern service account permissions: enumerate every tool granted to each agent, document the business justification, and review quarterly
• Make fail-open or fail-closed policy an explicit documented choice per agent, not a default — include the rationale in your SOC 2 description
• Add AI-specific incident scenarios to your incident response plan: prompt injection, data leakage via tool calls, model provider outage affecting business processes
• Add AI model providers to vendor risk management: complete security questionnaires, review data handling terms annually, and confirm subprocessor lists

GDPR

• Document all AI agent processing in your Record of Processing Activities (RoPA) with lawful basis, data categories, retention periods, and transfer mechanisms for each agent
• Execute Data Processing Agreements with all model providers and confirm DPF enrollment for US providers before sending EU personal data to their APIs
• Implement data minimization in prompt construction: only include personal data fields the agent actually needs, validated against the task definition
• Build Art. 17 deletion capability before you need it: a per-individual log purge that removes prompt history from audit logs without breaking aggregate audit records
• Add human review workflows for any agent output that may constitute an Art. 22 automated decision with legal or similarly significant effect on an individual

PCI DSS

• Implement Luhn-valid PAN detection and in-process redaction before any prompt containing payment context reaches the LLM API — test with synthetic card numbers in staging
• Ensure audit logs never capture unmasked PANs — render unreadable at log ingestion time with at minimum the first 12 digits masked
• Update your CDE scope diagram to include AI agent processing components — model providers and inference infrastructure must appear explicitly, not just application servers

EU AI Act

• Classify each AI agent against Annex III high-risk categories with documented rationale — review the classification annually and when the agent's capabilities change significantly
• For high-risk agents: implement Art. 12 automatic logging capturing operating period, model version, input data reference, and outputs — with minimum 6-month tamper-evident retention
• For high-risk agents: implement Art. 14 human oversight as a technical mechanism — a specific control that allows an operator to pause, override, or halt the agent in real time
• Maintain technical documentation per Art. 11 covering architecture, training data description, performance metrics, and known limitations — updated with each model or system change