Healthcare Data Migration Planning: A Practical Guide
Practical data migration planning guide for healthcare teams. Scope, mapping, compliance, validation, cutover and rollback decisions explained end to end.
October 15, 2026Read more
HIPAA became enforceable long before modern generative AI existed, and the record since then is the reason a healthtech team can't rely on vendor assurances alone. By October 31, 2024, OCR had received more than 374,321 HIPAA complaints and initiated over 1,193 compliance reviews, while its 2024 Report to Congress recorded 30,256 new complaints and $8,763,831 in settlements and civil money penalties from compliance reviews (HHS enforcement highlights; 2024 Report to Congress). The engineering problem is straightforward to describe and easy to get wrong in production: once an LLM touches PHI, you need controls, logs and evidence as well as a signed agreement. This article covers the architecture-level pieces that make HIPAA-compliant AI defensible, from BAAs and data minimization to prompt-injection controls, SMART on FHIR scope handling, and human review.
If you're building in healthcare, the implementation details matter as much as the model choice. Ekipa's healthcare engineering work sits in that space, where EHR integration, workflow controls, and compliance evidence all have to line up.
What HIPAA-compliant AI means in 2026
HIPAA compliance for AI depends on how the whole system handles PHI. OCR's enforcement history shows why, because covered entities and business associates have to prove they control PHI across the full system lifecycle, not only at the moment a model is called. A BAA matters, but it is only one layer of the system.
The control trail matters more than the label
A practical way to frame HIPAA-compliant AI is simple. Every PHI-bearing path needs an owner, a policy, and evidence. That means data-flow maps, access logs, retention rules, model-provider agreements, and corrective-action procedures. In production, these are engineering artifacts that the system produces as it runs. If they do not exist, you do not have proof the system stays controlled when the model, prompt, or retrieval layer changes.
If a control cannot be shown in configuration, logs or a documented review step, it probably does not exist in production.
The same standard applies to observability and incident handling. OCR's complaint and review volume is a reminder that informal privacy promises do not survive audits well, especially when multiple services, tenants, and support paths can touch the same clinical payload. For a broader software-development view of those obligations, Ekipa's HIPAA compliance for software development piece is a useful companion.
Most of this work belongs before launch, when product, security and operations can still agree on the controls before a model has access to live data. That alignment is the focus of AI implementation support.
When an AI provider becomes a business associate
HIPAA's trigger is straightforward. If an AI provider creates, receives, maintains, or transmits PHI on your behalf, it is acting as a business associate and needs a BAA before it touches clinical data. That follows the HHS framing of how HIPAA applies to covered entities and their business associates (HHS HIPAA overview).
The BAA binds the provider only. Your team still owns minimum-necessary access, de-identification where possible, retention, logging, and whether output can safely flow back into the EHR. If the app sends the wrong fields, or keeps prompts forever in a debugging bucket, the agreement does not correct the design.
Using a consumer chatbot for clinical data is a different case. Those products are built for general use, not for a HIPAA-covered workflow with scoped access, auditability, and customer-controlled retention. Before wiring PHI into any workflow, confirm the provider will sign a BAA for that specific product, because the safe boundary has to be built into the architecture, with the contract as only one layer.
Responsibility splits three ways:
| Party | What it owns |
|---|---|
| AI provider | Signs the BAA, honors retention and training limits, and documents what the service does with PHI. |
| Your organization | Decides what PHI enters the prompt, how output is reviewed, and where logs, embeddings and transcripts are stored. |
| Engineering team | Enforces authentication, scope checks, encryption and audit trails at each hop. |
If revenue cycle or claims workflows sit next to clinical text, the covered path needs to stay narrow. Separate PHI-handling services from general admin tools so the audit trail stays clear and the exposed surface stays limited.
LLM providers that sign BAAs and the conditions they attach
BAAs exist on some major model APIs, but the covered path is usually narrower than the marketing page suggests. The test is whether the service, plan, and endpoint are all inside the HIPAA contract chain, with retention, training, and access terms written down before any PHI flows. Consumer chat interfaces are out of scope.
The conditions usually fall into a few buckets: retention settings, the plan or service tier, generally-available-only coverage, and feature-level exclusions. Each provider name in the table links to its official HIPAA page (checked October 16, 2026). Check the exact service path you use, because the same company may offer both covered and uncovered interfaces.
| Provider (as of October 2026) | BAA path | Key conditions |
|---|---|---|
| OpenAI API | Standard BAA accepted in the API Platform settings, plus Modified Retention provisioning | Only endpoints on OpenAI’s HIPAA-eligible list are covered (for example /v1/responses, /v1/chat/completions and /v1/embeddings). Consumer ChatGPT is not covered; ChatGPT for Healthcare and ChatGPT Enterprise with Regulated Workspace are BAA-eligible. |
| Anthropic (Claude) API | BAA for HIPAA-ready services: the first-party API, Claude in Microsoft Foundry and HIPAA-ready Enterprise plans | The organization’s Primary Owner signs the BAA, then Anthropic enables HIPAA-ready API access. Not all API features are covered (see Anthropic’s HIPAA implementation guide), Claude Console and beta features are excluded, and some models require 30-day retention. |
| Azure OpenAI Service | HIPAA BAA included by default in the Microsoft Product Terms and Data Protection Addendum | No separate signature is needed, but coverage applies only to in-scope services. Confirm the service and feature you use are listed in the HIPAA BAA scope in Microsoft’s Service Trust Portal. |
| Amazon Bedrock | AWS BAA, accepted in AWS Artifact | Bedrock is on AWS’s HIPAA Eligible Services list, and its generally available features are eligible unless AWS excludes them. Treat model invocation logs as containing PHI. |
| Generative AI on Gemini Enterprise Agent Platform (formerly Vertex AI) | Google Cloud BAA | Only products on Google’s covered-products list are in the BAA, and pre-GA offerings must not be used with PHI. The Gemini API in Google AI Studio is not on that list. |
A BAA means the provider is willing to sit in the HIPAA chain. It does not make the full workflow safe by itself. Healthtech teams still have to control data flow, key management, log retention, and output handling, because those decisions sit in your architecture, not in the contract.
The clean procurement move is to ask for the exact service, exact plan, and exact retention behavior in writing before PHI enters the pipeline. If the answer is vague, treat the path as uncovered until legal and engineering have verified it.
For product teams, the more useful question is simple. Can this provider path support narrow scopes, auditable access, and prompt content that stays inside the approved use case? If it cannot, the BAA only covers part of the risk.
Data minimization and de-identification in AI pipelines
In an AI pipeline, the minimum necessary standard becomes a prompt-design rule. The prompt should carry only the fields the task needs, the retrieval index should hold only the approved corpus, and prompt and response logs should be treated as potential ePHI unless they're explicitly controlled like clinical data. That means retention, access, and deletion settings have to be designed for logs, not inherited from the source record.
HHS recognizes two HIPAA de-identification paths, Safe Harbor and Expert Determination (HHS de-identification guidance). Safe Harbor removes specified identifier categories. Expert Determination requires a qualified person with statistical and scientific expertise to conclude that re-identification risk is very small, including through linkage with reasonably available information.
For AI pipelines, the practical issue is that quasi-identifiers can still leak meaning. Dates, rare diagnoses, free-text narratives, geography, clinician names, and unique encounter patterns can make a record easy to re-identify even when obvious identifiers are gone. If you're training, fine-tuning, or testing, that risk needs review at the dataset level and again at the model-output level.
A worked hypothetical helps. Suppose a triage assistant needs to draft a summary for a clinician. The input set is reduced to age band, relevant diagnosis codes, medication list, and recent vitals. Dates are generalized, clinician names are removed, and a rare-diagnosis flag is suppressed unless the reviewer has a documented reason to see it. The output still gets checked for memorized text before it leaves the system.
De-identification is only defensible when the exception path is visible. If a reviewer can reintroduce identifiers on demand, that reintroduction needs its own log and approval.
Technical safeguards for AI workflows that handle PHI
Encryption alone proves little. The deployed system has to show who accessed which PHI, when it was retained, how it was deleted, and whether that evidence survives configuration changes. That is the standard the Security Rule pushes you toward, across ingestion, temporary files, prompts, model infrastructure, outputs, logs, backups, and admin interfaces (HHS Security Rule risk assessment guidance; HHS HIPAA and FTC Act guidance).
What the configuration should show
Identity and access management: least-privilege service accounts, role-based access, and a clean split between user, service, and admin permissions.
Encryption: explicit settings for data in transit and at rest, plus documented key ownership.
Audit logging: immutable or tamper-resistant logs for prompts, responses, tool calls, and admin actions.
Retention rules: separate policies for prompts, outputs, telemetry, embeddings, and support traces.
Session controls: automatic timeout, tenant isolation, and revocation paths for active sessions.
Review workflow: a named reviewer for clinical output and a defined escalation path when the output is ambiguous or unsafe.
If prompts or responses are logged for debugging, treat them as ePHI from the first byte, because debugging copies are often where exposure starts.
The question is whether the system can prove access, retention, deletion, and auditability without gaps. The risk analysis should map those controls to actual workflows, not to a generic “HIPAA-ready” label.
A reference architecture with SMART on FHIR scopes and human review
A HIPAA-safe AI workflow starts with a narrow authorization boundary and a human at the end of the write-back path. A clinician opens an EHR-integrated assistant to draft a follow-up summary and suggest next-step tasks. The app launches through SMART on FHIR, the authorization server returns token claims, and the AI service builds a prompt only from the FHIR resources the session is allowed to see. Before anything is written back, the output is filtered and a clinician approves it.
SMART on FHIR gives the app a concrete authorization model. In App Launch 2.0 syntax, scopes like patient/Observation.rs or user/Encounter.rs describe the resource and operation boundaries, and the authorization result can be narrower than what the app requested (HL7 US Core scopes). Launch context matters too, because EHR launch mode, standalone mode, launch/patient, openid, and fhirUser each shape what identity and patient context the app can safely use (SMART launch context).
The failure mode is scope drift. If the app passes an access token into a prompt service without preserving the granted scope, the model layer can see more than the EHR intended. The safer pattern is to validate the returned scope, bind it to the session, and have the API gateway enforce the same boundary on downstream reads and writes. That keeps the authorization decision outside the prompt text, where it belongs.
A production flow usually has five steps.
Launch and token validation: the EHR issues context, and the app checks the granted scope, not the requested one.
Prompt assembly: only the minimum FHIR fields enter the prompt, and retrieved text is treated as untrusted content.
Model call: the model gets de-scoped context, not direct database access.
Output filtering: the system checks for PHI leakage, unsupported claims, and unsafe instructions.
Human review: a clinician approves any write-back to the EHR before the action executes.
Prompt injection defenses belong inside that sequence. Input allow-listing, system prompt isolation from retrieved content, and strict treatment of retrieval text as untrusted all reduce the chance that a malicious note, pasted instruction, or embedded prompt can steer the model outside the approved action set.
Clinical write-back should always pass through a human review step.
HIPAA-compliant AI checklist and conclusion
A deployable system needs evidence for every control. The checklist is simple to state and unforgiving to skip: a signed BAA for each PHI-handling provider, minimum-necessary fields per prompt, a documented de-identification path, explicit encryption and retention settings, audit logging for prompts and responses, SMART scope validation, prompt-injection tests, and human review of clinical output.
The main engineering implication is that HIPAA compliance for AI is a property of the whole system. The provider contract matters, but the app still owns scope, retention, review, and incident handling. The best deployments make those controls visible in configuration and logs so compliance can be demonstrated with evidence when an auditor asks.
This article is general information, not legal or regulatory advice. If you're planning a PHI-bearing AI workflow and want to pressure-test the architecture before implementation, book a consultation with Ekipa's HealthTech engineers.
Frequently asked questions about HIPAA-compliant AI
Can OpenAI's API be used with PHI?
Yes, once your organization has an executed OpenAI BAA and is provisioned with Modified Retention; the endpoints on OpenAI’s HIPAA-eligible list can then process PHI (as of October 2026). Your application still has to enforce minimum necessary access, logging and human review.
Is consumer ChatGPT allowed for clinical data?
No. Consumer ChatGPT plans are not covered by a BAA (OpenAI offers BAA-eligible ChatGPT for Healthcare and ChatGPT Enterprise with Regulated Workspace instead), so keep PHI out of consumer interfaces and use only the documented BAA-covered service path.
What evidence does an auditor expect for HIPAA-compliant AI?
Auditors usually want the BAA, the risk analysis, access and audit logs, retention settings, and proof that human review exists for clinical output. HHS audit protocol materials are a better reference point for how that evidence is reviewed than vendor summaries.
How should prompt retention be configured?
Prompt and response logs need explicit retention limits, restricted access, and deletion procedures because they can contain ePHI. Keep them separate from general application telemetry so they do not linger by default.
How do SMART on FHIR scopes help with LLM prompts?
They keep the AI app inside the patient, user, resource, and operation boundaries the EHR authorized. The app should inspect the granted scope returned by the authorization server and enforce it before a prompt ever reaches the model. That prevents scope drift, where a request starts narrow but the model is fed broader patient context than the token allows.
