AI is not HIPAA compliant out of the box, and the single most repeated piece of advice about making it compliant is wrong. A healthcare AI setup becomes HIPAA-eligible when the provider has signed a Business Associate Agreement, the account is provisioned on the specific retention configuration that agreement requires, the deployment runs in an approved region with encryption, protected health information is minimized where possible, and access is logged and controlled.
That third word matters. Most guidance tells you to turn zero data retention on. For the two providers healthcare teams reach for most, that advice is either wrong or actively harmful. This guide explains what is actually required, which providers sign a BAA, what their agreements exclude, and where healthcare AI automation is safe to deploy first.
Written for operations and technology leaders, not lawyers. Treat it as general information, not legal advice, and run any specific setup past your compliance counsel before real data flows.
What HIPAA means for AI
HIPAA governs protected health information, or PHI: anything that identifies a patient and relates to their health, their care, or payment for that care. If an AI system touches PHI, the vendor providing that AI is handling data on your behalf and becomes a business associate under the law. That relationship has to be covered by a Business Associate Agreement, a contract in which the vendor commits to safeguarding the data and accepts liability for mishandling it.
That single fact rules out most casual AI use in healthcare. Pasting patient information into a consumer chatbot, with no BAA and default retention, is a breach waiting to happen. The path to compliant AI runs through the enterprise tiers of providers who will sign a BAA, and through the configuration discipline to match.
The retention setting almost everyone gets wrong
Here is where most healthcare AI guidance, including a lot of it written by vendors, goes wrong. The instinct is that less retention means more compliance, so teams reach for zero data retention as the safe default. Both major model providers say otherwise, in opposite directions.
OpenAI. HIPAA eligibility for the OpenAI API is contingent on the account being provisioned with Modified Retention. Covered endpoints can process PHI even when data is retained, once the BAA is executed. Zero retention is not the requirement, and it is not what makes the account eligible. The provisioning is. (OpenAI, HIPAA eligible products and functionality)
Anthropic. BAA-covered models require 30-day data retention and are incompatible with zero data retention. A team that reads generic advice, switches ZDR on, and assumes it is now safer has moved itself outside the agreement it signed. (Anthropic, Business Associate Agreements for commercial customers)
The underlying logic is not complicated once you see it. A BAA is a contract about accountability, and accountability needs a record. The provider commits to safeguarding data and accepts liability for mishandling it, and the retention window is part of how that commitment is auditable. Zero retention is the right setting for a lot of sensitive commercial work. It is not automatically the right setting for work under a BAA.
The practical takeaway: do not configure retention from a blog post, including this one. Read the provider’s own HIPAA implementation guide, and configure to that.
Which providers sign a BAA, and what they leave out
The major model providers now offer BAAs, so healthcare teams are not stuck choosing between compliance and capable models. As of August 2026:
| Provider | HIPAA-eligible products | Retention requirement | Notable exclusions |
|---|---|---|---|
| OpenAI | ChatGPT for Healthcare, ChatGPT Enterprise with Regulated Workspace, ChatGPT for Clinicians, ChatGPT FedRAMP, API with Modified Retention, API FedRAMP with Modified Retention | Account provisioned with Modified Retention | 20 specific endpoints are eligible; additional ChatGPT functionality outside the BAA is disabled by default and should stay that way for PHI |
| Anthropic | Claude Enterprise with HIPAA activated in org settings, Claude Platform API with BAA enabled by sales | 30-day retention; incompatible with zero data retention | Workbench, Claude Console, Claude Cowork and beta features excluded. Batch API, Files API and Computer Use not accessible to HIPAA-ready API users. Third-party integrations and external connectors not covered when data leaves |
| AWS Bedrock | Hosted models under the AWS Healthcare BAA | Per AWS BAA terms | Requires HIPAA-eligible region and managed encryption keys |
| Microsoft Azure OpenAI | Azure OpenAI under the Microsoft enterprise healthcare agreement | Per Microsoft BAA terms | Limited to HIPAA-eligible Azure regions |
| Google Vertex AI | Vertex AI under the Google Cloud Healthcare BAA | Per Google BAA terms | Covered services list is specific; verify per service |
That exclusions column is the part worth reading twice. It is standard to say “coverage is feature-specific” and move on. Look at what that actually means in practice: an Anthropic BAA does not cover the Workbench your engineers prototype in, or Console, or anything in beta, and it turns off Batch API and Files API access entirely for HIPAA-ready accounts. Those are not obscure edge features. They are the things a team reaches for on day two of a build.
Mapping which exact products, endpoints and configurations sit under the agreement is early, unavoidable work on any healthcare AI project. Assuming blanket coverage because a BAA exists is the single most common route to a compliance failure that nobody notices until an audit.
Where healthcare AI automation is safe to start
The safest place to start is high-volume, lower-risk administrative work, not clinical decisions. That is also, conveniently, where the economics are strongest, because administrative volume is what makes automation pay.
| Workflow | Risk tier | What it needs |
|---|---|---|
| Intake and documentation, turning forms, faxes and records into structured data | Low | BAA-covered configuration, human review before output drives anything |
| Scheduling, staffing and appointment logistics | Low | Standard covered setup; often little or no PHI if identifiers are stripped |
| Record summarization for a clinician to read | Medium | Human reads every output; summary never substitutes for the record |
| Drafting patient communications | Medium | Human approves before sending; no autonomous send |
| Coding and billing support | Medium to high | Audit trail, human sign-off, and the accuracy bar of a financial system |
| Anything that diagnoses, prescribes or determines care | High | Different regulatory category. Human decision-maker in the loop, much higher validation bar, and a design built around catching errors rather than moving fast |
The line to hold is that last row. When a wrong output affects someone’s health rather than someone’s paperwork, the cost of error changes the whole calculation, and the system should be designed around catching mistakes rather than throughput.
The setup that makes an LLM HIPAA-eligible
A signed BAA is necessary but not sufficient. The configuration around it is where compliance is won or lost. A defensible setup covers all of the following.
A BAA with every provider in the data path, including sub-processors. If your pipeline sends data through three services, all three need coverage, not just the model. This is where architecture diagrams earn their keep.
Retention configured to the provider’s HIPAA specification, not to a general instinct. See above. For OpenAI that means Modified Retention provisioning. For Anthropic it means accepting 30-day retention and not switching ZDR on.
A HIPAA-eligible region with encryption in transit and at rest, and customer-managed keys where the provider supports them.
PHI minimization and de-identification wherever the workflow allows it. The safest data to send a model is data that no longer identifies anyone. Where the workflow is answering questions from clinical or policy documents, retrieval keeps that data in your own store and pulls in only the passage needed for each question, which is a smaller exposure than it sounds. On a lending build we ran a local model purely to anonymize personal information before anything sensitive reached a hosted model, and the pattern transfers directly to health data. Anonymizing at the edge shrinks how much protected information ever leaves your controlled environment, which reduces both the risk and the surface area your compliance team has to reason about.
Audit logging and access control. You need to know who accessed what, when, and under which BAA-covered configuration, both to operate the system and to prove compliance if you are asked.
Deployment in your own cloud when sensitivity calls for it. For the most sensitive workloads, running the pipeline inside your own controlled environment removes an entire category of risk and a lot of argument.
How to run a HIPAA-safe first project
The safest way to start is to prove the compliance setup on data that cannot hurt you, before pointing it at data that can.
Before any of that, check you are actually ready to build: our company AI readiness checklist covers the data, infrastructure and governance questions that block healthcare projects most often.
Choose an administrative, high-volume workflow where a human reviews the output and a mistake gets caught rather than acted on. Before any real data flows, confirm the BAA covers the exact products and endpoints you plan to use, provision retention to the provider’s HIPAA specification, lock the deployment to an approved region with encryption, and stand up audit logging. Then strip or mask identifiers wherever the workflow allows, so the model sees the minimum it needs.
Run the first phase on de-identified or synthetic data to validate that the pipeline works and the controls behave as expected. Only once that holds do you connect real protected health information, and even then with a human reviewing outputs. Any gap in the setup then shows up while the data is safe, rather than after a breach.
Build the re-verification habit at the same time. Provider coverage changes: products get added, features move in and out of scope, and a configuration that was covered in March may not be in September. Compliance here is a standing process, not a launch checklist.
The five mistakes that cause most problems
Using a consumer AI tool with PHI, with no BAA and default retention. The most serious and still the most common.
Configuring zero data retention because it sounds safer, when the provider’s HIPAA path requires something else. Covered above, and worth the repetition.
Assuming a BAA covers everything a provider offers, when the exclusions list runs to beta features, developer tooling, and entire APIs.
Skipping de-identification when the workflow would allow it, and sending more identifying data to the model than the task actually needs.
Putting AI in a clinical decision path without a human check, which is a patient-safety problem before it is a compliance problem.
Frequently asked questions
Can I use ChatGPT with patient data? Only through a HIPAA-eligible OpenAI product with an executed BAA and the account provisioned with Modified Retention. That means ChatGPT for Healthcare, ChatGPT Enterprise with a Regulated Workspace, ChatGPT for Clinicians, or the API on Modified Retention. The consumer version should never receive protected health information. The same enterprise-tier distinction applies to every provider.
Should I turn on zero data retention for HIPAA? Check the provider’s HIPAA documentation first, because the answer is not universal and is often the opposite of the general advice. OpenAI requires Modified Retention provisioning. Anthropic’s covered models require 30-day retention and do not work with zero retention at all.
Does a BAA make my AI automatically compliant? No. The BAA is the contract. The configuration does the rest: correct retention, approved region, encryption, de-identification, logging and access control. Most real failures happen in the configuration, not the contract.
Should we deploy AI in our own cloud? For the most sensitive workloads, often yes. Running the pipeline inside your own controlled environment reduces exposure and simplifies several compliance questions at once. For lower-risk administrative work, a properly configured, BAA-covered managed service is usually sufficient.
Is this legal advice? No. This is general information about how HIPAA-eligible AI setups are typically built, current as of August 2026 and based on the providers’ own published documentation, which changes. Your obligations depend on your organization and your data. Confirm any setup with your compliance and legal teams before it touches real PHI.
Before you scope a healthcare AI project, work out which of your workflows sit in the low-risk tier in the table above. That list is usually shorter than teams expect and more valuable than they expect, and it is the honest starting point for a first workflow automation build.
If you want a second read on where your workflows land and how the compliant setup should be structured, that is what our AI strategy engagement does. Sometimes the finding is that a workflow needs a process fix or de-identification before any AI belongs near it, and we will tell you that before you spend on a build.