AI agent security and compliance in healthcare covers how software that answers patient calls and books EHR visits protects PHI and meets HIPAA. A phone tree routes calls, but an agent opens charts and speaks to callers. Sales claims about encryption and certifications often skip what the agent will disclose, and to whom. A practice can judge an agent on written evidence: the BAA, the audit scope, and the caller-verification procedure.

AI Agent Security and Compliance in Healthcare Start With PHI Access

An AI agent changes PHI during a live call, which a routing-only phone tree never does. The Assort Health platform writes intake, insurance, referral, and appointment data back to the EHR in real time and creates tasks for staff. Each of those actions needs a permission, a log entry, and an owner.

Chart access also creates disclosure risk. A caller who gives someone else's birth date could hear that patient's next appointment. That is an unauthorized disclosure, and the practice often learns of it only when the patient complains.

Where AI Agent Security and Compliance in Healthcare Meets HIPAA

HIPAA's Privacy and Security Rules apply to the agent because it handles PHI for the practice. Booking a visit needs open slots and scheduling fields, not the whole chart. Apply the Privacy Rule's minimum necessary standard and give the agent least-privilege EHR access limited to those fields.

The practice's risk analysis must also reflect the agent's access, because the agent is new technology touching PHI. Breach planning belongs in that analysis. The Breach Notification Rule allows up to 60 calendar days after discovery to notify patients of an unsecured PHI breach. Negotiate a shorter window, since the HHS model BAA uses 30 days.

Review Business Associate Agreements for AI Agent Platforms

A signed BAA must exist before an outside party handles PHI, and it should carry breach-notice obligations. Cover permitted uses, safeguards, unauthorized-disclosure reporting, termination rights, and PHI return or destruction at contract end under 45 CFR 164.504.

The BAA must also flow down to each subprocessor, usually the cloud host, the model provider, and the telephony service. Every party that touches PHI then answers to a BAA with the platform.

The Office for Civil Rights (OCR) treated MMG Fusion as a business associate because it received PHI and its software communicated directly with patients. Software companies get this scrutiny for the same reason.

Review Security Certifications: SOC 2, HITRUST, and More

Request a System and Organization Controls (SOC) 2 Type II report as the baseline and Health Information Trust Alliance (HITRUST) certification as a second attestation. Neither proves HIPAA compliance, and HHS recognizes no private HIPAA certification.

The table below has three columns: the attestation, what it tests, and what it doesn't prove.

What Each Security Attestation Covers

None of these attestations proves caller verification, and SOC 2 makes only the security criterion mandatory. Ask whether the AI agent, the EHR integration, and the model providers sit inside the audit scope.

Core Security Controls to Evaluate in an AI Agent

Within the audit scope, check how the platform protects PHI during calls, in the EHR integration, and in stored records.

Check PHI Encryption, Storage, and Retention

Under HHS guidance on unsecured PHI, lost PHI that was encrypted isn't "unsecured" and doesn't require patient breach notice. The Security Rule treats encryption as "addressable," not optional, so require documented reasons and an alternative if the platform skips it. Check call audio in transit and recordings, transcripts, and summaries at rest, and ask who holds the keys.

Encryption doesn't settle retention. State law governs it, because HIPAA sets no medical-record retention period. Each AI-handled call can produce audio, a transcript, a summary, and system logs. Require separate, configurable retention for audio and transcripts, plus certified deletion at contract end.

Evaluate Access Controls and Audit Logging

A shared login hides who changed a disputed booking, the agent or a coworker. HIPAA requires unique user identification. Imprivata and draft guidance from the National Institute of Standards and Technology (NIST) National Cybersecurity Center of Excellence say an AI agent may need its own identity and credentials.

Scope that service account to reading open slots, booking visits, and writing tasks. Check EHR permission scopes and which staff roles can play recordings or read transcripts.

HIPAA calls for activity records in systems holding electronic PHI (ePHI) but doesn't specify fields. Request a sample export showing who acted, including the agent, what happened, when, and why. Ask for a standard format such as Fast Healthcare Interoperability Resources (FHIR) AuditEvent that your security team can ingest.

AI-Specific Risks: Model Training, LLM Subprocessors, and Prompt Injection

AI agents differ from phone systems in their exposure to model training and provider retention, and caller input can be used to manipulate them. A large language model (LLM) is the model behind the agent's conversation, and its provider is usually a subprocessor.

Model training. Training on customer data without patient authorization or a defensible treatment, payment, or operations basis is a potential HIPAA violation. Get a written no-training commitment from the platform and every model provider.

Retention at the model provider. Even without training, the provider of a "HIPAA-eligible" model may keep PHI. Some providers retain application programming interface (API) inputs for abuse monitoring for up to 30 days by default. Confirm the provider approved zero data retention, which means nothing is kept after the model responds. Read the written terms, because BAA-covered models don't all qualify.

Prompt injection. Caller input can be crafted to change model behavior, which is a different risk from provider retention. The Open Worldwide Application Security Project (OWASP) says foolproof prevention may not exist. Don't authorize access based on model output. Enforce authorization outside the model, in EHR access limits and platform rules.

How AI Agents Verify Patient Identity on Calls

The agent must run chart-based identity checks before it shares protected details, and failures go to staff. HIPAA requires verifying the identity and authority of unknown requesters but names no method (45 CFR 164.514), so the practice sets the standard. Caller ID is spoofable and isn't proof.

Even with identity checks, a guesser gets through if the agent accepts a correct birth date after two wrong guesses. Check retry limits and whether failed attempts get logged. The Assort Health platform verifies the patient before disclosing any protected details.

After verification, limit disclosure to task-needed details: appointment date, time, and provider. Leave out diagnoses and insurance details. For minors, check parental authority as well as identity, including state-law exceptions where a minor consents to care. HIPAA doesn't require identity proof from other family involved in care, so practices set their own rules.

Test Proxy Calls for More Than One Patient

Callers on pediatric calls usually aren't the patients. When a parent asks about two children, the agent must confirm parental authority and verify against each child's chart before disclosing anything. It must transfer to staff if either check fails. Test that exact call before go-live.

What Should an AI Agent Security and Compliance Review in Healthcare Ask?

Ask these questions on contractual terms, audit scope, data handling, identity verification, and incident response:

• BAA and breach notice. Will your BAA cover AI processing and flow down to every subprocessor? Will it require breach notice within 30 days?
• Audit scope. Will you share the scope section of your SOC 2 Type II or HITRUST report, identifying the systems tested?
• Data and subprocessors. Which subprocessors touch call audio, transcripts, or PHI? Does each have written zero data retention and no-training commitments?
• Identity verification. Which identifiers and retry limits apply, and how does the agent handle proxies and log failed attempts?
• Incident response. Does your plan cover prompt injection and PHI exposure through integrations, with exportable access logs for investigation?

A verbal assurance on a sales call doesn't bind the platform the way a signed BAA does. Request the written terms and reports behind each answer.

How Assort Health Answers These Security Questions

Request Assort Health's signed BAA. Through its Trust Center, request a SOC 2 Type II report, HIPAA attestation, penetration test executive summary, and certificate of insurance. Use them to check unique account authentication, restricted production access, encryption, data deletion at contract end, and US hosting. The subprocessor check extends beyond hosting to Google Cloud, OpenAI, Anthropic, and Twilio. Ask whether they're Assort Health's only subprocessors and whether each is US-based.

Outbound campaigns need the same review before launch. The Assort Health platform runs proactive outreach by phone and text to a defined patient list and writes each outcome back to the EHR. After deploying outreach, Annapolis Internal Medicine booked 61% of its flu shot appointments through it, which is why its BAA, subprocessor terms, and text and voicemail rules belong in the review.

Book a demo with Assort Health and give patients 24/7 scheduling that verifies each caller and writes every booking to the EHR.

Frequently Asked Questions

Are Recordings of Patient Calls With an AI Agent Considered PHI?

Yes, when the recording captures patient identifiers, for example a patient's name, date of birth, or reason for visit. No HHS guidance addresses AI call recordings specifically. The call's transcript, summary, and logs also contain PHI, and each copy needs its own access and retention rules. Ask Assort Health to confirm its customer data deletion terms through its Trust Center.

Does Using an AI Agent Change a Healthcare Organization's Own HIPAA Obligations?

Yes, it adds duties without removing any. The practice still needs a signed BAA and an updated risk analysis that names the agent. It also keeps its own breach response duties. A platform's compliance claims don't replace the practice's own risk analysis.

Can AI Agents Leave Voicemails With Appointment Details Under HIPAA?

Yes, within limits. HHS guidance allows messages on answering machines when they carry only needed information, for example the practice name, a callback number, and enough to confirm an appointment. A diagnosis doesn't belong in the message. Ask the platform to show the exact voicemail script before go-live.

Do Patients Need to Be Told They're Speaking With an AI Agent?

Not under HIPAA, but state requirements may apply. Check whether Texas's disclosure requirement for AI used in a health care service or treatment applies. For Utah, check whether disclosure is required on request or prominently for high-risk interactions. For California, check whether the rule for generative AI clinical messages excludes scheduling.

Who Is Liable If an AI Agent Platform Has a Data Breach?

Both the platform and the practice can be. The platform is directly liable under HIPAA for its own Security Rule failures and for late breach notice. The practice can share liability if it knew, or should have known, of a pattern of BAA violations and didn't act. Indemnification and cyber insurance terms, in turn, split breach costs between the parties.

Can AI Agents Send Text Message Reminders Under HIPAA?

Yes, if the text stays short and carries only appointment details. Reminder texts fit the Federal Communications Commission (FCC) healthcare exemption if they contain no billing or marketing and follow HIPAA privacy rules. They must also honor opt-out replies such as "stop." Guidance hasn't addressed whether the exemption covers AI-written texts.