Data residency and AI: what changes when the weights stay in your VPC

Enterprise data residency policies were drafted in the SaaS era. They specify where customer data is stored, which jurisdictions can access it, and what consent applies. They were not drafted with LLM APIs in mind, and applying them to AI consumption produces ambiguity that legal teams find uncomfortable.

This is the framing we use on engagements with regulated multinationals that need clarity.

What data residency actually requires

A typical enterprise data residency clause:

"Customer data shall be stored, processed, and accessed only within [specified jurisdictions]. Transfers outside those jurisdictions require [specific safeguards, e.g., SCCs under GDPR, equivalent under other regimes]."

Three operations to consider: stored, processed, and accessed.

With a SaaS product, residency is generally clear — the vendor publishes which data centers host customer data and which regions process it.

With an LLM API:

Storage: the request payload may be stored briefly for retry, abuse detection, or 30-day rolling logs. OpenAI says US-based; Azure OpenAI offers regional storage.
Processing: inference happens on GPUs in specific data centers. Provider documentation specifies, but model weights and inference compute are distinct from customer data residency in subtle ways.
Access: the API provider's engineers and systems can access the request data during the storage window. This is "access by a third party" in most residency regimes.

The result: a clause that was clear under SaaS becomes ambiguous under LLM APIs. Legal teams either accept the ambiguity (most do, with vendor commitments as backstop) or refuse the API path entirely (some industries, especially in the EU and APAC).

What changes with on-prem

Self-hosted LLMs sidestep the analysis. The model weights live in the customer's environment; inference happens on customer hardware; the request and response never leave the customer's network. Standard data residency clauses apply unchanged — the AI is treated like any other internal service.

This is the clean answer. It's also operationally heavier than calling an API, which is why most teams have to weigh the trade-off rather than defaulting to on-prem.

Sub-issues that matter

A few specifics that come up in residency negotiations:

Model weights as part of "the system" vs. "customer data." The weights themselves aren't customer data — they're vendor intellectual property. They can move freely; they're not subject to data residency. What matters is whether customer data touches the weights' infrastructure.

Training vs. inference. Many residency clauses focus on training. "No customer data will be used to train models." OpenAI's enterprise commitments include this. But residency for inference (where the data physically goes during a single API call) is often less specified, leaving room for interpretation.

Multi-region failover. Cloud APIs may failover across regions during incidents. Customer data in the request might briefly traverse a non-approved region during the failover event. SaaS contracts typically address this; LLM API contracts sometimes do less clearly.

Logs and observability. API providers retain request logs for varying periods. Those logs may or may not be subject to the same residency constraints as the live request. Worth asking specifically.

Subprocessors. OpenAI uses Azure for some infrastructure; the request may briefly traverse Azure systems. Whether that counts as "vendor + subprocessor" or "two vendors" matters for the consent and audit story.

EU customer data under GDPR has the cleanest residency regime to work with. Three paths for using LLM APIs on EU customer data:

EU-resident inference. Some providers offer EU-region API endpoints. OpenAI does, Anthropic does. Helps but doesn't fully resolve — engineers in the US can still potentially access EU-region logs.
Standard Contractual Clauses + provider commitments. SCCs are the standard transfer mechanism. OpenAI's enterprise terms include them. This works for most enterprises but adds vendor management overhead.
On-prem inference inside the EU. Customer data never leaves the customer's environment. Cleanest GDPR posture; closest to SaaS-era thinking. The path most regulated EU enterprises end up on.

We've shipped engagements where the residency analysis was the headline reason for choosing on-prem over API. The cost math went either way; the residency analysis broke the tie.

What this looks like under HIPAA

HIPAA for healthcare LLM deployments has similar structure. Business Associate Agreements with API providers are available (OpenAI offers them), so API consumption is possible but adds:

BAA review and ongoing oversight
Documentation of all data flows
Incident response coordination with the vendor
Auditor questions about subprocessor handling

For larger healthcare systems, on-prem is the operationally simpler choice. For smaller practices, the vendor management is acceptable for the API path.

What this looks like under APAC regimes

China's Cybersecurity Law and Personal Information Protection Law (PIPL) are strict on cross-border data flows. Practical effect: customer data of mainland China subjects can't easily flow to OpenAI or Anthropic. On-prem (or in-region Chinese LLM providers) is the standard path.

Singapore, Hong Kong, Japan all have less strict residency regimes but increasing attention to AI-specific data handling. India's draft AI rules are evolving.

In APAC, the residency analysis sometimes pushes harder toward on-prem than US/EU clients face.

What we do on residency-driven engagements

A standard scope:

Map the data flows. What data goes into the AI request? What identifiers are present? What outputs come back? Where are they consumed?
Map the residency regime. What clauses apply? What enforcement risk is associated with non-compliance? What's the firm's existing posture for analogous services?
Triage the deployment path. Can a region-restricted API satisfy the clause with documentation? Can pre-redaction reduce sensitive content to a manageable level? Is on-prem the only acceptable path?
Build the technical architecture. If on-prem, deploy the model; if API with restrictions, document the controls.
Document the analysis. Memo to legal/compliance describing the architecture and the residency posture. This is the artifact internal audit will want.

The architecture work is often quick; the documentation work is what takes time. Engagement timelines for residency-driven deployments run 12-20 weeks, much of it on the compliance side.

What changes for the future

Three trajectories worth tracking:

Regional API offerings will improve. OpenAI, Anthropic, and Google are all expanding regional infrastructure. The "API only available in US-East" excuse will be less true in 2-3 years.

On-prem economics will improve. B200/B300 generation drops on-prem inference cost per token. Smaller firms can afford serious deployments. The on-prem option becomes more accessible.

Regulations will tighten. Most jurisdictions are moving toward more, not less, AI-specific regulation. The compliance complexity for API-based deployments will grow; on-prem postures will look increasingly attractive by comparison.

For new deployments today, we run the residency analysis early. The answer often changes the architecture. Better to know in week 2 than in week 12.