AI workloads are no longer “special cases.” Azure OpenAI, vector search, and GPU compute are becoming standard parts of enterprise applications — which means they need to live inside the same governed, private, cost-controlled landing zone as everything else. The mistake teams make is treating an AI service as a quick standalone deployment, then discovering it has a public endpoint, no network isolation, and no policy coverage.
This post shows how an AI landing zone sits on top of your platform foundation, and the handful of decisions that keep it secure and affordable.
Architecture diagram
The AI landing zone is just another workload subscription — it inherits identity, networking, monitoring, and policy from the platform, then adds the AI-specific services behind private endpoints.
flowchart TB
MG["Management Group"] --> B["AI Landing Zone (subscription)"]
subgraph Platform["Platform services (inherited)"]
P1["Identity / Entra ID"]
P2["Hub networking"]
P3["Monitoring / Log Analytics"]
P4["Azure Policy"]
end
Platform --> B
B --> W1["AI compute (GPU / AML)"]
B --> W2["Azure OpenAI (Private Endpoint)"]
B --> W3["Vector search (Private Endpoint)"]
B --> W4["Storage / data (Private Endpoint)"]
The four decisions that matter
1. Private connectivity by default
Every AI service — Azure OpenAI, the vector store, and the backing storage — should be reachable only over a Private Endpoint, with public network access disabled. Resolution flows through the platform’s Private DNS zones, so no AI traffic ever traverses the public internet. This is the single most important control, and it’s far easier to set up on day one than to retrofit.
Watch out: deploying Azure OpenAI with default settings leaves a public endpoint enabled. Disable public access and link the Private DNS zone explicitly, or your “private” model is one leaked key away from exposure.
2. Identity over keys
Use managed identities and Entra ID authentication for service-to-service calls instead of API keys. Keys get copied into notebooks, pipelines, and chat messages; managed identities can’t leak the same way and are governed by conditional access and PIM. Reserve keys for the rare case that genuinely can’t use identity, and store them in Key Vault behind a Private Endpoint.
3. Governance that understands AI
Extend your existing Azure Policy guardrails to the AI estate:
- Deny AI resources with public network access enabled.
- Restrict deployable model regions for data-residency compliance.
- Require diagnostic settings so prompts, completions, and usage flow to Log Analytics.
- Enforce tagging for cost attribution and content-safety configuration.
4. Cost and capacity, on purpose
AI spend is lumpy and easy to lose track of. Decide deliberately between pay-as-you-go and provisioned throughput (PTU) for predictable, high-volume workloads, set budgets and alerts per subscription, and tag everything so each team sees its own consumption. Token usage flowing to Log Analytics makes this observable rather than a month-end surprise.
Tip: put a thin gateway (APIM or your own service) in front of Azure OpenAI. It gives you per-team rate limiting, central logging, prompt/response filtering, and the ability to switch models without touching every caller.
How it lands on the platform
Notice what the AI landing zone doesn’t do: it doesn’t define its own networking topology, its own identity provider, or its own monitoring stack. Those are inherited from the platform subscriptions. The workload team focuses purely on the AI services and the application — exactly the division of responsibility a good landing zone is designed to create.
Key takeaways
- Treat AI as a normal workload that inherits the platform foundation.
- Private Endpoints + disabled public access are non-negotiable.
- Prefer managed identities over API keys.
- Extend Azure Policy to cover AI-specific risks.
- Make token spend observable and choose your capacity model on purpose.
Haven’t set up the foundation yet? Start with Azure Landing Zones: A Practical Architect’s View.