Govern AI models and agents through an API gateway

Difficulty ▰▰▰▰▱ L400 (filled bars out of 5; L100 beginner → L500 expert)

When every team in an enterprise wants to use generative AI, the hard problem isn’t calling a model — it’s governing those calls: controlling who can use which model, capping spend, masking sensitive data, and proving what happened. In this lab you deploy the Citadel AI Governance Hub and put a single Azure API Management gateway in front of every model and agent, then use it to enforce and observe real governance controls.

Your scenario: you’re a platform engineer at Citadel, a company that runs a central AI platform for every product team. Teams keep spinning up their own model connections with no spend limits, no data protection, and no audit trail. Your job is to stand up the Citadel AI Governance Hub so those teams consume AI through the gateway — governed, metered, and observable — and then prove each control works.

You’ll start with the Core tasks that get you to a working, governed AI gateway as quickly as possible. From there, a set of Optional tasks lets you go deeper into the governance capabilities that matter most to you.

Note: Some of the technologies used in this lab are in preview or in active development. You may experience some unexpected behavior, warnings, or errors.

What you’ll learn

By completing the Core tasks of this lab, you’ll be able to:

• Onboard AI backends and route through the gateway so every model call goes through the Citadel Universal LLM API in API Management instead of hitting a model endpoint directly.
• Enforce access contracts that pin each workload to specific models (model-level RBAC) and cap its token spend with per-minute and quota limits.
• Observe usage, cost, and throttling across APIM Analytics, Application Insights, and Cosmos DB, so you can see exactly who called which model and what it cost.

The Optional tasks let you additionally:

• Protect sensitive data by masking or blocking PII in prompts at the gateway.
• Govern agent frameworks and route Microsoft Agent Framework, Foundry SDK, and LangChain agents through the gateway, and test the multi-provider Unified AI API.
• Publish and govern a hosted agent and expose it as a governed Agent-to-Agent (A2A) endpoint.
• Publish and govern an MCP server, exposing HR tools through APIM with a rate-limited access contract.

How this lab is organized

This lab is modular. Every task shares one deployment (azd up), one azd environment, and one Python environment, so you can work through it end to end or complete any single task on its own.

1. Start with Getting started — register resource providers, deploy the Citadel hub and a sample spoke with azd up plus one script, and set up your Python environment. You only do this once.
2. Do any task below. Each task page opens with a “Starting here on its own?” callout that tells you exactly what it needs (the deployed hub, an access contract from an earlier task, and so on), so you can jump straight in.

Lab at a glance

Complete the Core tasks first (about 1 hour of hands-on time after deployment) — they end with a working, governed AI gateway you can see usage and cost flowing through. Then expand any Optional tasks that interest you. The full lab, including all optional tasks, takes about 3 hours of hands-on time on top of the one-time azd up deployment.

⏳ Deployment dominates the clock. The one-time azd up (plus the spoke script) in Getting started takes about 30–45 minutes and is a prerequisite for every task. Kick it off first and read ahead while it runs.

Choosing your path — pick the tasks that fit the time you have:

• Core only (~1 hr): do Tasks 1–3. You’ll have a governed gateway with access contracts and full observability.
• Core + governance controls (~2 hr): also do Task 4 (PII) and Task 5 (agent frameworks + Unified AI API).
• Everything (~3 hr): add Task 6 (hosted agent + A2A) and Task 7 (HR MCP via APIM).

One gateway, growing governance: every task routes through the same Citadel API Management gateway you deploy once. Each task adds a new control on top of it — an access contract, a PII policy, a governed agent — so you’re always building on the same governed platform, never starting over.

Two ways to consume the gateway

Citadel’s API Management gateway deliberately exposes more than one API surface, and this lab shows you the main ones:

• The Universal LLM API (/models) — a single OpenAI-compatible endpoint that routes to the right backend based on the model field in the request body. This is the surface you onboard and test in Task 1, and the one most workloads use.
• The Unified AI API (/unified-ai) — a multi-provider wildcard surface that accepts Azure OpenAI, Foundry inference, Responses API, and Gemini-style paths through one gateway, enforcing path rules and model access along the way. You test this in Task 5.

Neither is “more correct” — they’re different levels of abstraction over the same governed backends. Seeing the Universal LLM API first is what makes the Unified AI API’s flexibility meaningful later.

Summary

Across this lab you:

• Onboarded AI backends and routed every model call through the Citadel Universal LLM API.
• Enforced access contracts that pin workloads to specific models and cap their token spend.
• Observed usage, cost, and throttling across APIM, Application Insights, and Cosmos DB.
• (Optionally) masked and blocked PII at the gateway, governed agent frameworks and the Unified AI API, and published governed agents as A2A and MCP endpoints.

Together these show the core idea of AI governance at scale: put every model and agent behind one gateway, and you get one place to control access, protect data, and prove what happened.

Clean up

If you’re finished, remove the deployed resources to avoid ongoing Azure costs. Do this in order — tear down the optional MCP resources first (if you ran Task 7), then the hub, then any spokes.

1. If you ran Task 7 (HR MCP), tear down its resources first because they live inside the shared hub network:

    pwsh -File ./workshop/mcp-hr/scripts/teardown-hr-mcp.ps1
   

2. Tear down the Citadel hub deployed by azd up:

    azd down --purge --force
   

3. Delete any spoke resource groups manually in the Azure portal — the spoke script deploys them separately, so azd down doesn’t remove them. Look for resource groups named like <hub-rg>-spoke-1.

Important: The --purge flag permanently removes soft-deleted resources (Key Vault, Cognitive Services). Verify in the portal that the hub and spoke resource groups are gone.