Task 5 — Govern agent frameworks and the Unified AI API

Part of the Govern AI models and agents through an API gateway lab. New here? Start with Getting started.

Starting here on its own? You need the Citadel hub and a sample spoke deployed, and — for the agent-frameworks notebook — the Sales-Assistant, HR-ChatAgent, and Support-Bot access contracts from Task 2. If you skipped Task 2, run 3. citadel-access-contracts-tests.ipynb first (it takes a few minutes). From the Labfiles/G-govern-ai-through-a-gateway folder, run python setup/check_env.py --task 5 to confirm the hub and spoke are deployed.

Continuing from a previous task? If you completed Task 2 and left its cleanup cell set to False, the three access contracts (and the Foundry connection name from Task 2 Step 4.2) are already in place — reuse them directly here.

Real workloads aren’t raw HTTP calls; they’re agents built on frameworks. This task proves the same governed gateway serves three different agent frameworks through their access contracts, and then explores the Unified AI API — a single wildcard surface that governs many provider API shapes at once.

Why route different frameworks through the same gateway?

Teams pick frameworks for their own reasons — Microsoft Agent Framework, the Foundry Agent SDK, LangChain, and more. If each one talked to models directly, governance would fragment across frameworks. Because Citadel’s access contracts issue a plain endpoint + key (optionally via Key Vault or a Foundry connection), any framework that can point at an endpoint is governed identically. The framework changes; the access contract and the controls don’t.

Route three agent frameworks through their contracts

  1. In VS Code, open 4. citadel-agent-frameworks-tests.ipynb from the workshop folder and select the workshop/.venv kernel.

  2. Run Steps 0–3 to initialize, verify your Azure CLI sign-in, initialize the APIM client, and retrieve the API keys for the three access contracts from Task 2.

  3. Run Step 4 to install the agent framework packages into the kernel (set the install toggle to False on later re-runs to skip re-installing).

  4. Run each framework section and watch the multi-turn conversation route through its own contract:

    Section Framework Contract Integration
    5 Microsoft Agent Framework (Sales-Assistant) Sales-Assistant Key Vault (endpoint + key)
    6 Microsoft Foundry Agent SDK (HR-ChatAgent) HR-ChatAgent Foundry project connection
    7 LangChain (Support-Bot) Support-Bot Local (direct endpoint + key)

    If Section 6 can’t find the Foundry connection, set its foundry_connection_name to the value Task 2 printed in Step 4.2 of the access-contracts notebook.

  5. Run Steps 8–9 to compare agent statistics and token efficiency across the three frameworks.

Validate the Unified AI API

The Unified AI API (/unified-ai) is a multi-provider wildcard surface: one gateway API that accepts several provider request shapes and enforces path rules, model access, and auth across all of them.

  1. Open 6. citadel-unified-ai-api-tests.ipynb and select the workshop/.venv kernel.

  2. Run Steps 0–4 to initialize, provision a dedicated access contract for the Unified AI API, and retrieve its key. Run Step 5 to discover available deployments through GET /unified-ai/deployments.

  3. Work through the numbered Tests, which exercise the provider patterns and the governance controls that make them safe to expose through one surface:

    • Tests 1–4 — the Azure OpenAI, Foundry Inference, Responses API, and Gemini path patterns.
    • Tests 5, 10 — API key authentication (valid, missing, and wrong keys).
    • Test 6 — a 30-second load test showing throttling.
    • Test 7 — the UAIG-* debug response headers that expose gateway state.
    • Test 8 — blocked path rejection: paths that match no configured API type return 403 PathNotAllowed.
    • Test 9 — model access control: a model outside the contract’s allowedModels list is rejected.
    • Test 11 — a streaming request, confirming UAIG-Is-Streaming is set.
Universal LLM API vs Unified AI API — what's the difference?

The Universal LLM API (Task 1) is one OpenAI-compatible surface that routes by the model field — great when your callers all speak the OpenAI chat/embeddings/responses shapes.

The Unified AI API is broader: a wildcard that accepts multiple provider path shapes (Azure OpenAI, Foundry inference, Responses, Gemini) through one gateway API, rejecting any path that isn’t explicitly allowed. It’s the surface you reach for when you need to front many providers and API styles behind a single governed endpoint.

Leave the optional Cleanup cells in both notebooks set to False unless you’re done with these contracts.

✅ Checkpoint: Three agent frameworks each held a governed conversation through their own access contract, and the Unified AI API enforced path rules, model access, and auth across multiple provider shapes. Your gateway governs agents regardless of how they’re built.

Next (optional): Task 6 — Publish and govern a hosted agent and an A2A endpoint