Build AI agents with portal and VS Code

In this exercise, you’ll build a complete AI agent solution using both the Microsoft Foundry portal and the AI Toolkit VS Code extension. You’ll start by creating a basic agent in the portal with grounding data and built-in tools, then interact with it programmatically using VS Code to use advanced capabilities like code interpreter for data analysis.

This exercise takes approximately 45 minutes.

Note: Some of the technologies used in this exercise are in preview or in active development. You may experience some unexpected behavior, warnings, or errors.

Prerequisites

Before starting this exercise, ensure you have:

An Azure subscription with sufficient permissions and quota to provision Azure AI resources
Visual Studio Code installed on your local machine
Python 3.13 or later installed
Git installed on your local machine
Basic familiarity with Azure AI services and Python programming

* Python 3.13 is available, but some dependencies are not yet compiled for that release. The lab has been successfully tested with Python 3.13.12.

Create a Microsoft Foundry Project

Microsoft Foundry uses projects to organize models, resources, data, and other assets used to develop an AI solution.

In a web browser, open the Foundry portal at https://ai.azure.com and sign in using your Azure credentials. Close any tips or quick start panes that are opened the first time you sign in, and if necessary use the Foundry logo at the top left to navigate to the home page.

Important: For this lab, you’re using the New Foundry experience.
In the top banner, select Start building to try the new Microsoft Foundry Experience.
When prompted, create a new project, and enter a valid name for your project (e.g., it-support-agent-project).
Expand Advanced options and specify the following settings:
- Microsoft Foundry resource: A valid name for your Foundry resource
- Region: Select one available near you**
- Subscription: Your Azure subscription
- Resource group: Select your resource group, or create a new one
* Some Azure AI resources are constrained by regional model quotas. In the event of a quota limit being exceeded later in the exercise, there’s a possibility you may need to create another resource in a different region.
Select Create and wait for your project to be created.
When your project is created, a welcome dialog may appear. Select Next to read through the welcome message, and then select Create agent.

You can also select Start building on the home page, and select Create agents from the drop-down menu.
Set the Agent name to it-support-agent and create the agent.

The playground will open for your newly created agent. You’ll see that an available deployed model is already selected for you.

Configure your agent with instructions and grounding data

Now that you have an agent created, let’s configure it with instructions and add grounding data.

In the agent playground, set the Instructions to:

 You are an IT Support Agent for Contoso Corporation.
 You help employees with technical issues and IT policy questions.
    
 Guidelines:
 - Always be professional and helpful
 - Use the IT policy documentation to answer questions accurately
 - If you don't know the answer, admit it and suggest contacting IT support directly
 - When creating tickets, collect all necessary information before proceeding

Download the IT policy document from the lab repository. Open a new browser tab and navigate to:
```
 https://raw.githubusercontent.com/MicrosoftLearning/mslearn-ai-agents/main/Labfiles/01-build-agent-portal-and-vscode/IT_Policy.txt
```
Save the file to your local machine.

Note: This document contains sample IT policies for password resets, software installation requests, and hardware troubleshooting.
Return to the agent playground. In the Tools section, select Add, and then add both File search and </> Code interpreter.
To the right of Add, select Upload files. Under Attach files, browse to and upload the IT_Policy.txt file you just downloaded, and then select Attach.
Wait for the file to be indexed. You’ll see a confirmation when it’s ready.
Now let’s add some performance data for the code interpreter to analyze. Download the system performance data file from:
```
 https://raw.githubusercontent.com/MicrosoftLearning/mslearn-ai-agents/main/Labfiles/01-build-agent-portal-and-vscode/system_performance.csv
```
Save this file to your local machine.
To the right of </> Code interpreter, select + Files, and then upload the system_performance.csv file you just downloaded.

Note: This CSV file contains simulated system metrics (CPU, memory, disk usage) over time that the agent can analyze.

Test your agent

Let’s test the agent to see how it responds using the grounding data.

In the chat interface on the right side of the playground, enter the following prompt:
```
 What's the policy for password resets?
```
Review the response. The agent should reference the IT policy document and provide accurate information about password reset procedures.
Try another prompt:
```
 How do I request new software?
```
Again, review the response and observe how the agent uses the grounding data.

Now test the code interpreter with a data analysis request:

 Can you analyze the system performance data and tell me if there are any concerning trends?

The agent should use the code interpreter to analyze the CSV file and provide insights about system performance.

Try asking for a visualization:

 Create a chart showing CPU usage over time from the performance data

The agent will use code interpreter to generate visualizations and analysis.

Great! You’ve created an agent with grounding data, file search, and code interpreter capabilities. In the next section, you’ll interact with this agent programmatically using VS Code.

Interact with your agent using VS Code

As a developer, you may spend some time working in the Foundry portal; but you’re also likely to spend a lot of time in Visual Studio Code. The AI Toolkit extension provides a convenient way to work with Foundry project resources without leaving the development environment.

Install and configure the VS Code extension

If you already have installed the extension for AI Toolkit, you can skip this section.

Open Visual Studio Code.
Select Extensions from the left pane (or press Ctrl+Shift+X).
Search the extensions marketplace for the AI Toolkit extension from Microsoft and select Install.
After installing the extension, select its icon in the sidebar to open the AI Toolkit view.

You should be prompted to sign in to your Azure account if you haven’t already.

Test your agent in VS Code

Before writing any code, you can interact with your agent directly in the extension interface.

In My Resources, under Microsoft Foundry select the three horizontal bar icon to select your default project.

If a default project is already active, the project name will appear next to the Microsoft Foundry section. You can select a different project by selecting the same Select project icon.
Expand the Microsoft Foundry section. Under Agents, you should see the it-support-agent you created in the portal. Select the agent name to open the Agent Builder interface.

The agent playground will appear in the Agent Builder interface, allowing you to interact with the agent and configure its settings without leaving VS Code.

In the playground chat pane, type a question such as:

 What is the policy for reporting a lost or stolen device?

Review the agent’s response. It should use the grounding data you uploaded earlier to provide relevant IT policy information.

Tip: You can use this built-in playground to quickly test your agent’s instructions and knowledge without writing any code.

Create a client application to interact with your agent

Now let’s create a client application that interacts with your agent programmatically.

In VS Code, open the Command Palette (Ctrl+Shift+P or View > Command Palette).
Type Git: Clone and select it from the list.

Enter the repository URL:

 https://github.com/MicrosoftLearning/mslearn-ai-agents.git

Choose a location on your local machine to clone the repository.
When prompted, select Open to open the cloned repository in VS Code.
Once the repository opens, select File > Open Folder and navigate to mslearn-ai-agents/Labfiles/01-build-agent-portal-and-vscode/Python, then choose Select Folder.
In the Explorer pane, open the agent_with_functions.py file. You’ll see it’s currently empty.
Add the following code to the file:

```python import os from azure.ai.projects import AIProjectClient from azure.identity import DefaultAzureCredential import base64 from pathlib import Path from dotenv import load_dotenv

def save_image(image_data, filename): “"”Save base64 image data to a file.””” output_dir = Path(“agent_outputs”) output_dir.mkdir(exist_ok=True)
```
 filepath = output_dir / filename
    
 # Decode and save the image
 image_bytes = base64.b64decode(image_data)
 with open(filepath, 'wb') as f:
     f.write(image_bytes)
    
 return str(filepath)
```
def main(): # Initialize the project client load_dotenv() project_endpoint = os.environ.get(“PROJECT_ENDPOINT”) agent_name = os.environ.get(“AGENT_NAME”, “it-support-agent”)

Save the agent_with_functions.py file (Ctrl+S or File > Save).

 if not project_endpoint:
     print("Error: PROJECT_ENDPOINT environment variable not set")
     print("Please set it in your .env file or environment")
     return
    
 print("Connecting to Microsoft Foundry project...")
 credential = DefaultAzureCredential()
 project_client = AIProjectClient(
     credential=credential,
     endpoint=project_endpoint
 )
    
 # Get the OpenAI client for Responses API
 openai_client = project_client.get_openai_client()
    
 # Get the agent created in the portal
 print(f"Loading agent: {agent_name}")
 agent = project_client.agents.get(agent_name=agent_name)
 print(f"Connected to agent: {agent.name} (id: {agent.id})")
    
 # Create a conversation
 conversation = openai_client.conversations.create(items=[])
 print(f"Conversation created (id: {conversation.id})")
    
 # Chat loop
 print("\n" + "="*60)
 print("IT Support Agent Ready!")
 print("Ask questions, request data analysis, or get help.")
 print("Type 'exit' to quit.")
 print("="*60 + "\n")
    
 while True:
     user_input = input("You: ").strip()
        
     if user_input.lower() in ['exit', 'quit', 'bye']:
         print("Goodbye!")
         break
        
     if not user_input:
         continue
        
     # Add user message to conversation
     openai_client.conversations.items.create(
         conversation_id=conversation.id,
         items=[{"type": "message", "role": "user", "content": user_input}]
     )
        
     # Get response from agent
     print("\n[Agent is thinking...]")
     response = openai_client.responses.create(
         conversation=conversation.id,
         extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
         input=""
     )
        
     # Display response
     if hasattr(response, 'output_text') and response.output_text:
         print(f"\nAgent: {response.output_text}\n")
     elif hasattr(response, 'output') and response.output:
         # Extract text from output items
         image_count = 0
         for item in response.output:
             if hasattr(item, 'text') and item.text:
                 print(f"\nAgent: {item.text}\n")
             elif hasattr(item, 'type'):
                 # Handle other output types like images from code interpreter
                 if item.type == 'image':
                     image_count += 1
                     filename = f"chart_{image_count}.png"
                        
                     # Download and save the image
                     if hasattr(item, 'image') and hasattr(item.image, 'data'):
                         filepath = save_image(item.image.data, filename)
                         print(f"\n[Agent generated a chart - saved to: {filepath}]")
                     else:
                         print(f"\n[Agent generated an image]")
                 elif item.type == 'file':
                     print(f"\n[Agent created a file]")

     # Check for files in the response and download them
     file_id = ""
     filename = ""
     container_id = ""

     # Get the last message which should contain file citations
     last_message = response.output[-1] 
     if (
         last_message.type == "message"
         and last_message.content
         and last_message.content[-1].type == "output_text"
         and last_message.content[-1].annotations
     ):
         # Extract file information from response annotations
         file_citation = last_message.content[-1].annotations[-1] 
         if file_citation.type == "container_file_citation":
             file_id = file_citation.file_id
             filename = file_citation.filename
             container_id = file_citation.container_id

     # Download the generated file if available
     if file_id and filename:
         file_content = openai_client.containers.files.content.retrieve(file_id=file_id, container_id=container_id)
         output_dir = Path("agent_outputs")
         output_dir.mkdir(exist_ok=True)
         file_path = output_dir / filename
         with open(file_path, "wb") as f:
             f.write(file_content.read())
         print(f"File downloaded successfully: {file_path}")

if name == “main”: main() ```

Configure environment and run the application

In the Explorer pane, you’ll see .env.example and requirements.txt files already present in the folder.
Duplicate the .env.example file, and rename it to .env.
In the .env file, replace your_project_endpoint_here with your actual project endpoint:
```
 PROJECT_ENDPOINT=<your_project_endpoint>
 AGENT_NAME=it-support-agent
```
To get your project endpoint: In VS Code, open the AI Toolkit extension, right-click on your active project, and select Copy Endpoint. If Copy Endpoint isn’t available in your installed version of AI Toolkit, open the Microsoft Foundry portal, go to your project, and copy the project endpoint from the project overview page instead.
Save the .env file (Ctrl+S or File > Save).
Open a terminal in VS Code (Terminal > New Terminal).

Install the required packages and login:

 pip install -r requirements.txt

 az login

Run the application:
```
 python agent_with_functions.py
```

Test the client application

When the agent starts, try these prompts to test different capabilities:

Test policy search with file search:

 What's the policy for password resets?

Request data analysis with code interpreter:

 Analyze the system performance data and identify any periods where CPU usage exceeded 80%

Request a visualization:

 Create a line chart showing memory usage trends over time

Ask for statistical analysis:

 What are the average, minimum, and maximum values for disk usage in the performance data?

Combined analysis:

 Find any correlation between high CPU usage and memory usage in the performance data

Observe how the agent uses both file search (for policy questions) and code interpreter (for data analysis) to fulfill your requests. The code interpreter will analyze the CSV data, perform calculations, and can even generate visualizations. Type exit when done testing.

Cleanup

To avoid unnecessary Azure charges, delete the resources you created:

In the Foundry portal, navigate to your project
Select Settings > Delete project
Alternatively, delete the entire resource group from the Azure portal