Create a generative AI app that uses your own data

Retrieval Augmented Generation (RAG) is a technique used to build applications that integrate data from custom data sources into a prompt for a generative AI model. RAG is a commonly used pattern for developing generative AI apps - chat-based applications that use a language model to interpret inputs and generate appropriate responses.

In this exercise, you’ll use Azure AI Foundry portal and the Azure AI Foundry and Azure OpenAI SDKs to integrate custom data into a generative AI app.

This exercise takes approximately 45 minutes.

Note: This exercise is based on pre-release SDKs, which may be subject to change. Where necessary, we’ve used specific versions of packages; which may not reflect the latest available versions.

Create an Azure AI Foundry project

Let’s start by creating an Azure AI Foundry project and the service resources it needs to support using your own data - including an Azure AI Search resource.

  1. In a web browser, open the Azure AI Foundry portal at https://ai.azure.com and sign in using your Azure credentials. Close any tips or quick start panes that are opened the first time you sign in, and if necessary use the Azure AI Foundry logo at the top left to navigate to the home page, which looks similar to the following image:

    Screenshot of Azure AI Foundry portal.

  2. In the home page, select + Create project.
  3. In the Create a project wizard, enter a suitable project name for (for example, my-ai-project) and if an existing hub is suggested, choose the option to create a new one. Then review the Azure resources that will be automatically created to support your hub and project.
  4. Select Customize and specify the following settings for your hub:
    • Hub name: A unique name - for example my-ai-hub
    • Subscription: Your Azure subscription
    • Resource group: Create a new resource group with a unique name (for example, my-ai-resources), or select an existing one
    • Location: Select Help me choose and then select both gpt-4 and text-embedding-ada-002 in the Location helper window and use the recommended region*
    • Connect Azure AI Services or Azure OpenAI: Create a new AI Services resource with an appropriate name (for example, my-ai-services) or use an existing one
    • Connect Azure AI Search: Create a new Azure AI Search resource with a unique name

    * Azure OpenAI resources are constrained at the tenant level by regional quotas. In the event of a quota limit being reached and no region is recommended for both models, select only one of them and use its recommended region. You will create another resource in a different region for the second model later in the exercise.

  5. Select Next and review your configuration. Then select Create and wait for the process to complete.

  6. When your project is created, close any tips that are displayed and review the project Overview page in Azure AI Foundry portal, which should look similar to the following image:

    Screenshot of a Azure AI project details in Azure AI Foundry portal.

Deploy models

You need two models to implement your solution:

  • An embedding model to vectorize text data for efficient indexing and processing.
  • A model that can generate natural language responses to questions based on your data.
  1. In the Azure AI Foundry portal, in your project, in the navigation pane on the left, under My assets, select the Models + endpoints page.

  2. Create a new deployment of the text-embedding-ada-002 model with the following settings by selecting Customize in the Deploy model wizard:

    • Deployment name: text-embedding-ada-002
    • Deployment type: Standard
    • Model version: Select the default version
    • AI resource: Select the resource created previously
    • Tokens per Minute Rate Limit (thousands): 5K
    • Content filter: DefaultV2
    • Enable dynamic quota: Disabled

    Note: If your current AI resource location doesn’t have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project.

  3. Repeat the previous steps to deploy a gpt-4 model with the deployment name gpt-4.

    Note: Reducing the Tokens Per Minute (TPM) helps avoid over-using the quota available in the subscription you are using. 5,000 TPM is sufficient for the data used in this exercise.

Add data to your project

The data for your copilot consists of a set of travel brochures in PDF format from the fictitious travel agency Margie’s Travel. Let’s add them to the project.

  1. Download the zipped archive of brochures from https://github.com/MicrosoftLearning/mslearn-ai-studio/raw/main/data/brochures.zip and extract it to a folder named brochures on your local file system.
  2. In Azure AI Foundry portal, in your project, in the navigation pane on the left, under My assets, select the Data + indexes page.
  3. Select + New data.
  4. In the Add your data wizard, expand the drop-down menu to select Upload files/folders.
  5. Select Upload folder and select the brochures folder.
  6. Select Next and set the data name to brochures.
  7. Wait for the folder to be uploaded and note that it contains several .pdf files.

Create an index for your data

Now that you’ve added a data source to your project, you can use it to create an index in your Azure AI Search resource.

  1. In Azure AI Foundry portal, in your project, in the navigation pane on the left, under My assets, select the Data + indexes page.
  2. In the Indexes tab, add a new index with the following settings:
    • Source location:
      • Data source: Data in Azure AI Foundry portal
        • Select the brochures data source
    • Index configuration:
      • Select Azure AI Search service: Select the AzureAISearch connection to your Azure AI Search resource
      • Vector index: brochures-index
      • Virtual machine: Auto select
    • Search settings:
      • Vector settings: Add vector search to this search resource
      • Azure OpenAI connection: Select the default Azure OpenAI resource for your hub.

  3. Wait for the indexing process to be completed, which can take a while depending on available compute resources in your subscription. The index creation operation consists of the following jobs:

    • Crack, chunk, and embed the text tokens in your brochures data.
    • Create the Azure AI Search index.
    • Register the index asset.

Test the index in the playground

Before using your index in a RAG-based prompt flow, let’s verify that it can be used to affect generative AI responses.

  1. In the navigation pane on the left, select the Playgrounds page and open the Chat playground.
  2. On the Chat playground page, in the Setup pane, ensure that your gpt-4 model deployment is selected. Then, in the main chat session panel, submit the prompt Where can I stay in New York?
  3. Review the response, which should be a generic answer from the model without any data from the index.

  4. In the Setup pane, expand the Add your data field, and then add the brochures-index project index and select the hybrid (vector + keyword) search type.

    Tip: In some cases, newly created indexes may not be available right away. Refreshing the browser usually helps, but if you’re still experiencing the issue where it can’t find the index you may need to wait until the index is recognized.

  5. After the index has been added and the chat session has restarted, resubmit the prompt Where can I stay in New York?
  6. Review the response, which should be based on data in the index.

Create a RAG client app with the Azure AI Foundry and Azure OpenAI SDKs

Now that you have a working index, you can use the Azure AI Foundry and Azure OpenAI SDKs to implement the RAG pattern in a client application. Let’s explore the code to accomplish this in a simple example.

Tip: You can choose to develop your RAG solution using Python or Microsoft C#. Follow the instructions in the appropriate section for your chosen language.

Prepare the application configuration

  1. In the Azure AI Foundry portal, view the Overview page for your project.
  2. In the Project details area, note the Project connection string. You’ll use this connection string to connect to your project in a client application.
  3. Open a new browser tab (keeping the Azure AI Foundry portal open in the existing tab). Then in the new tab, browse to the Azure portal at https://portal.azure.com; signing in with your Azure credentials if prompted.

  4. Use the [>_] button to the right of the search bar at the top of the page to create a new Cloud Shell in the Azure portal, selecting a PowerShell environment. The cloud shell provides a command line interface in a pane at the bottom of the Azure portal.

    Note: If you have previously created a cloud shell that uses a Bash environment, switch it to PowerShell.

  5. In the cloud shell toolbar, in the Settings menu, select Go to Classic version (this is required to use the code editor).

    Tip: As you paste commands into the cloudshell, the ouput may take up a large amount of the screen buffer. You can clear the screen by entering the cls command to make it easier to focus on each task.

  6. In the PowerShell pane, enter the following commands to clone the GitHub repo for this exercise:

     rm -r mslearn-ai-foundry -f
 git clone https://github.com/microsoftlearning/mslearn-ai-studio mslearn-ai-foundry

Note: Follow the steps for your chosen programming language.

  1. After the repo has been cloned, navigate to the folder containing the chat application code files:

    Python

    cd mslearn-ai-foundry/labfiles/rag-app/python

    C#

    cd mslearn-ai-foundry/labfiles/rag-app/c-sharp

  2. In the cloud shell command line pane, enter the following command to install the libraries you’ll use:

    Python

    pip install python-dotenv azure-ai-projects azure-identity openai

    C#

    dotnet add package Azure.Identity
dotnet add package Azure.AI.Projects --prerelease
dotnet add package Azure.AI.OpenAI --prerelease

  3. Enter the following command to edit the configuration file that has been provided:

    Python

    code .env

    C#

    code appsettings.json

    The file is opened in a code editor.

  4. In the code file, replace the following placeholders:
    • your_project_endpoint: Replace with the connection string for your project (copied from the project Overview page in the Azure AI Foundry portal)
    • your_model_deployment Replace with the name you assigned to your model deployment (which should be gpt-4)
    • your_index: Replace with your index name (which should be brochures-index)
  5. After you’ve replaced the placeholders, within the code editor, use the CTRL+S command or Right-click > Save to save your changes and then use the CTRL+Q command or Right-click > Quit to close the code editor while keeping the cloud shell command line open.

Explore code to implement the RAG pattern

  1. Enter the following command to edit the code file that has been provided:

    Python

    code rag-app.py

    C#

    code Program.cs
  2. Review the code in the file, noting that it:
    • Uses the Azure AI Foundry SDK to connect to your project (using the project connection string)
    • Retrieves the default Azure AI Search connection from your project so it can determine the endpoint and key for your Azure AI Search service.
    • Creates an authenticated Azure OpenAI client based on the default Azure OpenAI service connection in your project.
    • Submits a prompt (including a system and user message) to the Azure OpenAI client, adding additional information about the Azure AI Search index to be used to ground the prompt.
    • Displays the response from the grounded prompt.
  3. Use the CTRL+Q command to close the code editor without saving any changes, while keeping the cloud shell command line open.

Run the chat application

  1. In the cloud shell command line pane, enter the following command to run the app:

    Python

    python rag-app.py

    C#

    dotnet run

  2. When prompted, enter a question, such as Where can I travel to? and review the response from your generative AI model.

    Note that the response includes source references to indicate the indexed data in which the answer was found.

  3. Try a few more questions, for example Where should I stay in London?

    Note: This simple example application doesn’t include any logic to retain the conversation history, so each prompt is treated as a new conversation.

  4. When you’re finished, enter quit to exit the program. Then close the cloud shell pane.

Challenge

Now you’ve experienced how to integrate your own data in a generative AI app built with the Azure AI Foundry portal, let’s explore further!

Try adding a new data source through the Azure AI Foundry portal, index it, and integrate the indexed data in a client app. Some data sets you could try are:

  • A collection of (research) articles you have on your computer.
  • A set of presentations from past conferences.
  • Any of the datasets available in the Azure Search sample data repository.

Test your solution by submitting prompts that could only be answered by the data set you chose!

Clean up

To avoid unnecessary Azure costs and resource utilization, you should remove the resources you deployed in this exercise.

  1. If you’ve finished exploring Azure AI Foundry, return to the Azure portal at https://portal.azure.com and sign in using your Azure credentials if necessary. Then delete the resources in the resource group where you provisioned your Azure AI Search and Azure AI resources.