Get started with information extraction in Microsoft Foundry

Hi, I’m Anton.
I’ll be here to help you with hints and tips as you work through this lab, in which you’ll use Azure Content Understanding in Foundry to extract data from images.

If you want more interactive help, you can chat with me in the Ask Anton app.

This exercise takes approximately 25 minutes.

Create a Microsoft Foundry project

1. In a web browser, open Microsoft Foundry at https://ai.azure.com to start building; signing in using your Azure credentials.
2. If it isn’t already enabled, in the tool bar the top of the page, enable the New Foundry option.
3. If you do not have any existing projects, you will be prompted to create one. Create a new project with a unique name; expanding the Advanced options area to specify the following settings for your project (or you can select an existing project if you have one!):
   • Foundry resource: Enter a valid name for your AI Foundry resource.
   • Subscription: Your Azure subscription
   • Resource group: Create or select a resource group
   • Region: Select West US, Sweden Central, Australia East, or any of the regions in this list

   
   Tip: Depending on your permissions in the Azure subscription, you may need to clear the option to set up recommended resources.

4. Wait for your project to be created. It may take a few minutes. After creating or selecting a project in the new Foundry portal, it should open in a page similar to the following image:

   

Use Content Understanding to extract information from documents

Azure Content Understanding is a Foundry service that uses AI models to turn unstructured, multimodal content (documents, images, video, audio) into structured, usable outputs like JSON. It processes content by extracting, classifying, and generating fields with confidence scores and source grounding.

Open the Content Understanding playground in Foundry portal

1. In the Foundry portal, navigate to the tool bar at the top of the screen and select Build.

2. On the Build page, in the menu on the left-side of the screen (which you may need to expand), select the Services page.

   Microsoft Foundry Tools includes multiple AI Services (formerly known as Microsoft Cognitive Services) that support common speech, translation, language, and content understanding workloads.

   

3. Select Content Understanding to open the Content Understanding tool playground.

   

Use OCR to read text in an image

Suppose you want to find information related to a piece of computer hardware or some other item with information printed on it. A first step might be to digitize the text so you can use it to look up details on the Internet or in an AI assistant. You can use an AI technique called optical character recognition (OCR) to “read” text in images.

1. Select OCR/Read, and ensure that Document is selected in the Modality list, and OCR/Read is selected in the list of analyzers.

2. Select any sample image, and use the Run analysis button to extract text from it. When analysis is complete, view the results.

   

3. In the pane on the right, review the Markdown, Paragraphs, and Result tabs to see the data that has been read from the document by the analyzer.

4. In a new browser tab, download pcbs.zip from https://aka.ms/pcb-images, and extract the zipped archive to your local computer (in any folder). These files are images of printed circuit boards that contain text.
5. Upload any of the PCB images, and view it in the main content area of the app.

6. Run analysis on the uploaded image and review the results.

   

7. Repeat the process to analyze the other PCB images you downloaded.

   
   Tip: Try uploading any images that contain legible text.

   The OCR/Read analyzer extracts text from images. However, sometimes it may be useful to extract additional information about the layout of the text in the image.

8. In the list of analyzers, select Layout. Then select any of the sample images and use the Run analysis button to extract information from it. When analysis is complete, review the Markdown, Paragraphs, Tables, and Result tabs to see the ways in which the layout of the data in the document has been interpreted by the analyzer.

   Extacting the text and page layout is useful when the documents need to scan have a consistent, well-defined structure. In many cases though, you need to be able to identify which text values map to which data fields; so a more specific analyzer is needed.

Extract fields from documents

Now suppose you need to extract data fields from scanned receipts to help automate an expense claim solution. You can use OCR to identify text and its location in images, and then use a generative AI model to associate individual text values with specific data fields - such as company names, phone numbers, dates, amounts, and so on.

1. In the list of analyzer types, select Procurement, and then select the Receipt analyzer.

   
   Tip: Field extraction requires a custom model, so you may be prompted to deploy models during this process. Click Cancel when this happens.
   
   Do not run analysis - we’ll review the pre-prepared analysis results.

   

2. In the pane on the right, review the Fields, Markdown, Paragraphs, and Result tabs to see the data extracted from the document by the analyzer.

   The Fields tab displays a user-friendly version of the information from the raw JSON in the Results tab, which is how a client application would receive the results of analysis.

Understand how to extract content with the Python SDK

As a developer, you can also use code to extract meaning from content. The Foundry playground provides various code samples to get you started with information extraction with Azure Content Understanding.

1. Let’s take a closer look at the Python code for document layout analysis. In the Content Understanding playground, while viewing the results of the Receipt analyzer, select the Code tab. The following code is provided:

    import sys
    import json
       
    from azure.ai.contentunderstanding import ContentUnderstandingClient
    from azure.ai.contentunderstanding.models import AnalysisInput, AnalysisResult
    from azure.core.credentials import AzureKeyCredential
    from azure.core.exceptions import AzureError
    from azure.identity import DefaultAzureCredential
       
    def main() -> None:
        # Insert the following configurations.
        # 1) AZURE_CONTENT_UNDERSTANDING_ENDPOINT - the endpoint to your Content Understanding resource.
        endpoint = "https://content-project-resource.services.ai.azure.com/"
       
        # 2) CONTENT_UNDERSTANDING_KEY - your Content Understanding API key (optional if using DefaultAzureCredential).
        key = ""
       
        # 3) FILE_URL - you can replace this with your own URL.
        file_url = ""
       
        # ANALYZER_ID - the ID of the analyzer to use.
        analyzer_id = "prebuilt-receipt"
       
        # API_VERSION - the API version to use.
        api_version = "2025-11-01"
       
        # Set up Content Understanding client.
        credential = AzureKeyCredential(key) if key and "" not in key else DefaultAzureCredential()
        client = ContentUnderstandingClient(endpoint=endpoint, credential=credential, api_version=api_version)
       
        # [START analyze]
        print(f"Analyzing with {analyzer_id} analyzer...")
        print(f"  File URL: {file_url}\n")
       
        try:
            poller = client.begin_analyze(
                analyzer_id=analyzer_id,
                inputs=[AnalysisInput(url=file_url)],
            )
            result: AnalysisResult = poller.result()
        except AzureError as err:
            print(f"[Azure Error]: {err.message}")
            sys.exit(1)
        except Exception as ex:
            print(f"[Unexpected Error]: {ex}")
            sys.exit(1)
        # [END analyze]
       
        # [START output_result]
        print("=" * 50)
        print("Analysis result:")
        print("=" * 50 + "\n")
       
        max_display_lines = 50
        result_str = json.dumps(result.as_dict(), indent=2)
        ret_lines = result_str.splitlines()
       
        if len(ret_lines) > max_display_lines:
            print("\n".join(ret_lines[:max_display_lines]))
            print(f"\n {len(ret_lines) - max_display_lines} more lines to be displayed...\n")
        else:
            print(result_str)
        # [END output_result]
       
    if __name__ == "__main__":
        main()
   

   The code connects to the Content Understanding tool in your Foundry resource, and submits a document file to the prebuilt-receipt analyzer. The analyzer runs asynchronously, and returns the results of the analysis in the JSON format you saw in the Result tab.

Summary

In this exercise, you explored Azure Content Understanding in Foundry and learned how it transforms unstructured content into structured, usable data. You tried out three analyzers, each building on the previous one in capability:

• Read: Extracts raw text from documents without interpreting structure or meaning.
• Layout: Goes a step further by capturing structure and hierarchy.
• Receipt: A document -specific analyzer that uses a combination of capabilities to extract text values and map them to data fields.

You also learned how developers can integrate Content Understanding into applications using the Python SDK, which enables programmatic analysis of documents outside the Foundry playground.

Clean up

If you’ve finished working with the Content Understanding service, you should delete the resources you have created in this exercise to avoid incurring unnecessary Azure costs.

1. Open the Azure portal at https://portal.azure.com and select the resource group that contains the resources you created.
2. Select Delete resource group and then enter the resource group name to confirm. The resource group is then deleted.

If you used the Ask Anton app during this lab, we’d love you to tell us about your experience with it!