Process Azure Cosmos DB for NoSQL data using Azure Functions

The Azure Cosmos DB trigger for Azure Functions is implemented using a change feed processor. You can create functions that respond to create and update operations in your Azure Cosmos DB for NoSQL container with this knowledge. If you have implemented a change feed processor manually, the setup for Azure Functions is similar.

In this lab, you will

Create an Azure Cosmos DB for NoSQL account

Azure Cosmos DB is a cloud-based NoSQL database service that supports multiple APIs. When provisioning an Azure Cosmos DB account for the first time, you will select which of the APIs you want the account to support (for example, Mongo API or NoSQL API). Once the Azure Cosmos DB for NoSQL account is done provisioning, you can retrieve the endpoint and key and use them to connect to the Azure Cosmos DB for NoSQL account using the Azure SDK for .NET or any other SDK of your choice.

  1. In a new web browser window or tab, navigate to the Azure portal (portal.azure.com).

  2. Sign into the portal using the Microsoft credentials associated with your subscription.

  3. Select + Create a resource, search for Cosmos DB, and then create a new Azure Cosmos DB for NoSQL account resource with the following settings, leaving all remaining settings to their default values:

    Setting Value
    Subscription Your existing Azure subscription
    Resource group Select an existing or create a new resource group
    Account Name Enter a globally unique name
    Location Choose any available region
    Capacity mode Serverless

    📝 Your lab environments may have restrictions preventing you from creating a new resource group. If that is the case, use the existing pre-created resource group.

  4. Wait for the deployment task to complete before continuing with this task.

  5. Go to the newly created Azure Cosmos DB account resource and navigate to the Keys pane.

  6. This pane contains the connection details and credentials necessary to connect to the account from the SDK. Specifically:

    1. Notice the URI field. You will use this endpoint value later in this exercise.

    2. Notice the PRIMARY KEY field. You will use this key value later in this exercise.

  7. Select Data Explorer from the resource menu.

  8. In the Data Explorer pane, expand New Container and then select New Database.

  9. In the New Database popup, enter the following values for each setting, and then select OK:

    Setting Value
    Database id cosmicworks

  10. Back in the Data Explorer pane, observe the cosmicworks database node within the hierarchy.

  11. In the Data Explorer pane, select New Container.

  12. In the New Container popup, enter the following values for each setting, and then select OK:

    Setting Value
    Database id Use existing | cosmicworks
    Container id products
    Partition key /categoryId

  13. Back in the Data Explorer pane, expand the cosmicworks database node and then observe the products container node within the hierarchy.

  14. In the Data Explorer pane, select New Container again.

  15. In the New Container popup, enter the following values for each setting, and then select OK:

    Setting Value
    Database id Use existing | cosmicworks
    Container id productslease
    Partition key /id

  16. Back in the Data Explorer pane, expand the cosmicworks database node and then observe the productslease container node within the hierarchy.

  17. Return to the Home of the Azure portal.

Create Application Insight

Before you create the Azure Funtion Application, you will need to enable an Azure Application Insight so you can monitor the application funtion. The Application Insight will first need a Log Analytics workspace.

  1. In the search box search for Log Analytics workspaces.

  2. Select to +Create a new Log Analytics workspace.

  3. In the Log Analytics workspace dialog, enter the following values for each setting, and then select Review+Create and then select Create:

    Setting Value
    Subscription Your existing Azure subscription
    Resource group Select an existing or create a new resource group
    Name lab14laworkspace
    Location Choose any available region

  4. Once your Log Analytics workspace is created, in the search box search for Application Insights.

  5. Select to +Create a new Application Insight.

  6. In the Application Insights dialog, enter the following values for each setting, and then select Review+Create and then select Create:

    Setting Value
    Subscription (both entries) Your existing Azure subscription
    Resource group Select an existing or create a new resource group
    Name lab14appinsight
    Location Choose any available region
    Log Analytics Workspace lab14laworkspace

You should now be able to monitor your application function.

Create an Azure Function app and Azure Cosmos DB-triggered function

Before you can begin writing code, you will need to create the Azure Functions resource and its dependent resources (Application Insights, Storage) using the creation wizard.

  1. Select + Create a resource, search for Functions, and then create a new Function App account resource with the following settings, leaving all remaining settings to their default values:

    Setting Value
    Subscription Your existing Azure subscription
    Resource group Select an existing or create a new resource group
    Name Enter a globally unique name
    Publish Code
    Runtime stack .NET
    Version 6 (LTS) In-process
    Region Choose any available region
    Storage account Create a new storage account

    📝 Your lab environments may have restrictions preventing you from creating a new resource group. If that is the case, use the existing pre-created resource group.

  2. Wait for the deployment task to complete before continuing with this task.

  3. Go to the newly created Azure Functions account resource and navigate to the Functions pane.

  4. In the Functions pane, select + Create.

  5. In the Create function popup, create a new function with the following settings, leaving all remaining settings to their default values:

    Setting Value
    Development environment Develop in portal
    Select a template Azure Cosmos DB trigger
    New Function ItemsListener
    Cosmos DB account connection Select New | Select Azure Cosmos DB Account | Select the Azure Cosmos DB account you created earlier
    Database name cosmicworks
    Collection name products
    Collection name for leases productslease
    Create lease collection if it does not exist No

Implement function code in .NET

The function you created earlier is a C# script that is edited in-portal. You will now use the portal to write a short function to output the unique identifier of any item inserted or updated in the container.

  1. In the ItemsListener | Function pane, navigate to the Code + Test pane.

  2. In the editor for the run.csx script, delete the contents of the editor area.

  3. In the editor area, reference the Microsoft.Azure.DocumentDB.Core library:

     #r "Microsoft.Azure.DocumentDB.Core"

  4. Add using blocks for the System, System.Collections.Generic, and Microsoft.Azure.Documents namespaces:

     using System;
 using System.Collections.Generic;
 using Microsoft.Azure.Documents;

  5. Create a new static method named Run that has two parameters:

    1. A parameter named input of type IReadOnlyList<> with a generic type of Document.

    2. A parameter named log of type ILogger.

     public static void Run(IReadOnlyList<Document> input, ILogger log)
 {
 }

  6. Within the Run method, invoke the LogInformation method of the log variable passing in a string that calculates the count of items in the current batch:

     log.LogInformation($"# Modified Items:\t{input?.Count ?? 0}");

  7. Still within the Run method, create a foreach loop that iterates over the input variable using the variable item to represent an instance of type Document:

     foreach(Document item in input)
 {
 }

  8. Within the foreach loop of the Run method, invoke the LogInformation method of the log variable passing in a string that prints the Id property of the item variable:

     log.LogInformation($"Detected Operation:\t{item.Id}");

  9. Once you are done, your code file should now include:

     #r "Microsoft.Azure.DocumentDB.Core"
    
 using System;
 using System.Collections.Generic;
 using Microsoft.Azure.Documents;
    
 public static void Run(IReadOnlyList<Document> input, ILogger log)
 {
     log.LogInformation($"# Modified Items:\t{input?.Count ?? 0}");
    
     foreach(Document item in input)
     {
         log.LogInformation($"Detected Operation:\t{item.Id}");
     }
 }

  10. Expand the Logs section to connect to the streaming logs for the current function.

    💡 It can take a couple of seconds to connect to the streaming log service. You will see a message in the log output once you are connected.

  11. Save the current function code.

  12. Observe the result of the C# code compilation. You should expect to see a Compilation succeeded message at the end of the log output.

    📝 You may see warning messages in the log output. These warnings will not impact this lab.

  13. Maximize the log section to expand the output window to fill the maximum available space.

    📝 You will use another tool to generate items in your Azure Cosmos DB for NoSQL container. Once you generate the items, you will return to this browser window to observe the output. Do not close the browser window prematurely.

Seed your Azure Cosmos DB for NoSQL account with sample data

You will use a command-line utility that creates a cosmicworks database and a products container. The tool will then create a set of items that you will observe using the change feed processor running in your terminal window.

  1. Start Visual Studio Code.

    📝 If you are not already familiar with the Visual Studio Code interface, review the Get Started guide for Visual Studio Code

  2. In Visual Studio Code, open the Terminal menu and then select New Terminal to open a new terminal instance.

  3. Install the [cosmicworks][nuget.org/packages/cosmicworks] command-line tool for global use on your machine.

     dotnet tool install cosmicworks --global --version 1.*

    💡 This command may take a couple of minutes to complete. This command will output the warning message (*Tool ‘cosmicworks’ is already installed’) if you have already installed the latest version of this tool in the past.

  4. Run cosmicworks to seed your Azure Cosmos DB account with the following command-line options:

    Option Value
    –endpoint The endpoint value you copied earlier in this lab
    –key The key value you coped earlier in this lab
    –datasets product 
     cosmicworks --endpoint <cosmos-endpoint> --key <cosmos-key> --datasets product

    📝 For example, if your endpoint is: https­://dp420.documents.azure.com:443/ and your key is: fDR2ci9QgkdkvERTQ==, then the command would be: cosmicworks --endpoint https://dp420.documents.azure.com:443/ --key fDR2ci9QgkdkvERTQ== --datasets product

  5. Wait for the cosmicworks command to finish populating the account with a database, container, and items.

  6. Close the integrated terminal.

  7. Close Visual Studio Code.

  8. Return to the currently open browser window or tab with the Azure Functions log section expanded.

  9. Observe the log output from your function. The terminal outputs a Detected Operation message for each change that was sent to it using the change feed. The operations are batched into groups of ~100 operations.

  10. Close your web browser window or tab.