How to manage your Azure IoT Hub

Lab Scenario

Contoso’s Asset Monitoring and Tracking Solution is working great. The system provides continuous monitoring throughout the packaging and shipping process. You have implemented Group Enrollments within DPS to provision devices at scale, and when the container arrives at the destination, the IoT device “decommissioned” through DPS so that it can re-used for future shipments.

To help manage device utilization and other characteristics of the solution, the IT department has asked your team to implement Azure monitoring and logging services within the IoT solution.

You agree to begin by implementing some simple metrics that can be reviewed with the IT folks before you commit to any additional workload.

In this lab, you will implement monitoring to track the number of connected devices and telemetry messages sent, as well as send connection events to a log. In addition, you will create an alert that will be triggered based upon the average number of devices connected. To test the system, you will configure 10 simulated IoT Devices that will authenticate with DPS using a Device CA Certificate generated on the Root CA Certificate chain. The IoT Devices will be configured to send telemetry to the the IoT Hub.

The following resources will be created:

Lab 17 Architecture

In This Lab

In this lab, you will complete the following activities:

  • Verify Lab Prerequisites
  • Enable diagnostic logs.
  • Enable metrics.
  • Set up alerts for those metrics.
  • Download and run an app that simulates IoT devices connecting via X.509 and sending messages to the hub.
  • Run the app until the alerts begin to fire.
  • View the metrics results and check the diagnostic logs.

Lab Instructions

Exercise 1: Verify Lab Prerequisites

This lab assumes that the following Azure resources are available:

Resource Type Resource Name
Resource Group AZ-220-RG
IoT Hub AZ-220-HUB-{YOUR-ID}
Device Provisioning Service AZ-220-DPS-{YOUR-ID}
Storage Account az220storage{your-id}

If these resources are not available, you will need to run the lab17-setup.azcli script as instructed below before moving on to Exercise 2. The script file is included in the GitHub repository that you cloned locally as part of the dev environment configuration (lab 3).

The lab17-setup.azcli script is written to run in a bash shell environment - the easiest way to execute this is in the Azure Cloud Shell.

  1. Using a browser, open the Azure Cloud Shell and login with the Azure subscription you are using for this course.

    If you are prompted about setting up storage for Cloud Shell, accept the defaults.

  2. Verify that the Azure Cloud Shell is using Bash.

  3. On the Azure Shell toolbar, click Upload/Download files (fourth button from the right).

  4. In the dropdown, click Upload.

  5. In the file selection dialog, navigate to the folder location of the GitHub lab files that you downloaded when you configured your development environment.

    In Lab 3: Setup the Development Environment, you cloned the GitHub repository containing lab resources by downloading a ZIP file and extracting the contents locally. The extracted folder structure includes the following folder path:

    • Allfiles
      • Labs
        • 17-How to manage your Azure IoT Hub
          • Setup

    The lab17-setup.azcli script file is located in the Setup folder for lab 17.

  6. Select the lab17-setup.azcli file, and then click Open.

    A notification will appear when the file upload has completed.

  7. To verify that the correct file has uploaded in Azure Cloud Shell, enter the following command:

     ls
    

    The ls command lists the content of the current directory. You should see the lab17-setup.azcli file listed.

  8. To create a directory for this lab that contains the setup script and then move into that directory, enter the following Bash commands:

     mkdir lab17
     mv lab17-setup.azcli lab17
     cd lab17
    
  9. To ensure the lab17-setup.azcli script has the execute permission, enter the following command:

     chmod +x lab17-setup.azcli
    
  10. On the Cloud Shell toolbar, to edit the lab17-setup.azcli file, click Open Editor (second button from the right - { }).

  11. In the Files list, to expand the lab17 folder and open the script file, click lab17, and then click lab17-setup.azcli.

    The editor will now show the contents of the lab17-setup.azcli file.

  12. In the editor, update the values of the {YOUR-ID} and {YOUR-LOCATION} variables.

    Referencing the sample below as an example, you need to set {YOUR-ID} to the Unique ID you created at the start of this course - i.e. CAH191211, and set {YOUR-LOCATION} to the location that makes sense for your resources.

     #!/bin/bash
    
     YourID="{YOUR-ID}"
     RGName="AZ-220-RG"
     IoTHubName="AZ-220-HUB-$YourID"
     DPSName="AZ-220-DPS-$YourID"
     DeviceName="asset-track"
     Location="{YOUR-LOCATION}"
    

    Note: The {YOUR-LOCATION} variable should be set to the short name for the region. You can see a list of the available regions and their short-names (the Name column) by entering this command:

    az account list-locations -o Table
    
    DisplayName           Latitude    Longitude    Name
    --------------------  ----------  -----------  ------------------
    East Asia             22.267      114.188      eastasia
    Southeast Asia        1.283       103.833      southeastasia
    Central US            41.5908     -93.6208     centralus
    East US               37.3719     -79.8164     eastus
    East US 2             36.6681     -78.3889     eastus2
    
  13. In the top-right of the editor window, to save the changes made to the file and close the editor, click , and then click Close Editor.

    If prompted to save, click Save and the editor will close.

    Note: You can use CTRL+S to save at any time and CTRL+Q to close the editor.

  14. To create the resources required for this lab, enter the following command:

     ./lab17-setup.azcli
    

    This script can take a few minutes to run. You will see JSON output as each step completes.

    The script will first create a resource group named AZ-220-RG, then your IoT Hub named AZ-220-HUB-{YourID} and Device Provisioning Service named AZ-220-DSP-{YourID}. If the services already exist, a corresponding message will be displayed. The script will link your IoT Hub and DSP. The script will then create a storage account named az220storage{your-id}.

    You should now be ready to proceed with Exercise 2 of this lab.

Exercise 2: Set Up and Use Metrics and Diagnostic Logs with an IoT Hub

Azure Resource logs are platform logs emitted by Azure resources that describe their internal operation. All resource logs share a common top-level schema with the flexibility for each service to emit unique properties for their own events.

When you have an IoT Hub solution running in production, you will want to set up various metrics and enable diagnostic logs. Then, if a problem occurs, you have data to look at that will help you to diagnose the problem and fix it more quickly.

In this exercise, you will enable diagnostic logs and use them to to check for errors. You will also set up some metrics to watch, and alerts that fire when the metrics hit a certain boundary.

Task 1: Enable diagnostics

  1. If necessary, log in to your Azure portal using your Azure account credentials.

    If you have more than one Azure account, be sure that you are logged in with the account that is tied to the subscription that you will be using for this course.

  2. On your Azure dashboard, click AZ-220-HUB-{YOUR-ID}.

    Your dashboard should have a link to your IoT Hub on the on the AZ-220-RG resource group tile.

  3. On the left side navigation menu, under Monitoring, click Diagnostic settings.

    Note: Current documentation suggests that Diagnostics may be disabled by default. If so, you may need to “Turn on diagnostics” in order to collect diagnostics data for your IoT Hub. When you click Turn on diagnostics, a Diagnostic settings blade will open.

  4. On the Diagnostics settings pane, under Name, click + Add diagnostic setting.

  5. In the Diagnostic settings name textbox, enter diags-hub

  6. Take a minute to review the options listed under Destination details.

    You can see that there are 3 options available for routing the metrics - you can learn more about each by following the links below:

    In this lab we will use the storage account option.

  7. Under Destination details, click Archive to a storage account.

    Additional fields are made available once you select this destination option, including the option to specify Retention (days) for the log categories.

    Note: Take a moment to review the notes about storage accounts and costs.

  8. For the Subscription field, select the subscription that you used to create your IoT Hub.

  9. For the Storage account field, select the az220storage{your-id} storage account.

    This account was created by the lab17-setup.azcli script. If it is not listed in the dropdown, you may need to create an account manually (check with your instructor).

  10. On the Diagnostic settings blade, under Category details, click Connections, and then click DeviceTelemetry.

  11. For each of the Log Categories that you selected, in the Retention (days) field, enter 7

  12. At the top of the blade, click Save, and then close the blade

    You should now be on the Diagnostics settings pane of your IoT Hub, and you should see that the list of Diagnostics settings has been updated to show the diags-hub setting that you just created.

    Later, when you look at the diagnostic logs, you’ll be able to see the connect and disconnect logging for the device.

Task 2: Setup Metrics

In this task, you will set up various metrics to watch for when messages are sent to your IoT hub.

  1. Ensure that you have your IoT Hub blade open.

    The previous task left you on the Diagnostics settings pane of the IoT HUb blade.

  2. On the left side navigation menu, under Monitoring, click Metrics.

    The Metrics pane is displayed showing a new, empty, chart.

  3. In the top-right corner of the screen, to change the time range and granularity for the chart, click Last 24 hours (Automatic).

  4. In the context menu that appears, under Time range, click Last 4 hours.

  5. In the same context menu, under Time granularity, click 1 minute, and under Show time as, ensure that Local is selected.

  6. To save your time settings, click Apply.

  7. Take a minute to examine the settings are used to specify the chart Metrics.

    Under the Chart Title and the toolbar for the chart, you will see an area to specify Metrics.

    • Notice that the Scope is already set to AZ-220-HUB-{YOUR-ID}.
    • Notice that Metric Namespace is already set to IoT Hub standard metrics.

    Note: By default, there is only one metric namespace available. Namespaces are a way to categorize or group similar metrics together. By using namespaces, you can achieve isolation between groups of metrics that might collect different insights or performance indicators. For example, you might have a namespace called az220memorymetrics that tracks memory-use metrics which profile your app. Another namespace called az220apptransaction might track all metrics about user transactions in your application. You can learn more about custom metrics and namespaces here.

    Your next steps is add a metric that will be used to monitor how many telemetry messages have been sent to your IoT Hub.

  8. In the Metric dropdown, click Telemetry messages sent.

    Notice the large number of metrics that are available for you to select from!

  9. Under Aggregation, ensure that Sum is selected.

    Notice there are 4 aggregation operations available - Avg, Min, Max and Sum.

  10. Take a moment to review your chart.

    Notice that the chart title has updated to reflect the metric chosen.

    You have completed the specification for the first metric. Next, you will add another metric to monitor the number of connected devices.

  11. Under the chart title, on the toolbar, click Add metric.

    A new metric will appear. Notice that the Scope and Metric Namespace values are pre-populated.

  12. In the Metric dropdown, click Connected devices (preview).

  13. Under Aggregation, ensure that Avg is selected.

    Your screen now shows the minimized metric for Telemetry messages sent, plus the new metric for avg connected devices. Notice that the chart title has updated to reflect both metrics.

    Note: To edit the chart title, click the pencil to the right of the title.

  14. Under the Chart Title, on the right side of the toolbar, click Pin to dashboard, and then click Pin to current dashboard

    Note: In order to retain the chart you have just created, it must be pinned to a dashboard.

  15. Navigate to the “AZ-220” dashboard and verify the chart is displayed.

    Note: You can customize the size and position of the chart by using drag and drop operations.

Now that you have enabled logging and setup a chart to monitor metrics, it is a good time for you to set up an alert.

Exercise 3: Configure an Alert

Alerts are used to proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues before the users of your system notice them.

In your asset tracking scenario, you use sensors to track the containers that are being shipped to customers. Each time a sensor is added in a shipping container, it is auto-provisioned through DPS.

For your upcoming proof-of-concept demonstration, you want to create an Alert that triggers when the number containers that are currently in transit approaches a capacity limit. To trigger the Alert, you will use the number of Device Connected events from IoT Hub.

In this exercise, you are going to add an alert that triggers when 5 or more devices have connected.

  1. In your Azure portal window, open your IoT Hub blade.

  2. On the left side navigation menu, under Monitoring, click Alerts.

    The empty Alerts page is displayed. Notice that the Subscription, Resource group, Resource and Time range fields are pre-populated.

  3. In the Time range dropdown, click Past hour.

  4. At the top of the Alerts pane, click + New alert rule

    The Create rule blade should now be displayed.

  5. Take a moment to review the Create rule blade.

    At the top of the blade are two fields - RESOURCE and HIERARCHY. Notice that these fields are pre-populated with properties from your IoT Hub. If you needed to change the pre-selected resource, you would click Select under RESOURCE.

  6. Under CONDITION, click Add.

    The Configure signal logic pane should now be displayed. Notice that there is a paginated table of available signals displayed. The fields above the table filter the table to assist in finding the signal types you want.

  7. Under Signal type, ensure that All is selected.

    If you open the Signal type dropdown, you would see that there are 3 available options: All, Metrics and Activity Log.

    Note: The signal types available for monitoring vary based on the selected target(s). The signal types may be metrics, log search queries or activity logs.

  8. Under Monitor service, ensure that All is selected.

    If you open the Monitor service dropdown, you would see that there are 3 available options: All, Platform and Activity Log - Administrative.

    Note: The platform service provides metrics on service utilization, where as the activity log tracks administrative activities.

  9. In the Search by signal name textbox, type connected

  10. Notice that the list of signals is immediately filtered based on your entry.

  11. Under Signal name, click Connected devices (preview).

    The pane will update to display a chart that is similar to what you created for Metrics. The chart displays the values associated with the selected signal (in this case Connected devices (preview)).

    Beneath the chart is the area that defines the Alert logic.

  12. Take a moment to review the options under Alert logic

    Notice that Threshold has two possible selections - Static and Dynamic. Also notice that Static is selected and Dynamic is unavailable for this signal type.

    Note: As the names suggest, Static Thresholds specify a constant expression for the threshold, whereas Dynamic Thresholds detection leverages advanced machine learning (ML) to learn metrics’ historical behavior, identify patterns and anomalies that indicate possible service issues. You can learn more about Dynamic Thresholds here.

    We are going to create a static threshold that raises and alert whenever the connected devices (preview) signal is equal to 5 or more.

  13. In the Operator dropdown, click Greater than or equal to.

    You may want to make note of the other options for this and the other fields.

  14. Under Aggregation type, ensure that Average is selected.

  15. In the Threshold value textbox, enter 5

    Note: The Condition preview shows you the condition under which the display will refresh based on the Operator, Aggregation type, and Threshold value settings that you entered. Below the Condition preview is the Evaluation based on area. These values determine the historical time period that is aggregated using the Aggregation type selected above and how often the condition is evaluated.

  16. Under Aggregation granularity (Period), ensure that 5 minutes is selected.

  17. Under Frequency of evaluation, ensure that Every 1 Minute is selected.

    Note: As the Frequency of evaluation is shorter than Aggregation granularity (Period), this results in a sliding window evaluation. This means that every minute, the preceding 5 minutes of values will be aggregated (in this case, averaged), and then evaluated against the condition. After a minute of time has passed, once again the preceding 5 minutes of data will be aggregated - this will include one minute of new data and four minutes of data that was already evaluated. Thus, we have a sliding window that moves forward a minute at a time, but is always including 4 minutes of data that was also evaluated as part of an earlier window.

  18. At the bottom of the Configure signal logic pane, to configure the alert condition, click Done.

    The Configure signal logic pane closes and the Create rule blade is shown. Notice that the CONDITION is now populated and a Monthly cost in USD is displayed. At the time of writing, the estimated cost of the alert condition is $0.10.

    Next, you need to configure the action taken when the alert condition is met.

  19. Take a moment to review the ACTIONS GROUPS (optional) area.

    Notice that no action group is selected. There are two options available - Add and Create.

    Note: An action group is a collection of notification preferences defined by the owner of an Azure subscription. An action group name must be unique within the Resource Group is is associated with. Azure Monitor and Service Health alerts use action groups to notify users that an alert has been triggered. Various alerts may use the same action group or different action groups depending on the user’s requirements. You may configure up to 2,000 action groups in a subscription. You can learn more about creating and managing Action Groups here.

  20. Under ACTIONS GROUPS (optional), click Create.

    The Add action group blade is displayed.

  21. Under Action group name, enter AZ-220 Email Action Group

    Note: An action group name must be unique within the Resource Group it is associated with.

  22. Under Short name, enter AZ220EmailAG

    Note: The short name is used in place of a full action group name when notifications are sent using this group and is limited to a max of 12 characters.

  23. Under Subscription, ensure that the subscription you have been using for this lab is selected.

  24. In the Resource group dropdown, click AZ-220-RG.

    Note: Action Groups are usually shared across a subscription and would likely be centrally managed by the Azure subscription owner. As such they are more likely to be included in a common resource group rather than in a project specific resource group such as “AZ-220-RG”. We are using “AZ-220-RG” to make it easier to clean up the resources after the lab.

    The next area, Actions is used to define a list of actions that will be performed whenever this action group is invoked.

  25. Under Action name, enter AZ220Notifications

  26. Open the Action Type dropdown, and then review the available options.

  27. In the Action Type dropdown, click Email/SMS/Push/Voice.

    Immediately, the Email/SMS/Push/Voice blade is displayed showing you the action details for this action type. Notice that you can choose up to 4 methods for delivering the notification.

  28. On the Email/SMS/Push/Voice blade, click Email, and then enter an email address that you have easy access to.

  29. Click SMS, and then enter the Country code and the Phone number for the phone that you wish to use to receive the SMS alert.

  30. Skip Azure app Push Notifications and Voice.

  31. Under Enable the common alert schema, click Yes.

    Note: There are many benefits to using the Common Alert Schema. It standardizes the consumption experience for alert notifications in Azure today. Historically, the three alert types in Azure today (metric, log, and activity log) have had their own email templates, webhook schemas, etc. With the common alert schema, you can now receive alert notifications with a consistent schema. You can learn more about the Common ALert6 Schema here.

    Important: Given the benefits, you may wonder why the common alert schema is not enabled by default - well, when you select Yes you will see a warning Enabling the common alert schema might break any existing integrations. Bear this in mind in your own environments.

  32. At the bottom of the Email/SMS/Push/Voice blade, to save the action configuration, click OK.

    The Add action group blade should now list your Action. Notice that the new action has a link to Edit details if changes are required.

    At this point, we could add multiple actions if we needed to launch some business integration via WebHooks or an Azure Function, however for this lab, a simple notification is enough.

  33. At the bottom of the Add action group blade, to create this action group, click OK.

    A few things happen at the same time. First, Add action group blade closes, leaving you on the Create rule blade, with the new Action Group added to the list of ACTIONS.

    Then, in quick succession, you should receive both an SMS notification and an email, both of which inform you that you have been added to the AZ220EmailAG action group. In the SMS message, you will note that you can reply to the message to stop receiving future notifications and so on - you can learn more about the options here. In the email, you have links that you can click to view the details on action groups and, towards the bottom of the email (in a smaller font) you can see the option to unsubscribe.

    Next, you will configure the ALERT DETAILS.

  34. On the Create rule blade, under Alert rule name, enter Connected Devices Greater or Equal To 5

    The name should be descriptive enough to identify the alert.

  35. Under Description, enter This alert is raised when the number of devices connected to the AZ-220-HUB-{YOUR-ID} hub is greater than or equal to 5.

    The description field is optional, but recommended.

  36. Under Severity, leave Sev 3 selected.

    In our scenario, this alert is informational and not indicative of any critical failure, therefore Sev 3 is the correct choice.

    Note: The severity level option are Sev 0 - Sev 4. Your business should have an established definition for each level.

    For example, Contoso may have defined these levels as follows:

    • Sev 0 = Critical
    • Sev 1 = Error
    • Sev 2 = Warning
    • Sev 3 = Informational
    • Sev 4 = Verbose
  37. Under Enable rule upon creation, ensure that Yes is selected.

    Note: It can take up to 10 minutes for a metric alert rule to become active.

  38. At the bottom of the blade, click Create alert rule.

    The Alerts pane of your IoT Hub should now be displayed. A message in the middle of the should be telling you that you have no alerts, and you should see that a Manage alert rules(1) button has been added below that status message.

It is now time to configure the environment needed to trigger the alert.

Exercise 4: Simulating the Sensors

To simulate Contoso’s asset tracking system, you need to simulate the IoT devices that are placed inside in shipping containers. As each device is activated, it should use automatic device provisioning to connect to the Iot solution and start sending telemetry. In order to automatically connect, each device will need its own X.509 certificate that is part of a chain to the root certificate used to create a group enrollment.

In this exercise, you will verify the existing environment, perform any necessary setup, generate 10 device certificates, and configure a console application that will simulate the 10 devices.

Note: In lab 6 of this course (Lab 6-Automatic Enrollment of Devices in DPS) you configured DPS to use X.509 resources. If you still have that configuration available, you may be able to skip over one or more of the tasks below.

Task 1: Verify DPS Configuration

  1. In your browser, navigate to the Azure Portal and login to your subscription.

  2. On your Dashboard, check the “AZ-220-RG” resource group tile for the AZ-220-DPS-{YOUR-ID} Device Provisioning Service.

    Note: If AZ-220-DPS-{YOUR-ID} does not exist, return to Exercise 1 in this lab and run the setup script.

  3. On your resource group tile, click AZ-220-DPS-{YOUR-ID}.

  4. On the left side navigation menu, under Settings, click Certificates.

  5. With the Certificates pane open, follow these instructions:

    • If the certificates list is empty, move directly to Task 2 of this exercise - Task 2: Verify OpenSSL.
    • If a certificate named root-ca-cert is listed, continue to the next step.
  6. For the certificate listed, check the value under Status, and follow these instructions:

    • If the certificate status is Unverified:
      • click the certificate to view the details, then click Delete.
      • Enter the Certificate Name to confirm the deletion and click OK.
      • Move directly to Task 2 of this exercise - Task 2: Verify OpenSSL.
    • If the certificate status is Verified, continue to the next step.
  7. On the left side navigation menu, under Settings, click Manage enrollments

  8. On the Manage enrollments pane, to view the list of enrollment groups in DPS, click Enrollment Groups.

  9. If the simulated-devices enrollment group is listed, move directly to the next Exercise - Exercise 5: Simulate Devices

  10. If the simulated-devices enrollment group does not exist, follow these instructions:

    • If you have a verified certificate named root-ca-cert, move directly to Task 5 of this exercise - Task 5: Create an Enrollment Group.
    • If you Did Not find a verified certificate above, continue with Task 2 - Task 2: Verify OpenSSL.

Task 2: Verify OpenSSL

In the following steps you will verify that OpenSSL tools installed in an earlier lab are still available.

  1. In your browser, navigate to the Azure Shell and login to your subscription.

  2. At the shell prompt, enter the following command:

     cd ~/certificates
    

    If you see an error that states No such file or directory, move directly to Task 3 of this exercise - Task 3: Install OpenSSL Tools.

  3. At the Cloud Shell command prompt, enter the following command:

     cd certs
    

    If you see an error that states No such file or directory, move directly to Task 4 of this exercise - Task 4: Generate and Configure x.509 CA Certificates using OpenSSL.

  4. If the certs folder is available, move directly to Task 5 of this exercise - Task 5: Create an Enrollment Group.

Task 3: Install OpenSSL Tools

  1. In the cloud shell, enter the following commands:

     mkdir ~/certificates
    
     # navigate to certificates directory
     cd ~/certificates
    
     # download helper script files
     curl https://raw.githubusercontent.com/Azure/azure-iot-sdk-c/master/tools/CACertificates/certGen.sh --output certGen.sh
     curl https://raw.githubusercontent.com/Azure/azure-iot-sdk-c/master/tools/CACertificates/openssl_device_intermediate_ca.cnf --output openssl_device_intermediate_ca.cnf
     curl https://raw.githubusercontent.com/Azure/azure-iot-sdk-c/master/tools/CACertificates/openssl_root_ca.cnf --output openssl_root_ca.cnf
    
     # update script permissions so user can read, write, and execute it
     chmod 700 certGen.sh
    

Task 4: Generate and Configure x.509 CA Certificates using OpenSSL

The first X.509 certificates needed are CA and intermediate certificates. These can be generated using the certGen.sh helper script by passing the create_root_and_intermediate option.

  1. In the cloud shell, ensure that you are in the ~/certificates directory.

  2. At the Cloud Shell command prompt, to generate the CA and intermediate certificates, enter the following command:

     ./certGen.sh create_root_and_intermediate
    

    This command command generates a CA Root Certificate named azure-iot-test-only.root.ca.cert.pem and places it in ./certs directory.

  3. At the Cloud Shell command prompt, to download the azure-iot-test-only.root.ca.cert.pem certificate to your local machine (so it can be uploaded to DPS), enter the following command:

     download ~/certificates/certs/azure-iot-test-only.root.ca.cert.pem
    
  4. In the Azure portal, open the AZ-220-DPS-{YOUR-ID} Device Provisioning Service.

  5. On the Device Provisioning Service blade, in the left side navigation menu under Settings, click Certificates.

  6. On the Certificates pane, at the top of the blade, click Add.

    Note: If you see an existing certificate, select and delete it.

  7. On the Add Certificate pane, select the x.509 CA Certificate file in the Certificate .pem or .cer file upload field.

    This is the azure-iot-test-only.root.ca.cert.pem CA Certificate that was just downloaded.

  8. In the Certificate Name field, enter root-ca-cert

    This name could be the same as the name of the certificate file, or something different. This is a logical name that has no correlation to the Common Name within the x.509 CA Certificate.

  9. Click Save.

    Once the x.509 CA Certificate has been uploaded, the Certificates pane will display the certificate with the Status of Unverified. Before this CA Certificate can be used to authenticate devices to DPS, you will need to verify Proof of Possession of the certificate.

  10. To start the process of verifying Proof of Possession of the certificate, click root-ca-cert.

  11. On the Certificate Details pane, click Generate Verification Code.

  12. Copy the newly generated Verification Code value.

    Note: You will need to leave the Certificate Details pane Open while you generate the Verification Certificate. If you close the pane, you will invalidate the Verification Code and will need to generate a new one.

  13. Open the Azure Cloud Shell, if it’s not still open from earlier, and navigate to the ~/certificates directory.

    Proof of Possession of the CA Certificate is provided to DPS by uploading a certificate generated from the CA Certificate with the Validate Code that was just generated within DPS. This is how you provide proof that you actually own the CA Certificate.

  14. At the Cloud Shell command prompt, to create the Verification Certificate (passing in the Verification Code), enter the following command:

     ./certGen.sh create_verification_certificate <verification-code>
    

    Be sure to replace the <verification-code> placeholder with the Verification Code generated by the Azure portal.

    For example, the command run will look similar to the following:

     ./certGen.sh create_verification_certificate 49C900C30C78D916C46AE9D9C124E9CFFD5FCE124696FAEA
    

    This command generates a Verification Certificate that is chained to the CA Certificate with the Verification Code. The generated Verification Certificate named verification-code.cert.pem is located within the ./certs directory of the Azure Cloud Shell.

  15. At the Cloud Shell command prompt, to download this Verification Certificate to your local machine (so it can be uploaded to DPS), enter the following command:

     download ~/certificates/certs/verification-code.cert.pem
    
  16. In the Azure portal, navigate back to the Certificate Details pane for the CA Certificate.

  17. In the Verification Certificate .pem or .cer file field, click verification-code.cert.pem.

    This is your newly created and downloaded Verification Certificate file.

  18. On the Certificate Details pane, click Verify.

    With the Proof of Possession completed for the CA Certificate, notice the Status for the certificate in the Certificates pane is now displayed as Verified.

Task 5: Create an Enrollment Group

  1. In the Azure portal, ensure that you have your AZ-220-DPS-{YOUR-ID} Device Provisioning Service blade open.

  2. On the left side navigation menu, under Settings, click Manage enrollments.

    There should not be any enrollment groups listed.

  3. At the top of the blade, click Add enrollment group.

  4. On the Add Enrollment Group blade, in the Group name field, enter simulated-devices

  5. Ensure that the Attestation Type is set to Certificate.

  6. Ensure that the Certificate Type field is set to CA Certificate.

  7. In the Primary Certificate dropdown, click root-ca-cert.

    Verify that the Select the IoT hubs this group can be assigned to dropdown includes your AZ-220-HUB-_{YOUR-ID}_ IoT Hub. This will ensure when the device is provisioned, it gets added to this IoT Hub.

  8. In the Initial Device Twin State field, modify the properties.desired JSON object to include a property named telemetryDelay with the value of "1". This will be used by the Device to set the time delay for reading sensor telemetry and sending events to IoT Hub.

    The final JSON will be like the following:

     {
         "tags": {},
         "properties": {
             "desired": {
                 "telemetryDelay": "1"
             }
         }
     }
    
  9. At the top of the blade, click Save.

Now that the environment is setup, it’s time to generate our device certificates.

Exercise 5: Simulate Devices

In this exercise, you will be generating X.509 certificates from the root certificate. You will then use these certificates in a console application that will simulate 10 devices connecting to DPS and sending telemetry to an IoT Hub.

Task 1: Generate Device Certificates

You will now generate and download 10 device certificates.

  1. Open the Azure Cloud Shell and login with the Azure subscription you are using for this course.

  2. At the Cloud Shell command prompt, to create and then move into a directory name monitoring, enter the following commands:

     mkdir monitoring
     cd monitoring
    
  3. To create an empty file in which you will copy the device generation script, enter the following commands:

     touch gen-dev-certs.sh
     chmod +x gen-dev-certs.sh
    
  4. On the Cloud Shell toolbar, click Open editor.

    The button to open the Cloud Shell editor is the { } icon, second from the right.

  5. Under FILES, to edit the contents of the gen-dev-certs.sh file, click monitoring, and then click gen-dev-certs.sh

    The gen-dev-certs.sh file is currently empty.

  6. Paste the following code into the cloud editor:

     #!/bin/bash
    
     # Generate 10 device certificates
     # Rename for each device
     # download from the Cloud CLI
     pushd ~/certificates
     for i in {1..10}
     do
         chmod +w ./certs/new-device.cert.pem
         ./certGen.sh create_device_certificate asset-track$i
         sleep 5
         cp ./certs/new-device.cert.pfx ./certs/new-asset-track$i.cert.pfx
         download ./certs/new-asset-track$i.cert.pfx
     done
     popd
    

    This script will create and download 10 device certificates.

  7. To save the edited gen-dev-certs.sh file, press CTRL-Q.

    If prompted to save you changes before closing the editor, click Save.

  8. At the Cloud Shell command prompt, to run the gen-dev-certs.sh script, enter the following command:

     ./gen-dev-certs.sh
    

    While the script runs, you will see the output from the certificate generator and then the browser should automatically download each certificate in turn.

    Note: If your browsers asks what you want to do with the files, click Save for each file.

    Once it completes, you will have 10 certificates available in your browser download location:

    • new-asset-track1.cert.pfx
    • new-asset-track2.cert.pfx
    • new-asset-track3.cert.pfx
    • new-asset-track4.cert.pfx
    • new-asset-track5.cert.pfx
    • new-asset-track6.cert.pfx
    • new-asset-track7.cert.pfx
    • new-asset-track8.cert.pfx
    • new-asset-track9.cert.pfx
    • new-asset-track10.cert.pfx

With these certificates available, you are ready to configure the device simulator.

Task 2: Add Certificates to Simulator

  1. Copy the downloaded X.509 Device Certificate files to the lab 17 Starter folder.

    In Lab 3: Setup the Development Environment, you cloned the GitHub repository containing lab resources by downloading a ZIP file and extracting the contents locally. The extracted folder structure includes the following folder path:

    • Allfiles
      • Labs
        • 17-How to manage your Azure IoT Hub
          • Starter

    The Starter folder for lab 17 includes the SimulatedDevice.csproj and Program.cs files. The project will need to access this certificate file when authenticating to the Device Provisioning Service. The files need to pe placed at the root of the project folder:

    After copied, the certificate files will be located in the following locations:

     /Starter/new-asset-track1.cert.pfx
     /Starter/new-asset-track2.cert.pfx
     /Starter/new-asset-track3.cert.pfx
     /Starter/new-asset-track4.cert.pfx
     /Starter/new-asset-track5.cert.pfx
     /Starter/new-asset-track6.cert.pfx
     /Starter/new-asset-track7.cert.pfx
     /Starter/new-asset-track8.cert.pfx
     /Starter/new-asset-track9.cert.pfx
     /Starter/new-asset-track10.cert.pfx
    
  2. Open Visual Studio Code.

  3. On the File menu, click Open Folder.

  4. In the Open Folder dialog, navigate to the lab 17 Starter folder, click Starter, and then click Select Folder.

    Note: If Visual Studio Code suggested loading assets or performing a Restore, follow the suggestions.

  5. In the EXPLORER pane, to open the Program.cs file, click Program.cs.

    You should also see the certificate files listed.

  6. In the code editor, locate the GlobalDeviceEndpoint variable.

    Notice that it’s value is set to global.azure-devices-provisioning.net. This is the Global Device Endpoint for the Azure Device Provisioning Service (DPS) within the Public Azure Cloud. All devices connecting to Azure DPS will be configured with this Global Device Endpoint DNS name.

     private const string GlobalDeviceEndpoint = "global.azure-devices-provisioning.net";
    
  7. Locate the dpsIdScope variable.

     private static string dpsIdScope = "<DPS-ID-Scope>";
    

    You will need to replace the <DPS-ID-Scope> placeholder value with the actual value.

  8. Return to your browser window containing the Azure Cloud Shell.

  9. At the Cloud Shell command prompt, to display the ID Scope of your DPS service, enter the following command:

     az iot dps show --name AZ-220-DPS-{YOUR-ID} --query properties.idScope
    

    Note: Be sure to replace {YOUR-ID} with the ID you created at the start of this class

  10. Copy the output of the command

  11. Return to Visual Studio Code.

  12. Replace the <DPS-ID-Scope> value with the value that you copied from Azure Cloud Shell.

    private static string dpsIdScope = "0ne000A6D9B";
    

This app is very similar to the app used in the earlier lab L06-Automatic Enrollment of Devices in DPS. The primary difference is that instead of just enrolling a single device simulator and then sending telemetry, it instead enrolls 10 devices, one every 30 seconds. Each simulated device will then send telemetry. This should then cause our alert to be raised and log monitoring data to storage.

Task 3: Run the Simulator

  1. In Visual Studio Code, on the Terminal menu, click New Terminal.

  2. At the Terminal command prompt, to run the app, enter the following command:

     dotnet run
    

    You should see output that shows the first device being connected via DPS and then telemetry being sent. Every 30 seconds thereafter, and additional device will be connected and commence sending telemetry until all 10 devices are connected and sending telemetry.

  3. Return to the DPS group enrollment in the Azure Portal.

  4. In the simulated-devices enrollment group, to view the connected devices, click Registration Records.

    You should see a list of the devices that have connected. You can hit Refresh to update the list.

    Now that you have the devices connected and sending telemetry, you can await the triggering of the alert once you have 5 or more devices connected for 5 mins. You should receive an SMS message that looks similar to:

     AZ220EmailAG:Fired:Sev3 Azure Monitor Alert Connected Devices Greater or Equal to 5 on <your IoT Hub>
    
  5. Once the alerts have arrived, you can exit the application.

    Either press CTRL+C in the Visual Studio Code terminal, or close Visual Studio Code.

    Note: When the devices are disconnected, you will receive messages informing you the alert has been resolved.

Now, it’s time to check the storage account and see if anything has been logged by Azure Monitor.

Exercise 6: Review Metrics, Alerts and Archive

In this exercise, you will examine some of the reporting and logging resources that you configured earlier in this lab, and review the event data has been recorded in the short time that has elapsed.

Task 1: See the Metrics in the Portal

  1. In the Azure portal, open the Metrics chart that you pinned to the dashboard by clicking on the chart title.

    The chart will open and fill the page.

  2. Change the time values to the Last 30 minutes.

    Notice that you can see Telemetry messages sent and *Connected devices (preview)** values, with the most recent numbers at the bottom of the chart - move you mouse over the chart to see values a specific points in time.

Task 2: See the Alerts

To use the Azure Portal to review alerts, complete the following steps.

  1. In the Azure Portal, navigate back to you Dashboard.

  2. On the Azure portal toolbar, in the search box, type monitor

  3. In the search result pane, under Services, click Monitor.

    The Monitor - Overview page is displayed. This is the overview for all of the monitoring activities for the current subscription.

  4. On the left side navigation menu, near the top of the list, click Alerts.

    This alerts view shows all alerts for all subscriptions. Let’s filter this to the IoT Hub.

  5. Near the top of the blade, under Subscription, select the subscription you are using for this class.

  6. In the Resource group dropdown, click AZ-220-RG.

  7. In the Resource dropdown, click AZ-220-HUB-{YOUR-ID}.

  8. In the Time range dropdown, click Past hour.

    You should now see a summary of alerts for the last hour. Under Total alert rules you should see 1, the alert you created earlier. Below this, you will see a list of the severity categories as well as the count of alerts per category. The alerts we are interested in are Sev 3. You should see at least one (if you have stopped and restarted the device simulator, you may have generated more that one alert).

  9. In the list of severities, click Sev 3.

    The All Alerts page will open. At the top of the page you will see a number of filter fields - these have been populated with the values from the preceding screen so that only the Sev 3 alerts for the selected IoT hub are shown. It will show you the alerts that are active, and if there are any warnings.

  10. Under Name, to select your Sev 3 alert, click Connected Devices Greater or Equal To 5.

    A pane will open showing a Summary of the details for the alert. This includes a chart illustrating why the alert fired - a dash line shows the threshold value as well as the current values for the monitored metric. Below this are details of the Criterion and other details.

  11. At the top of the pane, below the title, click History.

    In this view you can see when the alert fired, the action group that was invoked, and any other changes such as when the alert is resolved and so on.

  12. At the top of the pane, below the title, click Diagnostics.

    If there were any issues related to the alert, addition details would be shown here.

Task 3: See the Diagnostic Logs

Earlier in this lab, you set up your diagnostic logs to be exported to blob storage. It is a good time to check and see what was written.

  1. Navigate to your Dashboard, and then locate your “AZ-220-RG” resource group tile.

  2. In the list of resources, select the Storage Account that was created earlier - az220storage{your-id}.

    The Overview for the storage account will be displayed.

  3. Scroll down until you can see the metrics charts for the Storage Account: Total egress, Total ingress, Average latency and Request breakdown.

    You should see that there is activity displayed.

  4. On the left hand navigation menu, to view the data that has been logged, click Storage explorer (preview).

  5. In the Storage Explorer pane, expand the BLOB CONTAINERS node.

    When Azure Monitor first sends data to a storage account, it creates a container called insights-logs-connection.

  6. Under BLOB CONTAINERS, click insights-logs-connection.

    The contents of the container will be listed to the right.

    Logs are written to the container in a very nested fashion. You will need to open each subfolder in turn to navigate to the actual log data. The structure is similar to that show below:

    • resourceId=
      • SUBSCRIPTIONS
        • **** - this is the ID for the subscription that generated the log
          • RESOURCEGROUPS - contains a folder for each resource group that generated a log
            • “AZ-220-RG” - the resource group that contains the IoT Hub
              • PROVIDERS
                • MICROSOFT.DEVICES
                  • IOTHUBS
                    • AZ-220-HUB-{YOUR-INITIALS-AND-CURRENT-DATE} - contains a folder for each year where a log was generated
                      • Y=2019 - contains a folder for each month where a log was generated
                        • m=12 - contains a folder for each day where a log was generated
                          • d=15 - contains a folder for each hour where a log was generated
                            • h=15 - contains a folder for each minute where a log was generated
                              • m=00 - contains the log file for that minute

    Drill down until you get to the current date and select the most recent file.

  7. With the file selected, on the toolbar at the top of the pane, click Download.

  8. Open the downloaded file in Visual Studio Code.

    You should see a number of lines of JSON.

  9. To make the JSON easier to read, press F1, enter Format document and select Format document from the list of options.

    The JSON will show a list of connection and disconnection events similar to:

     {
         "time": "2019-12-26T14:32:45Z",
         "resourceId": "/SUBSCRIPTIONS/AE82FF3B-4BD0-462B-8449-D713DD18E11E/RESOURCEGROUPS/AZ-220/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/AZ-220-HUB-DM121619",
         "operationName": "deviceConnect",
         "category": "Connections",
         "level": "Information",
         "properties": "{\"deviceId\":\"asset-track9\",\"protocol\":\"Amqp\",\"authType\":\"{\\\"scope\\\":\\\"device\\\",\\\"type\\\":\\\"x509Certificate\\\",\\\"issuer\\\":\\\"external\\\",\\\"acceptingIpFilterRule\\\":null}\",\"maskedIpAddress\":\"67.176.115.XXX\",\"statusCode\":null}",
         "location": "westus"
     }
     {
         "time": "2019-12-26T14:33:12Z",
         "resourceId": "/SUBSCRIPTIONS/AE82FF3B-4BD0-462B-8449-D713DD18E11E/RESOURCEGROUPS/AZ-220/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/AZ-220-HUB-DM121619",
         "operationName": "deviceConnect",
         "category": "Connections",
         "level": "Information",
         "properties": "{\"deviceId\":\"asset-track10\",\"protocol\":\"Amqp\",\"authType\":\"{\\\"scope\\\":\\\"device\\\",\\\"type\\\":\\\"x509Certificate\\\",\\\"issuer\\\":\\\"external\\\",\\\"acceptingIpFilterRule\\\":null}\",\"maskedIpAddress\":\"67.176.115.XXX\",\"statusCode\":null}",
         "location": "westus"
     }
     {
         "time": "2019-12-26T14:37:29Z",
         "resourceId": "/SUBSCRIPTIONS/AE82FF3B-4BD0-462B-8449-D713DD18E11E/RESOURCEGROUPS/AZ-220/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/AZ-220-HUB-DM121619",
         "operationName": "deviceDisconnect",
         "category": "Connections",
         "level": "Information",
         "properties": "{\"deviceId\":\"asset-track8\",\"protocol\":\"Amqp\",\"authType\":null,\"maskedIpAddress\":\"67.176.115.XXX\",\"statusCode\":null}",
         "location": "westus"
     }
     {
         "time": "2019-12-26T14:37:29Z",
         "resourceId": "/SUBSCRIPTIONS/AE82FF3B-4BD0-462B-8449-D713DD18E11E/RESOURCEGROUPS/AZ-220/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/AZ-220-HUB-DM121619",
         "operationName": "deviceDisconnect",
         "category": "Connections",
         "level": "Information",
         "properties": "{\"deviceId\":\"asset-track4\",\"protocol\":\"Amqp\",\"authType\":null,\"maskedIpAddress\":\"67.176.115.XXX\",\"statusCode\":null}",
         "location": "westus"
     }
    

    Notice that each individual entry is a single JSON record - although the overall document is not a valid JSON document. Within each record you can see details relating to the originating IoT Hub and properties for each event. Within the properties object, you can see the connecting (or disconnecting) deviceId.