Monitor Application Performance with Azure Load Testing
Student lab manual
Lab requirements
-
This lab requires Microsoft Edge or an Azure DevOps supported browser.
-
Set up an Azure DevOps organization: If you don’t already have an Azure DevOps organization that you can use for this lab, create one by following the instructions available at Create an organization or project collection.
-
Identify an existing Azure subscription or create a new one.
-
Verify that you have a Microsoft account or a Microsoft Entra account with the Owner role in the Azure subscription and the Global Administrator role in the Microsoft Entra tenant associated with the Azure subscription. For details, refer to List Azure role assignments using the Azure portal and View and assign administrator roles in Azure Active Directory.
Lab overview
Azure Load Testing is a fully managed load-testing service that enables you to generate high-scale load. The service simulates traffic for your applications, regardless of where they’re hosted. Developers, testers, and quality assurance (QA) engineers can use it to optimize application performance, scalability, or capacity. Quickly create a load test for your web application by using a URL, and without prior knowledge of testing tools. Azure Load Testing abstracts the complexity and infrastructure to run your load test at scale. For more advanced load testing scenarios, you can create a load test by reusing an existing Apache JMeter test script, a popular open-source load and performance tool. For example, your test plan might consist of multiple application requests, you want to call non-HTTP endpoints, or you’re using input data and parameters to make the test more dynamic.
In this lab, you’ll learn about how you can use Azure Load Testing to simulate performance testing against a live-running web application with different load scenarios. Lastly, you’ll learn how to integrate Azure Load Testing into your CI/CD pipelines.
Objectives
After you complete this lab, you will be able to:
- Deploy Azure App Service web apps.
- Compose and Run a YAML-based CI/CD pipeline.
- Deploy Azure Load Testing.
- Investigate Azure web app performance by using Azure Load Testing.
- Integrate Azure Load Testing into your CI/CD pipelines.
Estimated timing: 60 minutes
Instructions
Exercise 0: Configure the lab prerequisites
In this exercise, you will set up the prerequisites for the lab, which consist of a new Azure DevOps project with a repository based on the eShopOnWeb.
Task 1: (skip if done) Create and configure the team project
In this task, you will create an eShopOnWeb Azure DevOps project to be used by several labs.
-
On your lab computer, in a browser window open your Azure DevOps organization. Click on New Project. Give your project the name eShopOnWeb and choose Scrum on the Work Item process dropdown. Click on Create.
Task 2: (skip if done) Import eShopOnWeb Git Repository
In this task you will import the eShopOnWeb Git repository that will be used by several labs.
-
On your lab computer, in a browser window open your Azure DevOps organization and the previously created eShopOnWeb project. Click on Repos>Files , Import. On the Import a Git Repository window, paste the following URL https://github.com/MicrosoftLearning/eShopOnWeb.git and click on Import:
- The repository is organized the following way:
- .ado folder contains Azure DevOps YAML pipelines
- .devcontainer folder container setup to develop using containers (either locally in VS Code or GitHub Codespaces)
- infra folder contains Bicep&ARM infrastructure as code templates used in some lab scenarios.
- .github folder container YAML GitHub workflow definitions.
- src folder contains the .NET 8 website used on the lab scenarios.
- Go to Repos>Branches.
- Hover on the main branch then click the ellipsis on the right of the column.
- Click on Set as default branch.
Task 3: Create Azure resources
In this task, you will create an Azure web app by using the cloud shell in Azure portal.
- From the lab computer, start a web browser, navigate to the Azure Portal, and sign in with the user account that has the Owner role in the Azure subscription you will be using in this lab and has the role of the Global Administrator in the Microsoft Entra tenant associated with this subscription.
- In the Azure portal, in the toolbar, click the Cloud Shell icon located directly to the right of the search text box.
- If prompted to select either Bash or PowerShell, select Bash.
Note: If this is the first time you are starting Cloud Shell and you are presented with the You have no storage mounted message, select the subscription you are using in this lab, and select Create storage.
-
From the Bash prompt, in the Cloud Shell pane, run the following command to create a resource group (replace the
<region>
placeholder with the name of the Azure region closest to you such as ‘eastus’).RESOURCEGROUPNAME='az400m09l16-RG' LOCATION='<region>' az group create --name $RESOURCEGROUPNAME --location $LOCATION
-
To create a Windows App service plan by running the following command:
SERVICEPLANNAME='az400l16-sp' az appservice plan create --resource-group $RESOURCEGROUPNAME \ --name $SERVICEPLANNAME --sku B3
-
Create a web app with a unique name.
WEBAPPNAME=az400eshoponweb$RANDOM$RANDOM az webapp create --resource-group $RESOURCEGROUPNAME --plan $SERVICEPLANNAME --name $WEBAPPNAME
Note: Record the name of the web app. You will need it later in this lab.
Exercise 1: Configure CI/CD Pipelines as Code with YAML in Azure DevOps
In this exercise, you will configure CI/CD Pipelines as code with YAML in Azure DevOps.
Task 1: (skip if done) Create a Service Connection for deployment
In this task, you will create a Service Principal by using the Azure CLI, which will allow Azure DevOps to:
- Deploy resources on your Azure subscription.
- Have read access on the later created Key Vault secrets.
Note: If you do already have a Service Principal, you can proceed directly to the next task.
You will need a Service Principal to deploy Azure resources from Azure Pipelines. Since we are going to retrieve secrets in a pipeline, we will need to grant permission to the service when we create the Azure Key Vault.
A Service Principal is automatically created by Azure Pipelines, when you connect to an Azure subscription from inside a pipeline definition or when you create a new Service Connection from the project settings page (automatic option). You can also manually create the Service Principal from the portal or using Azure CLI and re-use it across projects.
- From the lab computer, start a web browser, navigate to the Azure Portal, and sign in with the user account that has the Owner role in the Azure subscription you will be using in this lab and has the role of the Global Administrator in the Microsoft Entra tenant associated with this subscription.
- In the Azure portal, click on the Cloud Shell icon, located directly to the right of the search textbox at the top of the page.
-
If prompted to select either Bash or PowerShell, select Bash.
Note: If this is the first time you are starting Cloud Shell and you are presented with the You have no storage mounted message, select the subscription you are using in this lab, and select Create storage.
-
From the Bash prompt, in the Cloud Shell pane, run the following commands to retrieve the values of the Azure subscription ID and subscription name attributes:
az account show --query id --output tsv az account show --query name --output tsv
Note: Copy both values to a text file. You will need them later in this lab.
-
From the Bash prompt, in the Cloud Shell pane, run the following command to create a Service Principal (replace the myServicePrincipalName with any unique string of characters consisting of letters and digits) and mySubscriptionID with your Azure subscriptionId :
az ad sp create-for-rbac --name myServicePrincipalName \ --role contributor \ --scopes /subscriptions/mySubscriptionID
Note: The command will generate a JSON output. Copy the output to text file. You will need it later in this lab.
-
Next, from the lab computer, start a web browser, navigate to the Azure DevOps eShopOnWeb project. Click on Project Settings>Service Connections (under Pipelines) and New Service Connection.
-
On the New service connection blade, select Azure Resource Manager and Next (may need to scroll down).
-
The choose Service Principal (manual) and click on Next.
- Fill in the empty fields using the information gathered during previous steps:
- Subscription Id and Name.
- Service Principal Id (appId), Service principal key (password) and Tenant ID (tenant).
- In Service connection name type azure subs. This name will be referenced in YAML pipelines when needing an Azure DevOps Service Connection to communicate with your Azure subscription.
- Click on Verify and Save.
Task 2: Add a YAML build and deploy definition
In this task, you will add a YAML build definition to the existing project.
- Navigate back to the Pipelines pane in of the Pipelines hub.
-
Click New pipeline (or Create Pipeline if this is the first one you create).
Note: We will use the wizard to create a new YAML Pipeline definition based on our project.
- On the Where is your code? pane, click Azure Repos Git (YAML) option.
- On the Select a repository pane, click eShopOnWeb.
- On the Configure your pipeline pane, scroll down and select Starter Pipeline.
- Select all lines from the Starter Pipeline, and delete them.
-
Copy the full template pipeline from below, knowing you will need to make parameter modifications before saving the changes:
#Template Pipeline for CI/CD # trigger: # - main resources: repositories: - repository: self trigger: none stages: - stage: Build displayName: Build .Net Core Solution jobs: - job: Build pool: vmImage: ubuntu-latest steps: - task: DotNetCoreCLI@2 displayName: Restore inputs: command: 'restore' projects: '**/*.sln' feedsToUse: 'select' - task: DotNetCoreCLI@2 displayName: Build inputs: command: 'build' projects: '**/*.sln' - task: DotNetCoreCLI@2 displayName: Publish inputs: command: 'publish' publishWebProjects: true arguments: '-o $(Build.ArtifactStagingDirectory)' - task: PublishBuildArtifacts@1 displayName: Publish Artifacts ADO - Website inputs: pathToPublish: '$(Build.ArtifactStagingDirectory)' artifactName: Website - stage: Deploy displayName: Deploy to an Azure Web App jobs: - job: Deploy pool: vmImage: 'windows-2019' steps: - task: DownloadBuildArtifacts@1 inputs: buildType: 'current' downloadType: 'single' artifactName: 'Website' downloadPath: '$(Build.ArtifactStagingDirectory)'
-
Set the cursor on a new line at the end of the YAML definition. Make sure you position the cursor at the indentation of the previous task level.
Note: This will be the location where new tasks are added.
- Click Show Assistant from the right hand side of the portal. In the list of tasks, search for and select the Azure App Service Deploy task.
-
In the Azure App Service deploy pane, specify the following settings and click Add:
- in the Azure subscription drop-down list, select the service connection you just created.
- Validate App Service Type points to Web App on Windows.
- in the App Service name dropdown list, select the name of the web app you deployed earlier in the lab (**az400eshoponweb…).
- in the Package or folder text box, update the Default Value to
$(Build.ArtifactStagingDirectory)/**/Web.zip
. - Expand Application and Configuration Settings and add the value
-UseOnlyInMemoryDatabase true -ASPNETCORE_ENVIRONMENT Development
-
Confirm the settings from the Assistant pane by clicking the Add button.
Note: This will automatically add the deployment task to the YAML pipeline definition.
-
The snippet of code added to the editor should look similar to below, reflecting your name for the azureSubscription and WebappName parameters:
- task: AzureRmWebAppDeployment@4 inputs: ConnectionType: 'AzureRM' azureSubscription: 'SERVICE CONNECTION NAME' appType: 'webApp' WebAppName: 'az400eshoponWeb369825031' packageForLinux: '$(Build.ArtifactStagingDirectory)/**/Web.zip' AppSettings: '-UseOnlyInMemoryDatabase true -ASPNETCORE_ENVIRONMENT Development'
Note: The packageForLinux parameter is misleading in the context of this lab, but it is valid for Windows or Linux.
-
Before saving the updates to the yml-file, give it a more clear name. On top of the yaml-editor window, it shows EShopOnweb/azure-pipelines-#.yml. (where # is a number, typically 1 but could be different in your setup.) Select that filename, and rename it to m09l16-pipeline.yml
-
Click Save, on the Save pane, click Save again to commit the change directly into the master branch.
Note: Since our original CI-YAML was not configured to automatically trigger a new build, we have to initiate this one manually.
-
From the Azure DevOps left menu, navigate to Pipelines and select Pipelines again. Next, select All to open all pipeline definitions, not just the Recent ones.
Note: if you kept all previous pipelines from previous lab exercises, this new pipeline might have reused a default eShopOnWeb (#) sequence name for the pipeline as shown in below screenshot. Select a pipeline (most probably the one with the highest sequence number, select Edit and validate it points to the m09l16-pipeline.yml code file).
- Confirm to run this pipeline by clicking Run from the appearing pane and confirm by clicking Run once more.
- Notice the 2 different Stages, Build .Net Core Solution and Deploy to Azure Web App appearing.
-
Wait for the pipeline to kick off.
-
Ignore any Warnings showing up during the Build Stage. Wait until it completes the Build Stage successfully. (You can select the actual Build stage to see more details from the logs.)
-
Once the Deploy Stage wants to start, you are prompted with Permissions Needed, as well as an orange bar saying:
This pipeline needs permission to access a resource before this run can continue to Deploy to an Azure Web App
- Click on View
- From the Waiting for Review pane, click Permit.
- Validate the message in the Permit popup window, and confirm by clicking Permit.
- This sets off the Deploy Stage. Wait for this to complete successfully.
Task 2: Review the deployed site
- Switch back to web browser window displaying the Azure portal and navigate to the blade displaying the properties of the Azure web app.
- On the Azure web app blade, click Overview and, on the overview blade, click Browse to open your site in a new web browser tab.
- Verify that the deployed site loads as expected in the new browser tab, showing the eShopOnWeb E-commerce website.
Exercise 2: Deploy and Setup Azure Load Testing
In this exercise, you will deploy an Azure Load Testing Resource in Azure, and configure different Load Testing scenarios for your live-running Azure App Service.
Task 1: Deploy Azure Load Testing
In this task, you will deploy an Azure Load Testing Resource into your Azure subscription.
- From the Azure Portal (https://portal.azure.com), navigate to Create Azure Resource.
- In the ‘Search Services and marketplace’ search field, enter Azure Load Testing.
- Select Azure Load Testing (published by Microsoft), from the search results.
- From the Azure Load Testing Page, click Create to start the deployment process.
- From the ‘Create a Load Testing Resource’ page, provide the necessary details for the resource deployment:
- Subscription: select your Azure Subscription
- Resource Group: select the Resource Group you used for deploying the Web App Service in the earlier exercise
- Name: eShopOnWebLoadTesting
- Region: Select a region that is close to your region
Note: Azure Load Testing service is not available in all Azure Regions.
- Click Review and Create, to have your settings validated.
- Click Create to confirm, and get the Azure Load Testing resource deployed.
- You are switched to the ‘Deployment is in progress’ page. Wait for a few minutes, until the deployment completes successfully.
-
Click Go to Resource from the deployment progress page, to navigate to the eShopOnWebLoadTesting Azure Load Testing resource.
Note: If you closed the blade or closed the Azure Portal during the deployment of the Azure Load Testing Resource, you can find it again from the Azure Portal Search field, or from the Resources / Recent list of resources.
Task 2: Create Azure Load Testing tests
In this task, you will create different Azure Load Testing tests, using different load configuration settings.
- From within the eShopOnWebLoadTesting Azure Load Testing Resource blade, navigate to Tests. Click the +Create menu option, and select Create a URL-based test.
- Complete the following parameters and settings to create a load test:
- Test URL: Enter the URL from the Azure App Service you deployed in the previous exercise (az400eshoponweb…azurewebsites.net), including https://
- Specify Load: Virtual Users
- Number of Virtual Users: 50
- Test Duration (minutes): 5
- Ramp-up time (minutes): 1
- Confirm the configuration of the test, by clicking Review and Create, (Don’t make any changes in the other tabs). Click Create once more.
- This kicks off the Load Testing tests, which will run The test will run for 5 minutes.
- With the test running, navigate back to the eShopOnWebLoadTesting Azure Load Testing Resource page, and navigate to Tests, select Tests and see a test Get_eshoponweb…
- From the top menu, click Create, Create a URL-based test, to create a 2nd Load test.
- Complete the following parameters and settings to create another load test:
- Test URL: Enter the URL from the Azure App Service you deployed in the previous exercise (eShopOnWeb…azurewebsites.net), including https://
- Specify Load: Requests per Second (RPS)
- Requests per second (RPS): 100
- Response time (milliseconds): 500
- Test Duration (minutes): 5
- Ramp-up time (minutes): 1
- Confirm the configuration of the test, by clicking Review + create, and Create once more.
- The test will run for about 5 minutes.
Task 3: Validate Azure Load Testing results
In this task, you will validate the outcome of an Azure Load Testing TestRun.
With both quick tests complete, let’s make a few changes to them, and validate the results.
- From Azure Load Testing, navigate to Tests. Select either of the test definitions, to open a more detailed view, by clicking on one of the tests. This redirects you to the more detailed test page. From here, you can validate the details of the actual runs, by selecting the TestRun_mm/dd/yy-hh:hh from the resulting list.
- From the detailed TestRun page, identify the actual outcome of the Azure Load Testing simulation. Some of the values are:
- Load / Total Requests
- Duration
- Response Time (shows the outcome in seconds, reflecting the 90th percentile response time - this means that, for 90% of the requests, the response time will be within the given results)
- Throughput in requests per second
- More below, several of these values are represented using dashboard graph line and chart views.
- Take a few minutes to compare the results of both simulated tests with each other, and identify the impact of more users on the App Service performance.
Exercise 2: Automate a Load Test with CI/CD in Azure Pipelines
Get started with automating load tests in Azure Load Testing by adding it to a CI/CD pipeline. After running a load test in the Azure portal, you export the configuration files, and configure a CI/CD pipeline in Azure Pipelines (similar capability exists for GitHub Actions).
After you complete this exercise, you have a CI/CD workflow that is configured to run a load test with Azure Load Testing.
Task 1: Identify ADO Service Connection Details
In this task, you will grant the required permissions to the Azure DevOps Service Connection’s Service Principal.
- From the Azure DevOps Portal(https://dev.azure.com), navigate to the eShopOnWeb Project.
- From the down left corner, select Project Settings.
- Under the Pipelines section, select Service Connections.
- Notice the Service Connection, having the name of your Azure Subscription you used to deploy Azure Resources at the start of the lab exercise.
- Select the Service Connection. From the Overview tab, navigate to Details and select Manage Service Principal.
- This redirects you to the Azure Portal, from where it opens the Service Principal details for the identity object.
- Copy the Display Name value (formatted like Name_of_ADO_Organization_eShopOnWeb_-b86d9ae1-7552-4b75-a1e0-27fb2ea7f9f4) aside, as you will need this in the next steps.
Task 2: Grant Service Principal Permissions
Azure Load Testing uses Azure RBAC to grant permissions for performing specific activities on your load testing resource. To run a load test from your CI/CD pipeline, you grant the Load Test Contributor role to the service principal.
- In the Azure portal, go to your Azure Load Testing resource.
- Select Access control (IAM) > Add > Add role assignment.
- In the Role tab, select Load Test Contributor in the list of job function roles.
- In the Members tab, select Select members, and then use the display name you copied previously to search the service principal.
- Select the service principal, and then select Select.
- In the Review + assign tab, select Review + assign to add the role assignment.
You can now use the service connection in your Azure Pipelines workflow definition to access your Azure load testing resource.
Task 3: Export load test input files and Import to Azure Repos
To run a load test with Azure Load Testing in a CI/CD workflow, you need to add the load test configuration settings and any input files in your source control repository. If you have an existing load test, you can download the configuration settings and all input files from the Azure portal.
Perform the following steps to download the input files for an existing load testing in the Azure portal:
- In the Azure portal, go to your Azure Load Testing resource.
- On the left pane, select Tests to view the list of load tests, and then select your test.
- Selecting the ellipsis (…) next to the test run you’re working with, and then select Download input file.
- The browser downloads a zipped folder that contains the load test input files.
-
Use any zip tool to extract the input files. The folder contains the following files:
- config.yaml: the load test YAML configuration file. You reference this file in the CI/CD workflow definition.
- quick_test.jmx: the JMeter test script
- Commit all extracted input files to your source control repository. To do this, navigate to the Azure DevOps Portal(https://dev.azure.com), and navigate to the eShopOnWeb DevOps Project.
- Select Repos. In the source code folder structure, notice the tests subfolder. Notice the ellipsis (…), and select New > Folder.
- specify jmeter as folder name, and placeholder.txt for the file name (Note: a Folder cannot be created as empty)
- Click Commit to confirm the creation of the placeholder file and jmeter folder.
- From the Folder structure, navigate to the new created jmeter subfolder. Click the ellipsis(…) and select Upload File(s).
- Using the Browse option, navigate to the location of the extracted zip-file, and select both config.yaml and quick_test.jmx.
- Click Commit to confirm the file upload into source control.
Task 4: Update the CI/CD workflow YAML definition file
In this task, you will import the Azure Load Testing - Azure DevOps Marketplace extension, as well as updating the existing CI/CD pipeline with the AzureLoadTest task.
- To create and run a load test, the Azure Pipelines workflow definition uses the Azure Load Testing task extension from the Azure DevOps Marketplace. Open the Azure Load Testing task extension in the Azure DevOps Marketplace, and select Get it free.
- Select your Azure DevOps organization, and then select Install to install the extension.
- From within the Azure DevOps Portal and Project, navigate to Pipelines and select the pipeline created at the start of this exercise. Click Edit.
- In the YAML script, navigate to line 56 and press ENTER/RETURN, to add a new empty line. (this is right before the Deploy Stage of the YAML file).
- At line 57, select the Tasks Assistant to the right-hand side, and search for Azure Load Testing.
- Complete the graphical pane with the correct settings of your scenario:
- Azure Subscription: Select the subscription which runs your Azure Resources
- Load Test File: ‘$(Build.SourcesDirectory)/tests/jmeter/config.yaml’
- Load Test Resource Group: The Resource Group which holds your Azure Load Testing Resources
- Load Test Resource Name: ESHopOnWebLoadTesting
- Load Test Run Name: ado_run
- Load Test Run Description: load testing from ADO
- Confirm the injection of the parameters as a snippet of YAML by clicking Add
- If the indentation of the YAML snippet is giving errors (red squiggly lines), fix them by adding 2 spaces or tab to position the snippet correctly.
-
The below sample snippet shows what the YAML code should look like
- task: AzureLoadTest@1 inputs: azureSubscription: 'AZURE DEMO SUBSCRIPTION(b86d9ae1-1234-4b75-a8e7-27fb2ea7f9f4)' loadTestConfigFile: '$(Build.SourcesDirectory)/tests/jmeter/config.yaml' resourceGroup: 'az400m05l11-RG' loadTestResource: 'eShopOnWebLoadTesting' loadTestRunName: 'ado_run' loadTestRunDescription: 'load testing from ADO'
- below the inserted YAML snippet, add a new empty line by pressing ENTER/RETURN.
-
below this empty line, add a snippet for the Publish task, showing the results of the Azure Load testing task during the pipeline run:
- publish: $(System.DefaultWorkingDirectory)/loadTest artifact: loadTestResults
- If the indentation of the YAML snippet is giving errors (red squiggly lines), fix them by adding 2 spaces or tab to position the snippet correctly.
- With both snippets added to the CI/CD pipeline, Save the changes.
- Once saved, click Run to trigger the pipeline.
- Confirm the branch (main) and click the Run button to start the pipeline run.
- From the pipeline status page, click the Build stage to open the verbose logging details of the different tasks in the pipeline.
- Wait for the pipeline to kick off the Build Stage, and arrive at the AzureLoadTest task in the flow of the pipeline.
- While the task is running, browse to the Azure Load Testing in the Azure Portal, and see how the pipeline creates a new RunTest, named adoloadtest1. You can select it to show the outcome values of the TestRun job.
-
Navigate back to the Azure DevOps CI/CD Pipeline Run view, where the AzureLoadTest task completed successfully. From the verbose logging output, the resulting values of the load test will be visible as well:
Task : Azure Load Testing Description : Automate performance regression testing with Azure Load Testing Version : 1.2.30 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/load-testing/tutorial-cicd-azure-pipelines#azure-load-testing-task ============================================================================== Test '0d295119-12d0-482d-94be-a7b84787c004' already exists Uploaded test plan for the test Creating and running a testRun for the test View the load test run in progress at: https://portal.azure.com/#blade/Microsoft_Azure_CloudNativeTesting/NewReport//resourceId/%2fsubscriptions%4b75-a1e0-27fb2ea7f9f4%2fresourcegroups%2faz400m05l11-rg%2fproviders%2fmicrosoft.loadtestservice%2floadtests%2feshoponwebloadtesting/testId/0d295119-12d0-787c004/testRunId/161046f1-d2d3-46f7-9d2b-c8a09478ce4c TestRun completed -------------------Summary --------------- TestRun start time: Mon Jul 24 2023 21:46:26 GMT+0000 (Coordinated Universal Time) TestRun end time: Mon Jul 24 2023 21:51:50 GMT+0000 (Coordinated Universal Time) Virtual Users: 50 TestStatus: DONE ------------------Client-side metrics------------ Homepage response time : avg=1359ms min=59ms med=539ms max=16629ms p(90)=3127ms p(95)=5478ms p(99)=13878ms requests per sec : avg=37 total requests : 4500 total errors : 0 total error rate : 0 Finishing: AzureLoadTest
- You have now performed an automated Load Test as part of a pipeline run. In the last task, you will specify conditions for failure, meaning, we will not allow our deploy Stage to start, if the performance of the web app is below a certain threshold.
Task 5 : Add failure/success criteria to Load Testing Pipeline
In this task, You’ll use load test fail criteria to get alerted (have a failed pipeline run as result) when the application doesn’t meet your quality requirements.
- From Azure DevOps, navigate to the eShopOnWeb Project, and open Repos.
- Within Repos, browse to the /tests/jmeter subfolder created and used earlier.
- Open the Load Testing *config.yaml** file. Click Edit to allow editing of the file.
-
Replace
failureCriteria: []
with the following snippet of code:failureCriteria: - avg(response_time_ms) > 300 - percentage(error) > 50
- Save the changes to the config.yaml by clicking Commit and Commit once more.
- Navigate back to Pipelines and run the eShopOnWeb pipeline again. After a few minutes, it will complete the run with a failed status for the AzureLoadTest task.
-
Open the verbose logging view for the pipeline, and validate the details of the AzureLoadtest. A similar sample output is below:
Creating and running a testRun for the test View the load test run in progress at: https://portal.azure.com/#blade/Microsoft_Azure_CloudNativeTesting/NewReport//resourceId/%2fsubscriptions%2fb86d9ae1-7552-47fb2ea7f9f4%2fresourcegroups%2faz400m05l11-rg%2fproviders%2fmicrosoft.loadtestservice%2floadtests%2feshoponwebloadtesting/testId/0d295119-12d0-a7b84787c004/testRunId/f4bec76a-8b49-44ee-a388-12af34f0d4ec TestRun completed -------------------Summary --------------- TestRun start time: Mon Jul 24 2023 23:00:31 GMT+0000 (Coordinated Universal Time) TestRun end time: Mon Jul 24 2023 23:06:02 GMT+0000 (Coordinated Universal Time) Virtual Users: 50 TestStatus: DONE -------------------Test Criteria --------------- Results :1 Pass 1 Fail Criteria :Actual Value Result avg(response_time_ms) > 300 1355.29 FAILED percentage(error) > 50 PASSED ------------------Client-side metrics------------ Homepage response time : avg=1355ms min=58ms med=666ms max=16524ms p(90)=2472ms p(95)=5819ms p(99)=13657ms requests per sec : avg=37 total requests : 4531 total errors : 0 total error rate : 0 ##[error]TestResult: FAILED Finishing: AzureLoadTest
-
Notice how the last line of the Load testing output says ##[error]TestResult: FAILED; since we defined a FailCriteria having an avg response time of > 300, or having an error percentage of > 20, now seeing an avg response time which is more than 300, the task will be flagged as failed.
Note: Imagine in a real-life scenario, you would validate the performance of your App Service, and if the performance is below a certain threshold - typically meaning there is more load on the Web App, you could trigger a new deployment to an additional Azure App Service. As we can’t control the response time for Azure lab environments, we decided to revert the logic to guarantee the failure.
- The FAILED status of the pipeline task, actually reflects a SUCCESS of the Azure Load Testing requirement criteria validation.
Exercise 3: Remove the Azure lab resources
In this exercise, you will remove the Azure resources provisioned in this lab to eliminate unexpected charges.
Note: Remember to remove any newly created Azure resources that you no longer use. Removing unused resources ensures you will not see unexpected charges.
Task 1: Remove the Azure lab resources
In this task, you will use Azure Cloud Shell to remove the Azure resources provisioned in this lab to eliminate unnecessary charges.
- In the Azure portal, open the Bash shell session within the Cloud Shell pane.
-
List all resource groups created throughout the labs of this module by running the following command:
az group list --query "[?starts_with(name,'az400m09l16')].name" --output tsv
-
Delete all resource groups you created throughout the labs of this module by running the following command:
az group list --query "[?starts_with(name,'az400m09l16')].[name]" --output tsv | xargs -L1 bash -c 'az group delete --name $0 --no-wait --yes'
Note: The command executes asynchronously (as determined by the –nowait parameter), so while you will be able to run another Azure CLI command immediately afterwards within the same Bash session, it will take a few minutes before the resource groups are actually removed.
Review
In this exercise, you deployed a web app to Azure App Service by using Azure Pipelines, as well as deploying an Azure Load Testing Resource with TestRuns. Next, you integrated the JMeter load testing config.yaml file to Azure Repos source control, and extending your CI/CD pipeline with the Azure Load Testing. In the last exercise, you learned how to define the success criteria of the LoadTest.