Benchmark and optimize a sales report generator
A sales report program works, but it slows down as the number of orders grows. In this exercise, you follow the optimization workflow from this module: measure the whole program, profile it to find the bottleneck, compare the slow code with a faster version, switch to the faster version, and measure again.
This exercise takes approximately 30 minutes.
Set up your workspace
You write and run your code in Visual Studio Code. The starter code is in a GitHub repository.
Tip: If you already cloned the repository in another exercise, skip to the next section.
-
Open a new Visual Studio Code window (File > New Window).
-
On the Welcome page, select Clone Git Repository…, or open the Command Palette (Ctrl+Shift+P) and run Git: Clone.
-
Enter the following URL and press Enter:
https://github.com/MicrosoftLearning/mslearn-python-programming.git -
Create a folder for the repository, such as
mslearn-python, and select Select as Repository Destination. -
When the clone completes, select Open.
Review the starter code
-
In the Explorer pane, open the
Labfiles/10-benchmark-sales-report-performancefolder and openbenchmark.py. -
Review the functions:
Function Purpose make_order_idsCreates a list of order IDs. IDs 0, 1, and 2 appear twice, so they’re duplicates. calculate_totalAdds up the sales amounts. find_duplicate_orders_slowFinds duplicate order IDs by comparing every ID with every other ID. find_duplicate_orders_fastFinds duplicate order IDs by checking each ID once. build_reportBuilds the report text. It currently calls find_duplicate_orders_slow.The
if __name__ == "__main__":block at the bottom creates 5,000 orders and prints the report. It also contains three comments that describe the measurements you add. -
Open a terminal (Terminal > New Terminal), change to the exercise folder, and run the program:
cd Labfiles/10-benchmark-sales-report-performance python benchmark.pyOrders: 5003 Total sales: $125,075.00 Duplicate order IDs: [0, 1, 2]The report is correct. The question is how long it takes.
Measure the whole report
A single timing varies from run to run, so you use timeit to run the report several times and average the result.
-
Beneath the
# Time three runs of the whole report and print the average per runcomment, add the following lines:runs = 3 total_time = timeit.timeit(lambda: build_report(order_ids, sales_amounts), number=runs) print(f"Whole report: {total_time / runs:.4f} seconds per run")[!TIP] As you add code, be sure to maintain the correct indentation. Use the existing comments as a guide, entering the new code at the same level of indentation.
timeit.timeit()runs a functionnumbertimes and returns the total time in seconds.- The
lambdawraps the call tobuild_reportso thattimeitcan run it with its arguments. - Dividing by
runsgives the average time for one run.
-
Run the program. Your time differs, but it’s roughly a third of a second:
Orders: 5003 Total sales: $125,075.00 Duplicate order IDs: [0, 1, 2] Whole report: 0.3384 seconds per runThat’s your baseline. After you make a change, you compare against this number.
Profile the report to find the bottleneck
The report runs three steps, and you don’t know which one takes the time. Guessing can send you to the wrong place, so you profile the program with cProfile.
-
Beneath the
# Profile the whole report and sort the results by time spent in each functioncomment, add the following line:cProfile.run("build_report(order_ids, sales_amounts)", sort="tottime")cProfile.run()runs the code in the string and prints how many times each function was called and how long it took.sort="tottime"lists the functions that spent the most time first. -
Run the program and look at the top of the profile output:
5015 function calls in 0.367 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.366 0.366 0.366 0.366 benchmark.py:16(find_duplicate_orders_slow) 5005 0.001 0.000 0.001 0.000 {built-in method builtins.len} 1 0.000 0.000 0.367 0.367 {built-in method builtins.exec} 1 0.000 0.000 0.366 0.366 benchmark.py:37(build_report)Your numbers differ, but the pattern is the same. These columns matter most:
Column Meaning ncallsHow many times the function was called. tottimeTime spent inside the function itself, not counting the functions it calls. cumtimeTime spent in the function and everything it calls. find_duplicate_orders_slowhas almost all of thetottime. It’s the bottleneck. Makingcalculate_totalfaster wouldn’t help, because it takes almost no time.
Compare the slow and fast versions
Now that you know which function to fix, measure it on its own at different data sizes. This shows how each version scales.
-
Beneath the
# Time both duplicate finders at three data sizes and print the resultscomment, add the following lines:for size in [500, 1000, 2000]: ids = make_order_ids(size) slow = timeit.timeit(lambda: find_duplicate_orders_slow(ids), number=3) fast = timeit.timeit(lambda: find_duplicate_orders_fast(ids), number=3) print(f"{size:5d} IDs: slow {slow:.4f}s, fast {fast:.4f}s")- The loop builds a list of each size and times both functions on it.
- Each function runs three times for each size.
-
Run the program. The comparison prints after the profile output:
500 IDs: slow 0.0111s, fast 0.0001s 1000 IDs: slow 0.0435s, fast 0.0002s 2000 IDs: slow 0.1878s, fast 0.0003sEach time the number of IDs doubles, the slow version takes about four times longer. That’s the O(n²) pattern: it compares every ID with every other ID. The fast version stays under a millisecond because it checks each ID once and uses a set to remember the IDs it has seen. That’s an O(n) algorithm.
Switch to the faster function
You chose the better algorithm because the profile showed where the time goes. Now use it in the report.
-
In
build_report, beneath the# Use the faster function to find the duplicate order IDscomment, update the code to the following:duplicates = find_duplicate_orders_fast(order_ids)The report now calls
find_duplicate_orders_fast. The rest of the function stays the same. -
Run the program and look at the first lines of the output:
Orders: 5003 Total sales: $125,075.00 Duplicate order IDs: [0, 1, 2] Whole report: 0.0004 seconds per runThe report shows the same duplicate order IDs, so the result is unchanged. The time per run drops from about 0.34 seconds to under a millisecond.
-
Look at the profile output.
find_duplicate_orders_slowno longer appears, and the whole program takes about a millisecond:5014 function calls in 0.001 seconds
You measured the program, found the bottleneck with a profiler, compared two algorithms, and confirmed the improvement with the same measurement. Always measure again after a change, to prove that it helped.
Practice with GitHub Copilot
Use GitHub Copilot to practice reading and extending your measurements. Open Copilot Chat (Ctrl+Alt+I). Copilot can make mistakes, so run the program and check each answer against your own results.
-
Check your reading of the profile. Enter:
“Explain the cProfile output for
benchmark.pyline by line. Which function is the bottleneck, and how do thetottimeandcumtimecolumns show that?”Compare the answer with what you concluded. Does Copilot point to the same function?
-
Remove the repeated timing code. Enter:
“Write a function
compare_functions(slow_func, fast_func, sizes)that uses timeit to time two functions at each size and prints the results. Use it to replace theforloop at the end ofbenchmark.py.”Run the program. Do the timings match the ones you saw before?
-
Add another implementation. Enter:
“Add a third duplicate finder that uses
collections.Counter. Time all three finders at 2,000 order IDs, and time only the two faster ones at 200,000 order IDs. Tell me the Big O of each.”Run the program. Does the Big O that Copilot gives match the timings you see? The slowest finder is left out of the larger test on purpose, because it would take a long time at 200,000 IDs.