Build and Tune Hybrid Search in C#

In this exercise, you build hybrid search over the CosmicWorks product catalog in Azure Cosmos DB for NoSQL. You compare keyword and vector results, combine them with RRF, and adjust the weights while measuring request charge and latency.

This exercise takes approximately 60 minutes to complete.

Before you start

To complete this exercise, you need an Azure subscription with permission to create resources and assign roles.

If your lab environment isn’t set up yet, follow Set up your lab environment to install Visual Studio Code, Git, the Azure CLI, and PowerShell 7.

You also need the .NET 10 SDK or later installed.

The setup script creates a Microsoft Foundry resource, project, and text-embedding-3-small deployment. Your subscription needs quota for that model in the Foundry region.

Set up your Azure Cosmos DB resources

This exercise creates its own Azure Cosmos DB account and deletes it at the end, so it doesn’t use the shared account from the rest of this learning path.

1. Start Visual Studio Code.

2. If you don’t have the lab code yet, clone the repository for DP-420: open the command palette with Ctrl+Shift+P, run Git: Clone, and enter the following URL. Choose a local folder when prompted. Otherwise, open the folder from your previous clone.

    https://github.com/microsoftlearning/dp-420-cosmos-db-dev
   

3. Once the repository is cloned, open that local folder in Visual Studio Code.

4. In the Explorer pane, browse to the Allfiles/Labs/Shared folder.

5. Open the context menu for the folder and select Open in Integrated Terminal. If the terminal isn’t PowerShell, select the dropdown beside the + in the terminal toolbar and choose PowerShell.

6. Sign in to the Azure CLI and follow the sign-in prompt.

    az login
   

7. Set variables for the resource group and region. If your lab environment provides a resource group, use that name. Otherwise, use a new group that contains only this exercise’s resources.

    $resourceGroup = "ResourceGroup1"
    $location = "eastus"
   

8. Run the setup script.

    ./setup.ps1 -ResourceGroup $resourceGroup -Location $location -NamePrefix dp420lab17 -LabProfile search -AccountOnly -EnableFoundry -EmbeddingOnly -FoundryLocation $location
   

   The script generates a unique account name, such as dp420lab17a7f3k9.

9. Wait for the script to finish. This takes 5-10 minutes.

10. Record the Account name, Account endpoint, Foundry account, and OpenAI endpoint values the script prints. The Cosmos DB endpoint looks like https://<your-account-name>.documents.azure.com:443/. The script also saves the Foundry values in logs/foundry-<account-name>.json.

11. Set the account name the script printed, then open that account in the Azure portal.

     $accountName = "<your-account-name>"
    

12. Under Settings, select Features. Confirm Vector Search for NoSQL API is enabled. Allow up to 15 minutes for activation before creating containers.

13. Resume setup against this same account to create the database, container, and Cosmos DB role assignment.

     ./setup.ps1 -ResourceGroup $resourceGroup -Location $location -AccountName $accountName -LabProfile search -SearchFeaturesReady -EnableFoundry -EmbeddingOnly -FoundryLocation $location
     ./verify.ps1 -ResourceGroup $resourceGroup -AccountName $accountName -LabProfile search -EnableFoundry -EmbeddingOnly
    

    Continue after both commands succeed.

After both stages, the script creates the following resources:

The applications use DefaultAzureCredential for Microsoft Entra ID authentication. Make sure the identity it selects has the role assignments listed above.

⚠ Use the dedicated account created for this exercise, not a shared training or production account.

1. Set variables for the account name and its endpoint so later commands can use them.

    $accountName = "<your-account-name>"
    $cosmosEndpoint = "https://$accountName.documents.azure.com:443/"
   

Task 1: Confirm the embedding deployment

Confirm that the embedding deployment is ready before loading the catalog.

1. In the Allfiles/Labs/Shared terminal, load the saved settings.

    $foundry = Get-Content "./logs/foundry-$accountName.json" -Raw | ConvertFrom-Json
    $openAiName = $foundry.FoundryAccountName
    $openAiEndpoint = $foundry.OpenAiEndpoint
    $foundry | Format-List FoundryAccountName, OpenAiEndpoint, EmbeddingDeployment, EmbeddingDimensions
   

2. Check the deployed model’s state.

    az cognitiveservices account deployment list `
        --name $openAiName `
        --resource-group $resourceGroup `
        --query "[].{Deployment:name,Model:properties.model.name,Version:properties.model.version,State:properties.provisioningState}" `
        --output table
   

   Confirm that text-embedding-3-small reports Succeeded. Use OpenAiEndpoint, not the project endpoint, in the application code that follows. The model’s default output length matches the container’s 1,536 dimensions.

📝 New role assignments can take several minutes to take effect.

Task 2: Load the catalog with searchable text and embeddings

The catalog descriptions repeat the product names. This exercise combines each name and category into searchText and embeds that same text. The container’s indexes read /searchText and /embedding.

1. Create a project and open its folder in the integrated terminal.

    dotnet new console --name hybrid-lab
    cd hybrid-lab
   

2. Install the required packages.

    dotnet add package Microsoft.Azure.Cosmos
    dotnet add package Azure.Identity
    dotnet add package Newtonsoft.Json
    dotnet add package Azure.AI.OpenAI
   

3. Replace the contents of Program.cs with the following code. Replace the two placeholder values with your endpoints.

    using Azure.AI.OpenAI;
    using Azure.Identity;
    using Microsoft.Azure.Cosmos;
    using Newtonsoft.Json.Linq;
    using OpenAI.Embeddings;
   
    string cosmosEndpoint = "<cosmos-endpoint>";
    string openAiEndpoint = "<openai-endpoint>";
    string deployment = "text-embedding-3-small";
    string dataUrl = "https://raw.githubusercontent.com/AzureCosmosDB/CosmicWorks/main/data/database-v4/product";
   
    // Reuse a credential for both services; each SDK requests its own token audience.
    DefaultAzureCredential credential = new DefaultAzureCredential();
   
    EmbeddingClient embeddingClient = new AzureOpenAIClient(new Uri(openAiEndpoint), credential)
        .GetEmbeddingClient(deployment);
   
    Container container = new CosmosClient(cosmosEndpoint, credential)
        .GetContainer("cosmicworks", "productSearch");
   
    using HttpClient http = new HttpClient();
    JArray products = JArray.Parse(await http.GetStringAsync(dataUrl));
   
    Console.WriteLine($"Downloaded {products.Count} products.");
   
    // Use the same name-and-category text for both search methods.
    for (int start = 0; start < products.Count; start += 50)
    {
        List<JObject> batch = products.Skip(start).Take(50).Cast<JObject>().ToList();
        List<string> texts = batch
            .Select(product => $"{product["name"]} {product["categoryName"]}")
            .ToList();
   
        OpenAIEmbeddingCollection embeddings = await embeddingClient.GenerateEmbeddingsAsync(texts);
   
        // Pair each vector with its source product; each upsert is a separate write.
        for (int i = 0; i < batch.Count; i++)
        {
            batch[i]["searchText"] = texts[i];
            batch[i]["embedding"] = JArray.FromObject(embeddings[i].ToFloats().ToArray());
   
            // Preserve the ID and partition key so reruns replace the same logical item.
            await container.UpsertItemAsync(
                batch[i], new PartitionKey(batch[i]["categoryId"]!.ToString()));
        }
   
        Console.WriteLine($"Loaded {start + batch.Count} of {products.Count}.");
    }
   

4. Run the application.

    dotnet run
   

    Downloaded 295 products.
    Loaded 50 of 295.
    Loaded 100 of 295.
    Loaded 150 of 295.
    Loaded 200 of 295.
    Loaded 250 of 295.
    Loaded 295 of 295.
   

Task 3: Measure each retrieval method on its own

Compare keyword and vector results for these two shopper requests.

Each query returns up to five products. Keyword search matches terms in searchText; vector search ranks by embedding similarity.

1. Replace the contents of Program.cs with the following code, keeping the same project and packages and replacing the two placeholder values with your endpoints.

    using System.Diagnostics;
    using Azure.AI.OpenAI;
    using Azure.Identity;
    using Microsoft.Azure.Cosmos;
    using OpenAI.Embeddings;
   
    // Edit each search phrase and its keyword terms together when testing scenarios.
    var cases = new[]
    {
        (
            Label: "Hill riding",
            Text: "I need some tires to ride up the hill.",
            Terms: new[] { "tires", "ride", "hill" }
        ),
        (
            Label: "Exact bike",
            Text: "Touring-1000 Blue, 50",
            Terms: new[] { "Touring", "1000", "Blue", "50" }
        )
    };
   
    string cosmosEndpoint = "<cosmos-endpoint>";
    string openAiEndpoint = "<openai-endpoint>";
    string deployment = "text-embedding-3-small";
   
    DefaultAzureCredential credential = new DefaultAzureCredential();
   
    EmbeddingClient embeddingClient = new AzureOpenAIClient(new Uri(openAiEndpoint), credential)
        .GetEmbeddingClient(deployment);
   
    Container container = new CosmosClient(cosmosEndpoint, credential)
        .GetContainer("cosmicworks", "productSearch");
   
    foreach (var testCase in cases)
    {
        // Generate once per case and reuse across methods; time this model call separately.
        Stopwatch embeddingTimer = Stopwatch.StartNew();
        OpenAIEmbedding embedding = await embeddingClient.GenerateEmbeddingAsync(testCase.Text);
        embeddingTimer.Stop();
        float[] queryVector = embedding.ToFloats().ToArray();
   
        Console.WriteLine($"\n{testCase.Label}: {testCase.Text}");
        Console.WriteLine($"Query embedding: {embeddingTimer.Elapsed.TotalMilliseconds:F1} ms");
   
        // Bind term values as parameters; generate only the placeholder names.
        string termArguments = string.Join(", ",
            Enumerable.Range(0, testCase.Terms.Length).Select(i => $"@term{i}"));
   
        // Rank products that contain at least one search term.
        QueryDefinition keywordQuery = WithTerms($@"
            SELECT TOP 5 c.id, c.name, c.categoryName
            FROM c
            WHERE FullTextContainsAny(c.searchText, {termArguments})
            ORDER BY RANK FullTextScore(c.searchText, {termArguments})",
            testCase.Terms);
   
        // Rank by meaning without requiring a keyword match.
        QueryDefinition vectorQuery = new QueryDefinition(@"
            SELECT TOP 5 c.id, c.name, c.categoryName
            FROM c
            ORDER BY VectorDistance(c.embedding, @queryVector)")
            .WithParameter("@queryVector", queryVector);
   
        await RunAsync("Keyword only", keywordQuery);
        await RunAsync("Vector only", vectorQuery);
    }
   
    QueryDefinition WithTerms(string sql, string[] terms)
    {
        QueryDefinition query = new QueryDefinition(sql);
        for (int i = 0; i < terms.Length; i++)
        {
            query.WithParameter($"@term{i}", terms[i]);
        }
        return query;
    }
   
    async Task RunAsync(string label, QueryDefinition query)
    {
        double charge = 0;
        List<string> lines = new List<string>();
        // Query execution is lazy; consume every page before totaling charge and elapsed time.
        Stopwatch timer = Stopwatch.StartNew();
        using FeedIterator<dynamic> feed = container.GetItemQueryIterator<dynamic>(query);
        while (feed.HasMoreResults)
        {
            FeedResponse<dynamic> page = await feed.ReadNextAsync();
            charge += page.RequestCharge;
            foreach (dynamic item in page)
            {
                lines.Add($"  {item.name}  |  {item.categoryName}");
            }
        }
        timer.Stop();
   
        Console.WriteLine($"\n{label}: {charge:F2} RUs; {timer.Elapsed.TotalMilliseconds:F1} ms");
        lines.ForEach(Console.WriteLine);
    }
   

2. Run the application.

    dotnet run
   

3. Review the Hill riding results. In a sample run:

   • Keyword only mixes road, mountain, and touring tires. It found the requested product type by matching the word tires.
   • In Vector only, the three mountain-tire variants appear first even though the query never uses the word mountain.

   Both methods returned tires. Keyword search uses the indexed words; vector search also takes the hill-riding context into account.

4. Review the Exact bike results.

   Both methods place Touring-1000 Blue, 50 first. An exact product name gives keyword search a strong match.

   The remaining keyword results include a yellow bike and a frame that share some terms. The remaining vector results are similar blue touring bikes, but include other sizes and the Touring-2000 model. Similarity doesn’t enforce an exact model or size requirement.

5. Review the cost and timing measurements.

   Vector search used 19.90 RU for each case, compared with 7.98 RU for the keyword hill-riding query and 14.06 RU for the keyword exact-bike query in this run.

   The database ms excludes Query embedding time. The sample hill-riding query takes about 1.16 seconds in Cosmos DB plus 8.8 seconds to generate the embedding. Keyword search doesn’t need that model call. Include both times when assessing response time.

   Timing can vary, especially on the first request while connections are established.

📝 This catalog has 295 vectors, below the 1,000-vector threshold for diskANN indexing. Vector queries use a full scan, so these measurements don’t represent indexed production performance.

Task 4: Fuse the two rankings with RRF

Combine the word-based and meaning-based rankings in one query with RRF.

1. In foreach (var testCase in cases), insert this code immediately after await RunAsync("Vector only", vectorQuery);, before the loop’s closing brace.

    // Fuse rank positions, not raw scores.
    QueryDefinition hybridQuery = WithTerms($@"
        SELECT TOP 5 c.id, c.name, c.categoryName
        FROM c
        ORDER BY RANK RRF(
            VectorDistance(c.embedding, @queryVector),
            FullTextScore(c.searchText, {termArguments}))",
        testCase.Terms)
        .WithParameter("@queryVector", queryVector);
   
    await RunAsync("Hybrid, equal weights", hybridQuery);
   

2. Run the application once. It prints all three lists for each case. In a sample run:

   • Hill riding: hybrid search keeps the mountain tires at the top, as vector search did. The road tires below them change order.
   • Exact bike: the requested bike stays first, followed by blue size variants of the same model. The yellow bike and frame don’t appear, but other sizes still do.
   • Cost: hybrid search uses 47.77 RU for hill riding and 52.28 RU for the exact bike, more than either method alone. It keeps the same first result as vector search for both queries.

   RRF combines rank positions from keyword and vector search rather than adding their raw scores. The hybrid query reuses the existing query vector, so it doesn’t make another embedding request.

Task 5: Weight the fusion and compare the results

The unweighted query treats keyword and vector rankings equally. Adding weights lets you favor one ranking without changing the search text, terms, or query vector. For the hill-riding request, giving vector search more weight can keep mountain tires near the top. Giving keyword search more weight can move road tires higher because they also match tires.

1. In foreach (var testCase in cases), insert this code immediately after await RunAsync("Hybrid, equal weights", hybridQuery);.

    // The first weight applies to VectorDistance; the second applies to FullTextScore.
    foreach (string weights in new[] { "[2, 1]", "[1, 2]" })
    {
        QueryDefinition weightedQuery = WithTerms($@"
            SELECT TOP 5 c.id, c.name, c.categoryName
            FROM c
            ORDER BY RANK RRF(
                VectorDistance(c.embedding, @queryVector),
                FullTextScore(c.searchText, {termArguments}),
                {weights})",
            testCase.Terms)
            .WithParameter("@queryVector", queryVector);
   
        await RunAsync($"Hybrid {weights}", weightedQuery);
    }
   

2. Run the application once. It prints the equal-weight and both weighted hybrid lists for each case. In a sample run:

   • Hill riding: equal weights and [2, 1] keep the mountain-tire variants together at the top. With [1, 2], a road tire moves ahead of one of the mountain tires.
   • Exact bike: the requested bike stays first with all three settings. The vector-heavy setting moves a similar blue bike from another model higher. The keyword-heavy setting brings in a yellow bike that shares the requested model and size. More keyword weight doesn’t require every term to match.
   • Cost: all three settings cost 47.77 RU for hill riding and 52.28 RU for the exact bike. Here, the weights change the order, not the RU charge. Timing varies between runs.

   Weights follow function order. With VectorDistance first and FullTextScore second, [2, 1] favors vector search and [1, 2] favors keyword search.

Clean up resources

Delete the resource group only if you created it for this exercise and it contains no resources you need to keep. This removes the Cosmos DB account, Foundry resource, project, and model deployment:

az group delete --name $resourceGroup --yes --no-wait

If your lab environment provided the resource group, keep it. Delete only the Cosmos DB and Foundry resources you created for this exercise, after confirming that no other application uses them:

az cosmosdb delete --name $accountName --resource-group $resourceGroup --yes
az cognitiveservices account delete --name $openAiName --resource-group $resourceGroup