Automated Benchmarking With BenchmarkDotNet For .NET

Hey guys! Ever wondered how to keep your .NET code running at peak performance? Well, benchmarking is the name of the game, and with BenchmarkDotNet, we've got a fantastic tool to help us out. In this article, we're diving deep into setting up continuous and automated benchmarking for your .NET projects, ensuring your code not only works but also runs like a well-oiled machine. We'll be using BenchmarkDotNet and a nifty GitHub Action to make this all happen, plus, we'll look at how to visualize those results for easy tracking.

Why Automated Benchmarking Matters

Alright, let's talk about why we even bother with automated benchmarking. In a nutshell, it's about measuring and monitoring the performance of your code over time. Think of it like a fitness tracker for your application. It helps you:

• Identify performance bottlenecks: Pinpoint those pesky areas of code that are slowing things down. No one wants a slow app, right?
• Track performance regressions: Catch any performance dips that might sneak in after code changes. Early detection is key!
• Make data-driven decisions: Ensure that any optimizations you make actually improve things. It's all about the numbers!
• Maintain code quality: By continuously monitoring performance, you can ensure that your code stays fast and efficient. This gives a guarantee of better user experience.

Now, imagine having this all automated. Every time you push a change, your code gets benchmarked, and you get instant feedback on its performance. That's the power of continuous benchmarking, and it's what we're setting up here. Without automation, you might miss a regression that causes user-facing issues. This approach helps in the early detection and prevention of performance issues.

Setting Up Your .NET Project for Benchmarking

First things first, you'll need a .NET project, or you can use an existing project. In this guide, we'll assume you have a project with a benchmark. If not, don't worry – creating one is super easy. Just follow the steps outlined below:

1. Install BenchmarkDotNet

Make sure that BenchmarkDotNet is installed to do the job. You can do this by using the following command in the project directory:

  dotnet add package BenchmarkDotNet

2. Create a Benchmarks Project

If you don't already have one, create a separate project specifically for your benchmarks. This helps keep things organized. You can do this by creating a new project with the command:

  dotnet new classlib -n OpenFeature.Benchmarks

Add all the required dependencies.

  dotnet add OpenFeature.Benchmarks package BenchmarkDotNet

3. Write Your Benchmarks

Inside your benchmarks project, create some benchmark classes. Each benchmark class will contain methods that you want to measure. For example:

  using BenchmarkDotNet.Attributes;
  
  public class MyBenchmark
  {
      [Benchmark]
      public void MyMethod()
      {
          // Code to benchmark
      }
  }

4. Configure Your Project

Configure your .csproj file to ensure the benchmarks are built correctly, and your results are published. This might involve setting up the output path, especially if you plan to publish the results to a GitHub page.

This simple setup lets you measure the time it takes to execute MyMethod. You can create more complex benchmarks with setups, tear downs, and parameters. The possibilities are endless!

Implementing Continuous Benchmarking with GitHub Actions

Now, let's get into the exciting part: automating the whole process. We're going to use GitHub Actions to run our benchmarks on a schedule and on every merge to the main branch. This is where martincostello/benchmarkdotnet-results-publisher comes into play. It's a fantastic GitHub Action designed to publish your BenchmarkDotNet results to a GitHub Page, making them easy to visualize. Follow these steps to set things up:

1. Create a GitHub Action Workflow File

In your repository, create a new workflow file under .github/workflows/. For example, benchmark.yml. This file will contain the instructions for our CI/CD pipeline. Here's a basic structure to get you started:

  name: Benchmark
  
  on:
    push:   # Run on every push to main branch
      branches: [ "main" ]
    schedule:
      - cron: "0 0 * * *"  # Run daily at midnight
  
  jobs:
    benchmark:
      runs-on: ubuntu-latest
  
      steps:
        - uses: actions/checkout@v3
        - name: Setup .NET
          uses: actions/setup-dotnet@v3
          with:
            dotnet-version: '7.0'
        - name: Build and Run Benchmarks
          run: |
            dotnet build test/OpenFeature.Benchmarks
            dotnet run --project test/OpenFeature.Benchmarks --configuration Release
        - name: Publish Benchmark Results
          uses: martincostello/benchmarkdotnet-results-publisher@v1
          with:
            github-token: ${{ secrets.GITHUB_TOKEN }}
            artifact-name: benchmark-results
            results-path: test/OpenFeature.Benchmarks/BenchmarkDotNet.Artifacts/results

2. Customize the Workflow

• on: This section specifies when the workflow will run. In this case, it's triggered on every push to the main branch and daily at midnight. Customize the branch names to match your repository's structure.
• jobs: The benchmark job runs on an ubuntu-latest runner.
• steps: This is where the magic happens:
  • actions/checkout@v3: Checks out your repository.
  • actions/setup-dotnet@v3: Sets up the .NET environment.
  • Build and Run Benchmarks: Builds your benchmarks project and runs the benchmarks. Be sure to replace OpenFeature.Benchmarks with the name of your project.
  • martincostello/benchmarkdotnet-results-publisher@v1: This action publishes your benchmark results to a GitHub Page. You'll need to configure the github-token, artifact-name, and results-path to match your setup.

3. Configure GitHub Pages

To see your benchmark results, you'll need to set up GitHub Pages for your repository. Go to the