Creating a Code Coverage Report for Multiple Projects

The current solution I’m working on is composed of 11 seperate projects, 3 of which are test projects. As part of our CI/CD deployment pipeline we want to be able to generate a code coverage report for these projects which is displayed as a comment on the PR in GitHub.

Or tests are built using xUnit which automatically adds the “coverlet.collector” package to the project, however it’s also necessary to add coverlet.msbuild to all the test packages to allow the coverage data to be gathered.

dotnet add package coverlet.msbuild

Within our workflow we already had a task to run tests so initally we just need to update this to collect coverage data using coverlet as well.

- name: Run all tests
  run: dotnet test MyProject/MyProject.sln /p:CollectCoverage=true /p:CoverletOutput=../Assets/Coverage/ /p:MergeWith="../Assets/Coverage/coverage.json" /p:CoverletOutputFormat=\"cobertura,json\" -m:1

This creates an “Assets/Coverage” folder in our route “MyProject” folder and creates 2 files in it. The first, “coverage.json” is a merged JSON file that contains the combined coverage results from all 3 test projects. A second file “coverage.cobertura.xml” is then generated from the “coverage.json” file and contains the data needed to produce the coverage report.

The -m:1 argument at the end of the command is very important if there are multiple test projects in the solution. By default the tests will run in parallel which will lead to inconsistant data being written to the merged JSON file. This argument changes the tests to run one after the other so that the data is recorded correctly.

In order to exclude the test projects from the coverage report we need to add the following to their csproj files.

<ItemGroup>
  <AssemblyAttribute Include="System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute" />
</ItemGroup>

Theoretically this can also be configured in a runsettings file but I haven’t managed to get that working.

If we want to add the data from this coverage file to our PR then we need to make use of a couple of 3rd party actions to generate the markdown and to post the comment.

- name: Code coverage report
  uses: irongut/CodeCoverageSummary@v1.3.0
  with:
    filename: MyProject/Assets/Coverage/coverage.cobertura.xml
    badge: true
    fail_below_min: false
    format: markdown
    hide_branch_rate: false
    hide_complexity: true
    indicators: true
    output: both
    thresholds: '60 80'

- name: Add Coverage PR Comment
  uses: marocchino/sticky-pull-request-comment@v2
  if: github.event_name == 'pull_request'
  with:
    header: codeCoverage
    recreate: true
    path: code-coverage-results.md

This turns the data from the xml coverage file into a markdown table and then posts it as a comment to the PR. It’s possible for the report task to also fail the build if the coverage isn’t high enough but unfortunatly our coverage is pretty low at the moment.

The second task adds and updates a sticky comment so that if multiple changes are made there’s only one comment added on the PR.