The current solution I’m working on is composed of 11 seperate projects, 3 of which are test projects. As part of our CI/CD deployment pipeline we want to be able to generate a code coverage report for these projects which is displayed as a comment on the PR in GitHub.
Or tests are built using xUnit which automatically adds the “coverlet.collector” package to the project, however it’s also necessary to add coverlet.msbuild to all the test packages to allow the coverage data to be gathered.
dotnet add package coverlet.msbuild
Within our workflow we already had a task to run tests so initally we just need to update this to collect coverage data using coverlet as well.
- name: Run all tests run: dotnet test MyProject/MyProject.sln /p:CollectCoverage=true /p:CoverletOutput=../Assets/Coverage/ /p:MergeWith="../Assets/Coverage/coverage.json" /p:CoverletOutputFormat=\"cobertura,json\" -m:1
This creates an “Assets/Coverage” folder in our route “MyProject” folder and creates 2 files in it. The first, “coverage.json” is a merged JSON file that contains the combined coverage results from all 3 test projects. A second file “coverage.cobertura.xml” is then generated from the “coverage.json” file and contains the data needed to produce the coverage report.
-m:1 argument at the end of the command is very important if there are multiple test projects in the solution. By default the tests will run in parallel which will lead to inconsistant data being written to the merged JSON file. This argument changes the tests to run one after the other so that the data is recorded correctly.
In order to exclude the test projects from the coverage report we need to add the following to their csproj files.
<ItemGroup> <AssemblyAttribute Include="System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute" /> </ItemGroup>
Theoretically this can also be configured in a runsettings file but I haven’t managed to get that working.
If we want to add the data from this coverage file to our PR then we need to make use of a couple of 3rd party actions to generate the markdown and to post the comment.
- name: Code coverage report uses: irongut/CodeCoverageSummary@v1.3.0 with: filename: MyProject/Assets/Coverage/coverage.cobertura.xml badge: true fail_below_min: false format: markdown hide_branch_rate: false hide_complexity: true indicators: true output: both thresholds: '60 80' - name: Add Coverage PR Comment uses: marocchino/sticky-pull-request-comment@v2 if: github.event_name == 'pull_request' with: header: codeCoverage recreate: true path: code-coverage-results.md
This turns the data from the xml coverage file into a markdown table and then posts it as a comment to the PR. It’s possible for the report task to also fail the build if the coverage isn’t high enough but unfortunatly our coverage is pretty low at the moment.
The second task adds and updates a sticky comment so that if multiple changes are made there’s only one comment added on the PR.