Code Coverage in .NET Core is tricky if you want to use it in a CI build and/or SonarQube. I blogged about .NET Core, SonarQube and Code Coverage – but this felt like a hack. I did a little more research and found a better way. My requirements were:
- Runs on Linux and Windows
- Displays a nice report in Azure Pipelines
- Supports Code Coverage in SonarQube/SonarCloud
- Tests build and run only one time
If you’re not interested in the details – check out my code coverage sample on GitHub. You must install two NuGet packages in your test project:
coverlet.msbuild and OpenCover. Then use the azure-pipelines.yml and modify it to your needs. If you are interested in the details read on…
There are different options to collect code coverage in .NET Core. The default is Visual Studio (a .coverage file). This does not display a nice report in Azure Pipelines – you have to download the file to your computer. If you want to use it with SonarQube you have to convert the files to XML like I did with my PowerShell script. Then there is Coverlet. You can use it together with the report generator to create nice reports and upload them to Azure Pipelines. But it is not supported by SonarQube. Another option is Opencover. This was the only option I found that can be converted to nice reports and is supported by SonarQube.
Here is my azure-pipelines.yml. The first part are the variables you can adjust to meet your project structure. The strategy is to run the build on Linux and Windows. If you just want to run it on one platform you can remove it and directly add the image name to vmImage under pool. The trigger causes the build to run on every check-in to master.
variables: buildConfiguration: "Debug" testProject: "tests/MySample.Tests" solution: "MySample.sln" strategy: matrix: linux: imageName: 'ubuntu-16.04' windows: imageName: 'vs2017-win2016' trigger: - master pool: vmImage: $(imageName)
The next are the build steps. The first is the SonarCloud Prepare Analysis task. You get the connection ID (line 4) from the service connection. Important is line 8: you must tell SonarCloud to use opencover format and the path where it can find the report file.
steps: - task: SonarCloudPrepare@1 inputs: SonarCloud: '66420b06-0308-4157-9b80-ef53c71c6596' organization: 'wulfland-github' projectKey: 'cov-demo' projectName: 'Coverage Demo' extraProperties: 'sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/coverage/coverage.opencover.xml'
Next are dotnet build and dotnet test. The –no-build argument tells dotnet test to not build the test project again. Use opencover as the output format.
- script: dotnet build $(solution) --configuration $(buildConfiguration) displayName: 'dotnet build $(buildConfiguration)' - script: | dotnet test --logger trx --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=$(Build.SourcesDirectory)/coverage/ $(testProject) displayName: 'dotnet test'
The next part is only so complex because the reportgenerator tool is different on Linux (reportgenerator) and windows (reportgenerator.exe). If you just have one platform this are only two lines:
dotnet tool install dotnet-reportgenerator-globaltool –tool-path .
./reportgenerator “-reports:$(Build.SourcesDirectory)/coverage/coverage.opencover.xml” “-targetdir:coverage/Cobertura” “-reporttypes:Cobertura;HTMLInline;HTMLChart”
As the report output use Cobertura, HTMLInline and HTMLChart.
- script: | dotnet tool install dotnet-reportgenerator-globaltool --tool-path . ./reportgenerator "-reports:$(Build.SourcesDirectory)/coverage/coverage.opencover.xml" "-targetdir:coverage/Cobertura" "-reporttypes:Cobertura;HTMLInline;HTMLChart" condition: eq( variables['Agent.OS'], 'Linux' ) displayName: Run Reportgenerator on Linux - script: | dotnet tool install dotnet-reportgenerator-globaltool --tool-path . .\reportgenerator.exe "-reports:$(Build.SourcesDirectory)/coverage/coverage.opencover.xml" "-targetdir:coverage/Cobertura" "-reporttypes:Cobertura;HTMLInline;HTMLChart" condition: eq( variables['Agent.OS'], 'Windows_NT' ) displayName: Run Reportgenerator on Windows
After that you can run the SonarCloud analysis and publish the Quality Gate Results. You also publish the test results and code coverage result. Set the code coverage to cobertura and point summary file and report directory accordingly.
- task: SonarSource.sonarcloud.ce096e50-6155-4de8-8800-4221aaeed4a1.SonarCloudAnalyze@1 displayName: 'Run Code Analysis' - task: SonarCloudPublish@1 displayName: 'Publish Quality Gate Results' - task: PublishTestResults@2 inputs: testRunner: VSTest testResultsFiles: '**/*.trx' - task: PublishCodeCoverageResults@1 inputs: summaryFileLocation: $(Build.SourcesDirectory)/coverage/Cobertura/Cobertura.xml reportDirectory: $(Build.SourcesDirectory)/coverage/Cobertura codecoverageTool: cobertura
That’s it. Now you have a nice report in the Azure Pipeline. If you click on a file you get a nice colored view of the code file. In Sonar you see the code coverage and can adjust your quality gate accordingly.
8 thoughts on “.NET Core Code Coverage done right”
I’ve followed the above but still not getting any coverage in SonarCloud 😦
I have multiple test projects in my case and so in the dotnet test task my parameters look like this: /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=$(Build.SourcesDirectory)\coverage\tests
And then my script looks like: dotnet tool install dotnet-reportgenerator-globaltool –tool-path .
.\reportgenerator.exe “-reports:$(Build.SourcesDirectory)\coverage\tests.opencover.xml” “-targetdir:coverage\Cobertura” “-reporttypes:Cobertura;HTMLInline;HTMLChart”
When the SonarCloud Run Code Analysis task runs it does find the files and adds them to the cache for later use:
INFO: Sensor C# Tests Coverage Report Import [csharp]
INFO: Parsing the OpenCover report D:\TfsBuildAgents\Agent4\_work\60\s\coverage\tests.opencover.xml
INFO: Adding this code coverage report to the cache for later reuse: D:\TfsBuildAgents\Agent4\_work\60\s\coverage\tests.opencover.xml
INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=0ms
But im still seeing 0.00% in sonar cloud. I’ve tried so much to get this right but no luck. Would love some help!
Have you set the the parameter of the sonar task correct?
Any luck with the settings?
great article! works like a charm
Excellent article – but there is one problem. You are not merging results of code coverage, so if you have multiple test projects, coverage.opencover.xml is being overwritten, and you get coverage only on the last test project that completed.
This can be fixed by merging test results as described here: https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Examples/MSBuild/MergeWith/HowTo.md
If anyone is interested, I’ve created an example project where this merging is implemented: https://github.com/kurtanr/CodeCoverageExample.
Thanks Robert. I’ll look into that.