How to run Pester tests in Azure DevOps with Test Results. Then link it to a Github pull request with continuous integration.

Azure DevOps Hosted Agent

In our example we will use the Microsoft hosted vs2017-win2016 build agent.

The pipeline files are written in YAML. The schema for the pipeline files is defined here. You should have a high level understanding of the schema.

Essentially, in our build pipeline for our PowerShell scripts we need to run Pester (PowerShell’s unit testing framework) and publish the test results.

In order to do this, we need to publish the Pester test results to an XML file with NUnit as the test runner for Azure DevOps tests to pick it up and show us a pretty visual of our test results.

Now you might be thinking why not just run invoke-pester in the azure-pipelines.yml file?!

Since we have to publish the results to a file, I prefer having a dedicated script but one could also do this by running an inline PowerShell step. The Invoke-PesterAzureDevOps.ps1 looks like this:

Hopefully, you are following…because that is all!!

Commit the change to publish the tests

Once you commit this azure-pipelines.yml file on your branch it should automatically run (Azure DevOps will see the file and run it).

I will explicitly say, you must have some working pester tests but as long as you, bada-bing, bada-boom any newly added tests will automatically run now too.

All code referenced is available in Github with a hello world script example and a few Pester tests available here.

The CI Build failed, but why?

Here are screenshots of what you can expect to see after you’ve pushed the pipelines file.

Ooops the build failed…let’s check it out…if I go to the summary page, I can see 1 test failed:

Now if I navigate to the tests tab, I can see exactly why the test failed:

It seems in the Pester module loaded into our Microsoft provided test agent, it doesn’t like the Should -Be for our assertion.

The agent is running pester version 3.4.0 as we can see from the stack trace. I had to change the Should -Be to Should Be (removing the dash).

Let’s try this commit again

Now all of our tests have successfully passed and we can see the description and the duration. I am ready to start the pull request process on my Github repo into my master branch. As soon as I open the pull request I can see the rocket logo for pipelines pending.

Github Pull Request

Azure DevOps CI job automatically kicks off

If I go back to my azure DevOps build pipeline, I can see my pull request build passed:

Github CI results are pulled in from Azure DevOps

Additionally, I can also see the outcome in my Github checks tab too:

Green Build! Ready to merge!