Pull Request Checks to Next Level with Deploys

In the previous blog post, I introduced the so-called pull request builds. Pull request build is run in a pull request and acts as one of the reviewers. If the build fails, the pull request can’t be completed and the master branch will stay clean. In this blog post I will introduce the next level check for pull requests in Azure DevOps: pull request deploys. We do deploy from the pull request build’s artifacts and that deploy will be like one of the pull request reviewers. I like to call this pull request deploy.

What can deploy check in a pull request? First, it will check if the deployed web app will start. For example, with C# it can do a smoke test for Startup.cs and Program.cs files which are run when the app is started. Unit tests won’t do that. We can also run automated tests (medium/large sized tests like integration or UI tests) against the deployed app. That is something that a build can’t verify with unit tests. If the app is REST API we can run some REST API calls against it and check the results. Power is that we can run those bigger tests before merging to the master branch.

While builds don’t need any environment, deploys need some environment to deploy to. That makes this more difficult than pull request build. There should be some temporary like environment for deployment. For example, there could be a dedicated environment for pull request deploys and it is used only for them. Or we can create an environment in release pipeline and delete it after pull request deploy and tests are done.

Initial Azure Pipeline

Here is our simple REST API code in C#. It is a simplified version of Visual Studio’s already simple template.

[Route("api/[controller]")]
[ApiController]
public class ValuesController : ControllerBase
{
  // GET api/values
  [HttpGet]
  public ActionResult<IEnumerable<string>> Get()
  {
    return new string[] { "value1", "value2" };
  }
}

We have also a build (and release) pipeline with YAML. It will first build the REST API (lines 8-25) and then deploy it to Azure (lines 27-53). In the deploy stage, it will “ping” the deployed app service to check that it has been deployed successfully. It is done with the curl command (lines 45-53).

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

stages:
- stage: 'Build'
  displayName: 'Build stage'
  jobs:
  - job: 'Build'

    steps:
    - task: DotNetCoreCLI@2
      displayName: 'Restore, build and publish'
      inputs:
        command: 'publish'
        projects: '**/*.sln'
        publishWebProjects: false
        arguments: '--output $(Build.ArtifactStagingDirectory)'
        zipAfterPublish: true

    - publish: '$(Build.ArtifactStagingDirectory)'
      displayName: 'Publish build artifacts'
      artifact: drop

- stage: 'Deploy'
  displayName: 'Deploy stage'
  dependsOn: Build
  jobs:
  - deployment: Deploy
    environment: dev
    strategy:
      runOnce:
        deploy:
          steps:
          - download: current
            artifact: drop
          - task: AzureWebApp@1
            displayName: 'Azure App Service Deploy'
            inputs:
              azureSubscription: 'pull-request-deploy-rg'
              appName: 'prdeploydemo'
              package: '$(Pipeline.Workspace)/drop/*.zip'
          - script: |
                   http_response=$(curl -s -o /dev/null -w "%{http_code}" https://prdeploydemo.azurewebsites.net/api/values)
                   echo $http_response
                   if [ $http_response != "200" ]; then
                       exit 1
                   else
                       exit 0
                   fi
            displayName: 'ping'

There is one thing we need to note! If we now configure pull request build (see the previous blog post about it), it will also deploy to that app service. If that is any app service that is in use, it can have any pull request’s code. I highly recommend doing the pull request deploy to some temporary app service that is not in use.

We have to add a condition to the deploy stage: deploy only if it is the master branch. This can be done by adding conditions to run only if the branch is the master. Added also displayName to make this more clear, especially in Azure DevOps web page.

- stage: 'Deploy'
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  displayName: 'Deploy to dev from master'

Now when the pull request build is run, it won’t anymore do the deployment to dev environment because the pull request build is not run from the master branch.

Deploy to Temporary Environment from Pull Request

To not mess the app that is in use, we will deploy to a temporary environment (named “prdeploy-temp”) from the pull request. We will add the following to our build pipeline:

- stage: 'PR_deploy'
  condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/pull/'))
  displayName: 'Deploy to temp'
  dependsOn: Build
  jobs:
  - deployment: Deploy
    environment: temp
    strategy:
      runOnce:
        deploy:
          steps:
          - download: current
            artifact: drop
          - task: AzureWebApp@1
            displayName: 'Azure App Service Deploy to temp'
            inputs:
              azureSubscription: 'pull-request-deploy-rg'
              appName: 'prdeploy-temp'
              package: '$(Pipeline.Workspace)/drop/*.zip'
          - script: |
                   http_response=$(curl -s -o /dev/null -w "%{http_code}" https://prdeploy-temp.azurewebsites.net/api/values)
                   echo $http_response
                   if [ $http_response != "200" ]; then
                       exit 1
                   else
                       exit 0
                   fi
            displayName: 'ping temp'

Significant changes are highlighted:

• condition, deploy is done only from pull request branches. Pull request branches are named like refs/pull/909/merge, so comparing to start with refs/pull/ will match.
• appName, deploy is done to “prdeploy-temp” named Azure app service. (From master we deployed to “prdeploydemo”)
• Ping script’s URL is changed to ping “prdeploy-temp” Azure app service.

Now when we commit changes, it will deploy to temporary environment, and not to dev environment:

When Deployment Fails…

Everything was OK so far, but the most important thing is to find errors before merging. We will simulate an error in pull request deploy by commenting out following code in Startup.cs:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
  if (env.IsDevelopment())
      app.UseDeveloperExceptionPage();
  //app.UseRouting();   // this is commented out
  //Other code...
}

This change makes our REST API not work at all but unit tests still pass. When we commit the change, and pull request runs build and deploy, there is an error:

When we click the PR Build we will see that build was OK but deploy failed:

When we drill into logs, we can see that ping (curl) to our temporary environment has failed:

Pull request deploy saved us from merging this broken code! We will now fix it and then we can do the merge into the master branch. When merging, it won’t deploy to the temporary environment because we configured (or coded) condition to deploy to it only from pull requests.

Conclusion

This was just a simple example of how the pull request deploy can be used. In this case, there weren’t any tests, just a ping call to detect if the REST API was working. In the real world probably we will run some integration or UI tests in pull request deploy and ensure that our deployed application works before merging. This gives us a huge advantage and there will be even less broken code in the master branch.

In the example, we used the existing environment to deploy the REST API. We could also use infrastructure as a code to create an environment on-demand and tear it down after pull request deploy has succeeded. This would save on environment costs in the cloud, but on the other hand, it will slow the deploy because creating a new environment takes some time. I haven’t yet tried this but I would like to try it.

Here we used the YAML pipeline but this is also possible with a classic pipeline. In that case, there should be a separate release pipeline to be used in pull request deploy and it needs some configurations. Here is documentation on how it can be done: Deploy pull request builds using Azure Pipelines.

This was a continue to my previous blog post about pull request builds. With these two, pull request build and deploy, we can decrease the fail rate of our builds on the master branch because so much is already tested before merging.

Helpful Links

• Deploy pull request builds using Azure Pipelines
• Specify conditions
• YAML schema reference