Controlling Branches in Azure Devops Pipeline

 In my earlier post (https://arunvambur.blogspot.com/2021/11/building-master-pipeline-for.html) I had showed how to build the master-child pipeline in azure pipeline. Whenever there is a code commit in service pipeline, the azure pipeline gets triggered and invokes the master pipeline. The master pipeline contains many stages and many steps. How are we going control the stages to run only for certain branches? We don’t want to run every stage for every branch when there is trigger for that branch. For example, it is not useful publishing artifacts when there is a trigger in ‘feature’ branch. So how to skip the publish task from the master flow. This article describes about controlling the pipeline based on the branches that got triggered.

Branching strategy

There are many different strategies exists when it comes to version control, such as branching, merging and releasing the code. The most popular are GitFlow (https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow), GitHubFlow (https://guides.github.com/introduction/flow/) and ReleaseFlow(Release Flow: How We Do Branching on the VSTS Team - Azure DevOps Blog (microsoft.com)). Each flow has their own set of rules on when/how to create a branch, merge code between the branches and manage release etc. In this article we have our own strategy which is loosely based on ReleaseFlow.

The below diagram show the flow diagram,



Figure 1 - Release Flow

In our flow we have three branching strategy, ‘main’, ‘feature’ and ‘release’.  The ‘main’ branch is where all the feature code is merged. The main branch is our lifeline for our entire flow. Any new feature or code changes, a new feature branch ‘feature/*’ is created. ‘*’ indicates a short name for the feature. Once the feature branch is completed the code is merged and probably the branch is deleted. The release may happen automatically end of each sprint or it is based on business decision when to do a release. In either case a new release branch ‘release/*’ is created. ‘*’ indicates the release version number.

Why should we control the flow?

-          Not every branch has the same flow

-          It reduces the overall execution time of end-end devops process

 In our pipeline we configure to trigger where there is a code commit or pull request for all three branches. We have many stages in our pipeline, but we don’t want to run all the stages for all the branches. The below diagram shows the different stages for the branches,



Figure 2 - Branching Strategy

In the Feature branch we have two stages, Build and Unit Test.  We would like to run the pipeline when a developer commit his code to repository. Every time the code is committed, a developer may be interested in knowing the code gets compiled and build without any issues and it also run all the test cases. There might not be much useful to publish the code at this stage since the code is still under development. Next we have the Main branch were we need to run both the Build and Unit Test along with that we also need to publish the artifact. Finally we have the Release branch we need to run every stage and deploy our artifact somewhere.

Note: For simplicity there are other stages I haven’t considered here. For example in DevSecOps we have stages like code scanning for any security vulnerabilities and in containerization process we will be having image building, helm charts for k8s etc.

 The below flow chart shows how each stage get executed based on the conditions,



Figure 3 - Pipeline Flow Chart

Back to service configuration file

In our sample pipeline we have service-config.yaml file for all our service level configurations. We extended this file to include the branch level control as well.

Below is our configuration in our service-config.yaml,

build:

  branches:

    feature:

      build: true         # drives the build

      unitTest: true      # drives the unit test

      publishToLocal: true  # publish the build to local

      publishToAzure: true  # publish the build to azure artifactory

      deployToAzure: true # deploy the build to azure

    main:

      build: true        

      unitTest: true     

      publishToLocal: true 

      publishToAzure: true 

      deployToAzure: true

    release:

      build: true        

      unitTest: true     

      publishToLocal: true 

      publishToAzure: true 

      deployToAzure: true

Controlling the stages

Controlling the stages is by checking each variable is set to true and assign the corresponding branch name. This is performed in ‘read-service-config.yaml’. Part of the reading configuration code is shown below,

####################

#Read branch parameters

 

if($args[1] -eq $True)

{

    $branch="feature"

}

if($args[2] -eq $True)

{

    $branch="main"

}

if($args[3] -eq $True)

{

    $branch="release"

}

 

$value = $parsedYAML.build.branches.$branch.build

Write-Host "##vso[task.setvariable variable=pipeline.build;isOutput=true]$value"

 

$value = $parsedYAML.build.branches.$branch.build

Write-Host "##vso[task.setvariable variable=pipeline.build;isOutput=true]$value"

 

$value = $parsedYAML.build.branches.$branch.unitTest

Write-Host "##vso[task.setvariable variable=pipeline.unitTest;isOutput=true]$value"

 

$value = $parsedYAML.build.branches.$branch.publishToLocal

Write-Host "##vso[task.setvariable variable=pipeline.publishToLocal;isOutput=true]$value"

 

$value = $parsedYAML.build.branches.$branch.publishToAzure

Write-Host "##vso[task.setvariable variable=pipeline.publishToAzure;isOutput=true]$value"

 

$value = $parsedYAML.build.branches.$branch.deployToAzure

Write-Host "##vso[task.setvariable variable=pipeline.deployToAzure;isOutput=true]$value"

 

The next process is we should know from which branch the pipeline got triggered. The below code form service-pipeline.yaml shows the same,

variables:

  featureBranchTriggered: $[startsWith(variables['Build.SourceBranch'], 'refs/heads/feature/')]

  mainBranchTriggered: $[eq(variables['Build.SourceBranchName'], 'main')]

  releaseBranchTriggered: $[startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')]

 

The flag mainBranchTriggered, featureBranchTrigged and releaseBranchTriggered will capture if the respective branch got triggered and set the variable to true.

 

Summary

The sample code can be downloaded from github (https://github.com/arunvambur/AzurePipeline/tree/main/Controlling%20Branches). This article shows a simple pipeline with very minimal stages to show how to control the different stages for different branches. This pattern can be extended for complex stages.

Comments

Post a Comment

Popular posts from this blog

Debugging and Testing Helm Charts Using VS Code

Image
Introduction Helm charts are useful to manage, install and upgrade Kubernetes applications. Helm is a developer friendly templating Kubernetes resource files based on GO templates and YAML. The templates are written using any notepads and deployed using Helm. Writing helm commands on notepad is not user friendly. Using Visual studio code we can setup a workaround that will enable to code faster and see the generated Kubernetes YAML files.  The following document has the details of necessary steps and guide that will help you to debug and test your helm charts before directly installing in the Kubernetes cluster using Visual Studio Code. Download and configure Helm in windows Download the Helm.exe for windows from the link  https://github.com/helm/helm/releases  or  https://get.helm.sh/helm-v3.5.0-windows-amd64.zip . Unzip and copy the contents to some directory like 'C:\Users\<username>\Tools\Helm'. Configure the above directory URL to environment var...
Read more

Handle Multipart Contents in Asp.Net Core

Handle Multipart Contents in Asp.Net Core Introduction When we program a web API in ASP.Net core we usually we have the Content-Type header as ‘application/json’ if it is a REST API. Assume a scenario where we need to pass an image along with description about the image or metadata. We can either pass that information in query parameters or in headers, but query parameters or header has limited in size. If the metadata is a json object, then query parameters or header is not a best place to pass. How about sending both the json and binary image within the body itself. There is a Content-Type ‘multipart/mixed’ in HTTP protocol is where we can achieve this. This article shows you how to pass both the json data and the binary in the same http request. Content-Type=multipart/mixed Header In the below example I will show how to use http header content-type as ‘multipart/mixed.’ The purpose of the Content-Type field is to describe the data contained in the body fully enough that the ...
Read more

Validate appsettings in ASP.net Core using FluentValidation

Image
Introduction In the ASP.net core all the application configurations are maintained in appsettings.json. The appsettings.json contains many settings like database connections strings, logging settings, and authentication settings etc. Applications settings are injected into the application as IConfiguration. Application uses these settings to perform certain tasks. For example establishing a connection to database. What happens when you update a wrong database connection string settings in your appsettings.json file? Shouldn’t we validate the settings before establishing a connection to database? This article shows you how to validate the appsettings.json while application starts. Why should we validate appsettings.json? There are some benefits validating appsettings.json, ·        Centrally manage the validations Applications settings are consumed in many parts  of the application. The validations of these settings are scattered across the code. Instea...
Read more