TAP has an awesome GitOps PR flow, but OOTB it does not work with Azure DevOps. In this post we will see how we can make it work with Azure DevOps as well.
Why Azure DevOps Is Difficult
While Azure DevOps does have git repositories, There are 3 main issues with the Azure DevOps Git implementation.
1. They have not actually implemented the Git v1 API and require the use of the “multi ack” protocol which is not implemented in go-git which is the main git implementation library in go which is used in nearly all gitops tools.
2. The URL of a git repository does not follow the standard path format like is implemented in github, gitlab, gitea, etc. which makes templating difficult.
3. Azure DevOps does not support cloning repositories with the stanmdard “.git” suffix added to the URL, which in many platforms and solutions is added automatically to the URL and with Azure DevOps it simply does not work.
Beyond these specific low level issues in Azure DevOps, but very likely due to these issues, the tool which is used in TAP to create PRs in the GitOps flow which is jx-scm from the JenkinsX project does not support Azure DevOps.
Time to find a solution
How to create a PR
When searching how to create a PR via the command line in Azure DevOps, I found that the official method is to use the Azure CLI, and then adding the Azure DevOps extension. The issue i found is that this would increase the size of the container image used to create the PRs by 1.2GB!!!!!!!!!!
With this being immediately disregarded as a possible solution for me, I decided that using the REST API via simple curl commands is probably the way to go.
While Azure DevOps is a clunky Git implementation, the REST API is actually very well documented, so finding the right API was actually really easy.
Once i found the right API, I took a look at the OOTB Tekton ClusterTask which is used for pushing commits and opening a PR, and updated the script to support Azure DevOps as well.
The change that was needed is in the final step, which is called “open-pr” and the new script value needs to basically, check if the git provider type is azure devops, and if so use our custom logic and otherwise, simply use the default as it comes.
The final script looks like this:
#!/usr/bin/env bash
set -o errexit
set -o pipefail
head_branch=$(cat /workspaces/ws/commit_branch | tr -d '\n')
token=$(cat $(credentials.path)/.git-credentials | sed -e 's/https:.*://' | sed -e 's/@.*//')
if [[ "$(params.git_server_kind)" == "azure" ]]; then
JSON_FMT='{"sourceRefName":"refs/heads/%s","targetRefName":"refs/heads/%s","title":"%s","description":"%s"}\n'
bodyJson=`printf "$JSON_FMT" "$head_branch" "$(params.base_branch)" "$(params.pull_request_title)" "$(params.pull_request_body)"`
repo_api_path=`echo $(params.repository_name) | sed 's|/_git/|/_apis/git/repositories/|g' -`
uri="$(params.git_server_address)/$(params.repository_owner)/$repo_api_path/pullRequests?api-version=3.0"
echo $bodyJson
echo $uri
base64AuthInfo=$(echo -n "test:$token" | base64)
result=$(curl -X POST -u "test:$token" -H "Content-Type: application/json" -d "$bodyJson" "$uri")
echo $result | jq -r '.repository.webUrl + "/pullrequest/" + (.pullRequestId|tostring)' > $(results.pr-url.path)
else
jx-scm pull-request create \
--kind "$(params.git_server_kind)" \
--server "$(params.git_server_address)" \
--token "$token" \
--owner "$(params.repository_owner)" \
--name "$(params.repository_name)" \
--head "$head_branch" \
--title "$(params.pull_request_title)" \
--body "$(params.pull_request_body)" \
--base "$(params.base_branch)" \
--allow-update 2>&1 |
tee stdoutAndSterr.txt
cat stdoutAndSterr.txt | sed -n -e 's/^.*\. url: //p' > $(results.pr-url.path)
fi
As you can see, we are simply checking the parameter which is supplied in the tap-values file called git_server_kind and if it is set to “azure” we use our own custom logic, and otherwise we use the logic provided by VMware using the jx-scm CLI.
Solving the Path issue
This actually doesn’t need to be solved, but what it requires is that when filling in the TAP Values, the repository_name variable under the gitops section for the supply chain config, must be in the format <PROJECT NAME>/_git/<REPO NAME>.
While this could be solved and most likely hidden from the end user in multiple different ways, It would require changes in too many places, it simply was not worth the effort as this is a global setting and is not a value that is needed to be configured by end users on a day 2 day basis.
Solving the lack of Git v1 API support
As mentioned above, Azure DevOps does not work with go-git which is the default git implementation used in TAP specifically but also is the standard use in nearly all go based solutions that integrate with git.
Luckily TAP does support configuring the git implementation to use, with the default being go-git, but also supporting libgit2.
libgit2 does include support for the multi ack protocol which is what Azure DevOps supports, so that issue can be solved by adding the variable “git_implementation: libgit2” under the ootb_delivery_basic top level key as well as under the configured supply chains top level key.
libgit2 does work, however it should not be used unless no other option is available, as it is much less stable then go-git. but in this situation, we do not have a real choice.
Solving the “.git” suffix issue
This issue unfortunately requires changes to be made in multiple different locations, as the OOTB templates, add the “.git” suffix to repo URLs when templating the resources out.
The templates we need to change are the ones that create the gitrepository CRs and the ones that create the delivery CRs.
This means we have 2 ClusterSourceTemplates to update:
- source-template
- delivery-source-template
And we also need to update the ClusterTemplate that creates the delivery resource:
- deliverable-template
The change in all of them is the same. We have 2 YTT functions in all 3 of these templates called “git_repository” and “mono_repository”.
In these function we simply need to remove the addition of the “.git” at the end of the functions.
Once you have completed these changes, you are ready to use the GitOps flow and PR flow with Azure DevOps.
Example GitOps section in TAP Values files for Azure DevOps
An example of the relevant sections in the TAP Values that you can use once the above steps are completed would be
ootb_delivery_basic:
git_implementation: libgit2
ootb_supply_chain_testing_scanning:
git_implementation: libgit2
gitops:
server_address: https://dev.azure.com
repository_owner: vrabbi
repository_name: test-az-devops-tap/_git/test-az-devops-tap
branch: main
commit_strategy: pull_request
pull_request:
server_kind: azure
commit_branch: ""
pull_request_title: ready for review
pull_request_body: generated by supply chain
ssh_secret: git-creds
Summary
While TAP OOTB currently doesn’t support Azure DevOps in the GitOps PR flow, as I hope you can see from this post, the extensible nature of TAP, makes such integrations relatively very easy to do.
Building this solution was a matter of about 1.5 hours of investigation and trial and error, which is not bad considering the intricacies in solving this particular solution which I hope again shows just how flexible the platform really is, and how easy it is to customize it to your own needs!