Release management, part2: Designing scalable and testable build pipelines with BPMN and Jenkins Job Builder

This blog post aims to illustrate the process of designing build pipelines with Business Process Modeling Notation (BPMN) that is both scalable and testable. The central premise of this blog post is that Jenkins build pipelines are business processes and BPMN closely aligns with Jenkins and its plugin ecosystem. For example, with some clever use of Jenkins plugin, gateways and swimlanes in BPMN can be supported and realised. In the context of release management, Jenkins provides the executable platform for business processes, which benefit tremendously from automation.

In addition, scaling, testing, version controlling and deploying BPMN build pipelines as Jenkins jobs can be achieved through adopting Jenkins API, Git, and Jenkins Job Builder. All Jenkins jobs in the pipeline are written and templated in YAML and deployed via Git (say via an update-hook) when changes are detected. This significantly sped up the process of implementing and stringing together complex Jenkins jobs. This approach allows for early pipeline verification and ultimately enable devops teams to utilise Jenkins a lot more efficiently with greater freedom and control over their own build pipelines.

A quick BPMN primer

Designing build pipelines for release management is akin to doing any other business process modeling. BPMN offers a rich set of visual language to kick start this process. An example of BPMN is as below. See http://tynerblain.com/blog/2006/07/27/bpmn-gateways/ or http://www.bizagi.com/eng/downloads/BPMNbyExample.pdf for more information. Figure below is from the first link as outlined.

BPMN with joins and forks

BPMN with joins and forks

Nodes in BPMN represent events, activities and gates. The edges represent the relationship and the flow. This is fairly simple and offers a rich set of modeling tool to capture a complete business process. In addition, swimlanes represent multiple parallel flows. There is a strong similarity between UML activity diagrams.

Events are the active objects and they captures the start, end and intermediate events (such as timer events). Activities are split into tasks or subprocesses. They are user defined. Finally, gates are predefined set of logical operators designed to control the flow of the tasks. For example, exclusive, parallel, inclusive gateway and etc.

Designing your own build pipelines

In the context of release management, build pipeline refers to a series of test automation, compilation and deployment. These are tasks that are carried out prior to making the product available to the intended audience.

In the previous blog post in the series, https://macyves.wordpress.com/2014/05/20/release-management-part1-test-automation-with-vagrant-and-jenkins-build-pipeline/, test automation made up of the bulk of the release management process in the start. Each test oriented join job triggered multiple parallel downstream jobs. Only upon successful completion of all downstream jobs can the join job proceed to the next. This is modeled by the parallel gateway, which are used to defined forking and joining points in BPMN as below.

Forked unit testing  jobs

Forked unit testing jobs

Jenkins and plugins

While the previous blog post in the series outline the required plugin. No real explanations were given as to why and when to utilise them. There are a couple of key plugins that enables forking and joining, and the locking of common resources; join trigger plugin and throttle concurrent build plugin respectively. For example, the use of join trigger plugin is key to implementing parallel gates.

Once the plugin is installed, using it is as easy as accessing the usual Jenkins UI. The join plugin wiki page contains all the detailed information. https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin

For more information on the throttle plugin, see https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

Roll out your build pipelines with Jenkins UI? NO!!

Armed with Jenkins and a complete pipeline design in BPMN, one is ready to start defining Jenkins job in the UI. However, avoiding the UI all together is strongly recommended at this stage. The reason is that despite the convenience supplemented by Jenkins UI, fiddling around with it s both time consuming and error prone. It is most certainly not scalable if you have multiple people working on the same build pipeline model working in a clunky UI environment.

One solution to the problem is to use Jenkins API and define all jobs in XML. However, arguably it is just as clumsy and error prone. The XML model supported by Jenkins maps to its very own internal object models. To utilise it fully one would require a complete understanding of the underpinning relational model used by Jenkins.

Roll out your build pipelines with Git, and Jenkins Job Builder? Yes!

As with all things in software development life cycle, release management processes need to be repeatable, scalable and resilient. The following are the non-functional requirements for the build pipeline.

  • Testability and debugging support on non-production Jenkins setup
  • Tracking changes and rollback
  • Scalability

Enter Jenkins Job Builder. Jenkins Job Builder is a fantastic tool for writing Jenkins job. The supported format is YAML and is human readable. Further more, templating is supported, which translates to common job definitions and job reuse. For example, you could roll out multiple build pipelines based on branches of the source code. Ultimately, you can very easily setup a personalised build pipeline for each developer and their own personal branch with the same set of job templates. The power of Jenkins Job Builder is astounding as it supports a myriad of Jenkins plugin. Moving away from a UI to a text file oriented way of interacting with Jenkins means productivity increases dramatically and that build pipeline skeleton can be tested.

Coupled with Vagrant, using Jenkin Job Builder allows for the same set of Jenkins jobs to be deployed on any Jenkins server. The implication is that in theory debugging Jenkin related issues or test failures can be easily provisioned and reproduced in a VM. This means that anybody on the team can fire up a fully fledged release management platform with almost all the production trimmings. See previous blog entry for the Vagrantfile that is capable of provisioning the exact Jenkins master node required for building and testing the packages. For more information on Jenkins Job Builder, see http://ci.openstack.org/jenkins-job-builder/

Last but not least, integrating Jenkins Job Builder with Git via the update-hook mechanism would enable automated deployment of newly pushed YAML to production Jenkins. Upon failure, one can simply roll back or just discard that commit. Complete freedom and control.

Testability and debugging support on non-production Jenkins setup

Before mapping out all the detailed scripts and various packaging steps, create an overarching build pipeline template. Run it through and see that all the join jobs triggers the correct downstream jobs. This is essentially functionally testing and verification of the build pipeline itself. This template can of course be easily reused and so on. Furthermore, a Vagrantfile would help the developers create their own build server.

Tracking changes and rollback

By utilising Git, one would immediately gain the ability to track changes in the YAML files and rollback if needed. The content of the Git update-hook script is entirely up to you.

Scalability

Write the YAML as templated as possible. Use parameters and variables supported by Jenkins Job Builder. Scaling out different variants of the same build pipeline should be as easy as defining a new “project” or “job-group” in Jenkins Job Builder.

Examples

If you have read this far, you must be wondering if this is all too good to be true. Answer is yes. However, since each build pipeline is very specific to the product and release processes, these samples from OpenStack is a great starting point. https://github.com/openstack-infra/jenkins-job-builder/tree/master/samples

Advertisements

2 thoughts on “Release management, part2: Designing scalable and testable build pipelines with BPMN and Jenkins Job Builder

  1. Pingback: Release management, part 3: Supporting and scaling idempotent, isolated, and parallel jobs in Jenkins build pipeline with Docker | something else

  2. Pingback: Getting rid of monoliths: #lagom and microservice hooha | something else

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s