Getting started with Jenkins scripted pipelines (new automation system part – 15)

  1. Preface
    Hi all,
    As I continue with the automation project, I now got into a stage where I wish to be able to compose some pipelines with “variant” number of stages, meaning, I wish to have the flexibility to define a single pipeline, which will have a total of 7 stages (for instance), where upon some scenarios only a subset of these 7 total stages will actually run (for example, only the first 5 stages). From a short reading (and research) I came into a conclusion that the best way to achieve this “requirement” is to use scripted pipelines.  In this post I will demonstrate the basics settings and implementation required to compose a simple scripted pipeline that can have a variant number of stages.
  2. Preparations
    For the simple and minimal scripted pipeline I will demonstrate in this post I did not perform any specific installations to the entire environment.
    The Jenkins server I’m using is with version 2.204.2
  3. Common step capabilities
    After going over this answer regarding the differences between scripted and declarative pipelines, I came into decision that each Groovy class which will implement a specific stage of a pipeline will need to have the Script object (capability), so the base class I have composed (for now) for all further derived classes is as follows:

    package org.scriptedStages
    
    public class BaseScriptedStage
    {
    	protected final Script script
    	protected String stage_name
    
    	BaseScriptedStage(Script script, String stage_name)
    	{
    		this.script = script
    		this.stage_name = stage_name
        	}
    
    	public String get_stage_name()
    	{
    		return this.stage_name
    	}
    }
    
    1. The main member of this base class is the Script object (official Groovy documentation can be found here), which will enable each derived class (and the base class as well) “scripting” capabilities when its logic will be implemented.
  4. Shared library considerations
    As I mentioned in one of my previous post called Getting started with Jenkins shared libraries (new automation system – part 13) , the motivation behind why to places the Groovy files in a shared library repository and the structure of my shared library repository is as described in section 2. Following this structure, the Groovy files that will implement the different stages will reside under the /src/org folder of the shared libraries repository.
  5. Groovy objects as pipeline “stage”
    As I chose to implement the entire pipeline as a scripted pipeline –> it gives me the “flexibility” to “put” all the stage/step logic (and implementation) into (also) Groovy objects, thus making the majority of the pipeline implementation being “coded”. In this post I will implement and afterwards use in the pipeline, in two Groovy objects, each of which will implement a specific and “atomic” step I need for my pipeline.

    1. Git checkout stage (Groovy object)
      This object is required for the first stage of the pipeline which its role is to simply clone the repository (and a specific branch) into the Jenkins pipeline work space. It implementation is as follows:

      import org.scriptedStages.BaseScriptedStage
      
      public class CheckoutBranchFromAutomationRepoScriptedStage extends BaseScriptedStage
      {
      	protected final String repo_url = "ssh://git@somegitrepo.com:port/some/git/project/repo.git"
      	protected final String credentials_id = "some_SSH_user_name_with_private_key_credentials"
      
      	CheckoutBranchFromAutomationRepoScriptedStage(Script script, String stage_name)
      	{
      		super(script, stage_name)
      	}
      
      	void execute(String branch_to_checkout = null)
      	{
      		if (checkout_git_repo(branch_to_checkout))
      		{
      			println("repository:" + this.repo_url + " was checked out correctly")
      		}
      		else
      		{
      			println("was unable to checkout repository:" + this.repo_url)
      		}
      	}
      
      	private boolean checkout_git_repo(String branch_name)
      	{
      		println("about to checkout the following git repository:" + repo_url)
      		def branch_prefix = "*/"
      		def branch_to_checkout = "master"
      		if (branch_name != null)
      		{
      			branch_to_checkout = branch_name
      		}
      
      		branch_to_checkout = branch_prefix + branch_to_checkout
      		println("branch to checkout is:" + branch_to_checkout)
      		this.script.stage(this.stage_name)
      		{
      	    		script.echo "Triggering ${this.stage_name} stage"
      			script.checkout([$class: 'GitSCM', 
      	    				branches: [[name: branch_to_checkout]],
      	    				doGenerateSubmoduleConfigurations: false,
      	    				submoduleCfg: [],
      	    				userRemoteConfigs: [[credentialsId: this.credentials_id, url: this.repo_url]]
      					])
      
      		}
      
      		return true
      	}
      }
      
      1. Notes:
        1. As you probably noticed, this class derives from the BaseScriptedStage class.
        2. The “main” responsibility of this class (object) is implemented in the execute method (which is the actual API of this class) – it simply checks out the master branch from the repository which is kept as a class member, unless asked for a specific branch.
        3. In the private method that holds the actual “implementation”, you can notice the usage in the checkout method. This is the “heart” of this entire Groovy class.
    2. Create virtual environment stage (Groovy object)
      The second object (stage) I need for this simple pipeline is to create a virtual environment for my Python 3 project. This is implemented in the following Groovy class:

      import org.scriptedStages.BaseScriptedStage
      
      public class CreateVirtualEnvScriptedStage extends BaseScriptedStage
      {
      	private String virtual_env_name = "myPythonAutomationProject_env"
      	enum virtualEnvType
      	{
      	    VENV,
      	    VIRTUAL_ENV
      	}
      
      	CreateVirtualEnvScriptedStage(Script script, String stage_name)
      	{
      		super(script, stage_name)
      	}
      
      	// You can pass as many parameters as needed
          	void execute(String virtual_env_tool_to_use)
      	{
      		def virtual_env_enum = null
      		if (virtual_env_tool_to_use == "venv")
      		{
      			println("about to use " + virtual_env_tool_to_use + " to create the virtual environment")
      		    	virtual_env_enum = virtualEnvType.VENV
      		}
      		else if(virtual_env_tool_to_use == "virtualenv")
      		{
      		    	println("about to use " + virtual_env_tool_to_use + " to create the virtual environment")
      		    	virtual_env_enum = virtualEnvType.VIRTUAL_ENV
      		}
      		else
      		{
      			println("did not get a valid virtual environment tool")
      			return
      		}
      		
      		def created_correctly = create_virtual_env(virtual_env_enum)
         		if (created_correctly == true)
      		{
      			println("the virtual environemnt was created succsefully")
      		}
      		else
      		{
      			println("the virtual environemnt was NOT created succsefully")
      		}
              }
          
      	private boolean create_virtual_env(virtualEnvType virt_env_type)
          	{
              	String command = null
              	if(virt_env_type == virtualEnvType.VENV)
              	{
                  		command = "python3 -m venv " + virtual_env_name
              	}
              	else if(virt_env_type == virtualEnvType.VIRTUAL_ENV)
              	{
                  		command = "virtualenv -p python3 " + virtual_env_name
              	}
              	else
              	{
                  		println("got an invalid virtual environment tool type")
                  		return false
              	}
              
              	println("command to use is:" + command + ", in order to create the virtual environment")
      		this.script.stage(this.stage_name)
      		{
      	    		script.echo "Triggering ${this.stage_name} stage"
      			script.sh """pwd"""
      			script.sh """ls"""
      			script.sh """${command}"""
      		}
      
              	return true
          	}
      }
      
      1. Notes:
        1. As you probably noticed, this class ALSO derives from the BaseScriptedStage class in order to leverage the Script member class.
        2. Its role is to simply create a virtual environment with the desired tool according to the single argument provided to it.
  6. The actual scripted pipeline
    After the implementation of the object (Groovy classes) that holds the “code” for the different stages of the pipeline –> it is finally the time to implement the pipeline itself. Keeping things simple and minimal, the pipeline that will be discussed performs the following:
    – Executes two stages, which are implemented in the above two Groovy classes
    – Catches any exception thrown by any of them (if at all).
    – Finishes by cleaning the work space of the current run within a finally stage, which as its name state, will be executed anyways regardless of any error that might (or might not) occur throughout the pipeline execution. The entire imp,implementation of the pipeline is as follows:

    @Library("my-jenkins-shared-library")
    import org.scriptedStages.CreateVirtualEnvScriptedStage
    import org.scriptedStages.CheckoutBranchFromAutomationRepoScriptedStage
    
    
    node
    {
        try
        {
            new CheckoutBranchFromAutomationRepoScriptedStage(this, "Checkout repository").execute("master")
            new CreateVirtualEnvScriptedStage(this, "Create virtual environment").execute("venv")
        }
        catch (e)
        {
            echo "This will run only if failed"
    
            // Since we're catching the exception in order to report on it,
            // we need to re-throw it, to ensure that the build is marked as failed
            throw e
        }
        finally
        {
            echo "This will run anyways regardless of how the job terminated"
            cleanWs()
        }
    }
    
    1. Notes:
      1. The first line is the “importing” of the Jenkins shared library that holds the Groovy classes for the two stages of this pipeline.
      2. Lines 2-3 are the actual (more specific) import of each Groovy class that is used in this pipeline.
      3. The “entire” pipeline is within a try {} followed by catch {} and terminated with the finally {}. Within the finally “stage”, a cleaning of the work space is taking place. The cleanWs() method is a built-in method that can be used in Jenkins pipelines (scripted or declarative).
  7. Conclusions
    In this post the following were discussed:

    1. Introduction to Groovy classes and basic usage in the inheritance mechanism of the language.
    2. Usage in Jenkins shared libraries was described (again).
    3. Encapsulating a “stage” into a Groovy class by utilizing the Script built-in object.
    4. Basic implementation and semantics of a scripted pipeline.
    5. Usage in the finally {} of the pipeline to ensure some actions are taking place at the end of the pipeline regardless of its final “state”.Resources:
      a)  Good Q&A on StackOverflow regarding how to implement the post stage in scripted pipeline
      b) Extracting Jenkins stage into a shared library Q&A on StackOverflow

      The picture: Acueductos de Ocongalla, near the city of Nazca, Peru.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s