Vincenzo Marrazzo is a Test Automation Specialist. He has over 14 years of experience in various contexts both with open-source technologies and commercial ones. His primary activities are Test Automation and Performance Test. Vincenzo currently works at Capgemini Engineering Italy.

Become a JMeter and Continuous Testing Pro

Start Learning
Slack

Test Your Website Performance NOW! |

arrowPlease enter a URL with http(s)
May 03 2021

How to Download & Install Jenkins CI From Scratch

Jenkins-CI is one of the most popular Continuous Integration (CI) tools. One of its main benefits is the flexibility it enables when configuring a Continuous Integration / Continuous Testing pipeline. This article will explain how to install Jenkins. I will provide practical examples for different Jenkins environment configurations according to use cases: for personal debugging, for your department and for Cloud deployment. Moreover, I will explain about resources consumption and which plugins are a "must have" in every installation.

But first, let’s understand why we even need Jenkins.

 

Why Do We Need Jenkins? 

Which kind of activities can be executed with Jenkins? In theory, any SDLC activity (except assigning requirements, reading/ understanding requirements, coding requirements, coding unit/ component/ integration tests and committing code to version control).

By properly defining a development strategy and configuring Jenkins accordingly, Jenkins can automatically handle many "accessory" phases of software governance (e.g. release). This flexibility is enabled by the rich plugin ecosystem, continuously developed on Jenkins. 

Moreover, the Jenkins platform allows defining a multi-executor architecture to answer various potential needs. Two of most simple architectures can be described as:

  • Horizontal - many executors collaborate to handle parallel tasks
  • Vertical - Customized executors are dedicated to specific tasks

This provides both scalability and process isolation. Together with automation, this speeds up a lot of development processes and helps with code quality.

 

Installing Jenkins Based on Your Development Needs

Jenkins’s flexibility enables deployment and integration for multiple usage scenarios. Let’s see which ones.

 

Personal Debugging

Installing Jenkins for personal debugging requirements can be useful during the early development stage of the CI chain. For example, when we need to provide practical evidence using a Proof of Concepts (POC) on how CI introduces Return Of Investment (ROI) on development tasks.

Moreover we might have a CI chain already in place and in use, but we want to verify if the latest Jenkins build resolves bugs or introduces new expected features. It’s not recommended to alter production by adding a new Jenkins version without a preliminary investigation via a personal debug. 

The simplest way to install Jenkins is using the WAR file.

Follow these steps:

  1. Install Java Runtime Environment
  2. Ensure that java command is available on our shell
  3. Download Jenkins WAR file
  4. Execute command

java -jar jenkins.war

Now we have a running Jenkins instance running on our notebook ready to be used. This installation has the following features:

 

Pros

  • quick to start/reset
  • every configuration file is on a local path

Cons

  • resource shared on the entire OS
  • few tasks in parallel 
  • firewall can be an issue
  • 3rd party software

 

Typically this kind of installation does not include additional executors in addition to the main instance, because resources are limited.

 

CI/CD for the Engineering Department 

Let’s say our CI process successfully completes the prototype stage and already produces important feedback that it shares procedures/reports with our coworkers. Let’s also say we have up to ten CI processes, each one focused on different aspects of the same product.

However, the increasing complexity of CI chains is a new challenge for reliability and repeatability. Two tools can help us:

  • Jenkins pipeline - for simplifying complex coding CI processes with a smart Domain Specific Language (DSL) derived by Groovy
  • Version Control Management integration tools - for natively fetching customized pipeline code from the Git repository.

To include all the abilities listed above, follow these installation steps:

  1. Ensure you have a server that can easily be controlled from the department
  2. Ensure you have the necessary software installed (e.g. JRE and Git)
  3. Create a working directory for Jenkins
  4. Download the Jenkins WAR file
  5. Define an environment variable JENKINS_HOME with the path from point 3

JENKINS_HOME=/opt/jenkins_home

  1. Execute the command:

java -jar jenkins.war

Now we have a running Jenkins instance running on our server, ready to be used for our department activities. This installation has the following features:

 

Pros

  • quick setup 
  • backup and restore policies for Jenkins VM
  • running many parallel tasks according resources
  • 3rd party software and hardware defined properly

Cons

  • preparation of server machine in lab
  • might require additional IT skills for setting up the machine

 

In this type of installation, the Jenkins instance becomes a part of your shared services over the network and it can interact with others, for example by using Git to retrieve pipeline code. Therefore, you can pair Jenkins with executor instances to obtain various goals, like assigning an executor with close coupling with a hardware device. You might need your IT colleagues to resolve issues like server configuration.

Let’s see how to link executors to your Jenkins installation. The Executor is more and less a remote Java application on an external machine that interacts with Jenkins via socket (default value 50000). To install an executor we need two prerequisites:

  • an external machine with the right network rules (e.g. allowing port 50000)
  • Java Runtime Environment installed

Proceed as follows:

  1. Manage Jenkins -> Manage Nodes - New Node
  2. Insert an identification name for the node and select “Permanent Agent”
  3. Fill “Remote root directory” field (e.g. “/var/jenkins_home” for a Linux executor)
  4. Click on the name of the newly created node in the node list panel
  5. Find a command like this

java -jar agent.jar -jnlpUrl http://<jenkins_ip>:8080/computer/<executor_name>/slave-agent.jnlp -secret <secret_token> -workDir "/var/jenkins_home"

  1. Enter the executor machine shell and launch the command from point 5
  2. After a while, you will see the Jenkins instance availability of the new executor

 

Data Center Deployment

When our CI chain reaches a high level of maturity, the next step is to optimize business continuity and reduce the related operational costs by migrating Jenkins to the data center. This part typically involves extended support from the IT department, because the Jenkins installation becomes a corporate asset that requires Business Continuity policies (e.g. how to backup storage, server redundancy, scheduled update, etc.).

A necessary step will be to add monitoring facilities to the Jenkins installation. This will enable catching abnormal operations that might negatively affect the service. An example can be the Monitoring plugin, but there are many more.

The IT department can require technical support from the owner of the CI chain into early phases to provide accessibility to two kinds of software assets related to Jenkins-CI installation:

  • configuration files necessary to execute tasks (e.g. config.xml into each task folder)
  • information/documentation to configure the necessary modules, both internal (e.g. plugins) and external (e.g. custom Python shell, Git ssh key, etc.).

After completing the necessary migration to the data center, we will have a running Jenkins instance integrated into the corporate infrastructure:

  • listening on http://jenkins.<corp.domain>:8080
  • available everywhere using a proper VPN
  • with an existing task ready to be used
  • ${JENKINS_HOME} folder handled by IT department

 

Pros

  • maintaining the Jenkins instance by corporate IT
  • backup and restore policies dedicate to Jenkins configuration files
  • many parallel tasks according resources
  • integration with corporate services (e.g. LDAP, VPN)

Cons

  • such important improvement requires planning with IT

 

It’s important to also migrate the executors to avoid a conflict with the new Jenkins instance.

 

Serverless Deployment in the Cloud

Like many other generic applications, Jenkins can also be executed without instantiating a dedicated server like the Data Center resources. Cloud Service Providers (CSP) can provide Infrastructure as a Service (IaaS) without affecting operational costs related to Jenkins machines. The blazing advantage is reduced operational costs and without maintaining a machine!

The key to approach a serverless deployment is virtualization based on containerization. The same Jenkins instance does not see differences if it is:

  • installed on a real machine
  • or deployed on a virtualized environment that mimics a real machine

The latest scenario will use less resources and share hardware assets of various applications (e.g. machine, network connection, disk). 

To give a raw idea about cost saving on machine cost, take into account that renting a virtual machine can cost dozens times more than renting a single container at the same time. Moreover, containers natively support a redundancy strategy, to handle high volume and/or failure recovery. 

To provide an idea of the complexity of handling containers directly as an IaaS, let’s look at the Azure Cloud Shell. Make sure you’re subscribed to Azure services and that Azure Cloud Shell appears like a Linux-like shell. To obtain a running instance of Jenkins it is necessary to execute: 

az container create --resource-group DockerGroup --name jenkins --image jenkins/jenkins:2.235.1-lts --dns-name-label jenkins --ports 8080

After execution, we need to identify the public host to interact with Jenkins as usual. We can launch the command:

az container show --resource-group DockerGroup --name jenkins --query "{FQDN:ipAddress.fqdn,ProvisioningState:provisioningState}" --out table

As the reader can understand, this command runs Docker. Docker shell is quite straightforward and enables deriving necessary commands in the Cloud shell (e.g. add a volume). There are some restrictions with Docker but they are not relevant for our kind of application.

 

Pros

  • maintains Jenkins instance with corporate IT
  • maintains Jenkins machine with the Cloud Service Provider
  • provides backup and restore policies dedicates to Jenkins configuration files
  • running many parallel tasks according to resources
  • integration with corporate services (e.g. LDAP, VPN)

Cons

  • such an important improvement requires planning with IT
  • requires knowledge of a solution based on IaaS

 

What about executors? The executor can be ported to the CSP using a dedicated container. A more interesting scenario is when the executor continues to stay in the department network while the Jenkins instance is on the Cloud. In this latter scenario:

  • the department network must allow connecting from the Jenkins cloud instance
  • the executor requires a first time installation to the new Jenkins instance

 

How to Tailor Your Jenkins Installation with Plugins

The Jenkins plugins ecosystem is rich and answers many use cases. Identify which plugins work well and can help you with your needs. For example, the Extended Choice Parameter enables creating a dynamic parameter that fetches data directly from an external system (e.g. a list of files from the FTP folder). But, if network resources are saturated, the user experience for Jenkins users is poor and could result in an underestimated number of files in the remote folder. 

The following is a list of must have plugins for any scenario. Make sure you understand them before rushing to install them.

  • Credentials - credential sharing among tasks
  • Email Extension - personalised email formatting
  • Extended Choice Parameter- increases flexibility on task parameter creation
  • Git - allows interaction with the Git repository
  • Lockable Resources - introduces a layer of bookable resources shareable among tasks
  • Pipeline - allows developing tasks as a software asset with incredible flexibility

For task execution, you can execute sub processes but these might affect resources shared with Jenkins. The solution is to demand computational load for an executor, which handles the entire task with separated resources. 

Conclusion

Jenkins' killer feature is flexibility, and the installation procedure is no exception. Perhaps the real difficulty is deciding which kind of installation we need. Try to identify which of the proposed scenarios is the closest to what you need, and adapt missing parts. An incremental approach is probably the best way to familiarize yourself with Jenkins step by step.

You can also run your BlazeMeter performance tests in Jenkins. Get your free sign up.

 

   
arrowPlease enter a URL with http(s)

Interested in writing for our Blog?Send us a pitch!