Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
Apr. 18th, 2019

How to Scale Selenium Tests in a Kubernetes Cluster

We live in a world where nothing sits still. In today’s rapidly changing IT world, companies strive towards scalable and fault-tolerant distributed systems, and not only in regards to software development. A lot of digital transformations are going on in the Quality Assurance area as well. Companies are ‘shifting to the left’, focusing more on how they can automate their test processes to make them quicker, easier to configure and more efficiently deployed.

 

Selenium Grid: Advantages and Disadvantages

 

When it comes to web-application automation, Selenium Grid has become the first fundamental part of the revolutionizing tool sets, allowing to perform Selenium tests on different machines and with multiple browsers at the same time. In the good old days you would just plug a couple of computers together, run a couple of Selenium agent scripts on each of those machines and that was that. But nothing stays relevant forever in its initial form, and the wildly popular automation solution ‘Selenium Grid’ has been given a breath of fresh air. But before jumping into the most interesting part, let’s think about what was not perfect in Selenium Grid’s infrastructure in its original form:

 

  • The Grid has an ‘Achilles heel’ - Selenium Hub: The hub-and-node concept has a serious weak spot - Selenium Hub. Selenium Hub is a single browser access point that makes all nodes unavailable when the hub goes down.
  • Selenium Grid doesn’t scale well: It’s a widely known fact that the Selenium hub can only handle a limited number of connected nodes.
  • Stale infrastructure for ongoing test runs: Selenium Grid nodes usually end up in a poor state left up and running for quite a long time. Hence, the node restart is exactly what the doctor ordered, but it can only be done manually.
  • Frustrating version synchronization: Keeping the project up-to-date with the latest Selenium release and having it synchronized with the browser and webdriver version can cause difficulties and takes ages. Grid maintenance and updates are a time consuming node-by-node process.

 

Introducing Zalenium

 

This is where Zalenium comes in! According to its official page, Zalenium is a flexible and scalable container based Selenium Grid, with video recording, live preview, basic auth & a dashboard. Moreover, it has out-of-the-box Docker and Kubernetes integration. This fact makes Zalenium an attractive choice to get Selenium based infrastructure up and running.

 

When you dig into Zalenium, you will realize all the benefits it brings to web automation.

 

Selenium/Zalenium -Kubernetes Integration

 

Out-of-the-box integration with Kubernetes significantly reduces test execution time by spinning up Docker containers of the necessary web browsers and running them in parallel. This actually means horizontal scaling of your tests. Moreover, it lets you avoid the headache of Grid set up and maintenance. Therefore, you don’t have to deal with containers/pods directly. Instead, Kubernetes automatically manages the desired state of the system.

 

Running Zalenium in a Kubernetes cluster becomes a powerful solution to get Selenium based infrastructure up in a more efficient and simpler way. This is true especially if you’ve already orchestrated, shipped and run the company’s software with Kubernetes.

 

Let’s dive into Zalenium and take advantage of its integration with Kubernetes together. The desired configuration is represented in the diagram below.

 

Kubernetes Cluster configuration on the local machine


Kubernetes Cluster configuration on the local machine

 

Creating our Selenium based Zalenium-Kubernetes Test

 

The main prerequisite for this configuration is to have a running Kubernetes cluster. To experiment with Kubernetes locally, you can use Minikube. Minikube is a tool that allows spinning up a single-node virtual cluster on your machine.

 

The easiest way to set up Minikube on a Mac is to execute the following brew command (the installation steps for other platforms you can find here):

 

code 1

 

Alright!

 

code2  

 

In order to interact with the cluster, we can trigger the Kubernetes API via Kubectl. This tool allows running containerized applications on top of the Kubernetes cluster. If you setup Minikube with the command above, it downloads Kubectl as a dependency during the installation process.

 

code3

 

You just need to make sure that the sufficient version has been installed:

 

code4



For example, the Kubectl version I use is

 

code5

 

In case of separate Kubectl installation, use the instructions provided in the official documentation.

 

After we have all the necessary tools installed, we can instantiate our own Kubernetes cluster.

 

Just run the following:


minikube start

 

It will start a local Kubernetes cluster:


starting local kubernetesS
 

Check the status and IP of the cluster:

minkube status

 

Running Zalenium on Top of the Kubernetes Cluster

 

In order to spin up Zalenium in Kubernetes, we need to pull Zalenium and Selenium Docker images inside the cluster. For this to be possible, we should switch Docker to Kubernetes context by executing the docker-env command:


minikube docker

 

This will provide a cmd to use Docker from the command line of your local machine against the Kubernetes VM.

 

eval

 

Let’s download the Selenium and Zalenium Docker images for spinning them up in the cluster.

 

docker pull

 

As was mentioned before, guys from Zalando already prepared all necessary scripts to run Zalenium with Kubernetes using a one line command.

 

All the ‘magic’ scripts and configurations are located in the Zalenium GitHub repo:

 

git clone

 

So, create a resource from the cloned files:


kubectl create

 

It will start Kubernetes pods under a namespace ‘Zalenium’.

 

persistantvolume

 

Check that the service is up by listing all the container images in the cluster.

 

getpods

 

Moreover, you can open the  Kubernetes dashboard to verify all the running services under the ‘Zalenium’ namespace:

 

minikube dashboard

 

kubernetes dashboard

 

 

An Advanced (alternative) Way to Start Zalenium in Kubernetes using Helm

 

In order to simplify deploying applications to Kubernetes, we can use its own package manager  Helm, which helps with the installation and maintenance of any software on the cluster.

 

After Minikube is set up, install the Kubernetes package manager.

 

helm

 

Initialize Helm on both client and server sides


helm2

 

Helm packages, aka charts, might be installed in several ways. Zalenium developers have already provided a chart that bootstraps a Zalenium deployment on a Kubernetes cluster.

 

Navigate to the helm directory in the cloned zalenium repo:


cd zalenium

 

The chart’s installation command might include arguments for Zalenium customization. For example, in the string below the desired number of pods is increased to 3.


helm install

 

name my release

 

Great! All Zalenium pots are running on top of the Kubernetes cluster!

 

Web-based Test Execution with Zalenium

 

After you set up Zalenium, open the service zalenium/zalenium in the default browser via the command:


minikube service zalenium --namespace zalenium

 

You can get a URL of the service deployed on Kubernetes by the following command:


minikube zalenium2

 

Now all you need is to point the Selenium webdriver configuration to use this URL in order to run your tests with Zalenium.

 

If you want to practice Zalenium with an already existing test framework, let’s use the project from Yuri Bushnev ‘s blog about Top 15 UI Test Automation Best Practices.

 

You just need to modify the base.properties file in order to integrate the framework with Zalenium.


broswer chrome on grid


Now you are ready to run Selenium tests with the new configuration:


mvn clean verify

 

Zalenium also gives a great possibility of a live preview of your running tests. Moreover, you can interact with the dockerized container via VNC to watch a life stream of the test during the execution process. It can be accessed at the URL: 

 

http grid live

 

And it will look like this:

 

VNC Viewer

 

 

Moreover, Zalenium with its nice ‘video/logs recording’ feature enables to monitor the test from a browser. Just check the dashboard by the URL:

 

http dashboard

 

zalenium dashboard

 

 

Congratulations! You’ve just practiced Selenium tests running on the local Kubernetes cluster using Zalenium! But please notice that Minikube is a limited version of Kubernetes. This essentially means that you can not really scale your Selenium Grid by running Minikube (It’s restricted by the power of your local machine). But the installation process of Zalenium on Kubernetes is the same as on Minikube, so you can get a good understanding of this workflow.

 

The best way to run your test would be to deploy Kubernetes on a cloud provider like AWS and enjoy a  real cloud-native scalable Selenium Grid. If you are ready to try out a cloud service,  Amazon EKS User Guide has plenty of resources to get you started.



The combination of Zalenium and Kubernetes has become a powerful solution for testing web-based applications in a more efficient manner. In the QA space test automation is a rapidly growing area and shifting more towards a cloud-native approach.

 

Use BlazeMeter’s End User Experience Monitoring to easily run Selenium tests as part of a load test, to get insights into your front end performance under load. You can either add your URL after uploading your load testing script and we will run it in Selenium for you, or you can upload a Selenium script. The test results enable you to monitor your front end performance under a load: Get combined KPIs for HTTP/JMeter response times and for GUI page load times. Analyze requests in a Waterfall chart with webpage screenshots to get a complete client side breakdown.

 

Start testing now
 

You might also find these useful:

Interested in writing for our Blog? Send us a pitch!