Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
Oct. 20th, 2016

Headless Execution of Selenium Tests in Jenkins

As you probably know, Continuous Testing is becoming a more and more important part of the software development life cycle. Since any change to the product needs to be verified prior to going to production, the idea behind Continuous Testing is to make the whole workflow from a single commit to publishing the new release, 100% automated, and as a result - unattended.


When it comes to functional testing of web applications, the same rule should be applied. In addition to backend unit testing and performance testing, it makes sense to run a set of automated functional tests on the client side, to ensure that everything is working fine, all elements are visible and operating correctly, rendering speed is acceptable and so on. As of now, the most popular and powerful tool for browser automation is Selenium - a free and open source browser automation framework.


We already covered the point of having Selenium automated tests as a part of the Continuous Delivery process in the How to Automate Testing Using Selenium WebDriver, Jenkins and Allure post. Today’s article is about running Selenium tests in “headless” mode, i.e. when no browser window is being displayed.


The importance of having “headless” tests is:

- The majority of Linux server deployments don’t have GUI at all

- The same applies to Windows Core Servers

- When it’s not possible to open the browser in the foreground, for example when the build agent is being used for desktop applications testing which require focus, or you would like to continue using your machine normally while Selenium tests are in progress


Today you will learn how you can launch Selenium tests when Jenkins is in “headless” mode.


Quick Solution - Switch to Headless Selenium WebDriver Implementations


The fastest and the easiest way would be switching to “headless” Selenium WebDriver implementations such as:


- HtmlUnitDriver - wrapper for HtmlUnit - a Java-based browser implementation

- PhantomJSDriver - wrapper for PhantomJS, so called “headless WebKit”, non-GUI browser engine. By the way, WebKit is under the hood of such browsers as Chrome (and derivatives), Safari (including mobile versions), Dolphin browser (Android mobiles), etc.


However, in some situations switching to a headless browser might not be an option, like if the application is under test restrictions (for example application supports Internet Explorer browser only), there is a requirement to test plugin content (Flash, Silverlight, Java), or there is a requirement to support certain technologies like ActiveX or authentication types like single sign-on)


So the remaining part of the article will cover running Selenium tests using real browsers in headless mode on Jenkins nodes.


Test Project


All of the following examples assume the following minimal test scenario:


- Use Selenium Firefox Driver open as an instance of the Firefox browser

- Open BlazeMeter main page

- Print the global title 

- Close the browser

The test class is simple:


package com.blazemeter.junit.selenium;


import org.junit.Test;

import org.openqa.selenium.By;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.firefox.FirefoxDriver;


public class JMeterSeleniumDemoTest {



   public void openBlazeMeterTest() {

       FirefoxDriver driver = new FirefoxDriver();


       WebElement title = driver.findElement(By.xpath("//title"));


       System.out.println("*" + title.getAttribute("text") + "*");






And this is what it looks like in action:


selenium running headless mode


Now let’s see how this test can be executed in Jenkins on different operating systems.




In general, running Selenium tests on Windows is not possible. The Windows operating system family always has GUI, and even Windows Core Server deployments are capable of running GUI applications without any extra configuration (you just need to get the GUI application somehow using the command-line, for example using Invoke-WebRequest PowerShell cmdlet).


However, it is possible to run your test on behalf of a different user. This way, you can continue working while the test is being executed in the background, without having to worry that your normal keyboard and mouse activity will break the test, and vice versa.


We recommend you install Jenkins master (or node) as a Windows service. This will kill two birds with one stone:

- You will get failover and resilience if the machine is restarted after installing updates, if there is a power outage or if the Jenkins process dies for some reason.

- The process will be run under a different user account, for instance Local System, so the browser window won’t appear in the current session.


Here is the demo of the Test Project execution on a Jenkins node running as a Windows service


Test Project execution on a Jenkins node running as a Windows service


As you can see, no Firefox browser window is present, but the test execution is successful. The BlazeMeter page title, which is “JMeter and Performance Testing for DevOps | BlazeMeter” is being printed to the build log.


N.B. in some cases (for example if you need to use Internet Explorer browser or other applications requiring interacting with the Desktop) you may also need to set java.awt.headless Java system property to “false” for Jenkins master and/or slave processes.




Now let’s try running the same test on a Linux box:


headless execution on linux


As can be seen from the bottom of the build log, Jenkins failed to start the Firefox browser. Being a GUI application, Firefox requires a “display” to run at.


Linux operating systems use display server to draw windows, receive input and react with output. Any GUI-based program requires something called DISPLAY - a screen (or multiple screens) where output is rendered. If the Linux installation doesn’t assume GUI, you can use X virtual framebuffer to mock up the missing display.


If you don’t have the X virtual framebuffer installed, i.e. if Xvfb returns the “command not found” message, you will have to install it from your Linux distribution repositories.


The process of running X virtual framebuffer is as simple as:

Xvfb :N


Where: N stands for the number of the display. In Linux, display numbers are zero-based and normally the user is sitting on display 0 so you can start X virtual framebuffer on the display 0 as:

Xvfb :0 >& /dev/null &


The remainder of the command after 0 is to suppress Xvfb output. It prints quite a lot of noise to the command line, so if you add >& /dev/null & postfix you will see only Xvfb process id (14169 in my case).


linux jenkins command


The next step is to “tell” Jenkins to use this Display 0 for launching Firefox. On Linux systems it can be done by setting DISPLAY environment variable so you can either refer your Linux distribution documentation to learn how to set it, or do it on Jenkins level, for example under the Manage Jenkins -> Configure System -> Global Properties -> Environment Variables section.


jenkins display environment variable


Or in case of individual build agent the same settings live under

Manage Jenkins -> Manage Nodes -> %Node Name% -> Node Properties -> Environment Variables


To get to the configuration dialog click on the gear icon next to the node name.


jenkins configuration dialogue


Once you have launched the X virtual framebuffer and set up DISPLAY environment variable matching virtual display number, the build should succeed and you should see the BlazeMeter main page title in the build log.


jenkins console output build log




As MacOSX is a UNIX-based operating system the approach should be the same as for Linux. X window server implementation for MacOSX is XQuartz, however you may have to recompile it to support Xvfb feature and the browser you’re intending to use so it would use X11 server instead of Aqua.


Selenium and JMeter integration


If you want to combine your Selenium functional tests with JMeter load tests, it can be done via WebDriver Sampler. I believe there is no need to say that BlazeMeter was, is and will be 100% JMeter-compatible, including any plugins. If you don’t need the integration, just Selenium is also supported. Check out the Webdriver Test page for more details.


You also may be interested in viewing our webinar, Driving Headless Chrome with Selenium and Python


Finally, you can request a live 1-on-1 demo of how to leverage tools like Taurus, Selenium and Jenkins with BlazeMeter's performance testing. 

Interested in writing for our Blog? Send us a pitch!