Mixing Selenium Into Your Load Scenario
Selenium is an open source automated functional testing framework, which provides multiple facilities for web applications testing, across different browsers and platforms. Selenium is not just a single tool but a suite of software. The two most important parts of this suite are Selenium WebDriver and Selenium IDE. The first one is a set of language specific APIs for browser automation. The second is a Firefox addon that provides a “record & replay” feature.
In this blog post, when I will focus on the Selenium WebDriver, which implements a modern and stable approach to automating the browser's actions. The WebDriver is designed to provide a simpler and concise programming interface. It was designed with the goal to supply a well-designed object-oriented API, which provides improved support for advanced web-app testing problems.
This blog post will show how to enhance an existing Apache JMeter™ scenario with a Selenium WebDriver test script, to obtain a new test script that mixes different test requirements. This is done when there is a need to simulate some of the client-side Selenium data flows under a load. (When referring to Selenium I mean the Selenium WebDriver).
What are the best test conditions for adding Selenium to your JMeter script?
- When you can reuse an already developed Selenium code (e.g. from the functional test phase)
- When you don’t need to track every HTTP request, since in the Selenium Sampler each step presents only one tracking record.
You might suppose “I can execute Selenium and JMeter at same time via two command interfaces without using any plugin and achieve the same result”.
The answer is - yes, of course you can but you are executing two applications without any guarantee of the time correlation in the final report. The analysis of two final logs instead of one can be error prone, because it is affected by missing the time correlation of the executed steps. How many Selenium iterations were there? And how many HTTP Sampler ones?
How Does Selenium Integrate with JMeter?
JMeter has a plugin called “Selenium/WebDriver Support”, which you can find in the JMeter plugins manager. This plugin provides two components for integrating Selenium into a JMeter test script:
- Driver Configs - a set of JMeter configuration components, one for each Selenium supported browser (Firefox, Chrome, Internet Explorer and PhantomJS). Each configuration component requires that a specific Selenium WebDriver implementation is provided. The configuration component is executed only one time in the scope of the thread group, at setup, and independent of the iterations number. At the end of its execution, a running instance of the browser is launched under the control of the Selenium driver, through JMeter.
- WebDriver Sampler - a JMeter sampler that executes actions on a browser via the WebDriver. This sampler can include an entire Selenium script, and consequently more user actions on the browser can be performed. This sampler must be located under the same scope of the configuration element that allocates the Selenium WebDriver into JMeter, to resolve the correct one.
This is the test flow:
- Two Thread Group components start together. One for a “regular” JMeter script, the second one for the Selenium part of the test.
- In the first iteration, “Thread Group Selenium” executes "jp@gc - Chrome Driver Config” (or any other browser config) and the "WebDriver Sampler" component.
- The "jp@gc - Chrome Driver Config" component starts a "Google Chrome" browser instance, visible on the desktop.
- The "WebDriver Sampler" component executes the code that refers to the Selenium WebDriver. The Selenium WebDriver is resolved in the Thread Group scope, so the code is executed in the Chrome browser that was just created.
- After the "WebDriver Sampler"’s first execution, Thread Group Selenium" starts a second iteration (according to the test script) In the second iteration (and the following ones) "jp@gc - Chrome Driver Config" does not perform anything, because Chrome still open and the "WebDriver Sampler" continues to use it.
- Results can be viewed in the Listeners
Now let’s look at an example. Suppose we have an existing JMeter script that mixes two user flows in the same thread group. This thread group also has a “Constant Throughput Timer” and a “Throughput Controller” that fix the number of iterations per second. See the picture below:
Now suppose that we have an existing Selenium script that reproduces a test condition. This could be a user opening a browser, inserting credentials, filling an order form, etc. At this point we need to add a joint point for Selenium.
Add add another “Thread Group”, dedicated for Selenium only.
Add -> Threads -> Thread Group
Configure the time duration in this new thread group like you did in the other thread group in the JMeter test (maybe with a variable to centralize the test duration setting for the complete test). The only difference will be that only one thread is allocated to it. The choice to add only one thread is influenced by the number of Selenium WebDrivers allocated in the script. Only one driver can serve only one thread at a certain time.
We will now add the configuration element that starts the WebDriver instance with a Google Chrome browser.
Add -> Config Element -> jp@gc - Chrome Driver Config
Remember that, as requested by Selenium, it’s mandatory to provide a complete path to “chromedriver.exe”, also called “WebDriver implementation”. The selection of “WebDriver implementation” must be chosen according to the version of Google Chrome browser installed (e.g. 32bit or 64bit? running on Windows or Linux?...). For other browsers, there are different mandatory Selenium requirements.
At this point of the JMeter run, we should have a running browser controlled by Selenium. Now we will add the test logic, by adding the “WebDriver Sampler” into the thread group:
Add -> Sampler -> jp@gc - WebDriver Sampler
Now it’s time to add the Selenium code into the “Script” field. you can copy the code from your Selenium test and reuse it here.
#1 Creates an access point to the Selenium API
#2 Defines a JMeter sample that will be tracked in the listeners
#3 Landing on www.blazemeter.com
#4 Clicks on a page element “Blog”
#5 Searches an on page element “JMeter”
#6 Verification logic
#7 End of sample
To complete the script, add a Constant Throughput Timer to the Selenium thread group. The timer will limit the execution to a necessary rate. The limit rate is based on business requirements (e.g. one text search for ten minutes). You can also insert the rate limit for resource consumption on the test machine. This is important because Selenium Webdriver executes a complete browser, which could take up more memory and CPU if the rate is high. As a default, you can use a few iterations per minute.
Add -> Time -> Constant Throughput Timer
In the screenshot you can see we have configured all threads in the current thread group to execute the WebDriver Sampler three times a minute.
It’s important to decide the execution rate according to Selenium script average execution. If the rate is too high the single thread/driver will not be able to start a new execution, because it is still involved in the previous one. If you’re familiar with your test, you probably know the average execution time. If not, then you need to go through trial and error.
So if we take our previous example of three iterations in one minute:
- If a single iteration takes less than 20 seconds everything is ok because the WebDriver Sampler is completed entirely.
- If an iteration takes more than 20 seconds, the results collected are useless because the WebDriver Sampler was affected by a new iteration.
Now let’s run a script. Here are the results:
The second column describes how many samples are executed during the entire JMeter test execution. Notice that against many requests for HTTP samples, there are only a few WebDriver samples (four WebDriver Samples against hundreds of HTTP samples).
Selenium in JMeter - Things to Consider
1. The WebDriver plugin is a Selenium wrapper. So some features, especially the WebDriver configuration, are masked. Unclear situations/results should be analyzed while taking this layered environment into account.
3. The JMeter Webdriver plugin carries on the Selenium dependency jar library of an older version. This situation can potentially generate a compatibility issue on newer browsers with older Selenium.
4. Resources consumption for a low rate of WebDriver samples can be compared with thousands of HTTP samples.
Selenium is de facto the test automation standard for web applications. The possibility to mix it into an existing JMeter script, adds value to both your JMeter and Selenium tests. However, this convergence includes some draw back that should be evaluated, to better define the effort estimation and reachable goal.
That’s it! You now know how to mix Selenium into your JMeter load scenario. Learn more advanced JMeter from our free JMeter academy.