JMeter DSL: Bringing Performance Testing Closer to Developers

This blog post was originally published on April 26, 2022 and has since been updated with new information

JMeter is a very popular tool among non-developers testers. This is mainly because no programming knowledge is needed and it supports a large number of protocols and plugins, which increases its versatility. 

In addition, JMeter provides extensive documentation, boasts a broad ecosystem, and is flexible and very powerful. JMeter supports multiple protocols like for mainframe testing, video streaming. You can also implement your own plugins, if you have the relevant Java knowledge. You can use any Java library or any Java tool.

However, JMeter also has its limitations, and not being CI/CD or Git friendly is one of them. As seen here, running JMeter programmatically might be too verbose and hard to review as it's not friendly with any VCS format. In addition, creating test plans in JMeter GUI can be too slow for continuous development teams. The output is a large XML, which is hard to review, and integration with CI/CD requires plugins. Plus, JMeter lacks a built-in solution for assertion over all statistics, managing jmx files can be complicated and there is no modularization. This is where JMeter DSL, or "jmeter-java-dsl" comes in.

What is JMeter DSL?

JMeter DSL is an open-source developer-friendly tool for running performance tests as code, both locally and at scale. JMeter DSL is a Java API that takes advantage of not only JMeter using it as an engine but also coding benefits as well.

This means it provides all the JMeter potential and its protocols support as well:

• All the benefits from coding in an IDE, like inline documentation and autocompletion.
• Built-in features that ease CI/CD pipelines integration.
• Built-in cookie manager and cache manager. 
• When typing in an element, JMeterDSL automatically requires the developer or tester to add all the needed elements. For example, when adding Post, JMeter DSL requires adding a Content Type Header, which is an element that can be forgotten in the JMeter GUI.
• Modularization and parameterization. It even allows adding sampler templates that can be used across projects and adding assets like CSV files.
• A JMX file to DSL converter, for migrating JMX files. This also allows recording a scenario with the BlazeMeter recorder and converting to DSL.
• A simple design that allows programmers and testers to create much shorter tests and scripts that are easy to read and maintain.

Essentially, JMeter DSL is much more intuitive and convenient to use, compared to adding each element separately in the JMeter GUI. 

Follow the documentation here to learn how to use the DSL and also to acquire knowledge of good performance testing practices.

What is DSL?

Unlike other languages that are designed for general purposes, DSL or Domain Specific Language are expressive, human-readable programming languages that are customized for specific tasks like generating artifacts, configuring projects, and more. In our case, we used a DSL to configure our pom.xml file. JMeter DSL is Java-based and its purpose is to code performance tests.

In this post, we share the creation of a simple test with JMeter DSL. We will show the main functions, such as adding headers and assertions. We will also show how to run the test locally and in BlazeMeter, and see how to view the test results.

How to Use JMeter DSL

Locally Running jmeter-java-dsl

We will create a test for the BlazeDemo site that will simulate access to the home page and verify the response’s information.

Setup

In this example, we use the Apache Maven software. For other tools, check the user’s guide, which also contains additional tips and best practices.

If you’re starting from scratch, the first thing to do is create your test in the IDE. You can choose the language and build system of your choice - Java, Kotlin, Groovy, Maven, Gradle, etc.

1. To use the JMeter DSL there's no need to install anything. Just include the following dependency on your pom.xml file, which is the file that contains your project’s configurations:

<dependency><groupId>us.abstracta.jmetergroupId>
  <artifactId>jmeter-java-dslartifactId>
  <version>0.49.1version>
  <scope>testscope>
dependency>

2. We will use AssertJ and JUnit5 test libraries. So now, add the corresponding dependencies to the pom.xml file. These frameworks are needed in order to support the code written later:

<dependency><groupId>org.junit.jupitergroupId>
   <artifactId>junit-jupiterartifactId>
   <version>5.8.2version>
   <scope>testscope>
dependency>
 
<dependency>
   <groupId>org.assertjgroupId>
   <artifactId>assertj-coreartifactId>
   <version>3.21.0version>
   <scope>testscope>
dependency>

Simple HTTP test plan 

After the setup, let’s create our first test.

3. Add a test plan that will pass a thread group as a parameter. You can define the number of threads (concurrent virtual users), iterations, and samplers to be executed. This example uses 1 thread that will send 1 HTTP GET request to the BlazeDemo homepage. 

importstatic us.abstracta.jmeter.javadsl.JmeterDsl.*;importorg.junit.Test;publicclassSimpleScriptTest{@TestpublicvoidtestDSL()throws Exception {
   testPlan(
       threadGroup(1,1,
           httpSampler("http://blazedemo.com"))).run();}}

As mentioned, one of the advantages of using an IDE compared to the JMeter GUI is the ability to easily search for elements. When typing in ‘threadGroup’, for example, the IDE shows all the different Thread Group options and the most common use cases. It even connects to the documentation for additional information that can help you build the test. This significantly shortens the test building time and makes the process much more straightforward and easy to use.

4. Add a response assertion to the test to verify that the site is displaying the text “Welcome to the Simple Travel Agency!” as it should. Add the following to the httpSampler.

.children(responseAssertion().containsSubstrings("Welcome to the Simple Travel Agency!"))

5. Send the headers of the request using the method .header(), which in this case are:

@TestpublicvoidtestDSL()throws Exception {
 TestPlanStats stats = testPlan(
     threadGroup(1,1,
         httpSampler("http://blazedemo.com").children(responseAssertion().containsSubstrings("Welcome to the Simple Travel Agency!")).header("accept","text/html,application/xhtml+xml,application"+"/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application"+"/signed-exchange;v=b3;q=0.9").header("accept-encoding","gzip, deflate, br").header("accept-language","es-ES,es;q=0.9").header("cache-control","max-age=0").header("sec-ch-ua","\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google "+"Chrome\";v=\"98\"").header("sec-ch-ua-platform","Windows").header("user-agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) "+"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"))).run();}

HTTP Headers are sent to check that the right information is being received. They contain a lot of additional information about HTTP connection types, proxies, etc

6. Add acceptance criteria to your test results, like checking that the response time of the 99 percentile of the requests is less than 5 seconds.

This is the final script:

importjava.time.Duration;importorg.junit.Test;importus.abstracta.jmeter.javadsl.blazemeter.BlazeMeterEngine;importus.abstracta.jmeter.javadsl.core.TestPlanStats;importstatic org.assertj.core.api.Assertions.assertThat;importstatic us.abstracta.jmeter.javadsl.JmeterDsl.*;publicclassSimpleScriptTest{@TestpublicvoidtestDSL()throws Exception {
   TestPlanStats stats = testPlan(
       threadGroup(1,1,
           httpSampler("http://blazedemo.com").children(responseAssertion().containsSubstrings("Welcome to the Simple Travel Agency!")).header("accept","text/html,application/xhtml+xml,application"+"/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application"+"/signed-exchange;v=b3;q=0.9").header("accept-encoding","gzip, deflate, br").header("accept-language","es-ES,es;q=0.9").header("cache-control","max-age=0").header("sec-ch-ua","\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google "+"Chrome\";v=\"98\"").header("sec-ch-ua-platform","Windows").header("user-agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) "+"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"))).run();
 
   assertThat(stats.overall().sampleTimePercentile99()).isLessThan(Duration.ofSeconds(5));}

This means that this test will fail if it doesn't get the information that was specified or if the 99th percentile of the requests’ response time is less than 5 seconds.

7. Running the test is as easy as running any other Junit test, as it can be done in Maven or your preferred IDE. In IntelliJ just click the green play symbol next to the test:

These are the results in the console:

As we can see, we get the error range and time statistics (average, minimum and maximum time to perform the simulation). If JMeter is integrated into your CI/CD, you will see your results in those dashboards. 

How to Run JMeter DSL Tests at Scale in BlazeMeter

You can easily run a JMeter DSL test at scale in BlazeMeter and obtain an extended report. This will enable you to use the additional features provided (historic data tracking, real-time reporting, etc.).

8. To do so, include the following module as a dependency on the pom.xml file.

<dependency><groupId>us.abstracta.jmetergroupId>
  <artifactId>jmeter-java-dsl-blazemeterartifactId>
  <version>0.49.1version>
  <scope>testscope>
dependency>

9. To access your BlazeMeter account, you will need an authentication token. So the next step will be generating it, as shown here. As long as it doesn't expire, you only need to create it once.
10. Once you already have your token, replace each .run()line with .runIn(new BlazeMeterEngine(...)) in any existing jmeter-java-dsl test. See the following example:

importjava.time.Duration;importorg.junit.Test;importus.abstracta.jmeter.javadsl.blazemeter.BlazeMeterEngine;importus.abstracta.jmeter.javadsl.core.TestPlanStats;importstatic org.assertj.core.api.Assertions.assertThat;importstatic us.abstracta.jmeter.javadsl.JmeterDsl.*;publicclassSimpleScriptTest{@TestpublicvoidtestDSL()throws Exception {
   TestPlanStats stats = testPlan(
       threadGroup(1,1,
           httpSampler("http://blazedemo.com").children(responseAssertion().containsSubstrings("Welcome to the Simple Travel Agency!")).header("accept","text/html,application/xhtml+xml,application"+"/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application"+"/signed-exchange;v=b3;q=0.9").header("accept-encoding","gzip, deflate, br").header("accept-language","es-ES,es;q=0.9").header("cache-control","max-age=0").header("sec-ch-ua","\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google "+"Chrome\";v=\"98\"").header("sec-ch-ua-platform","Windows").header("user-agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) "+"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"))).runIn(new BlazeMeterEngine(System.getenv("BZ_TOKEN")).testName("Simple DSL test").totalUsers(10).holdFor(Duration.ofSeconds(120)).threadsPerEngine(25).testTimeout(Duration.ofSeconds(240)));
 
   assertThat(stats.overall().sampleTimePercentile99()).isLessThan(Duration.ofSeconds(5));}}

A detail to keep in mind is to set the auth token, which has a <KEY_ID:KEY_SECRET> format, as a custom environment variable “BZ_TOKEN”. You can get it with Sysem.getEnv(“BZ_TOKEN”) to make it cleaner. Even though inserting new BlazeMeterEngine(“KEY_ID:KEY_SECRET”) is valid as well, inserting sensitive data directly into a test is not a recommended security best practice. Using the token as a custom environment variable is safer than using it directly in a test.

11. Note that if you want to run your test in BlazeMeter, you must set a name for the test, the total number of users, the duration, etc. This test uses 10 concurrent users. Check BlazeMeterEngine for details on usage and other configuration options.

Once you run your test, it will automatically load the complete report on BlazeMeter. These are the results:

Besides scaling your tests, BlazeMeter gives really useful features like real-time reporting historic data tracking and engine health monitoring. You can observe key statistics of the tests compared with the baseline on the Summary Panel, a test overview, and trends of your KPIs.

How Can JMeter DSL Help Me?

JMeter DSL is a great option for integrating load tests to agile teams as it considerably eases the scripting and maintenance of performance tests in the repositories usually used in this kind of project. Also, it's a great choice for users who have more of a developer profile and are not used to JMeter’s UI. We know how irritating can be constantly switching from an IDE to a UI-oriented tool. That’s not all, you can also take advantage of the entire BlazeMeter platform, scale your tests and obtain detailed reports.

The best of this tool is that anyone can use it, having or not JMeter experience. The example provided is a good place to start and will guide you in your first steps. We consider this tool will be a major contribution to the software community so we encourage you to try this well-documented tool and start bringing performance testing closer to development.

START TESTING NOW