Noga Cohen is a Sr. Product Marketing Manager for CA BlazeMeter. She manages the BlazeMeter blog and other content activities. Noga focuses on creating technological content in the fields of performance, load testing and API testing, both independently and by managing writers who are developers. Noga has more than 5 years of experience in a wide scope of writing techniques: hi-tech, business, journalist and academic.

Become a JMeter and Continuous Testing Pro

Start Learning

Test Your Website Performance NOW!

arrow Please enter a URL with http(s)
Sep 08 2016

Automated Performance Tests in Jenkins Continuous Integration Environments


1. Why is Continuous Integration (CI) Important?


Continuous Integration (CI) is important for developers, as they can integrate code into a shared repository and verify each check-in with an automated build, deploy and test the process. This enables them to quickly find and remove bugs and measure the impact of each code change to application performance.


Users who want to add performance testing should identify their goals together with business analytics. This is important for determining which types of test should be run at each stage of the CI process.


Here are the types of tests you can run:

Load Tests are conducted to understand the behavior of the system under a specific expected load.

Stress Tests are used to understand the upper limits of capacity within the system.

Soak Tests determine if the system can sustain the continuous expected load.

Spike Tests determine if the system can sustain a suddenly increasing load generated by a large number of users.

Isolation Tests determine if a previously detected system issue has been fixed by repeating a test execution that resulted in a system problem.


So for example, for Continuous Delivery, running soak and spike tests in a pre-production environment helps understand how applications perform with continuous expected loads and spikes, with each production deployment.


To simplify inclusion of load tests into CI we suggest to use the Open Source test automation tool called Taurus. Let’s understand how Taurus works.


2. Using Taurus to Easily Run Load Tests


Taurus simplifies the automation of performance testing, is built for developers and DevOps, and relies on JMeter, Selenium, Gatling and Grinder as underlying engines. It also enables parallel testing, its configuration format is readable and can be parsed by your version control system, it’s tool friendly and tests can be expressed using YAML or JSON.


If you don’t have Taurus yet, install it.


Let’s look at this simple configuration in YAML.


This test ramps-up to 5 users for 30 seconds, then sustains this load for 1 more minute. The number of hits per second (throughput) is 3, and they send requests to the url: http://aws-uut:8000/



- concurrency: 5
  throughput: 3 
  ramp-up: 30s
  hold-for: 1m
    - http://aws-uut:8000/


This is a clear and simple test. The same test in JMeter would be very long, with lots of text that is hard to understand, and changes made into it would be unclear.


You can also split the file into 2 (or more) config files which will make it easier to maintain the test code with the source code, especially if many developers are working on the same test.



      - label: Home
        url: /
        method: GET
        think-time: 500ms



- concurrency: 5
  throughput: 3 
  ramp-up: 30s
  hold-for: 1m
    - include-scenario: my_scenario


The first config file defines the scenario.


The second config file defines the execution parameters.


More complex scenarios can include assertions for validating response data, group requests into transactions and pass-fail criteria.


To run the test, type bzt and the name of the YAML file.


In this case: bzt simple-travel.yml scenario-home-page.yml


Give it a few seconds to run and look at the dashboard. These include response times, percentiles, response codes and local CPU.


This is the dashboard for the test after 83% of the test. All response codes are 200s, CPU is  6% and memory is 70%. The two labels that are being hit are the Home Page and the Reservation page, and it’s successful with no failures.


load test results in Taurus


After the test is finished, the results are shown collectively:


load test results in taurus

Now, let’s understand how Taurus works with BlazeMeter.


3. Viewing Taurus Results on BlazeMeter


By using the -report parameter, we can send the results to BlazeMeter in real-time.


Taurus is working in the background, but we can see all the reports and analyze the results on BlazeMeter, in a more sophisticated and attractive fashion.


load test results reports on blazemeter


Finally, let’s see how it all comes together with Jenkins so you can include it in your Continuous Integration process.


4. Using Jenkins with Taurus and BlazeMeter for Load Testing


To use Jenkins for load testing, first install Taurus onto the Jenkins machine. Make sure you also install Jenkins.


Now, open Jenkins.


1. Click ‘Configure’ on the left.




2. Choose the Jenkins Build Triggers tab on top.


3. Scroll down to the Build section and enter the command for the test.


jenkins execute shell build


In this case: bzt simple-travel.yml -report


If you have a complete CI deployment process, make sure the test is kicked off with it.


That’s it.


To see the test details, check out the Console Output.


jenkins console output


Results can also be shown on BlazeMeter.


We can add details easily by changing the test config file in one place on Taurus.


The test will now run on Taurus and on Jenkins. Congratulations! Now you can see the results and analyze load testing as a part of your full development process.

arrow Please enter a URL with http(s)

Interested in writing for our Blog?Send us a pitch!