Konstantine is the Head of QA department at ISS Art. He has over 25 years of experience in the field of IT and Quality Assurance. Konstantine is specialized in a variety of fields including load testing, functional and automated testing, security and other types of testing

Learn JMeter in 5 Hours

Start Learning
Slack

Test Your Website Performance NOW!

arrow Please enter a valid URL

How to integrate JMeter into CircleCI

As software products grow in complexity, so does the whole process of release management and quality assurance. Nowadays, Continuous Testing and Continuous Integration (CI) have become an industry standard for development and delivery process automation by combining deployment and testing activities into one cycle. Regular merging of developers’ working copies and automatic testing makes issue detection faster, and debugging processes easier.

 

Continuous Integration with CircleCI

 

In this article I want to focus on including JMeter performance tests into your CI flow with CircleCI.

 

CircleCI is a modern and powerful CI solution. It features very fast integration jobs, wide support for third-party tools, built-in integration with the most popular hosting services and fine jobs tuning.

 

After you’ve added any project to CircleCI you can start to configure integration jobs. In CircleCI, jobs are configured in config.yml file, that you should place in your repository into the hidden folder .circleci. The most basic config.yml may look as follows:

version: 2
 jobs:
   build:
     docker:
       - image: circleci/ruby:2.4.1
     steps:
       - checkout
       - run: echo "My first job"

 

Now, after every commit to your repository, CircleCI will spin up circleci/ruby:2.4.1 Docker image, checkout the code from your repository and return the "My first job" message.

 

Using config.yml file you can configure complex jobs known as workflows. As an example:

version: 2
 jobs:
   one:
     docker:
       - image: circleci/ruby:2.4.1
     steps:
       - checkout
       - run: echo "First job"
   two:
     docker:
       - image: circleci/ruby:2.4.1
     steps:
       - checkout
       - run: echo "Second job"
 workflows:
   version: 2
   one_and_two:
     jobs:
       - one
       - two:
          requires:
            - one

 

Here, we construct two jobs and configure a workflow to run them concurrently.

JMeter with CircleCI

Using the workflow functionality you may construct very complex build configurations, consisting of consequent and parallel jobs.

 

Running Performance Tests in CI flow

 

Running automated unit and UI tests is a critical part of any CI process. But what if stable performance is crucial for your software? It’s very useful to have performance tests included in your CI workflow. By doing so, you can have regular reports on your software performance, so you can detect performance degradations more quickly and precisely.

 

When including performance testing in your CI flow consider that you will most likely need a dedicated load generator system. In order to reduce infrastructure costs you don’t need to run performance tests every commit - regular daily runs is just enough.

 

How to run a JMeter script in CircleCI

 

I will explain how we can include a JMeter script into the CircleCI workflow.

  1. First of all, prepare your JMeter script and add it to your project repository under src/test/jmeter/ folder.

2. Prepare pom.xml file and add it to your project root

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.jmeter</groupId>
    <artifactId>maven</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>com.lazerycode.jmeter</groupId>
                <artifactId>jmeter-maven-plugin</artifactId>
                <version>2.8.0</version>
                <executions>
                    <execution>
                        <id>jmeter-tests</id>
                        <goals>
                            <goal>jmeter</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>jmeter-check-results</id>
                        <goals>
                            <goal>results</goal>
                        </goals>
                        <configuration>
                            <generateReports>true</generateReports>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

 

Here we use JMeter Maven Plugin to launch our JMeter script and generate the report.

3. Add config.yml file to .circleci/ folder

 

# Java Maven CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/openjdk:8-jdk

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/postgres:9.4

    working_directory: ~/repo

    environment:
      # Customize the JVM maximum heap limit
      MAVEN_OPTS: -Xmx3200m

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "pom.xml" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: mvn dependency:go-offline

      - save_cache:
          paths:
            - ~/.m2
          key: v1-dependencies-{{ checksum "pom.xml" }}

      - run: mvn verify

      - store_artifacts:
          path: target/jmeter/reports

 

Here we configure our job to load Docker image circleci/openjdk:8-jdk, check-out a code from a repository, download dependencies, running our script and saving results in target/jmeter/reports.
 

4. Now it’s time to log in to CircleCI.

5. Go to Add Project tab

CircleCI in JMeter

 

6. Select your project from the list and press the Set Up Project button

7. Select Maven (Java) option and press the Start Building button

 

CircleCI with JMeter

 

8. Now, your job has started, you can go to Jobs and see results.

 

CircleCI with JMeter

 

You can view the generated HTML report by going to the Artifacts tab

 

CircleCI with JMeter

 

10. Now, this job will be started every time a new commit is pushed to your repository. If you want to specify your own triggers, you can set up a schedule for your workflow in the config.yml file.

 

For example, add the following workflow to your config.yml to run the performance test every day on 12am UTC:

workflows:
   version: 2
   nightly:
     triggers:
       - schedule:
           cron: "0 0 * * *"
           filters:
             branches:
               only:
                 - master
                 - beta
     jobs:
       - build

 

As you can see, including JMeter performance tests to your CircleCI builds is very simple and straightforward, yet highly configurable.

 

You can also customize your jobs by configuring pom.xml file (see this blog for additional info) and config.yml (see official documentation).

You can also use the powerful Blazemeter platform with your CircleCI workflow to enhance your performance testing. To do so, you need to:

 

1. Get a BlazeMeter Taurus installation file

2. Add a new directory to your repository, calling “blazemeter”

3. Put BlazeMeter Taurus installation file in this folder

4. Put your JMeter script in this folder

5. Create an My_Test.yml configuration file for BlazeMeter Taurus:

execution:
- concurrency: 10
  hold-for: 5m
  ramp-up: 2m
  scenario: with_script
  
scenarios:
  with_script:
    script: ~/src/test/jmeter/my_script.jmx

reporting:
- module: passfail
  criteria:
  - avg-rt>200ms, stop as failed
  - failures>0%, stop as failed
- module: blazemeter

modules:
  blazemeter:
    address: https://a.blazemeter.com
	data-address: https://data.blazemeter.com
	browser-open: none
	send-interval: 30s
	report-time-multiplier: 1000
	timeout: 5s
	artifact-upload-size-limit: 5
	public-report: false
	report-name: My Test
	test: My Test
	project: My Project

 

For more info on how to run BlazeMeter tests in CircleCI you can read more here. BlazeMeter supports over 20 open source tools, and can be used for end to end continuous testing. To get started for free go to www.blazemeter.com.

 

     
arrow Please enter a valid URL

Interested in writing for our Blog?Send us a pitch!