How to integrate JMeter into CircleCI
As software products grow in complexity, so does the whole process of release management and quality assurance. Nowadays, Continuous Testing and Continuous Integration (CI) have become an industry standard for development and delivery process automation by combining deployment and testing activities into one cycle. Regular merging of developers’ working copies and automatic testing makes issue detection faster, and debugging processes easier.
Continuous Integration with CircleCI
In this article I want to focus on including JMeter performance tests into your CI flow with CircleCI.
CircleCI is a modern and powerful CI solution. It features very fast integration jobs, wide support for third-party tools, built-in integration with the most popular hosting services and fine jobs tuning.
After you’ve added any project to CircleCI you can start to configure integration jobs. In CircleCI, jobs are configured in config.yml file, that you should place in your repository into the hidden folder .circleci. The most basic config.yml may look as follows:
Now, after every commit to your repository, CircleCI will spin up circleci/ruby:2.4.1 Docker image, checkout the code from your repository and return the "My first job" message.
Using config.yml file you can configure complex jobs known as workflows. As an example:
Here, we construct two jobs and configure a workflow to run them concurrently.
Using the workflow functionality you may construct very complex build configurations, consisting of consequent and parallel jobs.
Running Performance Tests in CI flow
Running automated unit and UI tests is a critical part of any CI process. But what if stable performance is crucial for your software? It’s very useful to have performance tests included in your CI workflow. By doing so, you can have regular reports on your software performance, so you can detect performance degradations more quickly and precisely.
When including performance testing in your CI flow consider that you will most likely need a dedicated load generator system. In order to reduce infrastructure costs you don’t need to run performance tests every commit - regular daily runs is just enough.
How to run a JMeter script in CircleCI
I will explain how we can include a JMeter script into the CircleCI workflow.
- First of all, prepare your JMeter script and add it to your project repository under src/test/jmeter/ folder.
2. Prepare pom.xml file and add it to your project root
Here we use JMeter Maven Plugin to launch our JMeter script and generate the report.
3. Add config.yml file to .circleci/ folder
Here we configure our job to load Docker image circleci/openjdk:8-jdk, check-out a code from a repository, download dependencies, running our script and saving results in target/jmeter/reports.
4. Now it’s time to log in to CircleCI.
5. Go to Add Project tab
6. Select your project from the list and press the Set Up Project button
7. Select Maven (Java) option and press the Start Building button
8. Now, your job has started, you can go to Jobs and see results.
You can view the generated HTML report by going to the Artifacts tab
10. Now, this job will be started every time a new commit is pushed to your repository. If you want to specify your own triggers, you can set up a schedule for your workflow in the config.yml file.
For example, add the following workflow to your config.yml to run the performance test every day on 12am UTC:
As you can see, including JMeter performance tests to your CircleCI builds is very simple and straightforward, yet highly configurable.
You can also use the powerful Blazemeter platform with your CircleCI workflow to enhance your performance testing. To do so, you need to:
1. Get a BlazeMeter Taurus installation file
2. Add a new directory to your repository, calling “blazemeter”
3. Put BlazeMeter Taurus installation file in this folder
4. Put your JMeter script in this folder
5. Create an My_Test.yml configuration file for BlazeMeter Taurus:
For more info on how to run BlazeMeter tests in CircleCI you can read more here. BlazeMeter supports over 20 open source tools, and can be used for end to end continuous testing. To get started for free go to www.blazemeter.com.