How to Use Grafana to Monitor JMeter Non-GUI Results - Part 2
In our latest article about JMeter Non-GUI Monitoring, we explained how to integrate Apache JMeter™ with Grafana to establish real-time monitoring of performance scripts. This can be done by using a local installation of the InfluxDB database as a metrics storage, together with a Backend Listener and the “GraphiteBackendListenerClient” implementation.
This Grafana integration via “GraphiteBackendListenerClient” requires developers to proceed with a list of non-trivial steps, to get it done. But our followers noticed (special thanks to Vincent D.) that there is one more easy way to perform an additional JMeter Grafana integration, which is simplified and doesn’t require any additional configuration of the InfluxDB.
We always appreciate feedback, so in this blog post we will explain a simpler way that uses the same JMeter Backend Listener, but with the “InfluxDbBackendListenerClient” implementation which takes less setup efforts.
This implementation uses API for pushing metrics. With the release of JMeter 3.2 by the Apache team, a new Backend Listener interface was introduced. This listener enables using InfluxDB HTTP API and pushing metrics directly to the database via Asynchronous HTTP calls, without any additional database configurations.
Step-by-Step With the InfluxDbBackendListenerClient Implementation
1. Install Grafana and InfluxDB
If you took a hands-on approach to our latest article about Grafana monitoring, you probably already have Grafana and InfluxDB installed on your local machine. If not, use the InfluxDB and Grafana installation guides to install both locally. The same goes for our performance script monitoring which will be exactly the same as we created in the last article. But actually, you can use any performance script that you already have ready in your arsenal.
2. How to Use InfluxDB’s HTTP API
In the last article, we showed an example of how to use the InfluxDB command line interface to make database queries. Another way to interact with the database is with InfluxDB’s HTTP API, which allows you to query and write data metrics from/to the database.
By default, InfluxDB exposes its HTTP API on the 8086 port. To check if it works for you, let’s ping that endpoint with a HTTP client like CURL (you can use any another HTTP client as well). With curl you can use the curl -i localhost:8086/ping command to send a HTTP GET request to the InfluxDB API endpoint, where the -i option prints the request output to the console:
If your InfluxDB instance is up and running, you should see a similar response from your ping request. With the InfluxDB HTTP API you can perform exactly the same operations that you can perform with the command line interface. Let’s create a database for our metrics and call it “blazedemo_performance”, by using the specified command: curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE blazedemo_performance
As you can see, it is quite simple. That’s why the Apache team thought - why not use the same HTTP API to push metrics from JMeter to InfluxDB? With the launch of JMeter 3.2, they introduced a new implementation of Backend Listener, which is called “InfluxdbBackendListenerClient”. It was added to JMeter 3.2 in Alpha state (an early version that might be unstable). But turns out is quite stable and is already used across the JMeter community without any critical issues.
3. Configure the JMeter Backend Listener
Once you have a performance script, all you need to do is to add the Backend Listener with the “org.apache.jmeter.visualizers.backend.influxdb.InfluxdbBackendListenerClient” value as a “Backend Listener implementation” config value. (Don’t panic, this value is specified in the List box and you don’t need to type it yourself). Add a few more specific configs:
- influxdbMetricsSender - the implementation class for sending metrics to InfluxDB. Currently there is only one available option so keep it as default
- influxdbUrl - the url of InfluxDB in format: http://[influxdb_host]:[influxdb_port]/write?db=[database_name]. As we have created the “blazedemo_performance” database and we are running it on local machine with default port then in our case url will be: http://127.0.0.1:8086/write?db=blazedemo_performance
- application - we can store metrics from different applications under one database. That’s why this param is used for grouping metrics from one specific app
- measurement - name of measurement that will be stored in InfluxDB (text-based line InfluxDB internal protocol to store metrics). Use default ‘jmeter’ for that property
- summaryOnly - put ‘true’ if you want to keep summary results only in the database and do not want to collect all the detailed metrics during test execution
- samplersRegexp - regexp expression to filter specific metrics which you want to store into database. Use default “.*” value if you want to store all metrics
- percentiles - metrics percentiles that should be sent to the database
- eventTags - a list of tags which will be stored in the ‘events’ measurement of InfluxDB
4. Run Your Performance Script
Once the configuration is in place, let’s run our JMeter performance script. You can run it via the JMeter UI or you can run it in the Non-GUI mode (check this article if you want to know how). As soon as we start our performance script, the Backend Listener starts to push metrics which will be collected in InfluxDB database with the name “blazedemo_performance” as we configured before.
After we start our script, we need to confirm that metrics are being collected. For that we can use HTTP API and this curl request: curl 'http://localhost:8086/query?pretty=true' --data-urlencode "db=blazedemo_performance" --data-urlencode "q=SHOW SERIES where the query ?pretty=true is used to show returns of the json response in this structured way:
5. Configure Grafana and Create Dashboards
Once we’ve confirmed the metrics are there, we can continue with the Grafana configuration. As we installed Grafana before, it should be up and available on http://localhost:3000, where you can use ‘admin’ as a default username and password to log in. If you remember from the last article, first of all you need to configure the data source. Even if you have done that step previously, this time we are using another database and therefore we need to add one additional data source, which is mapped to the new created database. To add the new data source in Grafana go to:
Menu -> Data Sources -> Add data source
If the data source was added successfully, you can start the dashboards creation:
Menu -> Dashboards -> New
During dashboard creation, you first need to choose the correct ‘Data Source’, based on the one that you have just added. On the second and third steps you need to specify measurement and application names, based on values that we have specified into the JMeter Backend Listener. On the last step you need to choose metrics that will be shown on the graph. It’s very simple, as it should be!
As you can see, the “InfluxdbBackendListener” is much simpler and requires less configuration to start using Grafana for JMeter tests monitoring. In comparison with the “GraphiteBackendListener” which we introduced in the last article, this way is much much simpler. But is it simple enough, or are there even more simple ways? You will be surprised, but YES!
Out-of-the-Box JMeter Grafana Dashboard
Developers from NovaTec-APM decided to make our life much easier, and created their own ready-to-use JMeter Grafana dashboard with all the key monitoring graphs and dashboard elements. You can find all the relevant information through this link. Using that predefined dashboard you don’t need to spend time on dashboards creation, as everything is already created for you to monitor detailed stats right away in just a few easy steps.
For this example we need to create a separate database for metrics storage: curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE blazedemo_load
First of all you need to download a separate Backend Listener implementation, which basically is a jar file that you can download from this link. Once the jar file is downloaded, you need to place it under “/lib/ext” of your JMeter home folder:
> cp /Users/ybushnev/Downloads/JMeter-InfluxDB-Writer-plugin-1.2.jar /usr/local/Cellar/jmeter/3.2/libexec/lib/ext
If your JMeter application is open, you need to restart to proceed with the installation of the additional plugin. After that, you can open the previously created “Backend Listener” and this time there will be additional option available in the “Backend Listener Implementation” field: “rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient”. The configuration of this listener client is pretty similar to implementations that we have been used before, and the configuration options are almost the same. In our case all we need to do is to provide the “testName” and “influxDBDatabase” names, respectively.
After the configuration is in place, you can run your performance script and confirm there are not any errors in the JMeter logs execution:
If you don’t see any errors, the metrics should be collected successfully and you can continue with the Grafana configuration. As well as we did that before, we need to add the additional data source that will be mapped to “blazedemo_load” database:
And now is the most awesome step. On the page of JMeter Load Test dashboard you can download the Grafana template in json format:
After that, open Grafana and instead of the new dashboard creation go to:
Menu -> Dashboards -> Import and choose “Download JSON”, specifying the data source that we have just created for that example:
After that, by clicking on “Import” we will automatically create outstanding JMeter monitoring without any additional configuration:
As you can see, there are always many ways to achieve your goals. Based on all 3 ways of JMeter and Grafana integration which we have tried in this and the previous article I would definitely recommend to go with the last “JMeterInfluxDBBackendListenerClient” integration. Yes, it requires you to install an additional plugin, but it is nothing in comparison to the benefits that this plugin provides you in the end. Thanks to feedback of our followers, we have now covered all relevant ways of JMeter integration with Grafana. Let us know which way you prefer and why in the comments section.
To try out BlazeMeter, which enhances JMeter and also provides real-time, collaborative and in-depth analysis, put your URL or JMX file in the box below. Your test will start in minutes.