Blog
April 24, 2024
Analyzing JMeter Test Results Using JMeter Aggregate Report Listener
Performance Testing
The process of creating and running a JMeter test is relatively simple:
- Create your test scenario, for example record it using JMeter’s HTTP(S) Test Script Recorder or BlazeMeter Chrome Extension.
- Perform correlation of dynamic parameters and parameterization of test data (user credentials and other input fields).
- Run test with 1-2 users/loops in GUI mode to ensure that it’s doing what it is supposed to be doing looking at request and response details in View Results Tree listener.
- Run the JMeter test in command-line non-GUI mode with the increased number of users according to your performance test type (load, stress, soak, etc.).
As the result you have a .jtl file containing test metrics for every single request including start time, elapsed time, connect time, latency, response code and message, etc.
JMeter .jtl result files are basically CSV files, so you could use a spreadsheet processing software like Excel for analysis and build tables and charts helping you to assess the system under test performance.
However, JMeter comes with special test elements called Listeners where you can load your .jtl results file and see test metrics in more human-readable form than a CSV file. One of the most popular Listeners is Aggregate Report
The JMeter Aggregate Report listener looks like a table with several columns showing different test metrics, it provides an overview of response times and throughput:
Let us take a closer look and see what each column means and how to read and understand the data.
Experience how BlazeMeter helps teams do JMeter testing at enterprise scale. Try BlazeMeter for free today >>
JMeter Aggregate Report Features
Label
Label is basically the name of a Sampler. For example, when you add HTTP Request sampler to your Thread Group by default it is called “HTTP Request”. And if you have more than one HTTP Request sampler in the Test Plan it will be quite hard to distinguish metrics coming from different samplers so it makes sense to provide reasonable names to Samplers. It can be done via “Name” input field:
The “Label” column of the JMeter Aggregate Report listener shows unique Sampler names (labels) for all Samplers which are in the Aggregate Report Listener’s scope. It allows you to analyze response time and throughput for individual Samplers or Transactions.
# Samples
The # Samples column provides the number of executions for each Sampler. For example, in my case I have 10 threads (virtual users) and 10 iterations in the Thread Group and it means that each virtual user executes all Samplers 10 times.
Given the test finishes successfully there should be 100 executions of each Sampler and 400 total executions.
Average
The Average column shows the average response time for each Sampler. It's the arithmetic mean of all response times for Samples with the particular Label. So, since I have 100 executions of “Complete Order” sampler to get the average response time, JMeter sums up the response times of all Samples and divides them by 100. Given the following response times: {3, 6, 9, 12, 15} the average response time will be (3 + 6 + 9 + 12 + 15) / 5 which is 9.
Median
Median is the middle value in the response times for Samplers with particular labels sorted from smallest to largest value. Given you have response times which look like {3, 6, 9, 12, 15} the median will be 9. For the response times {3, 6, 9, 12, 15, 18} median will be the average of 9 and 12 which is (9+12)/2 which is 10.5.
You can also think of median as of 50th percentile.
90% Line
The 90% Line shows 90th percentile value. Percentile is a statistical measure it means the value below which 90% of data falls. If you sort your response times for a Sampler with a particular label in ascending order the 90th percentile represents the point below which 90% of observations lie. This is often used to understand the distribution of data and identify values that are higher than most of the dataset.
95% Line
95% Line shows 95th percentile for the response times for the particular sampler label.
99% Line
99% Line shows 99th percentile for the response times for the particular sampler label.
These 90, 95 and 99 lines are not the only percentiles you can analyze, the percentiles are configurable via the following JMeter properties:
aggregate_rpt_pct1=90
aggregate_rpt_pct2=95
aggregate_rpt_pct3=99
If you need to see other values add the above lines to user.properties file (lives in “bin” folder of your JMeter installation) and after JMeter restart the Aggregate Report listener will show the new percentiles values.
Min
Min column simply shows the minimum response time for the sampler with the given label. For example if response times were {3, 6, 9, 12, 15} the Min column will show 3.
Max
Max column simply shows the maximum response time for the sampler with the given label. For example if response times were {3, 6, 9, 12, 15} the Min column will show 15.
Error %
Error % shows percentage of Samplers which were not successful. JMeter automatically treats HTTP Status Codes above 399 as failures. Also there could be requests failing due to Assertions or timing out or incorrectly setup (wrong URL).
Throughput
Throughput shows the number of requests per unit of time (second/minute/hour). The time period is dynamic and is being automatically adjusted for better readability.
Received KB/sec
Received KB/sec shows the incoming traffic in kilobytes per second. It depends on the size of the response and the throughput (number of requests per unit of time).
Sent KB/sec
Sent KB/sec is the same as “Received KB/sec” but for outbound traffic, it depends on the size of the request and the throughput.
Be aware that despite the Aggregate Report listener is informative and powerful it doesn’t tell the full story because you cannot correlate increasing load with increasing response time, cannot state when the errors start occurring, etc. Additionally, it would be good to see some charts showing relationships between different metrics and KPIs. Some are bundled with JMeter as Listeners, some can be generated in HTML Reporting Dashboard, and some are available via JMeter Plugins project.
BlazeMeter users get the full power of reporting including:
- Request Stats — Gives the same information as the Aggregate Report plus standard deviation, response codes and messages for failed requests, etc.
- Timeline Report — An interactive chart where you can choose from many metrics and KPIs and see them all at the same graph.
- Report Statuses — The informative overview of the test friendly for both developers and managers.
If you want to try out BlazeMeter reporting you can upload your existing test results to BlazeMeter using Existing Results Loader module of Taurus framework.
Bottom Line
BlazeMeter makes JMeter tests better — it is that simple.
The JMeter Aggregate Report is a great tool to help testers better analyze the data of their JMeter tests. By integrating BlazeMeter with JMeter, users can leverage the JMeter Aggregate Report to gain deeper knowledge of test statistics, test timeline reports, and reports of test statuses.
If JMeter is your preferred open source tool, then pairing it with BlazeMeter is a no-brainer. The best part? You can get started using BlazeMeter for FREE today!