Blog
February 24, 2022
Performance testing metrics are important. But they aren't always easy to understand. Until now.
Why Performance Testing Metrics Matter
Performance testing metrics are important to understand the effectiveness of your testing efforts.
Ask yourself...
Are your performance testing metrics helping you? Do errors occur in your test run? If they do, most chances are that the KPIs you are looking at are masked by the errors.
Why? Errors are usually returned either very quickly or extremely late, depending on the type of error. Timeout error, for example, is returned if the server takes too long to respond and would change the response time in your test to a higher value than it really is. On the other hand, errors that are immediately returned can make the response time look shorter than it really is.
We are not saying errors should be disregarded. Errors clearly indicate that something went wrong, and it must be investigated. However, you’ve put effort and resources into the test, and you need to be able to learn about the system performance from the test run even if it has error(s), so that a single error would not make you waste the whole test.
Back to topHow BlazeMeter Adds Clarity to Performance Testing Metrics
So, how can you see the real test results? A new feature in BlazeMeter was designed to allow you to temporarily eliminate background noises, focus only on requests that succeeded, and see what the test run would look like without errors.
Transactions Filter
On the top right corner of performance testing metrics reports, you will find a new filter called: Transactions. The Transactions filter allows you to filter the report data by requests that passed and requests that failed. Once applied, all the metrics, KPIs, and graphs in the report are being re-calculated based on the selected value.
In the Transaction filter you will find three options:
- Passed: calculates the report data based on requests that succeeded – http response code 200+
- Failed: calculates the report data based on requests that failed with http response code 400+, and other non-http errors such as Embedded Resource and Connection Lost.
- Failed – Assertions: calculates the report data based on requests that failed due to Assertions defined in the JMeter script.
What Filtering by Transactions Can Reveal
First, it can reveal the real response time of the system under test. In the example below, the report shows average response time of 174 ms and 90% response time of 420 ms, while when considering only Passed transactions and eliminating the errors impact, the 90% response time drops to 289 ms.
If that test contained a failure criteria on the 90% response time, that difference would have made a greater change, even turning the failure criteria status from Failed to Pass.
Back to topGet Performance Testing Metrics With BlazeMeter
While errors during test run are a meaningful red flag that must not be disregarded, temporarily eliminating them from the results can help you stick to the test goal and learn about the system performance. The new Transaction filter helps you do that, and better analyze the performance testing metrics.
Please note that this feature is only available to eligible BlazeMeter Enterprise customers. To learn more about JMeter check out BlazeMeter University.