It seems simplistic to say it: the more control a tester can have over the performance tests they are executing, the better and more efficient that performance test will ultimately be.
That control can come in the form of mock services that help remove testing bottlenecks. It can come in the form of streamlining test data usage with artificial intelligence. Or — as with the latest BlazeMeter release — it can come in the form stopping a performance test before it is completed when you know ahead of time that it will fail.
In this blog, we will discuss the newest BlazeMeter feature — Test Automation Failure Criteria — and how it benefits testing teams and helps them shift their testing left.
What is Test Automation Failure Criteria?
Traditionally, users would have to wait for a performance to be complete to optimize it, despite knowing it would fail.
Now, BlazeMeter users can issue a STOP order when their Taurus tests are reaching the failure threshold. Doing so saves two crucial things — time and resources.
BlazeMeter users can now convert Taurus’ Stop/Continue and Timeframe orders to BlazeMeter’s Failure Criteria with the newly available Stop and Sliding Windows, respectively.
Run Taurus tests in the cloud with a Pass/Fail module that includes a Stop order and have the test run in BlazeMeter — stopping when the test is reaching a threshold as defined in the Taurus yaml script.
How It Works
Like other Taurus pass/fail criteria, baseline-based criteria in YAML are translated to BlazeMeter failure criteria and appear in the test configuration UI.
KPI: the Key Performance Indicator defined in the pass/fail criteria
Condition: the comparison operator in the pass/fail criteria will be translated to the parallel phrase, e.g. “>=” will be translated to "Greater than or equal to".
Threshold: the value is calculated from the baseline and offset. For example, if average response time in baseline is 100 ms and the criteria had a “+5%” offset, the threshold would be baseline+5% which is 105 ms.
Use from baseline: true
Percentage offset = the percentage defined in the pass/fail without the +/- sign. The +/- sign will be used to add or reduce the offset percent from the baseline when calculating the threshold.
The Benefits of BlazeMeter’s Test Automation Failure Criteria
With this latest feature, BlazeMeter users now have more control over their performance tests. Test Automation Failure Criteria allows BlazeMeter users to:
Save Time: Identify when the Taurus test will failure before waiting for the test to conclude, allowing for faster test optimization and increased test velocity.
Create Better Tests: Creating better tests becomes easier when testers are able to view developing test results in real-time.
Save Resources: Creating a more efficient testing strategy by having more control over your performance tests will invariably lead to saving money and other valuable resources.
Bottom Line
At BlazeMeter, we are always striving to make the lives of testers easier. Through enhanced functionality, new features that streamline processes, and an unrivaled one-stop-shop platform, users are continually set up for success with BlazeMeter.
With BlazeMeter’s Test Automation Failure Criteria, testing teams can take better control of their performance tests by unlocking the ability to stop a test before it starts when a pre-defined threshold in the test script is about to be reached. Teams can now be far nimbler and more agile in their testing strategy.
Ready to take your performance testing to the next level? Start testing with BlazeMeter for FREE today!