The universal truth about performance tests is that they eat up machine resources. These resources include CPU, users, and memory. Consuming these resources at a fast rate can affect the health of your performance test — causing it to top out and fail much like an overstressed engine. Testers would unknowingly be stressing their machine CPU and memory too much.
Thus began the tedious process of tweaking performance test metrics so the consumption of resources would be more manageable (which likely takes several attempts) and start the test over again — with fingers crossed it works this time around.
Sounds frustrating and time-consuming, doesn’t it? That is because it is.
When BlazeMeter customers flagged these tests for failing for seemingly no reason at all, we listened. With the new Health Metrics feature in the BlazeMeter performance testing dashboard, users can now leverage automated recommendations to help them properly optimize their tests.
BlazeMeter’s Health Metrics feature removes the tediousness of recalibrating performance tests over and over again in the hopes that this iteration of the performance test metrics are the correct ones.
What does that mean? Let us dive in.
How BlazeMeter Health Metrics Optimize Performance Tests
BlazeMeter Health Metrics is a solution to a problem we have heard about from numerous customers. When performance tests failed, testers would often not know why. Metrics would redline suddenly, causing unusual results or outright failures.
When BlazeMeter support would be contacted, more often than not the culprit would be the engine health. After viewing the metrics, for example, if the CPU is running above 80%, support would send the user a message to address the problem by adjusting the number of users.
When conducting performance tests, it is recommended to maintain the average CPU and memory usage below 70%. The thresholds for engine instances are:
- 80% for CPU
- 85% for memory.
These averages are calculated using a 60-second moving average of the data points collected throughout the test duration.
To illustrate, consider a specific 60-second period within a test run, ranging from 1 minute and 20 seconds to 2 minutes and 20 seconds. In this interval, a total of 30 measurements are taken at 2-second intervals, divided into three sets of 10 consecutive measurements each. The first set averages 50%, the second 70%, and the third 90%. As a result, the 60-second moving average for this period amounts to 70%.
If the average of the 60-second moving averages recorded throughout the entire test duration surpasses the threshold set for a particular resource type (CPU or Memory), an on-screen alert will be displayed — indicating the need to adjust the number of users in the test.
Now, Health Metrics automates that.
How It Works
When a performance test is running and memory or CPU message starts to creep towards the threshold, BlazeMeter generates a warning that high usage was detected and provides a link to view the Engine Health.
With convenient filters, you can toggle between engines to identify the issue and proceed to debugging.
When the performance test concludes, the link is switched from Engine Health to a summary page that offers recommendations for an optimal test scenario: original value, suggested value, and suggested action.
Then, users simply hit “Apply” and will be taken to the load distribution screen to recalibrate their performance test with confidence.
Bottom Line
BlazeMeter’s primary motivation has always been the success of our customers. We built this automated Health Metrics feature to further help them along their performance testing journey.
BlazeMeter’s Health Metrics will help testers understand their infrastructure and determine if their testing efforts are outpacing their machines, engines, or other vital resources. It will help them make informed decisions that will impact their entire testing strategy.
Ready to get started? Experience BlazeMeter’s latest performance testing offering and get started for free!