Why ‘Normal’ Load Testing Isn’t Enough
Load testing is a vital part of any web-based application testing process.
It helps us determine a system’s behavior under both normal and anticipated peak load conditions. It also helps identify the maximum operating capacity of an application, potential bottlenecks and elements that may be contributing to performance degradation.
But is your load testing process good enough? Could you be missing key issues like serious memory leaks, failures in specific modules or graduated degradations of response times because your load testing is too limited?
To answer this question, it’s worth looking at the different types of load testing.
Normal Load Testing
This type of testing simulates typical user behavior for a short period of time, such as one hour. The normal load test is the basic scenario; it assumes the normal day-to-day usage scenario for a limited period of time to check that an application will behave as expected. This should be performed multiple times for each build to gain a general idea of how your application behaves.
Most developers & testers run these types of tests on their web and mobile apps. But if you’re only running these types of tests, you’re neglecting two key types of load testing which can identify a wider and deeper range of issues.
1. Soak Testing
Soak testing checks general application behavior under a typical load over a significant period of time. This identifies how the system will behave under sustained use. The main advantages of soak testing is that it reveals:
- Serious memory leaks that will eventually cause the application or operating system to crash
- Failures in closing connections between different layers or modules of your system, which might cause your application to crash
- General graduated degradation of response times - on prolonged tests or prolonged sessions an application tends to become less efficient as it handles more and more requests, log files grow in size, more data is pushed into the system, and so on.
However, carrying out soak testing and understanding the ensuing reports is more complicated, and it requires that the developers to be involved in the analysis of the data.
2. Stress Testing
Stress testing attempts to identify the point of failure in a specific system component likely to create a bottleneck or failure by placing an unusually high load on the system. If you have a specific module or action that is critical for your application, and you need to identify the potential breaking point or bottlenecks, then stress testing is important.
For example: the home page typically gets the largest number of hits, so you try to stress it to see when it will break. Another example might be the checkout page on an e-Commerce site. You create bursts of users hitting the page. If this page crashes, it will break a large part of the system. Although not the main purpose of stress testing, stress testing can also take normal user behavior and try to crash the system using multiple transactions on different components. This approach, however, is less effective than normal stress testing.
Load Testing Implementation
Before you implement load testing for your application, you need to understand the behavior of the application’s users and the relevant business scenarios. Using BlazeMeter’s load testing platform you can set the number of concurrent users and the ramp up. The ramp-up is the amount of time it takes to ramp up from 0 users to the maximum number of users.
For a normal load test, you would load your normal amount of users, with a medium size ramp up time. For example: 10 minutes for a one-hour test.
For soak testing, as you want to see more data about memory leaks and connections between layers, and at a wider resolution for the application's performance, you need to set a slower ramp-up. For example: for a 3-4 hour soak test, you might set a 45-minute ramp-up period. This will enable you to see any degradation or small bottlenecks that occur. In soak testing, it is also important to set the ramp-down time. In BlazeMeter, ramp-down is the time that it takes to kill users until you reach zero.
For soak testing, it is important to have a slow ramp-down time to view the changes in memory usage as the number of users decreases. For example: the absence of a correlation between the number of users and memory usage can be indicative of a memory leak or other issues.
For stress testing, you have no idea where your application is going to crash so, as a rule of thumb, multiply the normal number of users by two for testing purposes and apply a fast ramp-up to identify the breaking point. If it doesn't break, adjust the ramp-up and repeat the test to identify the breaking points.
Final Words of Advice
It’s important to include all the above types into your load test plans and preparations. Ideally, you’d start with a normal load test to get a general sense of how your application will behave, then go on to a soak test to identify how your system will cope under sustained use. All the while continually fixing your code in order to stabilize your application. Once that is done, the stress test will come into play to ensure that your application is ready to realize demand.