Performance Testing vs. Load Testing vs. Stress Testing
If you’re a developer, QA engineer or DevOps, you know you need to conduct different types of performance tests, to make sure each code change or feature added (a) doesn’t break the system, and (b) works.
But which types of performance tests should you conduct, what’s the difference between load tests and other types of tests and which test is suitable for which situation? In this blog post we’ll cover the answers to these questions, and more.
What is Performance Testing?
Performance testing is the general name for tests that check how the system behaves and performs. Performance testing examines responsiveness, stability, scalability, reliability, speed and resource usage of your software and infrastructure. Different types of performance tests provide you with different data, as we will further detail.
Before performance testing, it’s important to determine your system’s business goals, so you can tell if your system behaves satisfactorily or not according to your customers’ needs.
After running performance tests, you can analyze different KPIs, such as the number of virtual users, hits per second, errors per second, response time, latency and bytes per second (throughput), as well as the correlations between them. Through the reports you can identify bottlenecks, bugs and errors, an decide what needs to be done.
When should you use Performance Testing?
When you want to check your website performance and app performance, as well as servers, databases, networks, etc. If you work with the waterfall methodology, then at least each time you release a version. If you’re shifting left and going agile, you should test continuously.
This is an example of a performance test report on CA BlazeMeter. This is a good test, as the growing number of users doesn’t affect the response time, the error rate continues to be low and the hits per second rises with the number of users but remains stable.
What is Load Testing?
Load testing is testing that checks how systems function under a heavy number of concurrent virtual users performing transactions over a certain period of time. Or in other words, how systems handle heavy load volumes. There are a few types of open-source load testing tools, with JMeter being the most popular one.
When should you use Load Testing?
When you want to determine how many users your system can handle. You can determine different user scenarios that let you focus on different parts of your system, like the checkout webpage on your website or app for web load testing. You can also determine how the load behaves, like the geo-location users come from or how the load builds and sustains in the system. Basically, load testing is something you should do all the time, to ensure your system is always on point. That’s why it should be integrated into your Continuous Integration cycles, with tools like Jenkins and Taurus.
What is Stress Testing?
Stress testing is testing that checks the upper limits of your system by testing it under extreme loads. The testing examines how the system behaves under intense loads, and how it recovers when going back to normal usage, i.e are the KPIs like throughput and response time the same as before? In addition to load testing KPIs, stress testing also examines memory leaks, slowness, security issues and data corruption.
Stress testing can be conducted through load testing tools, by defining a test case with a very high number of concurrent virtual users. If your stress test includes a sudden ramp-up in the number of virtual users, it is called a Spike Test. If you stress test for a long period of time to check the system’s sustainability over time with a slow ramp-up, it’s called a Soak Test.
When Should You Use Stress Testing?
Website stress tests and app stress tests are important before major events, like Black Friday, ticket selling for a popular concert with high demand or the elections. But we recommend you stress test every once in a while so you know your system’s endurance capabilities. This ensures you’re always prepared for unexpected traffic spikes, and gives you more time and resources to fix your bottlenecks.
In this example we describe how to obtain the traffic spike using “Ultimate Thread Group” component. We suppose that system under test is under traffic at 3:00 minutes after test start. With Ultimate thread group we define additional threads running in fixed time windows using initial delay property (see below picture)
Using different types of performance tests ensures you are always aware of the issues you have and that you can plan how to deal with them. So don’t give up on any types of tests.
Learn more on how to use JMeter from our free Webinar JMeter Load Testing at Scale and from our blog. JMeter can be enhanced through BlazeMeter, with larger scalability and many geo-locations, better reporting and sharing capabilities, multiple integrations and recording options. Request a demo to learn more. To try out BlazeMeter and set up a trial account, put your URL in the box below and your test will start in minutes.