Performance Testing vs. Load Testing vs. Stress Testing
If you’re a developer or QA engineer or if you work in DevOps, you know you need to conduct different types of performance tests in order to make sure each code change or feature addition (a) doesn’t break the system and (b) works.
But which types of performance tests should you conduct, what’s the difference between load tests and other types of tests and which test is suitable for which situation? In this blog post we’ll cover the answers to these questions and more.
What is Performance Testing?
Performance testing is the general name for tests that check how the system behaves and performs. Performance testing examines responsiveness, stability, scalability, reliability, speed and resource usage of your software and infrastructure. Different types of performance tests provide you with different data, as we will further detail.
Before performance testing, it’s important to determine your system’s business goals, so you can tell if your system behaves satisfactorily or not according to your customers’ needs.
After running performance tests, you can analyze different KPIs, such as the number of virtual users, hits per second, errors per second, response time, latency, and bytes per second (throughput), as well as the correlations between them. Through the reports you can identify bottlenecks, bugs, and errors, then decide what needs to be done.
When should you use Performance Testing?
Run performance tests when you want to check your website and app performance, which may extend to testing servers, databases, networks, etc. If you follow the waterfall methodology, then test at least each time you release a version. If you’re shifting left and going agile, you should test continuously.
The following figure shows an example of a performance test report on BlazeMeter. This is a good test, given the growing number of users does not affect the response time, the error rate remains low, and the hits per second rise with the number of users, but remain stable.
What is Load Testing?
A load test is a type of performance test that checks how systems function under a heavy number of concurrent virtual users performing transactions over a certain period of time. In other words, the test measures how systems handle heavy load volumes. There are a few types of open-source load testing tools, JMeter being the most popular.
When should you use Load Testing?
Load test when you want to determine how many users your system can actually handle. You can configure tests to simulate various user scenarios which can focus on different parts of your system (such as a checkout page, for example). You can determine how the load behaves when coming from different geo-locations or how the load might build up, then level out to a sustained level. Load tests should be performed all the time in order to ensure your system is always on point, which is why it should be integrated into your continuous integration cycles (utilizing tools such as Jenkins and Taurus.)
What is Stress Testing?
A stress test is a type of performance test that checks the upper limits of your system by testing it under extreme loads. Stress tests examine how the system behaves under intense loads and how it recovers when going back to normal usage. Are the KPIs like throughput and response time the same as before spike in load? Stress tests also look for memory leaks, slowdowns, security issues, and data corruption.
Stress testing can be conducted through load testing tools by defining a test case with a very high number of concurrent virtual users.
Just as a stress test is a type of performance test, there are types of load tests as well. If your stress test includes a sudden, high ramp-up in the number of virtual users, it is called a Spike Test. If you stress test for a long period of time to check the system’s sustainability over time with a slow ramp-up, it’s called a Soak Test.
The following example shows how to create a traffic spike using JMeter’s “Ultimate Thread Group” component. We presume the system will be under traffic three minutes into the test. We define more threads to be added within fixed time windows using the “Initial Delay” setting.
When should you use Stress Testing?
Run stress tests against your website or app before major events, like Black Friday, ticket selling for a popular concert with high demand, or elections. We recommend stress testing every once in a while so you know your system’s endurance capabilities. This ensures you’re always prepared for unexpected traffic spikes and gives you more time and resources to fix your bottlenecks.
Utilize all the types of performance tests detailed here to ensure you are always aware of any issues and can have a plan for dealing with them.
Learn more from our free Webinar JMeter Load Testing at Scale and our blog. JMeter can be enhanced through BlazeMeter, which provides larger scalability, many geo-locations around the world, better reporting and sharing capabilities, multiple integrations, and recording options. Request a demo to learn more. To try BlazeMeter out, set up a free account, put your URL in the box below, and then your test will start in minutes.
This blog post was originally published in 2019 and was updated for accuracy in December 2020.
<?php include path_to_theme()."/templates/_test_wizard_url_widget.tpl.php" ?>