How to Prepare Your UI Vs. API Load Test
Before discussing UI and API load tests, there are a few key differences we should take into consideration. For a UI, you want to test the load of an actual user scenario. For example: logging in, adding items to a cart, filling out a form, and so on. Both tests are usually done by generating HttpWebRequests, however the API load test output will be in a JSON or XML format (no embedded objects involved) whereas a UI test returns the actual page.
UI Load Tests with JMeter
For the full scenario effect, we can use JMeter to simulate a great demand from virtual users. In this case, we highly recommend you use BlazeMeter’s Chrome extension to record the entire UI session. While recording a UI scenario, the extension records all of the HTTP/S requests that the browser makes, as well as the thinking time for each page, then creates a JMeter script, and automatically uploads it to BlazeMeter, where you can execute your JMeter load test. Learn how to use our Chrome extension here
For a complex scenario, like logging in or filling out a form, you will need to implement a dynamic variable. For example: in Drupal, you might want to catch the form build ID and pass it on with the login post request, because these are created in real time.
We also recommend incorporating the JMeter test with a Selenium test, in order to measure the real response time for any point in the flow. For example: if the load test reaches 20K users and hits a bottleneck with a 3 second response time, we want to understand how our real-time users are affected. The Selenium test report enables you to determine how long it actually took the browser to process the entire request including the image rendering time. It can teach us for example, how the 3 seconds response time (reported by JMeter) affected the actual user experience.
Prepare Your UI Load Test
To prepare for a UI test, you want to create a script for your target number of users, based on a real user scenario. Generally, we look at users vs. response time. The response time can be 500 ms or even 300 ms. In the worst case, it can take up to 3 to 3.5 seconds. Beyond that, the user experience is so distressing that canceling the test would be the best option.
Because JMeter downloads everything, it is important to calculate the number of users per JMeter engine. If your scenarios ask for an HTTP link that has a large number of JPEG files and you request to download all of the embedded objects, JMeter downloads everything, and saves it in the memory until the request ends. In this case you need to examine the load engines to determine the maximum feasible number of users per engine.
For UI load tests, response time is the KPI that you want to monitor. Of course, you also want to check for errors (such as 500/501, socket closed, socket timeout) to identify the bottlenecks on your web server.
API Load Tests
API load tests are not as common as old-fashioned UI load tests. Companies that use API for load testing do so because it is easier to maintain when you support multiple sites and devices (desktop and mobile sites).
Because we want to know how many requests our API server can handle in a timely fashion, the key parameter to consider in API load tests is the number of hits per second. If we look at response time for API, we will see a very short response time (from 5 to 200 ms). This is because the APIs and their responses need to be as fast as possible. It is also because you don’t typically download files from an API that makes the response time very short. When testing the API, we can integrate larger number of threads on a single load engine. In this case we might want to look at the CPU instead of the memory.
Prepare Your API Load Test Script
When setting the script, we usually want to use all of the restful API commands (CRUD). We also want to add assertions to our HttpWebRequests to verify the response. When a request is successful from the web server perspective, the server always returns "200:OK". Nonetheless, the data may indicate that the request was not OK; it might have contained some kind of error or fault. For this reason, we add assertions to our HttpWebRequest to verify whether or not a specific value exists in the response body and that there are no errors in the response.
In addition, we can add a constant throughput timer to allow us to control the number of hits per second. This can be enhanced using BlazeMeter. For example, assume that with 1,000 users, you see 500 hits per second, and you want to test how many hits per second our system can handle. Instead of using 1000 users, by adding 20K users and a small change to the script, BlazeMeter allows you to pace the hits during the load test. You could start with 500 and increase the load as you go in real-time to identify the breaking point. For analysis purposes, hits per second is the most important KPI for API testing.
Know your test KPIs and when to use them.
When running an API test, add assertions to the HttpWebRequest to examine specific values, find errors and understand the scenario.
To get the user experience for UI scripts, you should use Selenium web driver. For APIs you usually don't need the user experience, unless it's a mobile native app - then you can use BlazeMeter’s mobile visualisation feature.
- Measure users vs hits – the amount of concurrent virtual users per minute vs. the amount of hits per second.