How (& Why) to Get More Info on Your Load Test Errors
The application failed. A bug was found. What more do you need to know? If you’re running a load test and hoping to debug accordingly, the answer is “a lot!” Unfortunately, many load tests are conducted in ways that do not generate much (if any) actionable information apart from the main error. You need to know more. Here’s how to get it.
In this blog post, I wanted to offer a very brief step-by-step guide on how to get more information from your load test errors – things like response headers, response messages and so forth – using JMeter. Before I do, one very important point of clarification:
The tips I’m about to offer are not intended for large-scale load tests. Why? Because the changes described here will actually cause your script to affect performance on the load engine due to the high I/O.
With that out of the way, let’s take a closer look at the 2-step process (yes, it’s really that simple).
Step #1 - Configuring JMeter to Create a Detailed Error Log
Once you’ve opened your script in JMeter, go ahead and choose: add -> listener -> simple data writer
From there, you’ll want to click on the new element in the test plan (under the main configuration page) and ensure these configurations
Now select on the button labeled 'Configuration', and choose the following values
Step #2 - Viewing Results
At the conclusion of your test, go to the directory you ran JMeter from, and you'll see another artefact named 'errors.jtl'. Here is where all of your detailed errors can be found. You can also open a new instance of JMeter on your machine, and simply add a “view results tree” listener.
Click on the new test element, and from its main panel click on 'browse', and find the 'errors.jtl' file
Once the file has been opened by JMeter, you will see all the errors that happened during the test, with a lot more details, giving you and your team a much clearer view of what went wrong.
Easy enough, right? It’s also worth noting that you can get detailed error logs from BlazeMeter as well. Again, it’s critical to understand that running a JMeter script with these configurations causes a lot of I/O on the load engines, which might cause performance issues for JMeter itself, and thus decrease the hit rate that the engine is capable of, or even cause some errors in the test. The key point is this: use it for small scale tests and debugging sessions only. This process involves the following steps:
1. Create and run a test
2. When the test ends, go to the report page, click on the tab labeled 'Logs', find the section labeled 'jtls_and_more.zip", and click on 'download'
3. In the directory you can now find a file named 'errors.jtl', open it in the same way in JMeter, and view the errors
Why It Matters
In order to identify bottlenecks and resolve issues in a timely manner (ideally, during the actual test) developers, QA testers and Ops needs to have as complete a picture of the errors as possible. This process requires a great deal of collaboration and communication, and with incomplete or insufficient findings, lots of time will be wasted.