Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
Dec. 14th, 2014

3 Common Issues When Running JMeter Scripts and How to Solve Them

JMeter is an awesome tool that provides excellent scripting capabilities. You can simulate almost anything using JMeter scripting; log in, posting of forms, various browsers and so on, BUT, executing tests using JMeter is a totally different matter.

 

And finding the cause of failed tests can induce a series of painful headaches.


BlazeMeter’s simple tips can actually help you reduce your dependency on aspirin and other pain relievers.


1. The Console Freezes

 

The JMeter console running in GUI mode can consume a lot of CPU and memory. Memory consuming listeners such as “View Results Tree” or any of the graphical reporting listeners can cause the console to freeze without providing any prior alert. The GUI console is excellent for running a small load and developing and/or debugging your script. We don't recommend running high capacity loads using the GUI console as in most cases, it will freeze halfway through the test.

 

Symptoms

 

Severe headache.

 

The GUI console is not responding.

 

What not to do

 

Do not bang your head against any wall or hard surface.

 

Do not use the GUI console for large capacity load.

 

Do not have graphical reporting listeners enabled while running.

 

Do not have memory consuming listeners enabled while running.

 

What to do

 

Take a deep breath.

 

Use the GUI console for script development and debugging only.

 

Use the GUI console for small loads (<50 simultaneous users).

 

Run JMeter headless for any load over 50 simultaneous users.

 

2. The Console Doesn’t Collect Engine Data (it’s frozen again) - Using Distributed Architecture

 

Unless specifically configured, all the data retrieved while running a test is going through the JMeter console. So, when running a test using a JMeter distributed architecture that might include, (for example) 4 engines all the data that each engine generates is tunneled through the JMeter console. That is A LOT of data.

 

This means that the console needs to deal with all data gathered from all the JMeter engines - in parallel. This can be an extremelly heavy burden for the JMeter console.

 

Most likely, 2 things will happen:

 

1. 100% CPU utilization

 

2. Out of memory exception

 

In both scenarios the console will freeze and will stop reacting to the engines. This is obviously accentuated if the console is used in GUI mode (which as mentioned, consumes a lot of memory).
Either way, the test will continue to run and the engine will continue to generate the load, however, reporting is out of the question as the console is too busy gathering the data and crunching the numbers to generate a report.

 

There is a way to inform the engines to NOT send all the data to the console, but rather, streamline it and only pass the statistical outcome (you will lose most of the reporting).

 

Symptoms

 

Profuse sweating.

 

If running in GUI mode - The GUI console is not responding.

 

In any case, check for exceptions in the engine’s logs:

 

2011/12/13 11:08:56 ERROR - JMeter.samplers.BatchSampleSender: sampleOccurred java.rmi.ConnectException: Connection refused to host: 10.110.61.79;

 

This means that the connection was refused by the console and in all likelihood the console is frozen.

 

The limitations imposed on the console are imposed on the entire test regardless of the number of engines used.

 

What not to do

 

Do not work with sweat dripping down your face and fingers as this will affect the keyboard and you may push the wrong keys.

 

Do not run in GUI mode.

 

Do not overstress your testing environment


What to do

 

Use a soft cloth to wipe the sweat from your furrowed brow.

 

Run headless.

 

Monitor the console vital signs (i.e. CPU and memory consumption).

 

See that the console is responding (for example that it continues to write to a JTL file).

 

Assuming you use strong resources for your test, keep the maximum hits per second lower than 1000 per second.

 

Keep the amount of bandwidth lower than 300MBs.
 

If you are using medium size resources do not surpass 300HPs and 100MBs per the entire test.

 

All these limits should be imposed on the entire test.
 

3. The Engine is Running Out of Memory

 

When running a test using a JMeter distributed architecture an engine can often dump its core or produce an out of memory exception. It can also freeze before being able to write this exception to the log. This is usually caused due to a too high number of threads or a too intensive test.

 

Either way, the engine will stop generating traffic and should log the exception to its log file.

 

If this happens, it means that the engine has stopped working and will probably need rebooting. The test will continue however, this engine will not provide the traffic that is required.

 

Odds are that if one engine fails the others will follow suit and the next test will need to be reconfigured in a different way.

 

Symptoms

 

Stress builds up in throat.

 

If the engine was able to log the exception to its log file, you can find exception like this (example):

 

2011/12/13 11:25:19 ERROR - JMeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Java heap space

 

JMeter logs

 

What not to do

 

Do not scream at your computer.

 

Do not overstress your testing environment.

 

What to do

 

Find your center, focus on your inner chakra of serenity.

 

Monitor the engines vital signs (i.e. CPU and memory consumption).

 

See that the engine is responding (look at its log file).

 

Test a single engine to see that it can actually sustain the load before running the full blown test. If one engine will run well, it can imply that the others will run well.

 

Keep between 100-300 for a medium size server.

 

Keep between 200-600 for a strong server.

 

If such an exception is encountered reduce the number of threads.
 

Major Points:

 

1. Run headless unless developing or debugging a script.

 

2. Do not overstress your testing resources.

 

3. Monitor the health of the servers on a regular basis.

 

4. Run pilots - run a test using one engine only to verify  that all systems are go.

 

5. Run the full test at least one time before running a live test - be advised that tweaking the test configuration is a regular occurrence and you may have to adjust the test and setting several times before it's ready to run live.

 

6. Be patient. A well configured test will run smoothly and provide excellent intelligence about the performance of your system allowing you to run the test as many times as so desired.

 

Conclusion

 

Following strict guidelines will lead to successful testing 100% of the time. A test must be configured properly otherwise it will always be a hassle trying to locate the root cause as to why the tests went bad.

 

Learn more about JMeter in our free Webinar "How to Make Performance Testing with JMeter Simple"

 

and for advanced users "How to Create Advanced Load Scenarios with JMeter"

Interested in writing for our Blog? Send us a pitch!