Jun. 27th, 2017

What’s the Max Number of Users You Can Test on JMeter?

In this blog post we are going to discuss a question that actually doesn’t have an exact answer - what’s the maximum number of users you can test on Apache JMeter™? Limits usually depend on many factors, and the same goes for JMeter limitations. JMeter limits depend on the capabilities of your machine and network, the complexity of your performance scripts, the targeted number of simulated users and so forth.

 

In the next couple of paragraphs we are going to hit different JMeter limitations and find workarounds to overcome each one of them.

 

We are are going to use a pretty ordinary local machine:

  • CPU: 1.4 Ghz Intel Core i5
  • RAM: 4GB 1600 MHz

 

Create a Performance Script for 50 Users

 

But first things first. We need to create a performance script that can be used to hit different JMeter limitations. The simple script will simulate a load against the http://blazedemo.com/ web application. All we need to do is to create:

 

1. Thread Group - Start from 50 users, 2 minutes ramp-up and a 5 minutes test.

 

maximum jmeter users - creating a thread group

 

2. HTTP Request sampler - Named “Find flights request”, sending a Post request to blazedemo.com/reserve.php with the fromPort=Paris and the toPort=Buenos Aires. 

 

maximum jmeter users - add http request

 

3. View Results Tree listener - for debugging purposes.

 

maximum jmeter users - debug your script

 

4. Graph Results listener - to verify load metrics.

 

how many jmeter users can you test?

 

5. Put the HTTP sampler into an infinite loop request through the Loop Controller. This holds each thread’s load until the test is finished.

 

the largest number of users jmeter supports

 

6. Add an assertion element for validating the response. Based on our request, we should get a flight from “Virgin America” airline (you can check that out through the blazedemo site). We can use that as a text response assertion.

 

how many users can jmeter performance test?

 

Our script is done and we are ready to run it. So click the ‘start’ button.

 

After 5 minutes you should see that the test ran successfully and all threads were created and terminated successfully. Looks simple, right?

 

Run Your Load Test for 500 Users

 

Now let’s try to run the same script with 500 users. Change the number of users in the Thread Groups to 500 threads, and ensure the script ran successfully.

 

running jmeter for 500 users

 

Run Your Load Test for 1,000 Users

 

During our last test execution we simulated 500 users, using our own not so powerful laptop. Such a load usually is enough for an application that has a small target audience. But what if we need to simulate more? Let’s try to run the same script with 1,000 users this time.

 

A bad thing happened this time. While adding the 743th user, my JMeter UI application got stuck and stopped responding. In the JMeter logs the following errors appeared:

 

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid1824.hprof ...
Heap dump file created [147668693 bytes in 39.846 secs]

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread Group 1-743"

 

Such an error means that we reached the Java heap capacity allocated for the JMeter application. You can find more general information about Java heap here.

 

Tip #1: Use JMeter Listeners for Debugging Purposes Only

 

We have reached our first limitation, and we should learn this first piece of advice from it: “Use JMeter listeners for debugging purposes only!”.

 

JMeter listeners generate a lot of redundant load on your local machine, and they create lots of objects in the heap that might occupy most of the heap space. There are many other ways to monitor your scripts without generating a heavy load on your local machine. In this article you can find three ways to do it right. 

 

But our goal is to check how many users JMeter can simulate. To do that we can just remove listeners and verify if the script can be run successfully with the same 1,000 users as the target load.

We clicked the button and…... the script got stuck again. This time I found the script failed while adding the 921th user:

 

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid2083.hprof ...
Heap dump file created [150444856 bytes in 12.757 secs]

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread Group 1-921"

 

Tip #2: Run JMeter Tests in Non-GUI Mode

 

We see that this time we were closer, but still couldn’t simulate a load for 1,000 users. From this case we can learn the next piece of advice: “Run JMeter performance tests in Non-GUI mode” when you need to simulate a larger load.
JMeter GUI is wonderful when you need to create, edit and debug your tests, but is not a great solution for running created tests. The same article mentioned before explains how you can run your tests in Non-GUI mode and save lots of resources to simulate a larger load.

 

To run the test in Non-GUI mode you need to save it as a JMX file (let’s call it ‘JMeterLimitationsTest.jmx’) and execute this specified command:

 

jmeter -n -t JMeterLimitationsTest.jmx

 

Let’s run our script and… we can see that test execution went fine without any errors:

 

running jmeter for 1000 users

 

This is why it is highly recommended to run your load scripts in Non-GUI mode. By using that simple action, we can save lots of machine resources and avoid nasty memory issues.

 

Run Your Load Test for 2,000 Users

 

Okay, now let’s go further and increase the number of users by 2 times to reach a target load of 2,000 users. To do that, open the same JMX file by using the JMeter GUI, change the number of users in the thread group, and save the file with the same name.

 

We clicked on the test execution again and….

 

running jmeter for 2000 users

 

Tip #3: Increase JMeter Heap Space to Generate a Larger Load

 

We are out of memory again. This time it happened after simulating 1,191 users. Can we move further? Of course the answer is yes! We have lots of useful advices and now it’s time for one more: “Increase JMeter heap space to generate a larger load”.

 

The JMeter heap has default limits that can be easily increased without any harmful effect for test execution. Just open your JMeter runner script (located in the JMeter bin folder) and replace the value of max heap space (which goes after -Xmx flag) that has these default values:

 

HEAP="-Xms512m -Xmx512m"

 

Allocate 2 GB (2048 MB) RAM:

 

HEAP="-Xms512m -Xmx2048m"

 

Now we can run the same test again with 2,000 users. We clicked the button again and…

 

running jmeter for 2000 users

 

Tip #4: Use LAN instead of Wi-Fi Connections When Running Load Tests

 

We don’t have any out of memory issues anymore but you can see that after we ran almost 2,000 users, we started getting lots of failed requests for our test.

 

I’m not going to write a boring description of debugging the root of cause, but I will just give you the following advice: “Use LAN and never use Wi-Fi connection to simulate more users”, even if you think that your Wi-Fi is strong enough. All the previous test executions were performed while using Wi-Fi connection. The connection was working fine until this moment, but to go further we need to switch on the LAN internet connection.

 

The switch is done, now let’s try to execute the test again.

 

executing load test for thousands of users

 

This time the test passed without any issues. We can confirm that a regular laptop can simulate 2,000 users and we should now try to go for the next limitation.

 

Run Your Load Test for 10,000 Users

 

We simulated JMeter for more and more threads. As a result, we found that the max value of users we could simulate was around 8000-9000 users. If we added more, we got errors about available memory right after test execution  This time it is not about heap space, but rather based on the limits of my local machine’s RAM.

 

run your load test for 10,000 users

 

At this point there might also be many random errors, rendering test execution irrelevant.

 

In conclusion, we can say that you can simulate up to 10,000 users on a JMeter load test even on a regular laptop. This depends on following 4 simple rules:

  • Use JMeter listeners for debugging purposes only
  • Run JMeter performance tests in non-GUI mode
  • Increase JMeter heap space to generate a larger load
  • Use LAN instead of a Wi-Fi connection when running a load test

 

As you can see, the answer to the question ‘What’s the maximum number of users you can test on JMeter?’ varies. It depends on many factors: test complexity, script running method, internet and machine capabilities and so forth.

 

I have found the user limit on my local laptop is 8000-9000 users. But what if you need more? There are options to do that:

1. JMeter enables running scripts in distributed mode by using a few slave machines with your own main one, and then combining and outputting the results. With this method you can scripts horizontally without any limitations except for the number of slave machines you allocate for tests.

 

2. You can also always use cloud solutions. BlazeMeter provides an easy-to-use cloud testing platform, enabling you to run as many users as you wish.

 

To try out BlazeMeter, request a demo or put your URL or JMX file in the box below and your test will start in minutes.

 

Learn everything you need to know about JMeter for free from our JMeter Academy.

 

     
arrow Please enter a valid URL

Interested in writing for our Blog? Send us a pitch!