What is Performance Testing?

 

Performance testing and load testing are the practices of ensuring websites and apps perform under heavy loads, from different geo-locations and for different user scenarios. If you followed the latest Pokemon Go craze or heard about Macy’s crashing during the last Black Friday, you know the importance of Performance Testing and how crucial it is for businesses. Poor performance, whether website crashes or slow page loading, equals an immediate and long-term loss of revenue, as it creates a bad reputation and immediate churn.

 

An Introduction to JMeter

 

Therefore, it is important to carefully choose your load testing tools. JMeter is the most popular open-source load and performance testing tool. Open-source and JAVA-based, JMeter simulates browser behaviour (though it’s not a browser!) by sending requests to web or application servers for different loads. JMeter can also parse the responses. On your local machine, you can scale up to approximately 100 virtual users, but you can go up to more than 1,000,000 VUs with CA BlazeMeter, which is kind of a JMeter in the cloud.

 

This blog post is a basic tutorial about how to get started with JMeter. We will create a few basic JMeter scripts and go over different JMeter features like assertions and dynamic data. In the end, we will briefly talk about analyzing your results in reports.

 

Your First JMeter Script

 

First, download JMeter.

 

This is the JMeter interface:

 

 

The Test Plan is your JMeter script, and it determines the flow of your load test.

 

In this post, we will load test the demo we always use at CA BlazeMeter, of a mock travel agency named Simple. The Simple Travel Agency lets us search and choose flights, and we will simulate these actions through JMeter.

 

 

Thread Groups

 

To start building the test script, go back to JMeter and add a Thread Group.

 

Thread groups determine the user flow and simulate how users behave on the app. Each thread represents a user.

 

Right click - Add->Threads(Users)->Thread Group

 

 

Configure the Thread Group -

- Its name (for your convenience)

- Number of threads - the number of users you are testing. Let’s say 3.

- Ramp-up time - how much time you want to allow the Thread Group to go from 0 to 3 users. Let’s say 5 seconds.

- Loop count - How many times the test should be looped. Let’s say 1 time.

 

 

Samplers

 

We want to send a HTTP request to our site, in this case to our demo, so we add a HTTP Requests Sampler.

 

Right click - Add->Sampler->HTTP Request

 

 

Configure the Sampler -

- Its name (for your convenience)

- The server or IP you’re testing. In this case - blazedemo.com

 

 

Timers

 

When users click on your website or app, they naturally have pauses and delays. These can  be simulated with Timers.

 

Constant timers are the most common, and they determine how many milliseconds to wait before between requests.

 

Right click - Add->Timer->Constant Timer

 

 

In this case, we will wait for 300 milliseconds.

 

 

Listeners

 

After running our test, we want to see its results (obviously). This is done through Listeners, a recording mechanism that shows results, including logging and debugging.

 

The View Results Tree is the most common Listener.

 

Right click - Add->Listener->View Results Tree

 

 

We’re done! This is the basic script we created for hitting the BlazeDemo page:

 

 

Now click ‘Save’. Your test will be saved as a .jmx file.

 

To run the test, click the green arrow on top. After the test completes running, you can view the results on the Listener.

 

In this example you can see the tests were successful, because they’re green. On the right you can see more detailed results, like load time, connect time, errors, the request data, the response data, etc. You can also save the results if you want to.

 

 

Your Second JMeter Script

 

Congratulations! That was your first script. Now let’s create a second one, and add parameters to it. In our example, we want to also choose the flights.

 

 

How do you simulate that?

 

Go back to JMeter, add another HTTP Sampler, and this time, add the website path after the server. In our example - reserve.php.

 

Afterwards, add the relevant parameters from your website or app. In this case: fromPort, Boston; toPort, London.

 

 

We’ll also add a “View Results in Table” Listener, just to spice things up.

 

This is the second test plan, which includes hitting the Home page and then sending two parameters to the Reserve page.

 

 

Save and run the test.

 

In the results, you can see that each page is hit 3 times:

 

 

The View Results Table shows us additional data: start time, sent bytes, connect time, latency, etc.

 

 

Recording Scripts - Your Third JMeter Script

 

Congratulations! You can now create your own basic JMeter scripts. But as you might have understood, if you need to create a long user flow, it could take you a long time.

 

Therefore, a better way to create scripts is by recording them. To record, you can either use the JMeter recorder, or the BlazeMeter Chrome Extension. The BlazeMeter extension, free to use from the Chrome store, is more user friendly than the JMeter one, which requires you to set a proxy to redirect the traffic.

 

Through your Chrome, start recording and simulate the user scenario you want to test by clicking away. When you’re done, stop the recording, and edit as necessary.

 

 

Export your recording to JMX. Don’t forget to move the downloaded script to your preferred folder.

 

 

Open the .jmx file file in JMeter. You will be able to see your test plan, which was created from the recording and the .jmx file.

 

 

Note that this plan has new elements, like Cookie Manager and Cache Manager. They are here because browsers keep cookies and cache and they were captured in the recording. You can clear them up if you need to, and sometimes it’s even necessary to ensure you are really simulating new users.

 

 

Now add a listener and run the test.

 

Assertions

 

Assertions are elements that let you check for errors, or in other words - determine if your test passes or fails.

Let’s say we want to make sure a webpage returns the information we’re looking for. In our example, we want to make sure users who purchase flights receive a message on the confirmation page saying - Thank you for your purchase today!

 

 

Go to JMeter.

 

Add a Response Assertion (you know how to do it).

 

Add the exact characters you want users to see.

 

 

Save and run the test.

 

If the response contains the string, the test will pass. If not, the test will fail, and it will also detail why.

 

Dynamic Data

 

What happens when you want to create a dynamic script, which chooses different parameters each time you test, like passwords, login information or search criteria? This is what Dynamic Data through CSV files is for.

 

Create a CSV file on your computer, with the different variables you are testing. Put the file in the JMeter folder. In our case, we created a basic one, with departure and destination cities.

 

 

Got to JMeter. Right click - Add->Config Element ->CSV Data Set Config

 

 

Configure by adding the variable names. In our case, fromPort and toPort.

 

 

Go back to the HTTP Request (from our second script) and change the variable from the specific name (of the cities) to the general name.

 

 

The data tested will now come from the CSV file, and you will be able to see the dynamic results in the View Results Tree. In our example, it’s no longer Boston and London, but Philadelphia and Berlin, Portland and Rome, etc.

 

 

Scalability

 

After you built your test and checked it for a low number of users, it’s time to scale it up and check a large number of VUs. How many? That depends on your business goals.

 

In general, we recommend that in addition to your expected number of users, you bring your test to the limit. This lets you characterize your system’s strengths and weaknesses, enabling you to plan better and also react in real-time to unexpected bottlenecks and errors.

 

The best way to scale is through CA BlazeMeter, which lets you run heavier loads than JMeter. CA BlazeMeter lets you upload your JMeter scripts, scale your number of users, choose your region, set ramp-up time and determine how long the test will run.

 

Analyzing Results

 

After running your test on CA BlazeMeter, you will get rich and in-depth reports, with KPIs like throughput, error rate and connection time. Reports let you analyze trends and statistics, determine the health of your system and decide how you want to keep developing your product.

 

 

Congratulations! You just finished your basic JMeter tutorial. If you prefer viewing the content this blog post covered through our Get Started with JMeter  in 60 Minutes webinar, click here.

 

Want more basic training? Try out our free 5-day basic JMeter course.

 

Feeling ready to move forward? Check out our advanced load testing webinar.

 

Want to learn more about CA BlazeMeter? Request a demo here.