The JMeter Access Log Sampler - A Guide

One of the ways to create a performance test for a web application, is to record our steps and reproduce them in the Apache JMeter™ script. There are a lot of useful tools created exactly for this purpose, like the BlazeMeter Chrome Extension, and they can make load testing a lot easier.

But gaining access to the Access Log (no pun intended) file of your application web server and utilizing the access log sampler will make things even simpler.

• What Is the Access Log?
• What Is the Access Log Sampler?
• Using the Access Log Sampler in a JMeter Script
• Parsing Options
• Writing Your Own Access Log Filter
• How to Configure a JMeter Script
• When Not to Use the Access Log Sampler

What is the Access Log?

The Access Log (access.log) file is the file that records the requests users make to the a web application. Usually, each entry in the Access Log contains a server address, a user machine address, a request, a corresponding timestamp and a response code with a response message. But, DevOps can configure the file to store data in a number of different formats.

This is what an Access Log looks like:

127.0.0.1- test [23/Aug/2018:13:55:36+0700]"GET /index HTTP/1.0"2002326127.0.0.1- test [23/Aug/2018:13:55:50+0700]"GET /about HTTP/1.0"2003358127.0.0.1- test [23/Aug/2018:13:56:15+0700]"GET /profile HTTP/1.0"2004581

As you can see, the Access Log is a very useful tool to collect and analyze a data on real users’ actions, and can be used for traffic analysis. Now, how can we use this file and its data in our JMeter script? Here comes the Access Log Sampler.

What Is the Access Log Sampler?

The Access Log sampler gets an access log file, breaks it down to single lines, and optionally filters those lines. The sampler then parses the lines to JMeter requests, executes those requests and collects the responses to display in JMeter listeners.

Using the Access Log Sampler in a JMeter Script

Prerequisites

To get an access log from a web-server suitable for the standard parsers, your web server should be configured to write logs in the NCSA Common Log format. If your web server does not support this format, you can always try some of the numerous converter-applications, for example: RConvlog.

The Access Log Sampler Component

Right click -> Add -> Samplers -> Access Log Sampler

First of all, one thing needs to be addressed: the log sampler is not fully implemented yet, and the state of the component is considered a Beta version. This means it does the job, but several features are not there yet.

The component has the following fields:

• Protocol: specify the protocol you want to test here. Defaults to HTTP.
• Server: the IP address or hostname of the web server you want to access. Usually, entries in an Access Log will be the “127.0.0.1” (localhost) server address. This will not work if you try to perform a test from another machine, and not from the original server, where the Access Log was recorded, so it's necessary to change the server address. The value specified here will be used by JMeter as a host address to prepend any requests from your Access Log.
• Port: if the testing server is listening to a specific port - write it down here. Defaults to 80.
• Parse Images: Not implemented, because of the beta version.
• Parser: a java class that is used to read through your access log file and turn entries into JMeter requests. You can choose either the TCLogParser,  the SharedTCLogParser or the OrderPreservingLogParser, or create your own. For detailed information about available options see the Parsing Options section below.
• Filter: specify a java class that will tell JMeter how to filter entries from your access log. Two options are available - SessionFilter and LogFilter. SessionFilter doesn’t actually filter any requests but it modifies the cookie manager so that all requests from the same IP are processed by a single thread at a time. The second option, LogFilter, can be used to write your own filter - we will dive into this later in this blog.
• Log File: specify the path to your access.log file.

Parsing Options

Parsers are Java classes that tell JMeter how to construct requests from Access Log entries.

The current implementation of the standard log parsers in the sampler is quite simple: the parser looks at the first quoted string value in each line of the Access Log file and treats it like a request. The parser is able to retrieve ‘name-value’ pairs for parameters for the sampler from the request URL, so the request will be sent by JMeter with its original parameters.

There are three available log parser classes. The differences between them revolve around how they distribute parsed requests between threads:

• TCLogParser - each thread in the current Thread Group will take a whole access log file and process all of its entries from the beginning to the end. Unfortunately, the current implementation terminates threads as soon as the end of the file is reached, preventing threads from looping through the log file. Interestingly enough, this won’t happen if you specify a filter. This option is useful when you want to simulate multiple users repeating all recorded actions.
• SharedTCLogParser - each (random) thread will get the following unprocessed line from the log file. The first unprocessed line will be taken by the first thread, which finishes processing its current request. Then, another thread will follow. This parser allows looping through the log file, and is useful when you want to simulate multiple users performing recorded actions in semi-random order.
• OrderPreservingLogParser - it is the SharedTCLogParser, basically.

You can also extend parser classes and filter classes by creating a new class that will suit your purpose. All you need is to clone or download the JMeter source code and implement your own extension to the basic interfaces: LogParser for parsers and LogFilter for filters.

Writing Your Own Access Log Filter

In the following example we will write a very simple filter class that filters out all the requests from the access log that don’t have the response code “200”.

If we want to include only requests from the access log that are responded by the “200” code (OK response), then we can extend standard filters by writing our own class. Let’s see how.

1. Open the JMeter source code in any IDE you want. I use IntelliJ IDEA. The following steps might differ for other IDEs.

File->New->Project from Existing Sources…

Select a path to JMeter’s sources:

On all the other screens you can just press the “Next” button.

2. Add the build.xml file from the root directory to the Ant Build list:

3. Run “download_jars” Ant Build:

4. Open the project settings and add the jar libraries from the “bin” folder to your project:

5. In the Project view go to src/protocol/http/util/accesslog package.

6. Right click on the package and select Right Click-> New-> Java Class

7. Enter your choice for the name of your class.

8. To make the class suitable as a filter, we need to extend the basic implementation of LogFilter with the code below (the code in text appears below):

9. Write a simple constructor class. Here we need to specify a regular expression, which will be used to filter log entries. Then we pass our expression to a function, which enables filtering by it:

The resulting code:

package org.apache.jmeter.protocol.http.util.accesslog;publicclassOnlyOkFilterextends LogFilter {publicOnlyOkFilter(){
       String[] regexp ={"(.+)\"(.+)\" 200 (.+)"};
       includePattern(regexp);}}

10. Now, in order to debug our new class, run “run_gui” Ant Build:

This will start the JMeter GUI:

That’s it! We finished creating our filter. Now, we will configure our JMeter script to make sure our filter works.

How to Configure a JMeter Script

1. Add a simple Thread Group to the Test Plan:

Right click -> Add -> Threads -> Thread Group

2. Add the log sampler.

3. Configure your sampler:

• Specify the host
• Specify the path to your access log

The default parser is used here.

To get the path to your access log, download the log from the server (or ask DevOps for the file), then put it anywhere, and then specify the path to the location.

Don’t specify the filter yet. Let’s see how our sampler works without it to see the difference between how the sampler works with and without our filter.

4. Add the View Results Tree listener. Right click -> Add -> Listener -> View Results Tree

5. Set the Loop Count of your Thread Group to “Forever”:

6. Run your script and see the results in the listener:

We can see all the executed requests from the access log. Some of them return the “200 OK” response, some of them don’t. Take a note of the last response with an error - the sampler terminated the thread when the end of the log file is reached.

7. Now, let’s select our own filter class. We can see it here, because we launched JMeter from the edited sources from the IDE. In case of a regular launch, to see our log class here, we first need to build it as .jar file and put it in the lib/ext folder:

8. Run your script again and see the results in the listener:

Now we see only executed requests that return the “200 OK” response, just like we specified in our filter. Also, the thread continues indefinitely after the end of the file is reached (but an empty error request is still there). Our filter works!

When Not to Use the Access Log Sampler

The Access Log Sampler is a wonderful tool, however, there are instances where I don’t recommend using this sampler.

• If your application operates with sensitive data with high security requirements, it’s not a good choice to write it to the access log, where it is stored as a plain text, unencrypted. If you use encrypted data from HTTPS requests, they will be useless when injected into JMeter’s sampler.
• Standard parser classes can only parse url-encoded request data and do not process through the body data of any request.

All of the mentioned cases limit the usage of the sampler to test applications with high security requirements and with high dependency on the request body data.

Bottom Line

As we have seen, the Access Log sampler is a useful, yet slightly limited tool, without manual configuration. However, it’s easy to extend its boundaries with some simple coding. With a little effort it is possible to tune the parser and filter classes for your particular needs. Let us know how it goes in the comments section!

After creating your JMeter script, upload it to BlazeMeter and get scalability, advanced analytics and collaboration abilities.

START TESTING NOW

Related Resources:

• JMeter’s Dummy Sampler - The Ultimate Guide
• How to Configure JMeter Logging