December 10, 2020

Beanshell vs. JSR223 vs. Java For JMeter: Complete Showdown

Open Source Automation
Performance Testing

Beanshell. JSR223. Java request sampler. Which is best for JMeter scripting? And to extend baseline JMeter functionality? In this blog, you'll compare performance and analyze which one is the best. 

First, you might be familiar with Beanshell and Java... but what about JSR223?

What Is JSR223?

JSR223 is a scripting API that you can use for JVM languages.

JSR223 vs. Beanshell vs. Java Request Sampler

JSR223 is the best option when you're using Groovy. Groovy scripts can be compiled into native Java code (assuming some requirements are met) so Groovy script execution performance can be almost as fast as Java code. 

So if you’re about to use scripts for something once, quick 'n dirty (e.g. reading configuration files at the beginning of the test) you’re welcome to use Beanshell/Javascript/whatever you’re comfortable with.

But, if you’re about to do some extensive load testing through scripting (i.e. constructing massive HTTP requests from calculated data) you need to consider either Groovy or custom Java requests or a JMeter Sampler.

For comparison purposes, we’re going to use the same simple code which generates a 1Mb string of random alphanumeric characters. 

import java.security.SecureRandom; BlazeMeter blog
 

Comparison metrics will be collected using  10 users x 100 iterations with a  session duration of 1 hour (standard BlazeMeter session length), load generation will be done from single JMeter Console by execution of the above code and demonstrating key performance indicators of following engines:

  • Beanshell (as is)

  • JSR223 (Groovy as language, compilation caching enabled)

  • Java (as JMeter Java Request Sampler)

Including associated CPU/RAM cost on load generator side (BlazeMeter Console)

Tests will be using following environment:

  • Test Type - JMeter Test (Sandbox)

  • Threads - 10 threads per Thread Group

  • Server Type - Large

  • Session Time - 1 hour

  • Engine - Console Only

  • Server - 1

  • Threads per engine - 10

  • Rampup - no ramp-up

  • Iterations - 100

  • Server OS - Linux

  • Server CPU - 2x

  • Java 7 x64 1.7.0_03

  • JVM arguments - -server  -Xms3072m -Xmx6144m -XX:NewSize=64m -XX:MaxNewSize=128m  -XX:MaxTenuringThreshold=2  -Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000 -XX:PermSize=64m -XX:MaxPermSize=64m 

Beanshell Sampler

Configuration

For Beanshell, no pre-requisites are required, everything works out-of-the-box. All we need is to do is add a Beanshell Sampler and paste 1Mb random string generation code. After uploading test scripts to BlazeMeter's testing dashboard and setting appropriate threads, iterations and duration we got the following results:

Load Test Results

BlazeMeter's Load Test Results

JSR233 Sampler

Configuration

As mentioned, for scripting it’s recommended to use JSR233 + Groovy. Groovy isn’t shipped with JMeter, it needs to be downloaded separately.  To get started:

  1. Download latest groovy binary bundle from Groovy website download area

  2. Locate groovy-all-${VERSION}.jar under “embeddable” folder of distribution and drop it to JMeter/lib folder. Or upload it to BlazeMeter in the “Files” area, BlazeMeter will place it into the corresponding location

  3. Add JSR233 Sampler to Thread Group specifying “groovy” as Language

  4. Set Compilation cache key to something unique 

  5. Paste 1Mb random string generation code to Script area

Important to note: 

Use .groovy files instead of keeping the Groovy code inside the sampler. However if you need to have code directly in sampler, make sure that you have Compilation Cache Key set. If you have > 1 JSR233 Sampler – make sure that they use different keys

Don’t refer any variables as ${VAR} inside the Groovy script. Use either vars.get(“VAR”) or the Parameters stanza of JSR233 Sampler

Load Test Results

Load Test Results using JSR233 Sampler

Java Request

Configuration

Java Request is your own implementation of JavaSamplerClient, all described methods should have appropriate code. The absolute minimum is override of runTest() method but if you intent to parameterize your Java Request you need to provide appropriate logic  to read inputs and conditional interpretation of requests flow to determine whether Sampler passed or not. 

The example code for generation of 1Mb random string via Java Request will look as follows:

Java Request is your own implementation of JavaSamplerClient,

Compiled class needs to be placed in /lib/ext folder of your JMeter installation preferably in .jar form so JMeter could automatically pick it up, elsewise you’ll need to amend JMeter classpath. 

When using BlazeMeter just upload the .jar file altogether with your script and other extensions (if any) and the BlazeMeter engine will pick it up. 

2013/12/05 11:09:38 INFO - BlazeMeter: BlazeMeter startup script completed

2013/12/05 11:09:39 INFO - jmeter.protocol.java.sampler.JavaSampler: Created class: com.blazemeter.JavaRequest

Load Test Results

Java Request Load Report Results on BlazeMeter

Conclusion

Comparison Beanshell v JSR233 v Java

Beanshell is recommended to be used for once-only activities like reading configuration file somewhere in single-threaded setUp Thread Group or in situations, where no possibility to use alternatives exists. 

JSR233/Groovy is quite reasonable option for scripting but only with “compilation” feature.

And the winner is......... Java Request which provides blazing performance and cutting-edge productivity!

Try BlazeMeter For Testing

Want to learn more? You might be interested in viewing the webinar, Advanced JMeter Scripting - Writing Assertions in Groovy.

Check out BlazeMeter's Knowledge Base for loads of  JMeter, BlazeMeter and load testing tutorials!

START TESTING NOW