Eric Dapp is an Adoption Engineer at BlazeMeter and an instructor at the BlazeMeter University. He's got over 3 years experience in performance testing, helping teams realize their performance and automated testing goals.

Become a JMeter and Continuous Testing Pro

Start Learning

Test Your Website Performance NOW! |

arrowPlease enter a URL with http(s)

Mainframe Batch Testing with Zowe, JMeter, Taurus and BlazeMeter

Mainframe testing has historically been difficult to automate. With access to the mainframe typically being locked down from outside communication and with so much focus put on security, it’s understandable that much of mainframe testing remains manual. Some advancements, like the JMeter RTE Plugin, have been introduced. These plugins let users automate their GUI mainframe tests using JMeter and BlazeMeter. In this article, I’m going to focus on another aspect: mainframe batch testing, and how it can be accomplished by integrating open source Zowe with JMeter, and then run in BlazeMeter using Taurus.


Introducing Zowe

Zowe, part of the Open Mainframe Project, is an open source framework for z/OS, the operating system for IBM z/Architecture mainframes. Zowe lets you communicate with the mainframe via CLI commands, similar to how you might work with Google Cloud Platform using the gcoud CLI, or via SDKs available for Javascript and Python.


There’s a wide range of mainframe functionality that’s exposed using Zowe. In this article we’re only going to be scratching the surface of what’s possible to demonstrate when running mainframe batch test cases, i.e testing operations done in batches instead of one at a time. We'll be using JMeter and BlazeMeter to simplify automating those tests as part of a CI/CD pipeline. 


It should be noted that in order to follow along with the rest of this article you’ll need to have access to a mainframe that’s running the Zowe API Mediation Layer to handle the commands we’ll be sending, you will also need the Zowe CLI tool installed and a Zowe zosmf-profile defined pointing to your mainframe system.


Setting up the Zowe Test Script in JMeter

Let’s create a JMeter script. Our JMX is going to be very simple, it will only require a few components. When we’re done, you will have a JMeter script with a test structure that looks the following: 


zowe in JMeter


We’ll work from the inside out. 

1. Add a thread group to your test

2. Under that add a JSR223 Sampler

3. Copy the following Groovy code into the JSR223 Sampler element:


//	Prepare to construct command from JMeter Variable

def sout = new StringBuilder(), serr = new StringBuilder()
def command = "no command given"

// 	Run command using the oppropriate OS format

if ([''].contains("Windows")){
	command = ("cmd /c " + vars.get("command")).execute()"Found Windows")
else {
	command = vars.get("command").execute()"Found Not Windows")

//	Let command finish or timeout

command.consumeProcessOutput(sout, serr)

//	Log the output and errors"\n\nout>\n$sout\n\nerr>\n$serr")

//	Set the sampler and response data with command and output

SampleResult.setSamplerData("Command sent to MainFrame: " + vars.get("command"))


You can check out the comments in the code to see what each line is doing, but in essence, we’re taking a JMeter variable called “command” and running it on the operating system level.


We will be feeding this sampler commands via this variable. You’ll also notice the “zowe_timeout” variable used here, go to the Test Plan and set a user defined variable there called “zowe_timeout” and give it a value of ${__P(zowe_timeout, 60000)}. This will read from a JMeter property called zowe_timeout, but default to 60 seconds if not defined.


Technically, this is all you need. If you supply a Zowe command and your Zowe CLI installation has a zosmf-profile defined, this sampler will run and return the results which you can check by adding a View Results Tree to your test.


But let’s put in a little more elbow grease and make this test more flexible by reading our commands in from a CSV file. 


Zowe Command Listing

4. On the same level as our JSR223 Sampler, let’s add a CSV Dataset Config and point it to a file called zowe-commands.csv. In our case we want the test to stop when there are no more entries in the CSV. So, we have to set a couple options here to make that happen. Specifically, note the settings for Recycle on EOF (End Of File) and Stop thread on EOF.


open source zowe


You may also have noticed that we’re getting more than our commands from the CSV - we also have a variable for “label” and one for “assertion”. I’m going to use “label” to set the name of our sampler (so that our reporting makes sense) and I’m going to use “assertion” to carry a string that I want to be present in the output.


5. To accomplish this, name the JSR223 Sampler as ${label} (this is the syntax for JMeter variables).


6. Then, add a Response Assertion to the sampler, and under “Patterns to Test” use ${assertion}. 


zowe jmeter blazemeter


7. Now, create the zowe-commands.csv file in the same directory as our JMeter script with the following format:


< zowe cli command >,< sampler label >,< assertion string >

< zowe cli command >,< sampler label >,< assertion string >

< zowe cli command >,< sampler label >,< assertion string >


Our test will populate the Zowe commands, and match the label and assertion that we want. A simple example line to list files based on your High Level Qualifier would look like this:


zowe zos-files list data-set "< HLQ >.*",List Files,< HLQ >.ISPF.ISPPROF


zowe zos-files list data-set "< HLQ >.*" is the command to list all datasets that start with the HLQ (replace with your own to try this). “List Files” will be used as the label or name of our sampler, and “< HLQ >.ISPF.ISPPROF” will have to show up in the output otherwise our sampler will fail. It all fits together nicely at this point and we’re almost done!


8. One more thing we want to be able to do is loop through our CSV with a single user. So take what we’ve done so far, the CSV Dataset Config and the JSR223 Sampler and wrap them in a While Controller (you’ll find that under Logic Controllers).


9.In the condition, just put in one of the variables from your CSV file, ${label} as an example. This way the elements that are under the While Controller will execute as long as the variable ${label} exists.


BlazeMeter and Automating our Mainframe Tests

So, now we’ve got a working script that is interacting with our mainframe, we want to get that test on BlazeMeter. In BlazeMeter, we can easily automate this test with an API call or use a tool like the Jenkins Plugin to automatically execute this test.


10. Firstly, we’ll want to make sure we have deployed a BlazeMeter Private Location that has access to our mainframe. Make sure that the Private Location is set up to use API Functional and Performance functionalities since we can run this test as both of those.


11. In the test configuration, we’ll want to bring our JMeter script and the CSV file, but we’ll also need to include a YAML config from open source Taurus that will reference the JMX. The YAML will also let us install the Zowe CLI and create a zowe zosmf-profile as before the test starts.


12. Create the following Taurus YAML config in a text editor


- executor: jmeter
  scenario: zowe-jmeter
    harbor-<harborID>: 1    # Fill in Private Location ID that can reach the MainFrame

    script: MainframeBatchTest.jmx

- module: shellexec
  - npm install -g @zowe/cli
  - zowe profiles create zosmf-profile default --host <host> --port <port> --user <user> --password <pass> --reject-unauthorized false
#   ^^ for profile:
#   fill in details for host, port, username, password


With the profile command properly configured with the user, password, host, and port information in the above YAML, BlazeMeter will run this test, installing Zowe CLI, creating the required profile and looping through all of your Zowe CLI commands in JMeter. You will get reporting based on the labels and assertions you stated in your Zowe Commands CSV file.


zowe blazemeter mainframe testing


If our Zowe commands are using the --vasc (View All Spool Content) option, each of our commands will come back with the spool content from the mainframe like the following:


zowe mainframe batch testing


I’m running this test as a Functional Test on BlazeMeter in order to see the data for individual labels, but the same test configuration would work as a Performance Test as well.


With our test saved on BlazeMeter, it inherits all the benefits pertaining to automation. As an example, we can kick this test off with a call to the /tests/{testID}/start BlazeMeter API endpoint, or we could point the Jenkins BlazeMeter Plugin to the test ID to easily automate running this test. 


If we wanted to take this a step further, before we make the call to start this test, we can use the /tests/{testID}/files BlazeMeter API endpoint to replace the zowe-commands.csv with a new set of commands, labels, and assertions effectively letting you pass in your commands at run time. The way we’ve written this test really gives us a lot of flexibility when we’re thinking about automating our test cases.


That's it! You are now ready to start testing.

arrowPlease enter a URL with http(s)

Interested in writing for our Blog?Send us a pitch!