Travis Lloyd is a support engineer for BlazeMeter. Travis has more than 5 years of experience in development, support and QA. His main areas of expertise include Java, REST Services and functional and unit tests.

Become a JMeter and Continuous Testing Pro

Start Learning
Slack

Test Your Website Performance NOW! |

arrowPlease enter a URL with http(s)
Jul 01 2021

Automating Private Location Installations with BlazeMeter

 

Private locations are on-premise solutions for using the various components available in BlazeMeter on applications that can only be accessed behind a firewall. Creating and installing a few private location agents can be pretty simple if you use the method described here. But, creating more than that becomes more difficult.

 

This blog post will show you how to speed up this creation and installation process with the BlazeMeter APIs. With this method, you can create a number of private locations and agents much faster than going through the BlazeMeter UI. 

 

This method is also helpful in situations where you are not able to keep your private locations up and running at all times due to costs or other restrictions. Instead, you can spin up machines and install the necessary pieces on-demand.

 

Creating a Private Location

To create a new private location, you will need to use the Create a Private Location API, an account admin or workspace manager API key, your account ID, and the workspace ID where you will run your test. It’s always best to keep the machines from the same subnet in the same private location to avoid creating odd results in your performance tests.

 

Following the documentation linked, you will find that there are a number of required fields for the API to properly create a private location. Below, we will go over what is required to create each part of a private location.

 

1. Create the Private Location

Use the following API to create a dedicated private location (replace the {{accountId}} with your account ID, {{workspaceId}} with your workspace ID, the {{api_key_id}} with your API key ID, and {{api_secret_key}} with your API Secret key):

 

curl https://a.blazemeter.com/api/v4/private-locations’ \
	-X POST \
	-H Content-Type:application/json \
	--user {{api_key_id}}:{{api_secret_key}} \
	-d {consoleXms: 13312, consoleXmx: 13312, engineXms: 14336, engineXmx: 14336, funcIds: [ performance ], name: My Performance Private Location, slots: 1, threadsPerEngine: 1000, type: large, overrideCPU: 4, overrideMemory: 16384, accountId: {{accountId}}, workspacesId: [ {{workspaceId}} ]}

 

The body data in human readable format:

 

{
   "consoleXms": 13312,
   "consoleXmx": 13312,
   "engineXms": 13312,
   "engineXmx": 13312,
   "funcIds": [
       "performance"
   ],
   "name": "My Performance Private Location",
   "slots": 1,
   "threadsPerEngine": 1000,
   "type": "large",
   "overrideCPU": 4,
   "overrideMemory": 16384,
   "accountId": {{accountId}},
   "workspacesId": [
       {{workspaceId}}
   ]
}

 

NOTE: When creating a private location to handle performance testing, it is recommended to go with the Dedicated private location type, to ensure that each machine dedicates all its available resources for your performance tests. For other test types, it’s best to go with the Shared private location type (i.e. for GUI functional, mock services, etc.). The example provided is meant for a set of machines using 16 GB of memory and 4 CPUs that exist within the same location.

 

We set the JVM XMS and XMS settings to 3 GB of memory less than the full available memory to leave room for other functions that run during the test outside of the performance test. Then we set the private location to the Dedicated type, set the CPU and Memory settings to the max available for the 4 CPU and 16 GB of memory machine, and put it in the account and workspace where the performance tests will run. 

 

Some changes you could make is including multiple workspace IDs to share the private location across multiple workspaces, depending on who should have access to the location. You can also reduce the number of threads per engine to less than 1000 (if you already know what to expect from the machines you are utilizing). Changing from Dedicated to Shared will allow multiple smaller tests (i.e. CI testing or Functional tests), while changing the overrideCPU and overrideMemory values to something that matches the number of slots (or containers that can run functionalities) to be used on a single machine.

 

To see the details for all the attributes described above, visit the Create a Dedicated Private Location documentation. 

 

Once you run the above API, you should get a JSON formatted response back that looks something like below:

 

{
   "api_version": 4,
   "error": null,
   "result": {
       "id": "609d414c6c35756e2272d3d3",
       "name": "My Performance Private Location",
       "userId": 123456,
       "consoleXms": 13312,
       "consoleXmx": 13312,
       "engineXms": 13312,
       "engineXmx": 13312,
       "threadsPerEngine": 1000,
       "slots": 1,
       "type": "large",
       "funcIds": [
           "performance"
       ],
       "hidden": false,
       "overrideCPU": 4,
       "overrideMemory": 16384,
       "created": 1620918604,
       "updated": 1620918604,
       "accountId": {{accountId}},
       "workspacesId": [
           {{workspaceId}}
       ],
       "ships": []
   },
   "request_id": "609d414c9603b"
}

 

From this response, you will need to pull the result.id attribute, which is the harbor ID for this private location, for use in the next APIs that create the agent and the Docker command.

 

2. Create the Agent

Use the following API to create the agent in the new created private location (replacing the {{harborId}} with your harbor ID, the {{api_key_id}} with your API key ID, and {{api_secret_key}} with your API Secret key):

 

curl https://a.blazemeter.com/api/v4/private-locations/{{harborId}}’ \
	-X POST \
	-H Content-Type: application/json \
	--user {{api_key_id}}:{{api_key_secret}} \
	-d {name: My API Created Agent}

 

NOTE: For more information on the above API command, visit the Create an Agent link, which breaks down what can be provided to this API. You will also have the option to assign an IP with that agent to ensure that it is able to communicate to BlazeMeter and its various functionalities are able to perform their tasks (i.e. running a performance test against internal applications).  This may be necessary depending on your machine’s network.

 

If the agent was created, you should see a response similar to the one below:

 

{
   "api_version": 4,
   "error": null,
   "result": {
       "id": "60a26b8c7dc65c3e52759f05",
       "name": "My API Created Agent",
       "state": "empty",
       "hasJetlag": false,
       "useMasterFiles": true,
       "created": 1621257100,
       "updated": 1621257100,
       "ts": null
   },
   "request_id": "60a26b8caba43"
}

You will need to extract the result.id attribute (this will be the ship ID) from the above response in order to create the docker run command used to launch the installation process for the agent. 

 

3. Generate the Docker Command

To create the Docker command, use the Generate Docker Command API, which will look something like the below (replacing the {{harborId}} with your harbor ID, the {{shipId}} with your ship ID, the {{api_key_id}} with your API key ID, and {{api_secret_key}} with your API Secret key):

 

curl https://a.blazemeter.com/api/v4/private-locations/{{harborId}}/ships/{{shipId}}/docker-command’ \
	-X POST \
	-H Content-Type: application/json \
	--user {{api_key_id}}:{{api_key_secret}}

 

When running this command, you should get the following response:

 

{
   "api_version": 4,
   "error": null,
   "result": {
       "dockerCommand": "docker run -d --env HARBOR_ID=609d414c6c35756e2272d3d3 --env SHIP_ID=60a26b8c7dc65c3e52759f05 --env AUTH_TOKEN=37706e5dc995e88ff03cba7d5e6381262a0856ddb92a260d7d336dac176ca0c6 -u 0 --name=bzm-crane-60a26b8c7dc65c3e52759f05 --restart=on-failure -v /var/run/docker.sock:/var/run/docker.sock -w /usr/src/app/ --net=host blazemeter/crane python agent/agent.py"
   },
   "request_id": "60a26eefaf93d"
}

The docker run command you need for deploying the agent to your machine can be extracted from the result.dockerCommand attribute. 

 

NOTE: It’s important to remember that this is the most basic version of the docker run command and does not include anything needed for CA certificates and proxy settings. These settings will need to be added to the command following the documentation’s guidelines. These can be found in the optional installation steps under the On-Premise Machines -> Installation section in BlazeMeter’s documentation.

 

If you are handling the installation on multiple machines, you can repeat step 2 and step 3 for each machine you expect to install BlazeMeter agents on within this private location. If you are installing machines from multiple locations, then will want to repeat steps 1-3 for each location you plan to install the BlazeMeter agent to.

 

Shell Script for Creating a Private Location

With the information on what’s required to create a working private location and agent, the following shell script will create the private location, agent and docker run command. It will then log into the machine and install the agent.

 

# This script creates a private location, the number of agents based on the number of machines you want and gets the docker commands for each agent through the BlazeMeter APIs
 
# Fill in the following variables for your API key ID and API secret key
api_key_id="{{api_key_id}}"
api_key_secret="{{api_secret_key}}"
# Fill in the following variable for the user that has root access (defaults to root)
user=''
base_url="https://a.blazemeter.com/api/v4"
 
# Fill in the following variables for your account ID
accountId={{accountId}}
# Enter each workspace ID you want putting a space between each ID (i.e. (123456 234567 345678))
workspaceIds=( {{workspaceId}} )
printf -v workspaces '%s,' "${workspaceIds[@]}"
 
# Creates the private location
function create_private_location() {
 curl -X POST -s "$base_url/private-locations" -H "Content-Type: application/json" -d "{\"consoleXms\": 13312, \"consoleXmx\": 13312, \"engineXms\": 13312, \"engineXmx\": 13312, \"funcIds\":[\"performance\"], \"name\": \"My Shell Script Created Private Location\", \"slots\": 1, \"threadsPerEngine\": 1000, \"type\": \"large\", \"overrideCPU\": 4, \"overrideMemory\": 16384, \"accountId\": $accountId, \"workspacesId\": [${workspaces%,}]}" --user $api_key_id:$api_key_secret
}
 
# Creates the agent in the given private location
function create_agent() {
 curl -X POST -s "$base_url/private-locations/{$1}/servers" -H "Content-Type: application/json" -d "{\"name\":\"$2\"}" --user $api_key_id:$api_key_secret
}
 
# Creates the docker run command used to install the agent on a machine
function create_docker_com() {
 curl -X POST -s "$base_url/private-locations/{$1}/ships/{$2}/docker-command" -H "Content-Type: application/json" --user $api_key_id:$api_key_secret
}
 
harborId=$(create_private_location | grep '"id"' | cut -d ':' -f 2 | tr -d ',' | tr -d '"' | tr -d ' ')
echo "Created private location $harborId"
 
# Iterates through all the machine names you want to connect to in the machines.csv file
while read machine; do
 shipId=$(create_agent $harborId $machine | grep '"id"' | cut -d ':' -f 2 | tr -d ',' | tr -d '"' | tr -d ' ')
 echo "Created ship $shipId for machine $machine"
 docker_command=$(create_docker_com $harborId $shipId | grep "dockerCommand" | cut -d '"' -f 4)
 echo "This Docker command was created: $docker_command"
 # Checks is the $user variable contains a value, otherwise, defaults to root
 if [[ -z $user ]]; then
   echo "Logging into machine $machine as user root"
   ssh "root@$machine"
 else
   echo "Logging into machine $machine as user $user"
   ssh "$user@$machine"
 fi << EOF
 # Commands to run on remote host
 # Use the following command if you want to avoid passing your password over the network (needs to be stored in the password.txt file in the folder you start in when logging into the machine)
 # cat password.txt | docker login -u {{docker_user}} --password-stdin
 docker login -u {{docker_user}} -p {{docker_password}}
 $docker_command
EOF
done < machines.csv

 

This script also requires a machines.csv file in the same folder in order to loop through the machines you want to create. This file just needs to contain a list of your machines with an extra new line at the end to ensure the shell script can read all your machines in the list.

 

Once you run this script, it will create a new private location in the given account and workspace IDs and will then create an agent for each machine that is present in the machines.csv file. Each agent will take the name of the machine, so you know which agent is attached to which machine. For an example, you can see the screenshot of a private location with 2 agents installed in it:

 


 

For a fresh machine, there is a download process that occurs that will require some time to fully download and install all the necessary images for the functionality you want for the private location. To remove the download time, set up a snapshot that has the latest images for your functionality downloaded already. To find the latest versions available for a given private location, you can use the Private Location Versions API.

 

As has been shown, you can create any number of private locations and agents in BlazeMeter through the BlazeMeter APIs. Knowing this will help with creating private locations as part of a CI process or with deploying multiple agents at once.

 

Start now!

 

   
arrowPlease enter a URL with http(s)

You might also find these useful:

Interested in writing for our Blog?Send us a pitch!