What’s new for October 2023? 

[Performance] New Dynatrace integration

Dynatrace is an Application Performance Monitoring (APM) tool designed for analyzing the performance of application servers, database servers, and web services.

Because Dynatrace has deprecated their v1 APIs with Dynatrace version 1.243, and replaced it with v2 APIs, we have created a new Dynatrace integration to accommodate this change.

BlazeMeter now includes two versions of the Dynatrace APM integration:

• If you are using a Dynatrace version that is 1.243 or higher, use the Dynatrace APM integration. 

•  If you are using a Dynatrace version that is lower than 1.243, use the Dynatrace APM – deprecated integration. 

[Mock Services]: Reordering of processing actions 

After creating multiple processing actions, when the user decides to reorder them which means to change the order in which those processing actions are called can now be done. A simple intuitive way by which the processing actions can be reordered by simple drag and drop action. This enables customers to reorder the processing actions and decide the order in which those processing actions are called.   

[Mock Services]: Analytics updates 

The request and response body are now clearly shown in the Analytics view for each Mock Service. This is true for all the processing actions as well which enables the customers to see what requests were made and what responses were received as a part of not only the transaction but also the processing actions. Another change made is to also show the time at the seconds level as well. This is particularly helpful when a series of processing calls are made one after the other and this enables customers to know what calls were made and at what time. 

[Mock Services]: Import/Export Services enhancements 

When importing an already exported Service, all the transactions within the Service(s) including all the processing actions are imported as well. We recommend our customers to export (backup) their Services periodically. In the scenario that a single Service or multiple Services are accidentally deleted, the exported backup can be used to import the lost transactions. 

[Auto Correlation Recorder]:  Upload Correlation Rule Templates directly from JMeter to BlazeMeter 

Saving time just got easier! With the latest ACR plugin, you can now upload correlation rule templates directly to BlazeMeter with one click. No more saving locally and uploading separately. 

This enhancement streamlines the correlation template workflow allowing you to go from JMeter to BlazeMeter in seconds. Now you can focus on creating templates rather than hassling with a multi-step process. 

Say goodbye to the two-step upload grind. This new seamless experience lets you instantly access your templates across JMeter and BlazeMeter. Spend less time managing templates and more time innovating your tests. 

[Performance]: BlazeMeter integration with Datadog APM now available 

With Datadog APM becoming a notable APM in the industry and reflecting on its importance and value to our esteemed customers, we’ve taken a proactive step forward. 

In the first version of the integration, users can expect: 

• Comprehensive Performance Overview: Like other APM integrations, linking Datdog offers insights into both application and infrastructure performance. This occurs both during and after load testing. It empowers users with the capability to view specific Datadog metrics (like CPU, memory, and storage) right within the Timeline report, complemented by BlazeMeter’s own metrics. 
• Effortless Account Connection: Simply input your Datadog credentials in BlazeMeter to establish the link. 
• Customizable Metric Selection: Design a Datadog profile specifying which metrics are pertinent during the load test. 
• Smooth Test Integration: Seamlessly blend your Datadog setup into your performance tests. 
• Enhanced Report Insight: The Timeline report now furnishes a well-rounded view, encompassing both Datadog and BlazeMeter performance metrics. 

Sliding Window Feature: Enhancing Shift Left/Test Automation Failure Criteria 

In traditional testing scenarios, the assessment of failure criteria was typically done after the completion of the test. With the introduction of the Sliding Window Feature, we’re offering a more efficient approach. 

What to expect: 

Continuous Monitoring: Rather than evaluating only at the end of the test, the Sliding Window method checks throughout the test duration. If you’ve set failure criteria, such as an average response time exceeding 500, BlazeMeter monitors the performance in real-time, every minute. 

• Segmented Analysis: At specified intervals, like at minute 01:00, BlazeMeter evaluates the performance of the preceding minute, such as 00:00-01:00. This process is repeated at subsequent intervals. 
• Immediate Violation Detection: If the designated threshold is exceeded at any checkpoint, BlazeMeter promptly flags it as a ‘Violation’. 
• Clear Reporting: In the event of violations, the Timeline report highlights these periods with a distinct red rectangle, making it easier to identify and investigate areas of concern. 

The Sliding Window feature offers a more proactive approach to test evaluations, providing insights sooner and more frequently.

BlazeMeter’s support for Taurus’ “Stop” & “Timeframe” commands 

BlazeMeter has added support for Taurus’ “timeframe” and “stop”/”continue” pass/fail commands. These are now integrated with our “sliding window” and “stop” failure criteria configurations. Here’s what this means for you: 

• Automated CI/CD Process: This integration allows a more automated CI/CD process, complementing our shift left approach. 
• Efficient Testing: Tests can be terminated early when they fail, especially if initiated through Taurus CLI. This saves both time and resources.  

In the past, if you ran Taurus in the cloud via BlazeMeter, the “Stop” command was not recognized. This meant that the test would continue to run if a failure was detected. With this update, when using Taurus in the cloud, the “Stop” command is respected. This applies whether you are running a test from Taurus CLI or using yaml script directly in BlazeMeter. 

To use the stop function, the pass/fail module needs to specify the timeframe “for 1 minute” and include a stop command. Here are some examples of pass/fail modules that stop tests if the average response time exceeds 20 seconds: 

• avg-rt>20s for 1m, stop as fail  
• avg-rt>20s for 60s, stop as failed  
• avg-rt>20s for 1m, stop  

[Mock Services]: Conditional routing logic for processing actions 

Processing actions now include the ability to be fired based on conditional logic. When a transaction is matched, the first processing action is evaluated based on the optional conditional logic. The condition contains an expression that is validated against a value with the help of the selected operator. If multiple conditions exist, then all must be true for the processing action to fire.  

[Mock Services]: Ability to set response status dynamically 

The Mock Service response status code can now be set dynamically. This is done by specifying the logic for conditional status codes. Expressions are evaluated against the specified values which determines the response to be sent. When these conditional status codes are evaluated and none of them match, then the Fallback Status Code is sent back.  

[Mock Services]: Specifying an alias and password SSL calls 

For SSL connections, when a keystore is used, it generally contains multiple certificates. When the HTTPs call is being made, customers can provide the alias of the certificate which defines the correct certificate to be used for the current SSL call. For this feature to be used, the most important criterion is that the keystore should have aliases defined for certificates. Multiple certs are shown from an existing keystore from which the user can select the appropriate cert for the call. 

[Performance / Functional] Quick search for tests and reports

We’ve added a new Search field to Performance and Functional tabs! Now you can quickly search within the tab for tests and reports by their name, without having to go through “Show All Tests” or “Show All Reports” side bars. 

The field will display top 5 recently updated tests, and top 5 recently executed reports that match your search. To see the full list of tests or reports, click on “Show All Results” which will open the full list in a sidebar.

Looking for a more complex search, a way to find a test created by a specific user or during the last month? Stay tuned; We will add advanced search options to this field soon.

[Taurus / Performance Testing] K6 and Vegeta executors added

Taurus has added new executors: K6 and Vegeta. Now you can run your K6 and Vegeta scripts in BlazeMeter.

[Taurus / Performance Testing] MQTT protocol support added

Taurus has added support for the MQTT protocol. Now you can load test your IoT product with BlazeMeter.

"Disk Out of Space" Alert for Private Locations

Software updates require free disk space to be installed, and now you can get an alert and take action before free disk space drops below a threshold of your choice. To create an Agent alert, go to the Alerts page in Workspace Settings menu, and create a new alert. Like Test alerts, the notification channels for Agent alerts are Email, Slack, or both. Once an Agent alert is defined, you will be notified in case any agent in any private location in your workspace goes below the threshold.

For more details, see the documentation.

[API Monitoring] Enhanced Authentication for Client Certificates in API Monitoring Tests

We have enhanced the PEM-encoded Client Certificates based authentication, for API Monitoring Tests, to also accept a key file or a passphrase. 

You can optionally set up the passphrase as a Secret at the team or bucket level using the Secrets Management feature. This ensures that the passphrase need not be shared with every member of the team and is also not visible in Tests. This ties together two powerful features within BlazeMeter API Monitoring, designed to monitor your secure APIs and keep your API Tests secure.

Check out the docs to learn more. Requires a qualifying plan. Contact Sales to add Client Certificates support to your account.

[Performance Testing] Comparing performance tests to a Baseline

Comparison capabilities in performance testing are now enhanced with the option to define a Baseline for a test. Defining a test run as a baseline helps testers make sure the application performance remains stable as code changes. 

A test run can be defined as a Baseline from the (redesigned) test history tab, or from the report itself:

Once a baseline is defined, following test runs will be compared to the baseline, making it easier to identify degradations and bottlenecks, finding the related changes in code, and quickly taking actions to resolve them. You will find comparison data in the report Summary and in the Request Stats tab:

The Compare Report page and the Trend Charts tab in the test will display a visual representation of the comparison to the baseline.

BlazeMeter also helps you automate the decision making process, by allowing you to configure the failure criteria Threshold as a deviation from the baseline, so that test runs that significantly deviate from the baseline will automatically be marked as "Failed". To do that, check the “Use from Baseline” checkbox in the failure criteria section in the test, submit the deviation you are willing to accept, and the Threshold will automatically be calculated based on your selection. 

Watch a brief demo on the Baseline Comparison feature:

For more information, see the documentation.

[Mock Services] Inspection View for Transaction-Based Mock Services

Understanding which requests and responses were handled by your Mock Services is critical in cases when there is a need to debug why certain requests were or were not returned by a Mock Service. BlazeMeter Mock Services now provide an "Inspection view" for transaction-based Mock Services, which displays details about recent traffic handled by a specific Mock Service.

You can now quickly identify specific requests by full-text search and display corresponding response details. The Inspection view also includes responses served by real service in case of “redirect to live system” no match mode is selected. Such responses from live system can be saved as new transactions and pushed to the running Mock Service directly from the Inspection view screen.

For more information, see the documentation.

[Mock Services] Filter Mock Services by Status

Mock Services view now provides a “Filter by Status” option which helps in cases where there are many Mock Services defined. By using this filter option, it is easy to display a subset of Mock Services based on a certain status. For example, you can display only Mock Services that are running or stopped. More details here from BlazeMeter Docs.

[Mock Services] Display Historical Reports for BlazeMeter VSE Mock Services

Reporting for Mock Services running on BlazeMeter VSE now gives you the option to display reports for Mock Services that were previously running, but are now no longer available. This is useful if there is a need to display historical reports for Mock Services that were deleted. However, it is still important to understand how many transactions they handled or what were their hits per second characteristics.

To use this functionality, simply open a new analytics tab, define the desired time period and select "Mock Service" from the list of available Mock Services that were running in that particular timeframe. More details here from BlazeMeter Docs.

[Mock Services] Export Reports to PNG or PDF 

It is a very common need to share reports with others or store them for your own reference. Reports for Mock Services running on BlazeMeter VSE can be now exported and downloaded as PNG images or PDF documents. More details here from BlazeMeter Docs.

[Mock Services] Download vse_matches Logs from BlazeMeter UI

It is possible to download vse_matches logs for a selected BlazeMeter VSE directly from the Environments screen. More details here from BlazeMeter Docs.

[Mock Services] BlazeMeter VSE Support for Istio

You can now use Istio to help route Ingress traffic into the desired pod in containers and the cluster. More details here from BlazeMeter Docs.

Now when you display the “Show all tests” or “Show all reports” lists, you can resize them to display long names without truncation. Just drag the border on the right:

The APM integration for CA APM is now bi-directional and uniquely smart. It's our first integration to import APM metrics without requiring you to specify the metrics ahead of time. CA APM identifies the systems impacted as your test runs and automatically imports the relevant metrics for you. There's no better way to keep up with an always changing environment. Just point to your CA APM instance, launch your test and we’ll take care of the rest.

Behind the scenes, BlazeMeter injects headers to your test traffic and CA APM determines which systems are impacted. CA APM then sends back the relevant metrics, which we make available in the Timeline Report.  This new feature is layered on top of the existing outbound integration mode, where BlazeMeter test labels show up as Business Transactions in CA APM.

We have renamed the “Throughput RC” button to “Change RPS” to make it easier to find.

Was:

Now:

This button appears in tests where you have checked “Limit RPS” in the test configuration. It allows you to make mid-test changes to the RPS goal.  See Limit RPS in the knowledge base for more detail. 

We have re-designed the Stop Test dialog to make it more likely you will choose the “Graceful shutdown” option over the more destructive option, “Terminate servers”.

Workspace Alerts send alerts by email or Slack when test actions are taken and are triggered by size or duration thresholds. Now you can be notified of all test activity regardless of size or duration, by creating a Workspace Alert with duration set to "0" and concurrency set to "0".

To get alerts like this:

Create an alert (Settings > Workspace > Alerts) like this:

Tip: More detail on creating Slack Webhooks URLs for alerts has been added to our knowledge base.

We have renamed the “Multi Location” button to “Create CDN Test”:

Was:

Now:

This button appears in Test Configuration screens if your account plan includes this feature. Clicking it creates a 9-location multi-test designed to meet Content Delivery Network provider recommendations for an accurate test. Contact support@blazemeter.com if you have questions about this feature.

Download logs from multiple engines at once

We have made it easier to take a closer look at the logs, especially when it comes to tests with multiple engines!  The new Logs tab in the report now allows you to focus on a certain group of engines and download all their logs at once:

On download request, a zipped file will be generated, containing all the selected logs. Once the file is ready to download, you will also get a link to the file - which you can share with others or download directly.

[Test Data Management] Generate Synthetic data for UI Functional Scriptless tests

We’ve added a new Test Data view for UI Functional Scriptless testing. Get a better overview about data from attached CSV files and define data parameters to be synthetically generated in place to support specific test scenario needs. There are 50+ functions available to get real looking, random, or dynamic data for your test, which you can perform calculations and conditional logic on top of this data. For more information, see our documentation.

[GUI Functional testing] Data iterations preview now available in Scriptless tests

Better visualize how every iteration will combine data from various CSV files or synthetically generated data according to different iteration options. Now you can get a built-in data preview with the New Test Iterations settings dialog available in UI Functional Scriptless testing.

[GUI Functional testing] New browser versions now available 

We’ve added new browser versions including Chrome version 88, and Firefox version 85 - these versions are set as default.

[GUI Functional testing] New usage report available: "GUI Functional Tests - Number of Browser sessions"

The report shows the number of test sessions grouped by browser/version for a selected period.

[Mock Services] Ability to Bulk Start, Stop, and Delete Mock Services 

Managing Mock Services is even easier with the ability to bulk start, stop and delete functionality. For more information, see our documentation.

[Mock Services] Run MAR Mock Services to a BlazeVSE directly from the Asset Catalog

From the Asset Catalog, you can view what Mock Services are running on a BlazeVSE and deploy new ones. For more information, see our documentation.

[Mock Services] BlazeVSE License Entitlement and Virtual Service Reports

You are now able to specify a VSE as performance mode, and track metrics such as the total concurrent performance VSEs and the total functional transactions metrics. Additionally, we have added the ability to review reports and inspect MAR Mock Services deployed to a BlazeMeter VSE. For more information, see our documentation.

[Performance/Taurus] Automation Enhancement: Fail tests against the Baseline when running from Taurus

One of the most useful things about having a baseline to compare with, and fail the test in comparison to it (baseline-based failure criteria), is to have more accurate failures when automating your tests. 

If you are automating your BlazeMeter tests via Taurus (running on the Cloud), now you can define baseline-based failure criteria in the YAML and have your test fail automatically in BlazeMeter. 

Check out BlazeMeter documentation to learn more about this feature. 

[Mock Services] Helper Wizards for Request Matching and Dynamic Response Creation

The help wizards for XPath and JSON Path help you generate XPaths and JSON Paths for request matchers in a transaction so you don't have to enter the path manually. Just specify what type of value to match in tree-based selection fields and the path is generated for you.

The help wizard for responses helps you create dynamic values (Magic Strings) in the response content. The information is pulled from the request and rendered on the response so you don't have to worry about entering the syntax manually. 

Check out the docs to learn more.

[Mock Services] Cloning Transactions

You can now clone Transactions that are under a specific service to a new service. You can also find and replace a specific text in Transactions while cloning. Moreover, you can create a Mock Service that contains all the transactions that have been cloned.

[Mock Services] OpenShift Support for BlazeMeter Agent for Kubernetes

Mock Services now support the OpenShift Container Platform web console. You can create a new project, create a role for an agent and deploy the agent using OpenShift. You can then deploy Mock Services to your Private location as usual.

Check out the docs to learn more.

[Mock Services] Rename or Delete a Service

If you are a workspace manager or an administrator, you can now rename and delete services. See the docs for more info.

[Mock Services] Dependency on SV Bridge Component Removed for BlazeVSE

We removed the dependency on SV Bridge component for BlazeVSE. You don't need to enable SV Bridge anymore unless you want to use that bridge to connect to enterprise SV. More info here.

• Add a test to a Multi-Test (NTC) that’s already running
• Improved JMX Converter at converter.blazemeter.com
• Enhanced TeamCity plugin (version 2.3)

Add a test to a Multi-Test (NTC) that’s already running

One of the features long-time BlazeMeter power users love is the ability to add additional load or entirely new test logic to a large test that’s already running. No need to stop, reconfigure and re-launch: Just launch an additional test and join it to the Multi-Test in progress.  This feature is now live in New Test Create (NTC), also known as BlazeMeter “V4”.

Read Add Load/Logic Mid-Test in the Knowledge Base article for Multi-Test (NTC) to learn more.

Improved JMX Converter at converter.blazemeter.com

The BlazeMeter Labs project for generating JMX’s from HAR, PCAP or other files just got some great upgrades.  Check it out at converter.blazemeter.com

• Massive performance improvement - a 30MB file is converted in about 15 seconds!
• Improved support for PCAP files
• Improved file type detection
• New UI with a fresh & clean look:

File formats supported by the BlazeMeter JMX Converter:

• HAR JSON - from Mozilla Firefox traffic
• XML from Internet Explorer
• Selenium Builder input
• PCAP input
• JSON from the BlazeMeter Chrome Extension, HAR and Selenium Builder

Enhanced TeamCity plugin (version 2.3)

• You'll find a separate BlazeMeter tab in build results that contains a summary of the build and link to the BlazeMeter report
• Workspace selection added to build configuration
   

Mock Services: Bring your Own Certificate

We have expanded Mock Services HTTPs support.  In addition to using the SSL certificate that we provided, you can now configure your BlazeMeter agent or Kubernetes cluster to support a custom certificate for the Mock Service to use when communicating with whatever application is calling it.

Mock Services: Performance Scaling 

Mock Services can easily be configured to support horizontal scaling to meet your performance needs. Please check out our documentation on best practices on how to configure horizontal scaling and to find other recommendations about Mock Services horizontal and vertical scalability.
 

Moving our on-premise images from AWS S3 to Google Container Registry

Our latest version of BlazeMeter’s on-premise engine, requires access to the Google Container Registry and uses the following new addresses which should be whitelisted: 

• Gcr.io - 
  • Test connectivity by executing:  curl -v https://gcr.io
  • Response: 302 / 200 HTTP code
• Storage.blazemeter.com - 
  • Test connectivity by executing: curl -v https://storage.blazemeter.com/ping
  • Response: 200 (with a 'pong' XML)

Please update any special network configurations (e.g. Proxy, Environment variables, Firewall, etc.)

For any other assistance, contact BlazeMeter support and we will be happy to assist you with this update.

[General] Sharing Private Locations across Workspaces 

Sharing and unsharing Private Locations across Workspaces is now available directly through the UI (Settings > Account > Private Locations). 

Having the ability to decide which Private Locations should be shared with which Workspaces can contribute to better optimization and utilization of resources across the Account. 

Follow BlazeMeter’s Guide for more details on the new Private Location screen and on how to share, unshare and permanently delete a Private Location. 

[Test Data] Direct use of Service Data to drive tests 

Adding data driven Mock Services to tests allows to ensure that test data and mock service data are consistent and aligned where needed. In case, a test could be driven by a shared data model already provided by an associated Mock Service, BlazeMeter now recognizes that case and gives you an option to use Service data directly to drive the test.

[Mock Services] Additional Mock Services reports: Hits and Misses, Response time and Transaction/second throughput 

Mock Services Analytics tab for transactional Mock Services now contains additional three reports besides recently introduced “Transactions Daily Counts” report. Three new reports are Hits and Misses to display ratio of matched/not matched requests; Response time to provide data about how much time Mock Services took to respond back; and Transaction/second to display Mock Service throughput per second. 

All these new reports are available in “Reporting” tab in an Analytics card of your Mock Service.

[Mock Services] “Quick add” action for moving transactions to Mock Services 

Mock Services transactions shuttle control now provides easier and quicker way how to select transactions hosted in Mock Service. It is no longer needed to check transaction selection checkbox, then locate move button and click on it. You can use quick action to move transaction by a single click.

[Mock Services] Test Data parameters available in Edit Wizard for transaction response 

When dealing with long JSON or XML response bodies, the Edit Wizard for transaction response body is very helpful tool to find a particular value node in the response body and parametrize its value. Edit Wizard now provides list of available service data entities and parameters including preview of their values to streamline making transactions responses to be data-driven. 

We enhanced the Requests Stats Tab just for you!

We’ve upgraded the UX in the report’s requests stats tab to make it easier to customize, including an easier way to navigate between the pages, adapting the number of results per page and a new, advanced way to filter & sort your labels!

Account disabled notification screen

Sometimes, due to security or financial reasons, accounts can get disabled. In order to notify you, we created a new page that will show once an account you are part of is disabled. If this happens, please follow the instructions on the page to enable your account.

New booting screen improvements

We fixed bugs and improved flows based on your feedback including adding a scenario filter, naming EUX sessions, and more!

Infrastructure upgrade

We have successfully finished upgrading our database infrastructure to further enhance  performance and data security.

Mock Services: Support for Dynamic Responses

Mock Services already are a powerful tool to eliminate testing constraints such as unavailable  services or third party systems.  With the introduction of Dynamic Responses, we  have made Mock Services more robust and easier to use.  Now you can configure your transactions to respond based on the information that was shared in the request.  For example, if you were looking up an account based on a user ID, you do not need to create transactions for different IDs but can pass the ID from the request and ensure the response will include whatever ID was provided. Click here for BlazeMeter Docs for more information on how to utilize this feature.

Mock Services: Limit Port Ranges Supported

We recently introduced the capability to limit the port range that is allowed when spinning up a Mock Service.  Please make sure the range is wide enough to accommodate all the Mock Services you would like to run. Click here for BlazeMeter Docs to learn more.