What's new for June 2024?

[Performance]: Enhanced Report Comparison 

Project stakeholders often need a tabular comparison report to identify potential performance degradation in the system under test and pinpoint specific labels/endpoints where issues occur. 

Previously, aggregated data (Request Stats) could only be compared to a predefined baseline. However, this approach had limitations: 

• Comparison was unavailable if no baseline was defined prior to test execution. 

• Dynamic comparisons (e.g., comparing test run n to n-1) were not possible with a static baseline. 

• Comparisons across different tests or projects were not supported.  

BlazeMeter now offers enhanced Request Stats comparison, allowing users to select from three comparison options: 

1. Baseline comparison 

2. Previous test run comparison 

3. Any other report within the workspace 

This flexibility ensures that comparison insights are available to all users across a wider range of scenarios. 

Current Status and Future Plans: 

• Enhanced comparison is currently available for single performance tests. 

• Upcoming features include: enhanced comparison for multi-tests & real-time comparison during test execution 

Availability: 

This feature is accessible to all users across all plan types. 

[Performance]: Deep links to Errors tab  

Root cause analysis just got easier! Now we provide deep links from Request Stats and Timeline report to the Errors tab, allowing users to easily get to the root cause of a problem.  

Previously, when users wanted to see the errors of a certain label, they had to memorize the label name or write it on a note, then go to the errors tab and find the relevant label in the list to see the errors for that label. That made for tedious work in a test with dozens or hundreds of labels.   

With deep links to Errors tab, users can click on Error Percentage in Request Stats and be automatically navigated to Errors tab — with the selected label expanded by default. Similarly, a click on a data point of an Errors graph in Timeline tab will open the Errors tab with the selected label expanded by default. 

Note: If the label cannot be determined (ex. in case of overlapping graphs) — the aggregated “All” label will be expanded instead.  

Deep links to Errors tab are available to all users in all plan types. 

[Test Data]: New Enriched Pop-up for Data Profiler 

The Data Profiler (Automatic Data Creation Wizard) now features an enhanced user interface to streamline the process of replacing hard-coded values with dynamic data parameters. 

New Features: 

• Intuitive pop-up in the test editor when hard-coded values are detected 

• Clear display of discovered values and potential generated data 

• Real-time examples of generated data for immediate preview 

Benefits: 

• Improved visibility into Data Profiler capabilities 

• Faster decision-making for data parameterization 

• Enhanced user experience in creating dynamic tests 

[API Testing & Monitoring]: Exposed Complete JSON Response for OAuth2 Token 

To support advanced OAuth2 token management, BlazeMeter now provides access to the full JSON response from POST token requests. 

Key Advantages: 

• Ability to revoke or expire OAuth2 tokens after test completion 

• Greater control over authentication processes in API tests and monitors 

• Enhanced flexibility for implementing complex testing scenarios 

Availability: 

All these features are accessible to all users across all plan types. 

What’s new for June 2024?

[Performance]: Deep links to Errors tab  

Root cause analysis just got easier! Now we provide deep links from Request Stats and Timeline report to the Errors tab, allowing users to easily get to the root cause of a problem.  

Previously, when users wanted to see the errors of a certain label, they had to memorize the label name or write it on a note, then go to the errors tab and find the relevant label in the list to see the errors for that label. That made for tedious work in a test with dozens or hundreds of labels.   

With deep links to Errors tab, users can click on Error Percentage in Request Stats and be automatically navigated to Errors tab — with the selected label expanded by default. Similarly, a click on a data point of an Errors graph in Timeline tab will open the Errors tab with the selected label expanded by default.   

Note: If the label cannot be determined (ex. in case of overlapping graphs) — the aggregated “All” label will be expanded, instead.   

Deep links to Errors tab are available to all users, in all plan types. 

What’s new for May 2024? 

[Performance]: Baseline in Multi Tests

Defining a test run as a baseline helps testers make sure that the application performance remains stable as the code changes. Once a baseline is defined, subsequent test runs are compared to the baseline.  

With baselines, you can easily identify degradations and bottlenecks, find the related changes in code, and quickly take actions to resolve issues.  

Up until now, the baseline feature was available on single tests only. Now, we’ve added baseline support to multi tests as well, so that users using multi tests can also enjoy the feature.

Main Highlights:

1. Select a test run in a multi test and define it as a baseline for the multi test.
2. Have all future test runs automatically compared to the baseline. Baseline comparison data will be available in the Request Stats tab in the report.

Note: Unlike baseline for single tests, in multi tests there isn’t an option to define baseline-based failure criteria because failure criteria are defined on the single tests.

[API Testing & Monitoring]: Running the Radar Agent as a Container

We've streamlined the management and maintenance of the Radar Agent by introducing a containerized version. This change brings the Radar Agent closer to other BlazeMeter services, simplifying the management and administration of OPLs (Open Performance Labs) through common automation and pipelines.

For more details, please refer to our documentation.

[Correlation Recorder]: JSON Auto-Correlation Rules

The latest version of Correlation Recorder introduces the ability to create JSON-type auto-correlation rules. Users can now create and save templates with JSON rules, offering more control over correlations when JSON formats are present in the recording. This enhancement eliminates the need for Regex or manually creating a JSON post-processor.

What’s new for April 2024?

[Performance]: New Executive Summary 

We’re excited to bring you a powerful upgrade to your reporting tools: the new BlazeMeter Executive Summary. This new version of the executive summary enhances how you communicate test results, ensuring that your reports reflect the exact data and insights you need.    

What’s New:  

• Comprehensive Insights: The new executive summary includes all data from your performance report, incorporating any filters and view preferences you’ve applied. Now you can highlight the most relevant information without additional setup.  
• Enhanced Components: Enjoy detailed visualizations with components such as the Timeline, Request Stats, Engine Health, Failure Criteria, and Errors, all tailored to your settings.  
• Efficient Reporting: With the new summary, you no longer need to duplicate efforts in selecting what to display. Your report instantly mirrors your analysis preferences, saving you time and making reporting more straightforward.  

Why This Matters:  

• Targeted Insights: Produce summaries that truly represent your findings and conclusions, making it easier to communicate the right information to managers and team members.  
• Streamlined Process: By automatically applying your report settings to the executive summary, you get a consistent and comprehensive view of your test performance without extra work.  
• Flexibility: With both the new and traditional executive summaries available, you can choose which format best suits your needs while we continue to enhance this feature. 

[Service Virtualization]: Prioritizing transactions (unique priorities) 

Once transactions are assigned to a Virtual Service, the priority for each transaction can be defined uniquely. Matching of transactions is based primarily on the Request Matchers defined. If multiple transactions could be matched, then the new functionality of Unique Priorities can be used to assign unique priorities for those transactions. For example, transaction A has a priority of 4 and transaction B has a priority of 9. Since transaction A has a lower priority of 4, it will be matched and returned. However, you need transaction B to be the matched transaction. In that case, utilize the “Unique Priorities” feature to change the priorities and assign transaction B to be a lower priority than transaction A. In this case, you could move the transaction B such that its priority is lower priority than transaction A and thereby is matched first. 

[Service Virtualization]: Support for certs/keystore per environment 

A single transaction can be used across multiple Virtual Services. Every transaction can be configured with certs and/or keystores in processing actions like HTTP call and Webhooks. However, when that transaction is a part of multiple Virtual Services, if a different cert/keystore needs to be configured then that affects all the Virtual Services that have that transaction. So, rather than hardcoding the certs/keystore at the transaction level, this can be configured at the Environment level. Since parameters defined in the Environment level can be used with the syntax of $config.<paramname> inside transactions, that resolves the issue of hardcoding the certs/keystore into the transactions.  

Env2 has a parameter called mycert with dummy.jks, this parameter can be referenced in your Virtual Services with the Env2 selected as the current Environment as $config.mycert 

Env1 has a parameter called mycert with multiple-clients.jks, this parameter can be referenced in your Virtual Services with the Env1 selected as the current Environment as $config.mycert 

[Service Virtualization]: Revamped intuitive keystore and certs management 

All Keystore and Certs can now be uploaded, managed and deleted from the Certificates area of the Asset Catalog. See below 

Certificates can be opened, viewed and downloaded using this screen with the Actions column. 

Keystores can be viewed, edited, managed and deleted from the Actions column. Passwords for keystore assets can be added/updated in this section as well.  

Once a keystore and or cert is uploaded into the Asset Catalog ---> Certificates section, it can be used in the Virtual Services, transactions (processing actions) and the Environments. 

[Correlation Recorder]: JSON correlation rules 

In today's web environment, JSON is frequently used in both request and response data.  

Until now, when confronted with JSON scenarios, plugin users had to rely on complex regular expressions that sometimes weren’t precise for the correlation.  

The new version of the Correlation Recorder plugin introduces JSON-type correlation rules: JSON extractor and JSON replacement, using JSONPath. 

With the new version of Correlation Recorder, users can create JSON-type correlation rules instead of using Regex or manually creating a JSON post processor, save and use templates with JSON rules and have more control over correlation when JSON format is present in the recording. 

What’s new for March 2024?

[Correlation Recorder]: Test history and version control 

The BlazeMeter Auto Correlation Plugin just got even better! We've introduced Version Control for Correlations — designed to enhance the testing workflow and save significant debugging time.   

• Effortless Rollbacks: Applied a correlation that didn't work out? Now you can easily undo those changes and restore your test plan to its perfect state.  
• Track Your History: View the detailed history of your test plan, including all actions, steps, and versions, to keep a clear overview of your progress.  
• Precise Restorations: Select and restore the exact version of your test plan you need, ensuring optimal performance without compromising on your testing strategy.  

The new correlation recorder is designed to maximize efficiency and reduce your debugging time - giving you more time to focus on what truly matters. 

[Service Virtualization]: Mark optional params in Request Headers 

For any request that comes into a virtual service, there will be request parameters sent. Frequently there will be some parameters that are not sent while others are. Here we are providing the ability in the Request Matcher for Header params to be marked as required. Uncheck the Required checkbox field to indicate that the header parameter is optional. This would mean if the incoming request had those params they would be used for matching, but if they are missing then they are not considered for matching as they are marked optional. 

[Service Virtualization]: XML node attributes in selection wizard 

For XPath matchers, support has been added to show the XML node attributes. In the example below, the sample XML contains the node attribute called curr as curr=”USD”. In the Nodes Tree List, notice the Node called curr is displayed with the attribute value of USD. When selecting curr, you can set curr to be Specific, Anything or Matches Regex.  

[Service Virtualization]: CData matching and parsing 

A new Body Matcher called Matches XPath from CData has been added to match and parse the CDATA coming in the request to the Virtual Service. The Selection Wizard shows the parsed CDATA in the dropdown. One or more CData Nodes can be selected and Match Type and Match Value can be configured.  

[Service Virtualization]: URL/endpoints accept configuration params and test data params 

Every URL/Endpoint field in Service Virtualization supports parametrization using the Configurations tab as well as Test Data params. In the example below, look at the Live System Endpoints field. The param defined in Configurations tab is referenced as ${config.host} while Test Data params are referenced as ${path}, ${value}, ${gvalue} which constructs the Endpoint in this case. 

[Service Virtualization]: Capture response payload to be referenced in transactions  

The response payload can be referenced by ${response.body}. In the example below, transaction response body is set to name=${config.name}. This can be referenced within the transaction in the HTTP calls, Webhook calls, etc. Below is an example of the HTTP Call whose body contains reference to the transaction response body which in this case would be name=${config.name}
 


 

[Service Virtualization]: Looping on test data (multiple rows) 

Service Virtualization supports looping through a Test Data model. Previously only one row could be returned. This change now supports iterating through the dataset and retrieving more than one row based on filtering conditions. In the example below, the dataset contains 50 rows of user data which will be returned as shown in the example. 

Please check documentation for more information.  

[Service Virtualization]: Show version on Service Virtualization

There are times when the end users need to know what version their Virtual Service is running on. Once a Service Virtualization starts running from that time onwards we show the image version for each Virtual Service. Even if the Virtual Service is stopped, the version number will continue to be shown. 

What's new for February 2024?

[Performance] Single timeline selection for the report, includes textual input  

Analyzing a performance report often involves trimming the ramp-up time or focusing on a specific time frame during the test run. Until now, filtering the report was a repetitive, time-consuming process, especially in long-duration tests where it's hard to be accurate using a time slide bar. Additionally, the trimming process had to be performed on each report tab separately. 

To expedite this process, the datetime filter has been relocated to the top of the report. Now, users can filter the report once and gain a holistic view of it with almost a single click. Moreover, the filter now includes an input field in addition to the slide bar that enables users to type in the desired time range. 

[Performance] Multiple charts in Timeline report     

While there is only one chart in the Timeline tab, analyzing the report to find correlations and anomalies requires continuously selecting and unselecting labels and KPIs. Once this process is finalized, all selected labels and KPIs appear in the chart in different colors. 

BlazeMeter now supports customized, additional charts in the Timeline tab. Users can add more charts, name them, and even order them in the most convenient layout.  

With multiple timeline charts, users can now distribute the labels and KPIs between the different charts on the page. The report analysis process becomes faster, visualization becomes clearer, and conclusions are easier to highlight and share. 

[Performance] Deep links from Request Stats to Timeline report 

Often, after examining the request stats, you may want to delve deeper and analyze the performance of specific labels during the test run. 

Previously, switching from the aggregated data in the Request Stats tab to the label performance over time in the Timeline tab required manually selecting the desired label and metric in the Timeline tab. This process was tedious for tests with numerous labels.  

Now, we've introduced deep links between the Request Stats and Timeline tabs. Clicking on a cell in Request Stats automatically switches the view to the Timeline tab, with the relevant graph selected and displayed in the main chart at the top. 

What’s new for January 2024? 

[Performance]: Engine Health Metrics – Alerts & Recommendations for Private Locations 

BlazeMeter’s real-time Engine Health - Alerts & Recommendations feature covers any test run that was configured to run with Cloud Locations. 

We’ve further enhanced BlazeMeter’s monitoring capabilities to a level that allows us to accurately measure the CPU and Memory levels of Private Locations (K8s and Docker). 

Furthermore, BlazeMeter’s Engine Health - Alerts & Recommendations feature has been enhanced to support both Cloud Locations and Private Locations. The same logic applies to both: 

• The alert thresholds have been set at 80% for CPU and 85% for memory. These thresholds are calculated using a 60-second moving average of data points collected throughout the test duration for each engine. If the average of the 60-second moving averages recorded throughout the entire test duration exceeds the threshold for a specific resource type (CPU or memory), an on-screen real-time alert will be triggered.  
• During runtime, these alerts include a link to the Engine Health tab. We have introduced filters in this tab to detect engines with high levels of CPU or memory, streamlining the debugging process for more effectiveness. 
• Upon completion of the test, a recommendation screen will be available, providing suggestions to automatically remedy high CPU issues for relevant scenarios. Accepting each recommendation will dynamically adjust the Max Number of Users per engine value, a change that is highly likely to improve CPU utilization in future test runs. 

[API Testing & Monitoring]: Postman Collections Import now supports bulk import from ZIP file 

The import feature had posed a bottleneck for enterprise customers looking to migrate from Postman to BlazeMeter. That's why we've introduced bulk import functionality. Postman allows users to export all data into a ZIP file, and now we can efficiently process this file to import all collections at once! 

What’s new for December 2023? 

[Performance]: Apply "sliding window" on all Failure Criteria with a single click 

We’ve recently released enhancements for the Sliding Window option, including the support of the “Stop” order from Taurus. This sprint, we’ve added the ability to configure a “sliding window” on all failure criteria at once. This allows users that want all failure criteria to stop the test – can configure that quickly with minimal clicks. 

[Performance]: Datadog APM integration – sending BlazeMeter data to Datadog 

With Datadog APM becoming a notable APM in the industry, and reflecting on its importance and value to our esteemed customers (such as Disney, PwC, HomeSite – and many more), we’ve taken a proactive step forward. The first version of the integration included displaying Datadog metrics in the Timeline report.  

Now, with the second version of the integration, users can also send BlazeMeter row data to Datadog, so that they can manipulate the test data in Datadog dashboards.

What’s New for November 2023? 

[Mock Services] : Configure different data profiles for different mocks in the same service 

A mock service is often used across multiple testing environments like dev, test, QA, pre-prod and so on. If the parameters have hardcoded values in the various transactions of the mock service, then for each testing environment it is a manual task to update the values — which is a tedious task. Here, the new Configurations tab comes to the rescue with the ability for different profiles to be added for different testing environments by defining the name-value pairs, thereby externalizing them and moving away from the hardcoding approach. 

[Mock Services]: Mark optional params in Request Matchers 

For any request that comes into a mock service, there will be request parameters sent. Frequently, there will be some parameters that are not sent while others are. Now we have the ability in the Request Matcher for Body and Query Params to mark parameters as required. To indicate that the parameter is optional, uncheck the Required checkbox field. This would mean if the incoming request had those params they would be used for matching, but if they are missing then they are not considered for matching as they are marked optional. 

[Performance]: Change the number of users at run time 

Change the number of concurrent virtual users during load testing to enhance the quality of performance insights.  

This approach provides an enhanced view of an application's behavior under various conditions, helps in uncovering latent issues, and contributes to robust application performance optimization. Testers and developers alike can use this technique to fine-tune their applications, ensuring seamless operation even during periods of high user demand  

We’ve also enhanced the tracking and logging of Concurrency, Limit RPS and JMeter property changes by presenting the time and details of such changes under the Request Stats report. 

Follow BlazeMeter guide for more details. 

[Performance]: Engine Health Metrics - Alerts and Recommendations 

When conducting performance tests, it is recommended to maintain average CPU and memory usage below 70%. We’ve enhanced our in-app Engine Health monitoring and problem-solving capabilities by providing real-time alerts in case high levels of CPU or Memory have been reached.  

The alerts thresholds are set to 80% for CPU and 85% for Memory. These averages are calculated using a 60-second moving average of the data points collected throughout the test duration, for each engine. 

If the average of the 60-second moving averages recorded throughout the entire test duration surpasses the threshold set for a particular resource type (CPU or Memory), an on-screen real-time alert will be displayed. 

At run time, the alerts contain a link to the Engine Health tab, where we added filters to detect engines with high levels of CPU or Memory to make the debugging process more effective.  

Upon test end, a recommendation screen will become available with our recommendations to auto-remedy High CPU issues for relevant scenarios. Accepting each recommendation will automatically change the Max Number of Users per engine value, a change that with high certainty will improve the CPU utilization of future test runs. 

Follow BlazeMeter guide for more details.