Many SAP teams relegate performance testing to large projects only. While important in this setting, regular business changes and underlying platform updates can create unexpected system performance degradation. This exposes the business to potential poor service or inadequate capacity anytime there are updates or changes in SAP or the underlying infrastructure, and with potential consequences disruptive to the business. However, performance testing – also called load or volume testing – has a reputation as expensive and impractical to perform regularly, especially for SAP.
In recent years, SAP performance testing has become more efficient and more practical to perform on a regular, continuous basis. And doing it well is much easier today than in years past. At Perforce, we have extensive experience with load testing and SAP – and the tools for the performance testing challenges presented by complex SAP environments.
In this blog, we will explore some basic requirements and best practices for SAP performance testing along with practical solutions for adding continuous SAP performance testing to your next testing cycle.
Why You Should Be Doing SAP Performance Testing
Ideally, performance testing should be a regular part of the testing discipline for any change, and especially so for systems providing mission-critical services. In SAP, Workbench changes can modify code and database structure and present opportunity for performance issues to develop in previously well performing systems. Common underlying platform changes such as OS upgrades, network changes or hardware updates can have unpredictable consequences.
Project-driven requirements such as ECC to S/4HANA migrations, lift-and-shift to hyperscalers or migration to SAP RISE demands performance validation to ensure business continuity before go-live. Hearing the words SAP RISE and performance testing in the same article may strike surprise in some, but many large enterprises will elect to deploy S/4HANA on one of the RISE Private edition flavors. These solutions share many traits in common with traditional on-premises deployments, including customer responsibility for performance management. In many cases, RISE Private edition migrations can be treated as any large scale on-premises solution would be from a performance testing perspective.
In either case, performance testing early and often reduces the chances of a performance issue discovered late in the project lifecycle, where remediation is complicated and expensive, and the impact to delivery schedules significant.
SAP Performance Testing Considerations
Building a regular performance testing discipline requires a mix of investments in process, software and infrastructure, and while not exhaustive, any performance testing program will require at least these basic considerations:
Test Library
Performance testing depends on tests executed at scale, with a workload volume modeled after actual production activity. In SAP, this could be a mix of traditional SAP-GUI end user activities, Fiori (Web) user interface activities, and API-layer activities when the SAP system is used as a backend for integration, mobile, custom web interfaces or SAP BTP.
What’s important here is a distribution of real-world activities: limiting the scope of a performance test can return inaccurate performance assessments as the system caches and optimizes for the presented (and unrealistic) workload. This is where your functional test library can be especially useful: many functional tests are “end-to-end”, exercising a full-lifecycle workflow to validate the ability of the system to support the business across a diverse set of business processes and functional areas. Perforce BlazeMeter can consume Worksoft functional tests, converting them into a variety of useful end-to-end cases for performance testing and integrating this existing work into the performance testing cycle.
Test Data
Performance testing is notorious for consuming vast quantities of test data. A new order management platform may require tens of thousands of customers to test registration, and hundreds of thousands of units of inventory or other backend data to perform a realistic volume test. Because testing often can’t use actual customer data, a performance test will require a way to synthesize plausible customers for registration – name, address, payment information and other details. However, backend data – composed of known part numbers, inventory quantities and material dependencies – may need to be created using a system copy of an existing SAP system with sufficient inventory already in the system.
Many kinds of data in SAP involve intricate dependencies within the system itself and are not easily synthesized. These datasets are best created through the ordinary execution of process in the SAP system, either as a part of daily activity in production then copied and masked, or through preparation execution by a large volume of test scripts designed to perform the necessary activities for asset creation in the system. Since these cases can be time consuming to create initially, ideally a system reset/refresh with a snapshot capability is available to reset the runtime execution environment on demand.
Perforce takes a two-pronged approach to performance test data, using BlazeMeter for AI-Driven synthetic data creation, and our Delphix data virtualization technology for system copy and data masking requirements. In many cases, both are essential, as each provides a different kind of test data and addresses a different portion of the test data challenge, especially when load testing requires thousands of usable data scenarios. The BlazeMeter approach is simple – just ask for data in the format and quantity required and generative AI does the work of creating plausible test data. When data has system dependencies, the system copy and masking approach supported by Delphix is ideal, and as data is consumed during testing, Delphix resets the test environment to a known starting state, ready for the next test run.
Test Environment
Creating an environment for performance testing requires considerable planning. In the past, recommendations often included duplicating the production environment for size and scale. For SAP operations this may be a logistically and financially impractical request given the huge number of systems and integrations typical, especially in larger SAP estates with dependencies on 3rd party systems.
Service virtualization is a popular solution for abstracting dependencies away from the system under test while ensuring the integrity of the test process itself. Using BlazeMeter Service Virtualization, external dependencies such as 3rd party system integrations or even local applications external to the core SAP system under test are emulated. This eliminates the need to replicate an entire runtime environment for testing and reduces the lab footprint to the systems under change and test.
Service virtualization further simplifies the load testing and test data management problem by providing stabilized responses and helping to eliminate data sync and exhaustion challenges for integrated components.
Enterprises Choose BlazeMeter as Their Go-To Agile Performance Testing Solution
Based on dozens of customer reviews, BlazeMeter continues to shine as a rare example of a one-stop shop for performance testing that enables DevOps teams to test quickly and easily with its easy-to-use functionality, usability regardless of technical ability, flexibility, scalability, and cloud-based solutions. Download the eBook to explore why enterprises consistently choose BlazeMeter as their preferred performance testing solution.
How to Execute SAP Performance Testing
With everything in place, the final piece is the text execution itself. Here at Perforce, we use BlazeMeter to generate the functional load against the test target, and there are a couple of distinct advantages to this approach vs. traditional performance testing software. First, BlazeMeter leverages elastic cloud capacity for test execution, so planning for test execution infrastructure is vastly simplified: simply scale up and down as required. Second, BlazeMeter natively supports SAP-GUI, Web (Fiori) and API testing, so with a single solution we cover almost all SAP volume testing requirements.
With each test run, results are evaluated for error rates and general performance response times. We also recommend comparing test runs historically as a best practice for identifying “creep” in system performance not readily noticeable in any one given test. Finally, in addition to performance of the system from the end-user perspective, use SAP’s ST03 data – a rich discussion topic by itself for another blog, perhaps – to evaluate dialog and transaction performance on the server side: this can be a fantastic source of early detection and diagnosis of performance issues in SAP.
Bottom Line
The application of Perforce BlazeMeter for SAP performance testing may be new to many in the SAP market, but the technology and approach are ideal for today’s SAP systems and tomorrow’s hybrid architecture mixing on-premises and cloud technologies.
BlazeMeter is the idealsolution for continuous performance testing of SAP, and ideal for large SAP migration projects and go-live activities as well. Now comes the next step: Start SAP performance testing with the industry’s most flexible performance testing tool for FREE today!