Performance Testing in Scaled Down Environments. Part One: The Challenges
In an ideal world, we’d perform load testing on applications by running tests on an exact copy of the live production environment - including the application components, configuration and demand.
In the real world, we’re subject to constraints. In most instances, performance testing is run on applications that have already been developed and released. There are a couple of reasons for this. Firstly, the development and QA teams are focused on actually building applications, leaving little time for load testing. Secondly, it can be challenging to build a test environment, including real-life demand simulations, and defining the parameters of the test.
Testing in Production
Performance testing in a production environment is something small application vendors can do - but it requires a great deal of care. Running a performance test can obviously lead to issues, like poor service performance for active users. Plus, the application's database and file storage can pile up with 'junk' data from the performance test. A crash load can't be performed as a means of checking how the application will behave at its maximum capacity, because it would most likely result in downtime. As a result of these issues, users may be inclined to close the application or stop using the service altogether.
Testing in a Scaled Down Environment
A common solution, so far, has been to run tests in a smaller environment. The idea behind this method is simple multiplication. If a certain amount of users can be supported by X amount of infrastructure, then the same must be true if one of the variables is doubled or quadrupled. This linear approach sounds logical but, in reality, does not generate accurate results.
An application’s underlying infrastructure is constructed of many different components such as caches, web servers, application servers and disks(I/O). Bandwidth and CDNs also play a role in its function and therefore have to be taken into consideration during scaling. Each component behaves differently in the application according to how it was configured and scaled. However, the tiered structure makes it difficult to calculate how each should be tested and scaled.
Furthermore, there are two ways to scale the application. Scaling-up adds supplementary resources, like CPUs and memory, to a single computer. Scaling-out clusters additional computers together as one system to generate combined computing power. All of these options make it almost impossible to estimate actual data from performance testing in a smaller environment.
Performance Testing in the Cloud
The most efficient way to run a performance test of your application is by simulating an environment that is the most identical to that of production. It’s best to run this simulated performance test as early on in the development phase as possible. The cloud allows a temporary and unlimited amount of capacity to be used for performance tests. BlazeMeter’s Load Testing Cloud makes it possible to simulate high capacity load tests, effortlessly, on one manageable platform. Real world traffic will be simulated in the cloud and potential obstacles will be uncovered, at any scale, before they become a problem.