3 More Reasons Why Open Source Isn’t Enough for JMeter Experts: Part 2
JMeter is a free, open source tool that can be run directly from its user interface and has become the standard for performance testing in modern IT today. But is it enough?
In the first part of this series on why open source alone isn't enough for JMeter experts, we examined the disadvantages of open source tools and why advanced JMeter users rely on other tools in addition to JMeter. Here, in part two, we continue to examine JMeter’s capabilities and add three more reasons as to why open source tools alone are not enough for JMeter experts.
1. User Experience
When performing a load test for a website, whether it be a simple or elaborate one, it should be created as realistically and as intricately as possible. This can involve tedious work, where requests need to be created in a step-by-step fashion directly inside JMeter or with an external file, like CSV. It’s also possible to use the JMeter recorder (which is how it’s referred to as from version 2.9 and on), or JMeter proxy (which is how it’s referred to before version 2.9). This records a user session and provides all of the URLs, but in a disorganized way so you’ll need to work a lot to make up for it! For example: when an eCommerce website session is recorded, the URL is requested, and the webpage is recorded in the background, along with all of the sub requests that exist for each image file, java scripts that run on the client, CSS files, and so on. The final result includes lots of data from requests that need to be manually gathered, edited, and deleted in order to create a basic and ordered test template. In addition, JMeter is used to load test more important and advanced website elements like user logins and dynamic data. These can be added individually - “add a new URL, then a new http request sample, and so on.” An alternative to using the JMeter proxy is to add the desired elements manually for each request, requiring an intimate understanding of a website or web app’s code - which not every tester has.
2. CDN Load Testing
There are multiple reasons why CDN generally stays out of performance testing. Firstly, it is pricy to test if you want to create a large amount of traffic. There are also a lot of requirements to fulfill before performing CDN testing. If you’d like to perform a CDN load test in the U.S., it needs to run from nine different locations. This is one of Akamai’s many requirements for CDN load testing which, if not met, is confronted by QoS and traffic shaping policies that fail the load tests. Even if you're using Amazon, the largest cloud provider, for load testing, there still aren’t enough AWS regions available to generate this load. The parameters that need to be in place to create a distributed environment for CDN testing must be very precise. For example: if specific HTTP headers aren’t used, the CDN provider will identify traffic as DDOS and perform Traffic Shaping, which will skew the test results. Finally, you’ll need to receive certification from Akamai to run tests on their servers.
3. Mobile Network Emulation
These days, websites and online services are designed for mobile devices, sometimes as the users’ main devices. When testers test user experience and verify the load of a service’s backend servers, one of the key parameters that needs to be considered is the users’ mobile connections. Network characteristics like bandwidth and latency have a huge impact on different types of loads, their profiles, and their effects on backend servers. Testing a certain amount of users from a mobile network is very different than testing those with an unlimited connection (the same network or LAN with unlimited bandwidth). JMeter has some capabilities that cater to this. The ‘Command Line Parameters’ tests bandwidth and can be used for each load test, though some manual modifications and calculations are needed for further adjustment. No out-of-the-box solution impacts network latency in a simple way. Load testing without the ability to emulate users with a limitless network can skew your load test results and reports.
How To Overcome These Challenges
Fortunately, there are tools and platforms on the market to help. For the second part of this article, I will show how one such platform, BlazeMeter, can be used with JMeter to overcome these limitations.
BlazeMeter provides a multitude of tools that can be integrated to your load testing platform. One of which being the FollowMe Google Chrome Extension - an easy to use, state-of-the-art tool that generates a basic template for your load test. It works in a similar way to the JMeter recorder, but it’s a lot easier to use as it runs directly from your browser and doesn’t require any setup. It gives you a lot more control over what you prefer to record and generates the basic JMeter test scenario for you. This includes recording the thinking time that users spend on each page, generating a more realistic test than the JMeter recorder. Learn more
BlazeMeter is an Akamai certified load testing partner and has the ability to run from multiple cloud providers. It meets the requirements for CDN, has the ability to generate loads from multiple geographic locations, and is already set up and waiting for use. Nothing needs to be manually managed, nothing needs to be certified, and you can simply run your test.
BlazeMeter offers a very simple solution for network emulation, without additional and complex test configuration. A different network configuration can be selected from a few preset configurations for 3G networks with a good, bad, or average signal; LTE signals; or even a home DSL or cable connection. It can also be customized easily by selecting the bandwidth and latency values that you need. BlazeMeter then profiles and shapes your test with this network configuration. Learn more
The open source market leads the way for IT and DevOps today. Open source comes with great benefits - but also great limitations and challenges. These can be solved with companies like BlazeMeter, as they fill in the gaps and assist in commercial-grade products. We invite you to try it for free and to learn more about BlazeMeter’s capabilities.