Can You Run a Performance Test with 100,000 Real Browsers?
Quick question: How does your web app perform with 100,000 concurrent users? If you can answer that, then one of three things must be true:
1. You’ve actually had 100,000 concurrent users on your application under live conditions (if so, we hope it went well!)
2. You’ve leveraged a performance testing tool like JMeter that simulates real browsers.
3. You’ve constructed an elaborate, expensive testing lab with thousands of testing boxes running real browsers
We weren’t trying to put you on the spot. We were simply trying to point out – for people who cannot yet answer that question – that using real browsers for performance testing isn’t cost effective when testing a very large number of users. This might seem obvious to you, but for people who are just getting started with performance testing, it’s a fairly common mistake.
But why isn’t using real browsers massively scalable? And when does it actually make sense to run tests with real browsers? Let’s take a closer look. First, the times when it makes sense to use real browsers:
Single user tests in early stage code development to check for bugs
Tests to evaluate functionality alone, when responsiveness, performance and load times do not matter
Tests that do not need to account for how browsers themselves impact the functionality of dynamic apps
Performance tests using less than 10,000 concurrent users
And now, the limitations of using real browsers…
You could, of course, but that really isn't going to sound like a plausible scenario at most organizations, where the IT budget isn't exactly limitless.
It’s worth mentioning here that BlazeMeter's supports both solutions - browser simulation at the HTTP level for large scale testing of over one million concurrent users, and running real browsers which is perfect when the load required isn’t super-bowl scale. And as far as the deployment hardware goes, that's really up to you. You can either stay on-site, using your own systems for load generations to provide more security, or conduct your testing via public clouds. If you want to learn more on this topic, check out some of our past articles and videos here, here and here.
What limitations have you found with real browsers? Be sure to let us know in the comments section below.