How to Ensure WebEx Performance with Open Source Tools
Like many companies in the time of shelter in place and stay at home orders, Broadcom has moved the majority of our meetings online, including our global town halls. These high profile events are key to maintaining our collaborative and productive company culture when we are apart, but still working together.
The recent shift to online has put an incredible strain on web conferencing applications, who have seen their traffic spike over 500%. Our own WebEx implementation was no stranger to this increased demand as well as some performance hiccups, so we needed to be sure that both the application and the underlying infrastructure could seamlessly support the upcoming online town halls.
Hello? BlazeMeter - We Have a Test for You!
The BlazeMeter Continuous Testing Platform Team jumped in to help. BlazeMeter CT evolved from classic BlazeMeter - the industry-leading load and performance testing tool. We had the tool for the job. Timelines were tight, so we went to work, with extra support from our expert testing partner Abstracta.
The goal: ensure a great user experience for up to 3,000 concurrent users connecting to a WebEx event through audio, video, and media channels. BlazeMeter would generate this load and provide analytics and reports to help the team pinpoint any performance issues. For extra assurance, during this simulated peak traffic, we would “manually” validate each channel (ensure audio, video and media were online, without delay, etc.) and the infrastructure and network teams would confirm resource usage and network status.
Cheng Keat Lim from the Broadcom infrastructure team commented on the importance of testing to ensure apps are ready for peak traffic at any time, “Rightsizing the network infrastructure capacity to handle large town halls and all hands meetings to ensure great end-users experience is critical. Furthermore, when we host external customers and partners, poor connectivity and experience will impact our Broadcom brand reputation.”
A load test like this would typically be done with JMeter scripts, but we ran into several issues since WebEx is a proprietary application. We then turned to Selenium, which is both resource intensive and thus, expensive, for a test of this scale. But it worked! We were able to validate performance and the town halls went flawlessly. “This is a great example of how flexible and powerful BlazeMeter is. BlazeMeter embraces and enhances open source at scale. There is more than one way to get the job done. BlazeMeter helps you always ensure a five-star user experience,” said Refael Botbol Weiss, BlazeMeter head of experience and innovation.
Project Summary Goal:
Ensure WebEx performance for 3000 concurrent users connecting through audio, video and media channels.
Very visible project
Proprietary system under test
Use JMeter, the preferred open source technology for load and performance testing. But JMeter doesn't support Webex event protocol by default, or by using any developed plugin. We also couldn't implement the connection through code (using a Groovy or Java script for example) as the Webex connection protocol is a private specific communication protocol over UDP.
Use Selenium. We were able to automate the connection quickly with a Selenium script, however we did have some challenges. When we ran the script from our local load generator (a Linux VM) the audio channel could not connect. We solved this by installing these audio dummy drivers in the VM.
We had also had difficulty verifying that all the channels were connected once we established the connection to the event. We only could validate the media channel using a Selenium assertion (validating that the screen from the host was shared). The audio and video channels were also validated manually.
In the test environment:
- Selenium had a high resource consumption. Calibrating the test we found that we needed at least 600 VMs for the load engines.
- Configuring the 600 VMs required a custom shell script.
While Selenium was not ideal for a load test of this scale, it got the job done, and we look forward to a full JMeter plug in.
How To Get Started