Testing the Scalability of Real-time Mobile Transactions
Ensuring that scalability won’t be a problem tomorrow requires extensive, accurate testing today. At BlazeMeter, we strive to provide our clients with unmatched test capabilities. However, as developers push the boundaries of social media and mobile technology, even we sometimes have to adjust our service to help them validate theirs. Sound complicated? Consider our recent engagement with Sidecastr.
Validating a New Idea: Synchronizing Disjointed Experiences
Sidecastr offers what the entertainment industry has termed as a ‘second screen’ experience, keeping TV shows watched on one screen (such as your family room TV) in sync with scene specific social comments posted by fans to social networks like Twitter and Facebook — on any channel, at any time. This gives viewers an entirely new dimension as they watch their favorite shows, almost like moving from a 2D circle to a 3D sphere.
With Sidecastr, viewers get an enriched experience. While watching said shows, they get access to the best commentary and content from the most informed of fellow fans, celebs and friends – all curated by a team of superfans covering each new episode.
But here’s the kicker: the service works no matter when you watch the program. And no matter what platform you choose to watch it on. So regardless of whether you’re watching the show live, on a traditional TV or a laptop, recorded from a DVR, or streamed via Netflix/Amazon, you still get the best available fan-sourced reactions to it, sent right to your device, in sync with the show!
Pioneering the Use of Real-Time Triggers
How is that possible? The answer: every few seconds. While the viewer is watching TV, Sidecastr’s app samples ambient audio via the mobile device’s microphones, creates a small ‘digital fingerprint’, ships it off for analysis and precisely determines each viewer’s current point in the show. Then Sidecastr streams the exact social media content that’s appropriate for that point in time to the user’s device. The trick to keeping things in sync is the high frequency of sampling. Pretty cool, right?
Now, can you imagine the scalability issues that Sidecastr had to manage to develop this solution? What if we’re talking about the Super Bowl? Or the Oscars? How could Sidecastr – and its customers – know its service wouldn’t flake out under such a tremendous load?
The short answer is: They tested it with BlazeMeter.
The long answer is: We worked with them to precisely tailor synthesized activity to match their unique behavior profiles and application requirements.
Consider the data bursts mentioned earlier. Every seven seconds devices are uploading sound clips for analysis. The Sidecastr servers must receive the audio clips at the same time and the application has no more than seven seconds to analyze them and return a response to ensure that the social media content is always in sync with its viewers.
Creating a High Volume, Time-Sensitive Test Plan
To test the scalability of this function in a way that accurately simulated millions of independent end users’ mobile devices pumping in real time data to be processed on a fixed schedule, there could be no lag in the BlazeMeter servers synthesizing Sidecastr’s end user activity. We needed to run extremely high-volume (potentially tens of millions of users) that were dependent on real-time triggers (only a true viewing-content sync can deliver the intended value).
So we collaborated with Sidecastr, giving them more insight into how our service works, so they could better test theirs. We helped ensure that new threads, each of which represented one end user device, were generated to run - as much as possible - in parallel on each Blazemeter server, and also allocated multiple servers to guarantee responsiveness.
The results? Sidecastr completely finished their audio sync testing in under three months, from May to July of 2014.
Today Sidecastr has switched to validating scalability of serving content, and are now exhaustively testing functionality in that area.
“Being able to prove that we are able to scale rapidly from 0 to 60 during the evening primetime hours while handling an immense number of users is critical to our success” said Steve Brand, Founder and CEO of Sidecastr. “BlazeMeter helped us identify and correct issues in the early stages and validate the performance of our finalized solution. As we move forward with new features, their unique approach to testing will become an integral part of our entire development process.”
Are you running mobile testing? Get best practices and advice in our webinar Performance Testing for Mobile Apps and Websites