How Load Testing Helped Solve My Startup’s Teething Problems
This is a guest post written by Chen Atlas, Founder and CTO of laundry and dry-cleaning scheduling app Cleanly. This is Chen’s first post in a series of articles, in which he’ll be sharing his performance testing challenges and solutions.
My entire business is reliant on the smooth running of a single app - Cleanly. This app allows New York City residents to schedule the pick up and delivery of their dry cleaning and laundry on their smartphones. If the app goes down, there’s not much hope my customers will stick around for too long.
So load testing was a no-brainer when my team launched the beta version back in March. There were just a few problems. My ‘team’ consisted of just three people, myself, a CEO and COO, which meant that I was responsible for pretty much all of the development and testing. On top of that, I had a lot to check! I needed to ensure that the app and the main web landing page could handle hundreds of concurrent users – not to mention the logistics of all the deliveries. As such, I needed a simple and fast solution to test the app, website and APIs.
Improving the Backend to Increase the # of Concurrent Users
I’d used JMeter for performance testing before but I needed something with an easy UI and I didn’t want spend time writing scripts. I had hardly any time so I needed something easy. I started using BlazeMeter as it ran in the cloud and did almost all the work for me.
We were initially using the Apache infrastructure. I started by checking the metrics that I considered to be most important – the latency and errors.
Load Test Report Displaying Maximum Users and Errors (Apache)
Load Test Report Displaying Maximum Users and Latency (Apache)
I could see from these reports that when we hit around 200 concurrent users, the response time slowed dramatically and errors started to occur. This wasn’t good enough!
So, I decided to try a different technology. I switched over to Cherokee and ran the tests again.
Load Test Report Displaying Maximum Users and Errors (Cherokee)
Load Test Report Displaying Maximum Users and Latency (Cherokee)
I was amazed! Just by changing to Cherokee, I was able to get at least 100 more users.
Adding a Load Balancer for Better Performance Results
By using a load balancer with one extra instance, I quickly saw that we’d be able to cope with a maximum of 500 concurrent users.
CPU Usage for 300 Users (with Load Balancer)
CPU Usage for 500 Users (with Load Balancer)
This is a great number for what we currently need; It’s going to get a lot busier in the future – both in terms of the people coming to the app and on the logistics end as more people use our API. Fortunately, we can always add more instances to the load balancer as our business grows.
Load testing enabled me to see that our infrastructure simply wasn’t good enough for our needs...before it detrimentally affected our business.
App Performance: Future Considerations
Our startup has only been in business for seven months so we’re going to see big changes in the coming weeks, months, and years. We’re expanding our delivery zone so more people will be using our app and logging on to our website. As such, our logistics will get more widespread and complicated and we’ll have more delivery drivers using our API to get the data they need.
In whatever ways our business grows, performance testing will always be top of my priority list. After all, it doesn’t matter how many cool features you have or how fancy your UI is if your customer can’t use your app when he or she needs to!
As such, I’m going to keep running performance tests and making improvements accordingly. And, I’ll be happy to keep sharing my experiences with you to help you learn from both my mistakes and successes. Stay tuned!