4 Best Practices for Continuous Testing Implementation
Developing is in an interesting era, since we’re on the border between traditional testing and modern and continuous testing. We’re moving from the structure of the large, siloed Dev team with a centralized QA, and bottlenecks and delays in handoffs, to a new structure. This structure consists of small, autonomous and self-contained teams, which frequently ship software, use Continuous Integration tools to automate, and manage their own build environment to minimize bottlenecks.
But how do you get from traditional to modern? This blog post will cover 4 best practices for continuous testing implementation. For more information, check out the webinar “Continuous Testing - Onboarding and Best Practices” this blog post is based on.
1. Find the right continuous testing tools
Your tools are one of the most important components of your work. If your tools enable your work, elevate your abilities and maximize your productivity, you will get stuff done. If they become a barrier, not only won’t you be able to work, but you will get frustrated and stop trying.
Therefore, tools that smoothen the way to agility, shifting left, automation and collaboration are key to succeeding in moving to continuous testing. If shifting left becomes exciting and nifty, it will happen.
So make sure you find the right tools that enable you to develop, test and analyze continuously. You can check out this blog series we wrote about the DevOps Tools Ecosystem to get specific ideas, but in general we recommend the following tips:
- Try out tools before purchasing them, so you get their vibe and feel, and understand their capabilities. If you can use open-source tools, for example Apache JMeter™ for load testing, do so. Open-source tools are constantly evolving, they have a rich support community and they are developed according to your needs, because in the end you are the customer.
- Choose tools that integrate with existing tools you are using. Continuous testing and continuous integration is all about making things work together, so try to find tools that easily incorporate themselves into your working environment. We especially recommend you find tools that integrate with Jenkins, a wonderful and open-source CI tool, and GitHub (obviously).
- Choose tools that have a self-service platform. This lets you easily modify and upgrade whatever you need, immediately, without waiting for other people’s decisions.
- Find tools that have rich documentation - If you want to get started quickly, or want to find answers at any time of day or night when you are working, you need the tool to have easy access to a place that has answers.
- When testing, make sure you can easily set pass-fail criteria. Continuous testing is all about immediately identifying if things are working or not, so make sure you can set that up easily.
2. Think “Automation”
If you want testing to go quickly and smoothly, try to automate anything you can. This also saves you time and makes your work more interesting, because you’re probably automating the dull and repetitive work and not the exciting and creative parts of it.
We recommend you be in automation mode. Need to ensure the system is stable everyday? Automate tests over night. Need to make sure each change in the code doesn’t shake up your product? Automate every build. Need to drink coffee every morning before you communicate with humanity? You can automate that, too.
More and more tools enable automation. Taurus, an open-source automation testing tool, automates all open-source load testing tools like JMeter™, Gatling, Locust and Tsung, as well as functional testing Selenium. It also integrates with CI automation tool Jenkins, and with BlazeMeter.
Changing the Dev structure into smaller, atomic teams is crucial for making processes agile. But don’t forget that each of these teams is a component of a larger product, and that all teams need to collaborate together.
So share tests and assets across functional teams, make reports easy to access and shareable online (no emails!), open up roles and permissions as much as possible and use webhooks like Slack and HipChat to notify users when tests start and end. To learn more about incorporating Slack alerts in your development process, see here.
The more easy-to-chew information is shared among people, the more ideas and cooperation will come. The more developers feel they are responsible for a part of something that’s bigger, the more they will work together to move the product forward.
4. Define & Display Results
So you have the best tools there are, everything is automated and you’re sharing your work with each other, now what? It’s time to dive deep into the results. Those will show you if your code and product are working and if there are any gaps you need to fill.
So first, define quantifiable KPIs. These KPIs should be determined with Product and reflect the product’s business goals and the company’s business plans. They could include test coverage, number of pass-fail builds, average response time, etc.
Second, create dashboards that track these KPIs. The dashboard should present each KPI, its baseline and changes in it over time. The dashboard is supposed to make anything unusual pop out at you just by giving it a glance, and let you intuitively dive deeper into data when you give it some more time.
Third, display the results publically to create transparency and to easily identify gaps in test coverage. Showing the results on large monitors in hallways or in cubicle rooms is key to getting more engineers on board, as having the results before you mobilizes everyone to put all hands on deck and fix what needs to be fixed, instead of waiting.
Continuous testing requires a change in your way of thinking, but with the right tools and environment it’s smooth sailing to faster and more interesting development. Testing with BlazeMeter is a huge leap to the direction of continuous testing. To check out how it works, request a demo. You can also start testing immediately by putting your jmx file or URL in the box, and your test will start in minutes: