Continuous Testing Insights From the STARWEST Mega-Actionable Workshop

To successfully achieve continuous testing, there are several components that must be addressed. 

Your People

The key aspect of your continuous testing transformation is the team of developers and testers you assemble. Human beings are naturally creatures of habit, and introducing change can pose challenges. Developers are generally focused on writing code and then passing it off to testers. Testers are used to receiving the code in a certain environment and doing the testing. For some companies, bringing these two groups together is their biggest challenge.  

To ensure changes in your team’s test strategy are received as smoothly as possible, you should do the following:  

• Clearly communicate the vision of speed and quality to your developers and testers through meetings and consistent communication  

• Ensure that your developers and testers understand the benefits of change and how this can help them advance their career through increased knowledge and skills.  

• Empower developers with testing capabilities. Developers in this new agile approach need to understand that testing is now part of their responsibility.  

• Empower testers with more coding responsibilities. This will encourage inter-team collaboration. 

• If possible, keep your teams small. Agile testing means being nimble. 

Your Process

Time is one of — if not the most — valuable resource a testing team has. Manual test execution is slow and time-consuming. It is also not even an option when regression testing occurs in short iterations.  

The actual execution of tests must be automated so that this step can occur seamlessly during the development process and repeated frequently. Because developers now create small chunks of code in bursts, the testing process must be always available to verify subsequent code changes.  

Additionally, testing needs to move beyond the UI layer, to include application program interfaces (APIs) and become more data driven. That means automated testing must start much earlier in the application lifecycle and cannot wait to start when the UI is in place.  

With that said, do not attempt to remove all manual testing tasks, but adopt an “automated first” mindset. Start by focusing on the areas that will be run repeatedly. You can map out your software development lifecycle (SDLC) and identify automation opportunities. Maybe you will only automate 1 or 2 areas to begin with — that’s okay! Don’t try to automate everything at once. Small progress is better than rushed failure. 

Your Technology

Your testing toolkit is a vital part of your journey to continuous testing. Evaluating your current toolkit and potential additions to purchase will help you identify where across the entire SDLC to help your teams design, build, and test code faster — with less defects.  

There are many areas within the testing spectrum that can be improved and optimized. You and your team must evaluate where those areas are and then evaluate vendor tools that will meet your needs. Here are some basic guidelines to follow as you move through this process. 

Evaluate testing solutions that developers and testers will use. This means that they are part of the evaluation process. If you select tools without their input, you run the risk that the team won’t use them.  

You will most likely be looking at tools in these areas:

• User stories  

• Test case design  

• Test data management  

• Service virtualization   

• Test automation   

• Performance testing   

• Release automation  

The previously mentioned tools, if possible, should have the following characteristics:  

• Code-like environment (text-based, version control, easy-to-modify/copy)  

• Self-service platform  

• Documentation rich  

• Community-based support  

The latest advancements in artificial intelligence (AI) and machine-learning (ML) are important to consider as testing capabilities continue to evolve. AI and ML can be excellent tools for streamlining testing — if your organization allows for it. 

Don’t overlook open-source tools. In some cases, open-source tools may be the right fit for your particular needs and can be implemented into your continuous testing process. Open-source tools can offer significant flexibility in your testing. Look for tools that can integrate with your existing tools (i.e., Jenkins, Github). 

Implementing These Continuous Testing Insights

For software delivery to become truly continuous, a holistic approach to testing is required. Disparate tools and processes need to be joined up, from design and planning, through release and deployment. Only then can the wasted effort of going from one format and tool to another be avoided. 

When implementing a continuous testing ecosystem, you must ensure it includes the following:  

• Active requirements gathering, to capture the desired user functionality accurately and iteratively.  

• Optimized, automated test–case design to create the smallest set of test cases needed to validate the desired user functionality.  

• Self–service data provisioning, to locate, secure or create the data needed to execute every test case.  

• Service virtualization using comprehensive, accurate virtual data to create affordable, production–like systems for all tests.  

• Automated maintenance and traceability, so that when the design changes, the user stories, test cases, data and virtual endpoints can all be updated at the same time.  

• Accurate, automated test execution, easily managed across all systems, APIs and services.  

• Full orchestration and release automation, to align all the moving parts and seamlessly deploy, test and release code.  

• A centralized repository of up–to–date assets and tasks, enabling cross–functional collaboration and project–wide visibility.  

• Performance tests that closely mimic actual user behavior, that can be scaled beyond the level of stress.