Avoid these 5 pitfalls in Healthcare software testing
As a Quality Assurance (QA) Specialist in the healthcare industry, I’ve seen firsthand how software development and release cycles are being shortened by making the move to Agile techniques. Development-testing cycles are shorter, deliverables are more limited in scope and iterations are more frequent. Ultimately, testing is done earlier and more often in the software development lifecycle.
While this new development model has achieved significant popularity and momentum, there are challenges when adopting Agile in a domain like healthcare where software errors might lead to loss of life. Let’s examine some of the challenges our team faced when using an Agile model and how automated testing has helped us address them.
1. Requirements traceability during in-sprint testing
In Agile development, large requirements are split into smaller user stories and assigned to different developers who work on features. Requirements changes are commonplace after you’ve captured feedback from a couple of sprints.
Those changes, though, create a ripple effect. A single change in a desktop application requirement, for example, affects the entire workflow. This causes rework for all teams – especially when the software feature has interdependencies with other software modules and backend platforms.
When you are working manually and lack traceability and visualization, there is no way to know what you don’t know about the impact and how to respond. Do you modify the test inputs to suit the change in requirements? Do you scrap some portions entirely?
Frequent discussions among our Development and Operations teams helped us align requirements changes and avoid last-minute updates that might lead to in-sprint test failures. Unfortunately, though, this process was slow and prone to inaccuracies.
How Test Automation helps: Test cases can be generated and executed automatically off a local machine as changes are made to requirements. Reusable test scripts can be maintained and shared across distributed developer teams from a central repository so everyone can work efficiently and remain in sync.
2. Insufficient test coverage at scale for Agile teams
With smaller and faster Agile test sprints, our team invariably lacked insufficient time to carry out all our QA processes effectively. We essentially compressed critical, lengthy workflows into smaller timeframes. A typical workflow would consist of the following:
- Conducting a sleep study of up to 8 hours
- Uploading the study to the cloud and extracting KPI values from the graph
- Downloading from the cloud so physicians could review/modify the KPI values
- Generating reports and updating in cloud
- Storing data for a long period for compliance checking, etc.
As developer teams pushed batches of code for testing, there simply was too little time to create scripts and test the key workflow steps in each deployment. We would create the bare minimum of test scripts in the background and run them overnight for debugging and defects management. We had to compromise on the quality and scope of test cases and use record-and-playback features to create and execute test scripts manually.
How Test Automation helps: You have the test coverage you need to support QA on-demand. The speed of your testing workflow can easily match the speed of software development.
3. Unstable test environment for the CI/CD pipeline
Our software package includes two desktop applications and one web application that support about 10 user roles with different permissions. We found that smaller, faster sprints made our test environment less stable compared to the good-old waterfall model. Unfortunately, though, manually checking stability after each deployment was time-consuming and tedious.
Continuous integration had to be done by Agile developers, which added to our QA challenges. That meant we needed to extend test resources across distributed developer teams. But the lack of a stable test environment with reusable scripts made manual testing tougher. We were not able to replicate defects systematically and consistently or to understand why software functionality that passed testing earlier might not work minutes later.
We resorted to automating our daily, overnight regression test suites just to catch any unexpected instability, with alerts sent to relevant parties if execution failed. This helped QA to find roadblocks, without having to go through every single step in the workflow.
How Test Automation helps: You can virtually represent an actual production environment and continuously accelerate test automation across a CI/CD pipeline. You get a stable test environment, which is essential to any organization running mission-critical healthcare IT platforms 24x7.
4. Pain of maintaining automated test cases and test scripts
Continuous integration causes changes in the objects used in a software build – modifying the functionality, one sprint after another, until the desired result is achieved. This leads to inconsistencies in the test cases generated – breaking the scripts written to test the functionality of the feature being developed.
We tried to solve this by making it mandatory that QA be informed about any environmental or coding changes. But we needed to be able to generate and maintain test cases continuously as requirements changed.
How Test Automation helps: Test scripts are automatically aligned. You can define and execute automated test scripts from a developer’s local machine, and then reuse the same scripts to execute tests at an enterprise scale across the business – staying in lock-step across your software build.
5. Absence of test data for new, critical functionalities
Connected healthcare devices monitor and record patient data, including brain wave patterns, heart rate, pulse rate, blood pressure and other critical measures. Complex calculations rely on these vital readings, which are compared with a set of backend base values. Testing these connected devices, though, has many challenges – including the exponential growth in the types of devices and daily changes in device technologies. The task becomes how to put calculation logic into test scripts for every new build introduced.
Our team relied on time-consuming, manual testing of complex and critical functionalities and logic for device emulation. In addition, we were unable to use actual patient data from medical readings for preproduction tests. Doing so would create compliance issues. As a result, we might have new software features to test and be missing test data. What then?
How Test Automation helps: With the right automated test platform, you can readily mask sensitive patient data and can create synthetic test data to fill any gaps. Do not run the risk of non-compliance in this HIPPA conscious environment.
The Answer: Accelerated, Continuous Test Automation
It was clear our test team needed to adapt its culture and mindset to think differently about our work. Testing in the same way we did for waterfall development wouldn’t keep up with today’s fast-paced Agile development or help us leverage the true value that Agile and continuous testing can bring to DevOps. We needed to test early and continuously across the entire software development lifecycle.
For us, open source-based test automation was the clear solution. It has become the cornerstone of our continuous testing program, and the benefits are many. Here’s what you can expect when you make the move:
Your Test Center of Excellence (CoE) can decentralize testing and empower distributed developer teams to build better code. Developers can test early and often within their own allocated workspace using their independent development environment of choice.
Your CoE gets centralized visibility so you can bridge and broker test automation resources and remain relevant in this new Agile world, acting as the epicenter for delivering high-quality applications. You can guide developer teams on what and where to test, with scripts that define automated API and functional tests to be run from a local machine. You can start with a model-based testing approach and define the user stories developers work on. You can engineer requirements for comprehensive test coverage and can generate and maintain test cases that speak to the true nature of the features built.
Test artifacts can be stored in a shared repository and can be reused and executed at scale for performance and load testing on demand. You can use a virtualized representation of your actual production environment for systems tests – all while protecting sensitive data and ensuring compliance.
And importantly, you benefit from the many community-based contributions that popular open-source test automation technologies bring. Join our Apache JMeter™ Training Academy to learn tips and techniques that can save time and help you work more efficiently in an open-source environment.
Take the next steps
You might also find these useful:
Interested in writing for our Blog? Send us a pitch!