Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
Jan. 26th, 2017

The Ultimate DevOps Tools Ecosystem Tutorial - Part 4: Testing

Welcome to part 4 of our “Ultimate DevOps Tools Ecosystem Tutorial.” In this blog post series, we are covering the top DevOps and development process tools. In part 1, we introduced the DevOps work cycle. It’s divided into five stages: Plan, Develop, Test, Release and Operate. You can see the complete infographic here:


devops infographic


In part 2, we covered main tools from the “Plan” stage and in part 3 we went over the “Develop” stage. This time, we will go over tools from the “testing” stage. Part 5 is about the "Release" stage. Part 6 is about the "Operate" stage.


Testing examines the product and service and makes sure they work in real time and under different conditions, even extreme ones sometimes. This stage requires many different kinds of tests, mainly functional tests, performance or load tests and service virtualization tests. It’s also important to test compatibility and integrations with 3rd party services. The data from the tests needs to be managed and analyzed in rich reports, for improving the product according to test results.


Here are some of the top tools for the testing stage:


Load Testing - JMeter


JMeter is the most popular open-source load testing tool. JMeter enables users to create a test scenario that contains the actions and requests they want to test on their website or app. Then, the can modify it according to their business needs with elements like ramp-up time and timers, run it for multiple users and examine the results.


Pros - open-source, easy to use, robust, has a vibrant and involved support and development community, has multiple plugins and integrates with Continuous Integration tools like Jenkins

Cons - requires downloading, limited scalability, limited reporting, creating a test scenario can be meticulous


jmeter gui


Click here to see a comparison of different open-source load testing tools.


Load Testing - CA BlazeMeter


CA BlazeMeter has all the abilities JMeter has since it’s an enhanced and upgraded version of JMeter.


Pros - scalability (JMeter in the cloud), advanced reporting, collaboration options, multiple plugins and test recording features

Cons - not open-source


CA BlazeMeter reports:


blazemeter gui reports analysis


Functional Testing - Selenium


Selenium is an open-source functional testing tool, which automatically tests browsers and enables testing of web applications.


Pros - open-source, supports a variety of languages, operating systems and browsers, records scripts (through Selenium IDE), works with JMeter

Cons - requires high expertise to use, does not support windows based applications, lacks online support


selenium system


Functional Testing - Perfecto Mobile


Perfecto Mobile is a functional testing tool for mobile apps, which runs automated app tests on real devices.


Pros - High optimization and accuracy due to running tests on real devices

Cons - Testing speed can be slow


Service Virtualization - CA


CA Service Virtualization creates virtual assets that enable developers and teams to work in parallel on their systems, without having to write code, to duplicate infrastructure or dependencies.


Pros - quick setup, integrates with testing and Continuous Integration tools

Cons - on-premise and heavy weight


We’re more than half way through the DevOps cycle! Next time we’ll discuss the release stage, so stay with us.


Click here to see a demo of CA BlazeMeter (obviously our favorite testing tool!).

Interested in writing for our Blog? Send us a pitch!