In order to create automated testing in a continuous integration (CI) environment, with the objective of creating a continuous testing (CT) flow, you will need to familiarize yourself with a series of automation and testing tools. In this post, I’ll look at how to combine and leverage these tools to implement and automate testing in CI/CD.
Before we dive in, just a note that to better understand the rest of this blog post, you should already be familiar with the basics of programming languages and unit tests.
There are lots of tools and techniques that can be integrated in a CI environment. Here at BlazeMeter, we are using Selenium WebDriver Python and we integrate it with Jenkins. The testing framework we are working with is PyTest, which makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.
(Want to view a webinar on this topic? View the webinar recording at any time.)
Before writing your tests and being able to execute them on a nightly basis, there are several prerequisites which need to take into consideration.
Prepare Your Environment
In order to prepare your environment for testing, you should perform the following steps.
1. Choose your Browser
We will focus on ChromeDriver as a proof of concept, though there are other desktop browsers available out there such as FirefoxDriver, InternetExplorerDriver, SafariDriver, OperaDriver and PhantomJS. Download ChromeDriver and store it under the global /bin folder. In our case it will be /python27/bin.
2. Pick your Programming Language & IDE
We will focus on Python since this is the language we mostly use in BlazeMeter, but you may choose any language you desire as long as webdriver has bindings to it.
More info on the languages Selenium WebDriver supports can be found here
Additional info on Python can be found here
The relevant IDE (integrated development environment) for our purposes would be PyCharm.
3. Dive into Selenium Web Driver
Selenium WebDriver is one of the leading open source tools for automated testing. Generally speaking, Webdriver is one of the top GUI testing tools there is. If you haven't heard about WebDriver yet, this is your time to deep dive in order to utilize a high-quality automated testing and a stable product. Additional info on WebDriver can be found here.
4. Decide Which Testing Framework You Want
Depending on the programming language that you have chosen, the next step would be to choose the testing framework. We will focus on Py.Test since this is the testing framework we are working with in BlazeMeter.
Of course, there are many other testing frameworks from which to choose from.
In this post, we will describe tests created via Py.Test which interprets the test.py file and parses only the functions that contains ‘test’ in them. Additional info about how to write tests under PyTest convention can be found here.
Create your Tests
Once you have prepared your environment for automated tests, go ahead and create your first End-to-End (E2E) flow. Keep in mind that your tests should be as quick and short as possible in order to prepare for frequent changes in the code (i.e. login to your website and verify the URL once signed in).
Make sure to follow the py.test structure to run your tests as expected.
Here is a short example code for a small test using Webdriver Python vs. PyTest:
You can reference the full GitHub test repository here.
Run Your Tests Locally
Since we have created the tests under Py.Test convention, the correct way to run them in your local environment would be:
Option 1 - using the default values configured in parser.addoption under the conftest.py file.
The results should be something like this:
Option 2 - Run your tests using parameter as seen below:
Run Your Tests via Jenkins
As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.
NOTE: It is best to run your tests in a pre-configured virtual environment which installs requirements.txt prior to implementing them in Jenkins.
Create a Virtual Environment - information on how to create a virtual Python environment and its usage can be found here
Export Project Requirements - Additional information on how to export all relevant installed modules to requirements.txt can be found here
Install the Allure Reports Plugin to Jenkins
Allure is an open source framework designed to create test execution reports with good representation of the test execution output. The Allure Plugin allows you to automatically generate an Allure Report and attach it to any build during a Jenkins job run. Install the plugin and then let’s start working with Jenkins.
Create a New Jenkins Job
The Jenkins job you are now going to create should include all of the steps mentioned above, including:
- Create virtual environment
- Install requirements.txt
- Activate virtual environment
- Run tests
- Deactivate virtual environment
Create a new Jenkins job with the following SHELL script:
As seen in the screenshot below:
View Your Test Results in Allure Reports
After successfully integrating your project into Jenkins, the next challenge is to export the test results to a readable user friendly report. Allure allows to inject environment variable and to provide statistics on tests statuses.
It also has the option of embedding images, xml, json, txt, mp4 and many more attachments.
The execution of the py.test suits will be done using parameters or by their default values as described earlier.
In our case, we will use their default values as follows:
Allure has lots of cool features and parameters which can be embedded into the live report such as screenshots, logs files and many more.
The following GIF will give you a sense on how Allure is shown via Jenkins after a build is finished
Schedule Jobs on a Nightly Basis
Be sure to verify that the CI environment is configured to the latest release on a nightly basis. Additional information on how to configure a Jenkins job to run on a specific schedule can be found here.
Congratulations! All your tests are now configured in Jenkins. All that’s left is to schedule these tests to run automatically on your CI environment which should be with the latest build on a nightly basis.
In the next guide, I will explain how to create a master job in Jenkins that triggers all these jobs that you have created.
I hope this tutorial assisted you in achieving your goals.
NOTE: While this post was focused on unit and functional testing, Jenkins - combined with open source tools like Apache JMeter and Taurus, and of course BlazeMeter - is also a great tool for continuous performance testing as well. If you haven’t already, I encourage you to read more about how to run JMeter With Jenkins, running a JMeter test via Jenkins Pipeline, or how to easily automate performance tests in Jenkins.
You can also request a live 1-on-1 demo of how to leverage tools like Taurus, Selenium and Jenkins with BlazeMeter's performance testing.
Want to learn more? View our webinar Using Taurus to Automate JMeter and Selenium Tests.
Do not hesitate approaching if you have any feedback, questions or other concerns.
Happy automation testing!
You might also find these useful:
Interested in writing for our Blog? Send us a pitch!