How to Use PyTest Framework to Improve Selenium

When you think about choosing a framework for test automation, you would probably like to have a comprehensive solution that fits any type of test: unit, functional, end-to-end, acceptance and others. You would also want this framework to log events, share common data, track how much time test execution took, execute prerequisites and tear down steps, support data driven testing and show reporting. PyTest framework meets all these requirements and will do a great job executing your tests. Therefore, we recommend writing your Selenium WebDriver tests in PyTest.

Here, in this blog post, you will see how to use the most powerful instruments of PyTest framework, such as fixtures, incremental testing and parameterized fixtures. The pytest.mark decorator will also be shown, integrated in the other features. PyTest framework is not limited only to these features, but rather has much more, and you can find a full list with descriptions available on the official site. You can find the complete project shown in this blog post on GitHub, at this link.

To learn more basic Selenium and Python, read this blog post “How to Understand Your Selenium WebDriver Code and Run it with Taurus“.

Getting Started with PyTest Framework

To demonstrate all the major PyTest features, we will create tests to search and book flights on http://blazedemo.com with the help of Selenium. Selenium is a framework that includes tools and libs to manipulate elements on a web page: enter text, click on element, search entity in table, etc.

Below you can see the list of required packages and tools that need to be installed to go further.

1. Install Python from its downloading site. There are two major versions of python nowadays: 2.7.14 and 3.6.4. Code snippets in the blog post will be given for version 2.7.14 but they work for version 3.6.4. It’s up to the reader to choose which version to install.

2. Install python package manager (pip). This can be downloaded from its download page. All further installations in the blog post will be made with usage of pip so it’s highly recommended to install it.

3. For the development environment, the PyCharm Community edition will be used. You can download it from the Pycharm website. You can use any IDE of your choice since code snippets are not IDE dependent. 

Now, let’s create our project.

4. Create the project in PyCharm IDE with File -> New Project.

5. Specify the location for the project (the last part of the path will be the project’s name). Select the existing python installation in the “Interpreter” field.

The next step will be to define the project’s dependencies.

Python has an option to install all project packages to a virtual environment associated with it. This shows good design because you can keep the project’s packages pool with the exact versions, without interactions with other projects. In the blog post all packages will be installed in the virtual environment.

To create a virtual environment:

6. Install the Virtual Environment tool with the command pip install virtualenv in the prompt.

7. In the project root directory, create the environment with virtualenv ENV in the prompt where ENV is the environment’s name.

As a result you will notice a new folder in the project directory – ENV.

8. In the “ENV” directory you can find the separate python interpreter and pip installations. After the environment’s activation, all packages will be installed to this directory. The way you can activate an environment differs per operating system. In POSIX systems you can run the source bin/activate command from the environment root folder. In Windows systems, go to environment folder -> Scripts and execute activate.bat from the prompt.

If the environment is activated, your prompt will be prefixed with the environment’s name as below:

9. Execute two commands in the prompt: pip install PyTest and pip install selenium, to install PyTest and Selenium accordingly. The  PyTest and Selenium packages will be installed only for the project within the “ENV” environment, which in the example is “blazemeter-pytest”.

Now let’s learn more about how to use PyTest framework.

PyTest Framework Features

PyTest Fixtures

When we are developing tests with Selenium, the first issue we need to solve is when to load a browser. It’s not good practice to load a browser before each test. Rather, it is much better to load a browser before all tests and then close it after all tests are executed, as this will save resources and test execution time.

In PyTest we can solve these kinds of issues with a powerful instrument – fixtures. Fixtures allows you as the test engineer to maintain a fixed and well-known environment for testing. Fixtures in PyTest leverage the idea of dependency injection when your tests get prepared dependencies without taking care of setup and teardown. It’s a very convenient way to create independent tests.

So, following our current goal, let’s create the fixture to start the web browser before the tests and shut it down after. You can do this with fixtures by manipulating the fixture’s attribute scope. In general, a fixture’s scope defines how many times a fixture will be invoked. A fixture can have 4 scopes: module, class, session and function (the function scope is the default value). There are 2 major ways to manipulate,  the first option is to use scope=session and the other option is to use scope=class.

If the module scope is defined, the fixture will be created only once per module. The same applies for the class scope: only 1 fixture will be created per class object. With the session scope, the fixture will be created only once per test session. The session scope is what we are looking for to manage the Selenium WebDriver session.

To create a fixture that will be shared across several tests, we need to put it in the file conftest.py. Any fixture in it will be recognized automatically.

A fixture that starts the web browser looks like this:

@pytest.fixture(scope="session")
def driver_get(request):
    from selenium import webdriver
    web_driver = webdriver.Chrome()
    session = request.node
    for item in session.items:
        cls = item.getparent(pytest.Class)
        setattr(cls.obj,"driver",web_driver)
    yield
    web_driver.close()

The driver_get function is decorated with @pytest.fixture to indicate that it is a function that will be used as a fixture with scope=session. The request parameter refers to the test session, as fixture functions can accept the request object to introspect the “requesting” test function, class, module or session, depending on the scope.

In this method, at first we start with a Chrome driver. Make sure you have chromedriver executable in your PATH variable. To download the ChromeDriver to your system navigate to its download page. 

Later, for every object that is recognized as a test class instance, we set the attribute named “driver” that is the reference to the started chromedriver instance. After, we quit the fixture function with the yield statement. This is needed as after all tests have been executed, and we need to shut down the web driver. This is managed by the last line web_driver.close() as it will be executed after the last usage of the Selenium WebDriver.
 

[Note: There is another way you can shut down the browser after all tests are executed: by adding this finalizer to the request object:
 

request.addfinalizer(web_driver.close)

However, with the finalizer, an attempt to close the web driver will be made even if the web driver did not start due to any exception occurred. That is not what we are looking for, so the yield approach should be applied prior to the finalizer.]

Let’s see how this can be used in tests for the booking_test.py module (the test to blazedemo.com).

In PyTest, to make your function recognized as a test, the following rules should be applied by default:

1. The module name with the tests should begin or end with “test
2. The test class should begin with “Test”
3. Every test method or function should begin with “test”
    

Consider the following test class with two test methods: searching for the flight and choosing it for booking. In the “test_choose_any_flight” method an assertion is made to check if the user is taken to the flight reserve page.
 

@pytest.mark.usefixtures("driver_init")
class TestBooking:

    def test_search_flight(self):
        wait = WebDriverWait(self.driver, 3)
        self.driver.get("http://blazedemo.com/")
        wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"input[value='Find Flights']"))).click()

    def test_choose_any_flight(self):
        wait = WebDriverWait(self.driver, 10)
        wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input[value='Choose This Flight']"))).click()
        text = self.driver.find_element_by_tag_name("h2").text
        assert text == "Your flight from Paris to Buenos Aires has been reserved."

The Class TestBooking is decorated with @pytest.mark.usefixtures(“driver_init”). This decorator is useful when you don’t need direct access to the fixture’s instance but still need to execute an initialization or tear down the code in tests. Applying usefixtures to the class is the same as applying a fixture to every test method, but without direct access to the fixture’s instance. Since the fixture “driver_get” has a session scope, the web driver will be initialized only once. Now let’s go back to the fixture’s code and have a closer look at the following lines:
 

for item in session.items:
        cls = item.getparent(pytest.Class)
        setattr(cls.obj,"driver",web_driver)

In the lines above, for every test class object that is decorated with @pytest.mark.userfixtures(“driver_init”) we set the attribute “driver”. So in the test class we can access the web driver instance with self.driver. You can notice that we haven’t setup the WebDriver instance implicitly, instead we are just using it via reference to the self.driver. This is the implementation of the dependency injection: in the test class or function we don’t know how the WebDriver was initialized or what is going to be happen later, we are just using the instance.

Another way to initialize and shut down the WebDriver with fixtures is to use a fixture with the scope class.

The fixture with class scope:
 

@pytest.fixture(scope="class")
def driver_init(request):
    from selenium import webdriver
    web_driver = webdriver.Chrome("C:/chromedriver.exe")
    request.cls.driver = web_driver
    yield
    web_driver.close()

In comparison to the fixture with the session scope, the way we set the driver attribute is simpler: via “request.cls.driver = web_driver”.

But here we have one limitation - to initialize the web driver only once, this fixture should be applied to the shared base test class. All test classes should be extended from this base test class. Let’s review the code:

# Base class that keeps the reference to the web driver.
 

@pytest.mark.usefixtures("driver_init")
class BaseTest:
    pass

A test class with two tests:

class TestBooking(BaseTest):

    def test_search_flight(self):
        wait = WebDriverWait(self.driver, 3)
        self.driver.get("http://blazedemo.com/")
        wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"input[value='Find Flights']"))).click()

    def test_choose_any_flight(self):
        wait = WebDriverWait(self.driver, 10)
        wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input[value='Choose This Flight']"))).click()
        text = self.driver.find_element_by_tag_name("h2").text
        assert text == "Your flight from Paris to Buenos Aires has been reserved."

As you can see, the changes in the test class are minimal: we get rid of the decorator pytest.mark.usefixtures but have the added extension from the BaseTest class. The code of the test methods wasn’t changed.
 

Incremental Testing

You might have noticed that the  test “test_choose_any_flight” in the TestBooking class depends on the “test_search_flight”. If the first fails, there is no sense in running the second one. For this purpose, PyTest has an incremental marker that will mark all dependent tests as expected to fail, so they don’t get executed. To see how it works add the following lines to the confttest.py:

def pytest_runtest_makereport(item, call):
    if "incremental" in item.keywords:
        if call.excinfo is not None:
            parent = item.parent
            parent._previousfailed = item

def pytest_runtest_setup(item):
    if "incremental" in item.keywords:
        previousfailed = getattr(item.parent, "_previousfailed", None)
        if previousfailed is not None:
            pytest.xfail("previous test failed (%s)" %previousfailed.name)

Mark the test class with the @pytest.mark.incremental as shown below:
 

@pytest.mark.incremental
class TestBooking(BaseTest)

Add the assert that will fail:
 

assert 0

Now execute the tests.

In PyTest, to execute tests just type pytest in the prompt in the project root folder. This will execute all tests. To execute certain tests in your module, just type pytest booking_test.py to execute all the tests from booking_test module.

After the execution you will see the following report:

Only 1 test is marked as failed, the second is marked as expected to fail (xfailed).

This is a very useful instrument if you have a chain of tests, isn’t it?

Parameterized Fixtures

OK, the tests we have created in the blog post were run only in Chrome browser. But what if we want to run tests in Firefox and Chrome? For this purpose, we can use another useful feature of PyTest – parametrization of a fixture.

All tests that use fixture “driver_init” will be executed in Chrome and FireFox browsers.

The parameterized driver_init fixture looks like this:
 

@pytest.fixture(params=["chrome", "firefox"],scope="class")
def driver_init(request):
    from selenium import webdriver
    if request.param == "chrome":
        web_driver = webdriver.Chrome()
    if request.param == "firefox":
        web_driver = webdriver.Firefox()
    request.cls.driver = web_driver
    yield
    web_driver.close()

You can parametrize not only fixtures but test functions as well. The test “test_search_flight” navigates to the URL  defined in the body of test. Let’s make the URL the parameter with the help of @pytest.mark.parametrize:

@pytest.mark.parametrize("url", ["http://blazedemo.com/"])
def test_search_flight(self, url):
    wait = WebDriverWait(self.driver, 3)
    self.driver.get(url)
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"input[value='Find Flights']"))).click()

With the help of @pytest.mark.parametrize you can follow a data-driven approach and execute your tests on different data sets.

But what if I want to run only a single test? For this, you can just use @pytest.mark like this:

@pytest.mark.search
def test_search_flight(self, url):
    wait = WebDriverWait(self.driver, 3)
    self.driver.get(url)
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"input[value='Find Flights']"))).click()

And then execute the test with the “pytest -m search” command.

In the test “test_choose_any_flight” we have used the keyword assert to verify the expected result.

An assertion checks whether the argument is true. If the expected condition isn’t met, a detailed error message will be shown:

If we reserved an incorrect flight, then the informative error message would be shown.

But what if I want to check that a test fails with expected error? No problem, PyTest can handle this.

def test_choose_any_flight(self):
    wait = WebDriverWait(self.driver, 10)
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input[value='Choose This Flight']"))).click()
    text = self.driver.find_element_by_tag_name("h2").text
    with pytest.raises(AssertionError):
        assert text == "Your flight from Paris to Boston has been reserved."
    print "Seems like you have booked incorrect flight!"

By saying “with pytest.raises” we can capture the exception and handle it in our own way.

Now, if an incorrect flight was booked, there will not be AssertionError. This can be useful if you don’t want to fail a test due to the exception.

What about reporting? Again, no problem - just install the pytest-html report tool with the command pip intall pytest-html in the prompt and start your tests with pytest --html=report.html.

Run Your PyTest Framework Test with BlazeMeter

You can also upload your PyTest file to BlazeMeter and run your functional test:

By running your test in BlazeMeter you will be able to easily run your functional tests and also collaborate on your tests with your colleagues and management.

In this blog post, you saw how to use the major features of the PyTest framework. You can explore more functions following the official documentation discovering more and more options in usage of PyTest. PyTest offers a great flexibility in creating tests and hence is the great choice as test framework!

P.S. don’t forget to shut down the active environment ENV when finish working by “deactivate” command in the project root directory.

START TESTING NOW

Related Resources

• How to Run Selenium Tests in Docker
• How to Automate Testing Using Selenium WebDriver, Jenkins and Allure
• 6 Easy Steps to Testing Your Chrome Extension With Selenium
• How to Understand Your Selenium WebDriver Code and Run it with Taurus
• Headless Execution of Selenium Tests in Jenkins
• Using Selenium with JMeter's WebDriver Sampler
• How to Run a Selenium Test with JUnit
• Mixing Selenium Into Your Load Scenario