Noga Cohen is a Sr. Product Marketing Manager for CA BlazeMeter. She manages the BlazeMeter blog and other content activities. Noga focuses on creating technological content in the fields of performance, load testing and API testing, both independently and by managing writers who are developers. Noga has more than 5 years of experience in a wide scope of writing techniques: hi-tech, business, journalist and academic.

Become a JMeter and Continuous Testing Pro

Start Learning

Test Your Website Performance NOW!

arrow Please enter a URL with http(s)

How to Make Functional and Performance Testing Easy with Open-Source Testing Tools

Functional and performance testing are two important parts of testing your website or app, for discovering bottlenecks, errors and bugs. “Shifting left” and connecting the two types of tests in the Continuous Integration process improves your product’s quality and lowers costs. Running the tests separately takes more time, requires more meticulous work from you and prevents you from cross-analyzing trends and data through rich reports.


Taurus is an automation testing tool that makes it easy to configure test suites in a human-readable format, yaml or json, and integrate them in your continuous delivery process. With Taurus the tests can be run automatically and in parallel.


In this blog post, I will show you how to:

○ Define and run functional tests

○ Define and run performance tests

○ Integrate tests in a continuous delivery cycle with Jenkins


Defining and Running Functional Tests - A Selenium Tutorial


Our test scenario is run on the site. In the scenario, a user comes to the site, selects a flight, and purchases a ticket.  Here’s how to do this in three steps:


Step 1: Record the Use Case with Selenium


In this example, the use case we're interested in is a successful ticket purchase.


The simulated user loads the index page, chooses departure and destination, and presses the button to continue to the checkout page. The user enters their credit card data, and presses the button to purchase a ticket to the flight. The website should then show them a confirmation page.


You can record a Selenium script with Selenium IDE for this process, and add it to Taurus.


Step 2: Add the Tests to Taurus


You can run the Selenium scripts through Taurus by invoking the 'selenium' executor and adding them to the config file:


- executor: selenium
  # functional tests
    script: selenium/


Save the script as a yaml file, let’s say: example.yml.


Open your terminal and run 

bzt example.yml


As simple as that, Taurus will run it.


Step 3: Write Tests Natively in Taurus


The yaml or json format parsed by Taurus allows you to define each step of our use case above:


- concurrency: 10
  hold-for: 10s
  steps: 4
  scenario: usecase-purchase

    timeout: 5s
    retrieve-resources: true
    store-cache: false
    store-cookie: false
      User-Agent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36'
      Accept-Language: 'en-US'
      Accept-Encoding: 'gzip, deflate, sdch'
      Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp'
    - /index.php
    - url: '/reserve.php'
      method: POST
        Content-Type: application/x-www-form-urlencoded
        fromPort: Philadelphia
        toPort: Rome
    - url: '/purchase.php'
      method: POST
        Content-Type: application/x-www-form-urlencoded
        airline: United Airlines
        flight: '234'
        fromPort: Philadelphia
        price: '432.98'
        toPort: Rome
    - url: '/confirmation.php'
      method: POST
        Content-Type: application/x-www-form-urlencoded
        inputName: John Doe
        address: A Test Street
        city: Anytown
        state: SomeState
        zipCode: '54321'
        cardType: Visa
        creditCardNumber: '7463385993'
        nameOnCard: John Doe
        rememberMe: 'on'


While this step is optional for our example use case, you’ll find it to be extremely beneficial if you need to test performance.


Performance Testing Tutorial


You may want tests to ensure that the pages of your site will work as expected within reasonable response or load times. This means that you’ll decide the expected times for each page you want to check, and set one or more thresholds in Taurus for that page. If the threshold condition is reached, the test will be marked as failed and you can tell Taurus to take an action.


Performance testing using thresholds allows you to know immediately when you introduce a change that is too slow, like a terrible query that will slow down your application (I know, you'd never do such thing... but you can make your pipeline future-proof!).


You can do this in Taurus using the Pass/Fail module. For simplicity, we use the same scenario as above, and set values for the average response time of each page. Taurus will check and record the values for each pass. We can add to the above yaml:

  - module: passfail
      - avg-rt of IndexPage>10ms for 1s, continue as failed
      - avg-rt of ReservePage>15ms for 1s, continue as failed
      - avg-rt of PurchasePage>15ms for 1s, continue as failed
      - avg-rt of ConfirmationPage>15ms for 1s, continue as failed


In order to use these descriptive labels, we need to define them for the respective pages with the attribute 'label’. So we modify the initial yaml config to read in the pertinent part:

      - label: IndexPage
        url: '/index.php'
      - label: ReservePage
        url: '/reserve.php'
      - label: PurchasePage
        url: '/purchase.php'
      - label: ConfirmationPage
        url: '/confirmation.php'


The average response time is only one of the thresholds you can set. You can use others, such as fail:

      - fail of ConfirmationPage>50% for 10s, stop as failed


This is Taurus report screen for these example values:


Taurus dashboard ahowing functional and performance tests results


You can set the values to your liking, to serve your purpose. To find information about all the thresholds implemented in Taurus, and the explanation of each parameter, please see the Pass/Fail page in the documentation.


Adding Tests to Jenkins CI


By adding testing to Continuous Integration platforms like Jenkins, you will be able to see at any time if everything is still okay, for any commit you make. All you need to do is add a few commands to run bzt from the shell. You will be then able to see if your performance tests succeed or fail, giving you feedback every time your team changes something in your web application that can make things slower than the thresholds. This helps significantly because you know exactly which pull request caused the slowing down - and where.


Congratulations! You are now ready to define and run functional and performance tests, and integrate them in the CI cycle.


Want to learn? View our webinar Using Taurus to Automate JMeter and Selenium Tests

arrow Please enter a URL with http(s)

Interested in writing for our Blog?Send us a pitch!