Adrian Lopera-Valle, Ph.D., is a technical consultant with Scimitar Technical Consulting, Inc. Adrian's research experience allows him to identify and address clear objectives in complex projects, and to communicate efficiently about them.

Become a JMeter and Continuous Testing Pro

Start Learning
Slack

Test Your Website Performance NOW! |

arrowPlease enter a URL with http(s)
Dec 06 2021

Testing with Selenium and Other Automated Frameworks

Testing is an important stage of application development. In most projects, particularly in large ones, this phase can be repetitive and tiring for developers. When done manually, these tasks may introduce human error and decrease efficiency. To address this, automation tools, like Selenium, are used to test key performance indicators (KPI) by following a script. If you are interested in knowing more about Selenium and other automated testing tools, this blog is for you.

In this post, we compare manual and automated testing, compare three popular applications, and show you some of the best practices for test automation using Selenium.

Automated vs. Manual Testing

Browser testing is performed to check the functionality of an application across multiple browsers and operating systems. In manual testing, a quality assurance tester manually evaluates a software by looking for defects, following a test plan. On the other hand, automated testing uses automation frameworks, like Selenium, with scripted steps to perform the tests. The main difference between automated and manual testing is who does the testing itself. While this seems simple to distinguish, it may not always be clear how to choose one over the other. The key differences between these two approaches are:

  • Multiple manual tests take longer in most applications
  • Automation often comes with higher reliability and accuracy as human error is eliminated
  • Manual testing is intuitive and does not require much training
  • The initial investment of automated testing may be higher, although the return on the investment is often also higher
  • Exploratory tasks can only be done with manual testing, while automated testing is more effective for other performance testing tasks (load testing, stress testing, soak testing, etc)
  • Manual testing continues to be better for user interface projects
  • Manual testing may fit small projects, where testing is done once or twice

Introduction to Selenium Tools

Selenium is an open-source suite of tools for the automation of web browsers. It can be used to accelerate testing by providing users with an interface to write Selenium test scripts using languages like PHP, Python, and C#. Users can develop their own automated tests for different operating system and browser combinations. Selenium is highly adaptable, customizable, and compatible with existing browsers, testing frameworks, and operating systems, which makes it one of the most popular test automation tools on the market.

Selenium is composed of different projects, each of them offering different features for testing. The main projects of the Selenium “ecosystem” are:

  • WebDriver runs tests based on application programming interfaces (APIs) to control browsers. The tests are done with APIs of the browser, and not the WebDriver API, therefore the application is tested under highly realistic conditions. For example, in this paper, the authors created a data-driven framework to perform an automated test for email login using Selenium WebDriver.
  • Integrated development environment (IDE) is a tool to create full Selenium test cases, where the user's actions in the browser (Chrome and Firefox) are recorded and translated into Selenium commands. This project is considered to be the most efficient way to develop test cases because it is easy to use, saves time, and allows the user to learn Selenium script syntax.
  • Grid supports testing on multiple combinations of browsers and operating systems. This tool allows for test cases developed on WebDriver in a local end to be executed in a remote end.
  • Remote control (RC or Selenium 1) was the main Selenium project before WebDriver and others were introduced. While it is hardly used and no longer supported, it can still be used to write complex test cases.

The latest version of Selenium WebDriver was released in May 2021. You can learn more about what it brings in this article. To stay up to date with the development of Selenium, follow our blog.

The Key to Successful Selenium Testing

There is no universally perfect way of automating tests. With all the tools, design approaches, and scales of testing, it is hard to find a single “recipe” for successful testing. However, we want to give you some general points to increase your chances of successfully implementing Selenium WebDriver.

  1. Learn. Selenium documentation and SeleniumEasy are free resources.
  2. Design and plan how the test may be conducted before running a case.
  3. Maximize the browser window. Selenium will take screenshots during testing, it will be easier for you if the browser is clearly displayed.
  4. Set zoom to 100%. Changes in the browser zoom may change the location of objects and the coordinates of the mouse as the test is happening.
  5. Use real devices, instead of emulators if possible. This may make a difference in the accuracy of the results.
  6. Start tests on a fresh page. Make sure to refresh the page between tests.

Selenium vs. Cypress vs. Puppeteer

While Selenium is the most commonly used tool for automated testing, Puppeteer and Cypress have recently gained popularity. Puppeteer is an automation tool but only for Chrome browser, often used to run website tests. Cypress is a website testing tool built by and for front-end developers, which means that Cypress focuses on the graphical user interface (GUI) that users view and interact with. Here, we compare some relevant features of these frameworks.

 

Selenium

Cypress

Puppeteer

Language support

C#, JavaScript, Java, Python, and Ruby

JavaScript

Node.js

Browser support

Google Chrome, Firefox, IE, Safari, and any chromium-based browsers

Google Chrome, Firefox, and any chromium-based browsers

Only Chrome

Test framework

PyUnit, JUnit, TestNG, almost any language

Mocha JS

Jasmine or Mocha JS

Setup

Medium: Selenium and the browser drivers must be configured utilizing Selenium

Low: No additional browser drivers required

Driver dependencies

Dependent on specific browser drivers

No driver dependency

Multi-tab

Yes

No

Yes

Speed

Comparatively slow as commands are sent to the browser through the network

Fast as tests are executed within the browser.

Mobile testing

Yes

No

Yes

Licensing

Open-source

Open-source with some subscription-based features

Open-source

 

Choosing one of these tools over the others will depend on your particular needs.

  • Cypress will provide you with a built-in, front-end-oriented solution to best test the GUI users are exposed to. In addition, its performance tends to be higher as it is designed exclusively for testing.
  • Puppeteer can be the go-to automation framework for developers and testers provided they are working exclusively with Chrome.
  • While Selenium, as a set of features, is more complex, it allows developers to adapt tools to their particular needs, from front-end to server-end testing, including mobile emulation. The flexibility and wide range of integrations that come with Selenium, coupled with its open-source licensing, may explain why it is the leading tool for test automation. However, the selection of a particular software will depend on your particular project and scope.

For a complete performance comparison between the Selenium and Cypress, check out our recent blog.

You can also learn more general guidelines and recommendations here. Alternatively, you can see what to avoid when automating testing with Selenium here. Finally, you should know that the integration of Selenium with BlazeMeter may lead to better results. You can use the free Chrome extension recorder, which can record and run scripts, in Selenium, JMeter, or both together. You can learn how to get started here or install our chrome extension here.
 

   
arrowPlease enter a URL with http(s)

You might also find these useful:

Interested in writing for our Blog?Send us a pitch!