Test Automation Using Pytest and Selenium WebDriver
Test Automation Using Pytest and Selenium WebDriver: For all your cross-browser, multi-device testing needs, look no further than the powerful combination of Selenium WebDriver and pytest.
One of the challenges that developers face is ensuring that their web application works seamlessly across a different set of devices, browsers, and operating systems platforms. This is where cross-browser testing plays a very crucial role in testing the web application since it helps in testing across different combinations. Based on the target market, development and product teams need to chart out a plan for the various activities involved in cross-browser compatibility testing.
Selenium – Introduction and WebDriver Interface
As far as testing a web application is concerned, there a couple of web frameworks available that automate the tests performed across different web browsers. Selenium is a very popular framework that is primarily used for the automation testing of web applications. It is an open-source tool with which web testing can be performed against popular browsers like Chrome, Firefox, Opera, and Microsoft Edge. The framework can also be used if the test has to be performed on Internet Explorer (latest version or the legacy versions).
Selenium WebDriver is considered one of the core components of the Selenium framework. Selenium WebDriver API is a collection of open-source APIs/language-specific bindings that accepts commands and sends them to the browser, against which the testing is performed. The individual who is responsible for developing the tests need not bother about the architecture details or other technical specifications of the web browser since WebDriver acts as an interface between the test suite/test case and web browser (achieved using the browser-specific WebDriver).
Selenium WebDriver supports different programming languages like Python, C#, Ruby, PERL, and Java. The diagram below shows a simplified view of the Selenium WebDriver Interface. We have already covered the Selenium WebDriver architecture in-depth in our earlier post.
Pytest Test Framework – Introduction and Advantages
Python has a couple of test frameworks that ease the task of web application testing; unittest and pytest are the most widely used frameworks. unittest is a part of the standard library (in Python) and comes as a part of the Python installation. For test automation using pytest, the more popular of the two, with Selenium WebDriver, you need to install pytest separately. Here are some of the advantages of the pytest framework:
- Can be used by development teams, test teams, teams that are practicing Test-Driven Development (TDD), as well as in open-source projects.
- Can be used in simple, as well as complex, functional test cases for applications and libraries.
- Easy to port existing test suites to pytest for performing test automation using pytest with Selenium WebDriver.
- Compatibility with other test frameworks like unittest and nose, so switching to this framework is very easy.
- Supports parameterizing, which is instrumental in executing the same tests with different configurations using a simple marker. You can come up with more effective test cases/test suites with less repetitive code implementation.
- The highh number of asserts that provides more detailed information about the failure scenarios.
- Support of Fixtures and Classes. Using Fixtures, it becomes easy to make common test objects available throughout a module, session, function, or class. Fixtures and Classes will be covered in more detail in subsequent sections.
- Good and up-to-date documentation.
- xdist support through which test cases can be parallelized.
To summarize, Pytest is a software test framework which can be used to make simple, yet scalable test cases with ease.
Now that you are aware of the advantages of pytest over other test frameworks, let’s have a detailed look at the pytest framework and how it can be used with Selenium WebDriver framework in order to perform automated cross-browser testing for web applications.
Test Automation Using Pytest – Installation and Getting Started
As mentioned earlier, pytest is not a part of the standard Python installation and needs to be installed separately. In order to install pytest, you should execute the following command on the prompt/terminal:
pip install –U pytest
Once the installation is complete, you can verify whether the installation is successful, by typing the following command:
pytest --version
Below is the output when the above command is executed on Linux and Windows machine
PyCharm is a popular IDE that is used for pytest development. You can install the PyCharm Edu version for Windows, Linux, or macOS. For development, we are using PyCharm for Windows. Once PyCharm is installed, you should make sure that the default test runner is pytest. In order to change the default test runner, you should navigate to File -> Settings -> Tools -> Python Integrated Tools and change Default test runner for performing test automation using pytest with Selenium WebDriver.
Now that PyCharm Edu is installed and the default test runner is set to pytest, you need to install the Selenium package for Python to perform test automation using pytest with Selenium WebDriver. In order to install Selenium, you should invoke the command mentioned below in the terminal of PyCharm.
pip install -U selenium ( Syntax – pip install –U )
Shown below is the snapshot of the command execution:
Now that your development environment is all set, we look into some of the features and aspects of pytest.
Pytest – Usage, Exit Codes, and Compilation
pytest and py.test can be used interchangeably. In order to get information about the arguments that can be used with pytest, you can execute the command below on the terminal.
pytest --help
#Command to get help about the options that can be used with pytest command
# Details about fixtures pytest --fixtures #Shows the available built-in function arguments
When pytest code is executed, it results in one of the following exit codes:
| EXIT CODE | DESCRIPTION |
| 0 | Test cases/test suites are executed successfully and end result was PASS |
| 1 | Test cases/test suites were executed, but some tests FAILED |
| 2 | Test execution was stopped by the user |
| 3 | Unknown error occurred when the tests were executed |
| 4 | Usage of pytest command is incorrect |
| 5 | No tests were collected |
It is important that the file containing pytest code be named as test.py or test.py. In order to compile and execute pytest source code for performing test automation using pytest with Selenium WebDriver, you can use the following command on the terminal
pytest <test_file_name.py> --verbose --capture=no
Let’s have a look at some examples of test automation using pytest. We start with a very simple example –test_pytest_example_1.py.
#pytest in action – test_pytest_example_1.py
def function_1(var):
return var + 1
def test_success():
assert function_1(4) == 5
def test_failure():
assert function_1(2) == 5
In the above code snippet, we create a function named function_1 which takes one argument named var . There are two test cases: test_success() and test_failure() . The test cases are executed in serial order and the assert is issued on an execution of the test cases. Compile the code using the command mentioned below
pytest --verbose --capture=no test_pytest_example_1.py
As seen in the output, the result of the first test case is PASS (shown in blue) and a result of the second test case is FAIL (shown in red).
pytest makes use of the assert available in Python for verification of results. It gives out meaningful information which can be used for verification and debugging. pytest.raises is commonly used to raise exceptions; below is an example where a Factorial of a number is calculated. In one test case, a negative number is passed as an input to the factorial function and AssertionError is raised.
factorial_example.py contains the implementation that uses recursion in order to calculate factorial of the input number. Before the factorial is calculated, the input parameter check is performed. Assert would be raised in case the input number is negative.
def factorial_function(number):
# Perform a check whether the input number is positive or not, if it is not
# positive, raise an assert
assert number >= 0\. and type(number) is int, "The input is not recognized"
if number == 0:
return 1
else:
# recursive function to calculate factorial
return number * factorial_function(number – 1)
test_factorial_example.py is a pytest implementation which use factorial functionality. Three test cases are implemented – test_standard_library (output from factorial_function is compared with the output obtained from math.factorial module), test_negative_number (assertion is raised when the input number is negative), and (results of output from factorial_function are compared with specific values).
# Import the necessary modules/packages required for implementation
import pytest import math
from factorial_example import factorial_function
def test_factorial_functionality():
print("Inside test_factorial_functionality")
assert factorial_function(0) == 1
assert factorial_function(4)== 24
def test_standard_library():
print("Inside test_standard_library")
for i in range(5):
# verify whether factorial is calculated correctly
# by checking against result against standard
# library - math.factorial()
assert math.factorial(i) == factorial_function(i)
def test_negative_number():
print("Inside test_negative_number")
# This test case would pass if Assertion Error
# is raised. In this case, the input number is negative
# hence, the test case passes
with pytest.raises(AssertionError):
factorial_function(-10)
You can execute the code using the command py.test –capture=no test_factorial_example.py , either on the command prompt or on the Terminal of PyCharm IDE. As seen in the snapshot, all the test cases have passed and logs under “print statement” are output on the console
Test Automation Using Pytest – Fixtures (Usage and Implementation)
Consider an example where you have to execute certain MySQL queries on a database that contains employee information within an organization. The time taken to execute a query would depend on the number of records (i.e. employees) in the database. Before queries are executed, required operations (w.r.t database connectivity) have to be performed and the “returned handle” would be used in a subsequent implementation involving the database. Database operations can be CPU intensive (as the number of records increases); hence, repetitive implementation and execution should be avoided. There are two ways in which this issue can be solved:
- With the help of classic xunit style setup along with teardown methods.
- By using fixtures (recommended).
The xunit style of fixtures is already supported in unittest but pytest has a much better way of dealing with fixtures. Fixtures are a set of resources that have to set up before the test starts and have to be cleaned up after the execution of tests is complete. It contains a lot of improvements over the classic implementation of setup and teardown functions. The main advantages of using fixtures are
- Can be used by development teams, test teams, teams that are practicing Test-Driven Development (TDD), as well as in open-source projects.
- Can be used in simple, as well as complex, functional test cases for applications and libraries.
- Easy to port existing test suites to pytest for performing test automation using pytest with Selenium WebDriver.
- Compatibility with other test frameworks like unittest and nose, so switching to this framework is very easy.
- Supports parameterizing, which is instrumental in executing the same tests with different configurations using a simple marker. You can come up with more effective test cases/test suites with less repetitive code implementation.
- The highh number of asserts that provides more detailed information about the failure scenarios.
- Support of Fixtures and Classes. Using Fixtures, it becomes easy to make common test objects available throughout a module, session, function, or class. Fixtures and Classes will be covered in more detail in subsequent sections.
- Good and up-to-date documentation.
- xdist support through which test cases can be parallelized.
Ever since the launch of version 3.5, the fixtures of higher scope are prioritized above the lower scope fixtures in terms of instantiating. Higher scope fixture includes sessions, and lower scope fixture would include classes, functions, and others. You can even ‘‘parameterize" these fixture functions in order to execute them multiple times along with the execution of dependent tests.
Fixture parameterization has been widely used to write exhaustive test functions. Below is a simple code for test automation using pytest where setup() and teardown() of ‘resource 1’ is called, even when the test_2 is executed. Since this is a simple implementation (with fewer computations), there are not many overheads even when unnecessary setup and module calls are invoked, but it could hamper the overall code performance in case any CPU-intensive operations (like database connectivity) are involved.
#Import all the necessary modules import pytest
def resource_1_setup():
print('Setup for resource 1 called')
def resource_1_teardown():
print('Teardown for resource 1 called')
def setup_module(module):
print('\nSetup of module is called')
resource_1_setup()
def teardown_module(module):
print('\nTeardown of module is called')
resource_1_teardown()
def test_1_using_resource_1():
print('Test 1 that uses Resource 1')
def test_2_not_using_resource_1():
print('\nTest 2 does not need Resource 1')
Execute the test case ‘test_2_not_using_resource_1’ by invoking the following command on the terminal:
pytest --capture=no --verbose test_fixtures.py::test_2_not_using_resource_1
As observed from the output [Filename – Pytest-Fixtures-problem.png], even though “test_2” is executed, the fixture functions for “resource 1” are unnecessarily invoked. This problem can be fixed by using fixtures; we will have a look at these in the upcoming example.
As seen in the example below, we define a fixture function resource_1_setup() (similar to setup in xunit style implementation) and resource_1_teardown() (similar to teardown in xunit style implementation). The fixture function has “module scope” using @pytest.fixture(scope=’module’) .
#Import all the necessary modules import pytest
#Implement the fixture that has module scope
@pytest.fixture(scope='module')
def resource_1_setup(request):
print('\nSetup for resource 1 called')
def resource_1_teardown():
print('\nTeardown for resource 1 called')
# An alternative option for executing teardown code is to make use of the addfinalizer method of the request-context
# object to register finalization functions.
# Source - https://docs.pytest.org/en/latest/fixture.html
request.addfinalizer(resource_1_teardown)
def test_1_using_resource_1(resource_1_setup):
print('Test 1 uses resource 1')
def test_2_not_using_resource_1():
print('\n Test 2 does not need Resource 1')
We execute the code by triggering all the test cases. As shown in the output below [Filename – Pytest-Fixtures-all-tests-executed.png], “setup for resource 1” is called only for Test 1 and not for Test 2.
Now, we execute only test case 2, that is, test_2_not_using_resource_1() . As seen in the output below [Filename – Pytest-Fixtures-only-2-tests-executed.png], setup and teardown functions for Resource 1 are not called since the only test case 2 is executed. This is where fixtures can be highly effective since it eliminates repetitive code and execution of unnecessary code. Official documentation about fixtures in pytest can be found here.
Test Automation Using Pytest with Selenium WebDriver
When you are looking out for a test automation framework, you would probably require a test framework that meets all your requirements. The framework should have the ability to log events, generate test reports, and should have good community support. Pytest fulfils all these requirements and test automation using pytest with Selenium WebDriver is highly recommended as it does not involve a steep learning curve.
When you are planning to develop test automation using pytest with Selenium WebDriver, the first concern that you need to look into is when you should load the browser. Loading a new browser instance after each test is not recommended since it is not a scalable solution and might increase the overall test execution time. It is recommended to load the browser (under test) before the actual test cases have started and unloaded/closed the browser instance as soon as the tests are complete. This is possible by using Fixtures in pytest. As mentioned earlier, Fixtures make extensive use of a concept know as dependency injection, where dependencies can be loaded before the actual tests have started.
By default, fixtures have function scope, depending on the requirements; you can change the implemented fixture’s scope to a module, session, or class. Like the lifetime of variables in C language, the scope of fixtures indicates how many times the particular fixture will be created.
| FIXTURE SCOPE | EXPLANATION |
| Function | Fixture is executed/run once per test session |
| Session | One fixture is created for the entire test session |
| Class | Only one fixture is created per class of tests |
| Module | Fixture is created once per module |
Once the tests have been executed, you might be interested to capture the test results in a report format (like HTML). You need to install pytest-html module for the same
pip install pytest-html
Below is the snapshot of the command in execution:
Now that you have knowledge about pytest fixtures, Selenium, and Selenium WebDriver interface, let’s have a look at an example with all these things in action. Before you start the implementation, please ensure that you download Gecko driver for Firefox and ChromeDriver for Chrome from here and here respectively. In order to avoid mentioning the path/location where the drivers have been downloaded, make sure that you place these respective drivers at the location where the corresponding browsers are present. In the snapshot below, you can see that we have copied Geckodriver.exe in the location where Firefox browser (firefox.exe) is present.
Now that you have the setup ready, let’s get started with the implementation. Import all the necessary modules in the beginning so that you avoid errors. In our case, the modules imported are selenium, pytest, pytest-html. Two fixture functions – driver_init() and chrome_driver_init() have the “class” scope. As seen in the fixture function driver_init() , an instance of Firefox is created using GeckoDriver, whereas in chrome_driver_init() , an instance of Chrome browser is created using ChromeDriver. yield contains the implementation of teardown; code inside yield is responsible for doing the cleanup activity. A class is used to group test cases, in this case, there are two important classes, Test_URL() and Test_URL_Chrome() . The implemented classes are making use of the fixtures that were implemented using mark.usefixtures [ @pytest.mark.usefixtures(“driver_init”) ]. The test case performs a simple test of invoking the respective browser (Firefox/Chrome) and opening the supplied URL i.e. https://www.lambdatest.com/ Filename – test_selenium_webdriver-2.py
# Import the 'modules' that are required for execution
import pytest import pytest_html from selenium
import webdriver from selenium.webdriver.chrome.options
import Options from selenium.webdriver.common.keys
import Keys from time import sleep
#Fixture for Firefox @pytest.fixture(scope="class")
def driver_init(request): ff_driver = webdriver.Firefox()
request.cls.driver = ff_driver yield ff_driver.close()
#Fixture for Chrome @pytest.fixture(scope="class")
def chrome_driver_init(request):
chrome_driver = webdriver.Chrome()
request.cls.driver = chrome_driver yield chrome_driver.close()
@pytest.mark.usefixtures("driver_init") class BasicTest: pass class Test_URL(BasicTest):
def test_open_url(self):
self.driver.get("https://www.lambdatest.com/")
print(self.driver.title)
sleep(5)
@pytest.mark.usefixtures("chrome_driver_init")
class Basic_Chrome_Test:
pass class Test_URL_Chrome(Basic_Chrome_Test):
def test_open_url(self):
self.driver.get("https://www.lambdatest.com/")
print(self.driver.title)
sleep(5)
Since we require the test output in an HTML file, we make us of –html= argumentt while executing the test code. The complete command to execute test automation using pytest with Selenium WebDriver:
| 1 |
py.test.exe --capture=no --verbose --html=pytest_selenium_test_report.html test_selenium_webdriver-1.py
|
Below is the execution output, testcase test_open_url() is executed for the class Test_URL and Test_URL_Chrome() . The test report is pytest_selenium_test_report.html [Image – PyTest-Selenium-Output-1.png]. Here is a test report for further clarity.
As seen in the above implementation, the only difference between fixture function for Firefox and Chrome browser is the setting up of the respective browser. The majority of the implementation is same for both the browsers, so it becomes important to optimize the code by avoiding repetition of code. This is possible by making use of parameterized fixtures. As seen in the implementation [Filename – test_selenium_webdriver-1.py], the major change is addition of parameters to fixtures, as in @pytest.fixture(params=[“chrome”, “firefox”],scope=”class”) . Depending on the browser in use, the corresponding WebDriver is used to invoke the browser.
# Import the 'modules' that are required for execution
import pytest import pytest_html from selenium
import webdriver from selenium.webdriver.chrome.options
import Options from selenium.webdriver.common.keys
import Keys from time import sleep
#Fixture for Firefox @pytest.fixture(params=["chrome", "firefox"],scope="class")
def driver_init(request):
if request.param == "chrome":
web_driver = webdriver.Chrome()
if request.param == "firefox":
web_driver = webdriver.Firefox()
request.cls.driver = web_driver yield
web_driver.close()
@pytest.mark.usefixtures("driver_init") class BasicTest:
pass class Test_URL(BasicTest):
def test_open_url(self):
self.driver.get("https://www.lambdatest.com/")
print(self.driver.title)
sleep(5)
In our case, we are using the Chrome and Firefox browsers and the test case Test_URL() would be executed for each browser separately. As seen in the output, the test case is invoked once with parameters as “firefox” and “chrome.”
Cross-Browser Testing With Pytest, Selenium and Lambdatest
There is always a limitation on the amount of testing that you can perform on your local machine or test machines since thorough testing has to be performed on different kinds of devices, operating systems, and browsers. Setting up a local test environment is not a scalable and economical option. This is where your test team can utilize the power of Lambdatest’s cross-browser testing on the cloud capabilities.
You can perform manual as well as automated cross-browser testing of your web application or website on different browsers (even old versions) and devices. You can also perform real-time testing by using their Tunnel feature which lets you use their test infrastructure from the terminal. LambdaTest Selenium Automation Grid enables you to perform end-to-end automation tests on a secure, reliable, and scalable Selenium infrastructure. You can utilize the LambdaTest Selenium Grid to not only increase the overall code-coverage (via testing), but to also decrease the overall time required to execute your automation scripts written in Python.
Conclusion
Test automation using Pytest with Selenium WebDriver is a very favourable option as a framework that has good features with which test engineers can come up with implementation that is easy to implement and which is scalable. It can be used for writing test cases for simple scenarios as well as highly complex scenarios. A developer who is well-versed with the Python, unittest/other test frameworks based on Python would find pytest easy to learn. With pytest leverages concepts like dependency injection, there is less cost involved in the maintainability of the source code.
Since the number of devices are increasing with each passing day, it becomes highly impractical to manually test your code against different devices, operating systems, and browsers; this is where testers/developers can utilize Lambdatest’s cross-browser testing tool, which allows you to perform test automation using pytest with Selenium WebDriver effortlessly.
#selenium #python #testing #java
