pyviztest

A Visual Test Automation Library in Python. Works with Playwright and Selenium. Integrable with Allure Report.

Why Project pyviztest?

Forewords

In Python there was no robust and integrable Visual Test Automation library/module. Thankfully, Kumar Aaditya created a pytest fixture for Playwright using the power of pixelmatch library for Visual Testing, and Symon Storozhenko enhanced it to create pytest-playwright-visual. I really loved both, so I tried to realign the same concept by developing a library and added a few more functionality to make the use of it more flexible, and also compatible with Playwright and Selenium. All thanks go to both of them for being the harbinger of the visual testing libraries in python. I merely followed the path and improvized.

Features

So, previously created pytest-playwright-visual had the power to be summoned anywhere inside the tests which had it as a parameter, and to

1. • [x] Create Golden Snapshot (The reference image of the UI which needs to be compared with during test execution) of the AUT(Applicayion under test).
2. • [x] Update existing Golden Snapshot.
3. • [x] Take snapshot in runtime and compare it with the Golden Snapshot. If both do not match, it creates an image where the difference is highlighted.

In addition to these features, pyviztest is capable of doing a few more things!

4. • [x] Supports Windows, Linux and MacOS systems.
5. • [x] Supports Playwright and Selenium. You just need to provide the Page and WebDriver instances.
6. • [x] Supports custom snapshot directories.
7. • [x] Supports option to Create & Update snapshots(particular and all) without failing the tests.
8. • [x] Supports option to add multiple snapshot validation in a single test case, and if one of them fails it will still validate others.
9. • [x] Supports integration with Allure Report.
10. • [x] Supports auto-naming of the snapshots with respect to the test suite and test case name.
11. • [x] Provides api to return Golden, Actual and Difference Snapshots in bytes, to easily integrate with any custom report.
12. • [x] Supports comparing images with different in sizes without failing the test abruptly.
13. • [x] Supports comparing snapshots of multiple web elements together or the full page.

Upcoming Next:

1. • [ ] Support for Appium.
2. • [ ] Support for element specific identification.
3. • [ ] A fixture along with the usual library.

How To Use It:

Installing

Simple and easy to install because it's just another python library! Install using below pip command:

    $ pip install pyviztest

Configuration

1. Required: Import VisualTest from pyviztest.viztest

    from pyviztest.viztest import VisualTest

2. Required: The object of VisualTest is the driving factor of the visual tests.

    visualtest = VisualTest(driverpage=driver)

Note You may directly create an object of VisualTest class for each test separately and proceed with that object, or you can create a pytest fixture for dependency injection to initialize the VisualTest instance from inside that class. Using a separate class/fixture to implement dependency injection is always recommended. The constructor of VisualTest class can take 5 arguments i.e. snapshot_path, [MANDATORY]driverpage: WebDriver/Page, updatesnapshot, savefailuresnapondisk & allurereport. The usage of them are elaborated in below code snippet comments.

    class VisualTest(
        snapshot_path: str = '', #The path where all the snapshots will reside. If not provided, it will consider the parent directory of the test class as the snapshot path by default.
        driverpage: Any = '', #You MUST to provide either the driver object (if you are using Selenium for your tests) or page fixture/object (if you are using Playwright)
        updatesnapshot: bool = False, # If True, it will update all the existing golden snapshots or to create new golden snapshots for the current session of Visualtest class.
        savefailuresnapondisk: bool = True, #If True, all the failure snapshots will be saved inside the failure directories under snapshot_path as image files in PNG format.
        allurereport: bool = False #If True, allure report will be generated with Golden snapshots and failure snapshots.
    )

3. Required: Invoke the setpaths() function for each of the tests.

    visualtest.setpaths()

Note The setpaths function can take 2 optional arguments i.e. updatesnapshot and numberofclassesaftertestclass. The usage of them are elaborated in below code snippet comments.

    (method) def setpaths(
        updatesnapshot: bool = False, # Same as the updatesnapshot mentioned above. It provides the flexibility to update snapshots for a particular test case. If not given, it would take the default value during creation of VisualTest class object.
        numberofclassesaftertestclass: int = 0  # This parameter defines the number of classes between the test methods and where the visualtest_web() method is invoked.
    ) -> None

Warning numberofclassesaftertestclass parameter helps you to fetch the testcase name in order to generate the snapshot's name by default if you are not providing any specific name. By default the value of it is 0, that means it assumes that the visualtest_web() method is being invoked from inside the test mthod directly without any further abstraction. If you add N number of abstractions, then you should provide the value of the parameters as N. See point no. 4 for more info.

    visualtest.setpaths(numberofclassesaftertestclass=N)

4. Now that the configuration is done, you have to invoke visualtest_web() method wherever the webpage snapshots need to be validated.

    assert vt.visualtest_web(stepname="validateloginpage")

Note This method takes a minimum of 7 arguments i.e. stepname, threshold, fail_fast, updatesnapshot, fullpage, snapshot_of_locators and exclude_locators. The usage of them are elaborated in below code snippet comments.

    (method) def visualtest_web(
        *,
        stepname: str = '', # Optional, but it is recommended to provide the value to distinguish snapshots
        threshold: float = 0.1, # Optional, sets the threshold for the comparing the snapshots 0 to 1
        fail_fast: bool = False, # Optional, if you'd like to fail the comparison even if 1 pixel doesn't match, make it True
        updatesnapshot: bool = False, # Same as the updatesnapshot mentioned above. It provides the flexibility to update snapshots for a particular test step. If not given, it would take the default value during creation of VisualTest class object or setpaths().
        fullpage: bool = True, # Optional, to take snapshot of the full page or not
        snapshot_of_locators: list = [], # Optional, you may send set of locators or webelements to take snapshots of only those elements instead of a full page snapshot
        exclude_locators: list = [] # Optional, you may send set of locators to mask them in the snapshots, so during the comparison they will be excluded.
    ) -> bool

Warning fullpage and exclude_locators are only applicable for Playwright tests. They do not work for Selenium tests. Note This visualtest_web() method returns a boolean value as per below table:

| updatesnapshot(VisualTest()) | updatesnapshot(setpaths()) | updatesnapshot(visualtest_web()) | Snapshots Match? | visualtest_web() | | :----------: | :----------: | :----------: | :----------: | :----------: | | True | X | X | X | True | | False | True | X | X | True | | False | False | True | X | True | | False | False | False | True | True | | False | False | False | False | False |

Warning

Do not Change the value of numberofclassesaftertestclass if you don't understand the below concept well.

As mentioned above already, numberofclassesaftertestclass parameter helps you to fetch the testcase name in order to generate the snapshot's name by default if you are not providing any specific name. Let's look at an example:

       Project|
              |---src|
              |       |--testsuitename|
              |                       |---test_playwright.py|
              |                       |                     |---test_1()
              |                       |                     |---test_2()
              |                       |
              |                       |---test_selenium.py|
              |                                           |---test_1()
              |                                           |---test_2()
              |---VisualTestResults                     

In case of the folder structure depicted above, if the OS is Linux, `sna