Testing with Solara

Testing Application Logic

We recommend using pytest to test the application logic of your Solara components. To get inspiration for writing tests that cover component logic and their interactions with existing components, refer to the tests in the Solara repository.

Testing with a Browser

If you have custom components that depend on a connected browser because it is using JavaScript, we recommend using the Solara pytest plugin, which is installed by default when you install Solara. The plugin provides a fixture called solara_test that you can use to test your components. Here's an example:

import ipywidgets as widgets
import playwright.sync_api
from IPython.display import display

def test_widget_button_solara(solara_test, page_session: playwright.sync_api.Page):
    # this all runs in process, which only works with solara
    # also, this test is only with pure ipywidgets
    button = widgets.Button(description="Click Me!")

    def change_description(obj):
        button.description = "Tested event"

    button.on_click(change_description)
    display(button)
    button_sel = page_session.locator("text=Click Me!")
    button_sel.wait_for()
    button_sel.click()
    page_session.locator("text=Tested event").wait_for()

Run this test with pytest:

pytest tests/ui/test_widget_button.py --headed # remove --headed to run headless

This require playwright to be installed:

$ pip install playwright pytest-playwright
# $ pip install "solara[pytest]"  # if you haven't installed solara already
$ playwright install chromium

In this example, use the standard IPython display call to add your widget to the page.

Testing in the main Jupyter Environments

In case you want to test your component in the main Jupyter environments (e.g., Jupyter Notebook, Jupyter Lab, Voila, and Solara) to ensure it renders correctly, use the ipywidgets_runner fixture to run code snippets. Here's an example:

import ipywidgets as widgets
import playwright.sync_api
from IPython.display import display


def test_solara_button_all(ipywidgets_runner, page_session: playwright.sync_api.Page, assert_solara_snapshot):
    # this function (or rather its lines) will be executed in the kernel
    # voila, lab, classic notebook and solara will all execute it
    def kernel_code():
        import solara

        @solara.component
        def Button():
            text, set_text = solara.use_state("Click Me!")

            def on_click():
                set_text("Tested event")

            solara.Button(text, on_click=on_click)

        display(Button())

    ipywidgets_runner(kernel_code)
    button_sel = page_session.locator("button >> text=Click Me!")
    assert_solara_snapshot(button_sel.screenshot())
    button_sel.wait_for()
    button_sel.click()
    page_session.locator("button >> text=Tested event").wait_for()
    page_session.wait_for_timeout(1000)

Note that the function in the code will be executed in a different process, which will make it harder to debug and slower to run.

Limiting the Jupyter Environments

To limit the ipywidgets_runner fixture to only run in a specific environment, use the SOLARA_TEST_RUNNERS environment variable:

• SOLARA_TEST_RUNNERS=solara pytest tests/ui
• SOLARA_TEST_RUNNERS=voila pytest tests/ui
• SOLARA_TEST_RUNNERS=jupyter_lab pytest tests/ui
• SOLARA_TEST_RUNNERS=jupyter_notebook pytest tests/ui
• SOLARA_TEST_RUNNERS=solara,voila pytest tests/ui

Organizing Tests and Managing Snapshots

We recommend organizing your visual tests in a separate directory, such as tests/ui. This allows you to run fast tests (test/unit) separately from slow tests (test/ui). Use the solara_snapshots_directory fixture to change the default directory for storing snapshots, which is tests/ui/snapshots by default.

$ pytest tests/unit  # run fast test
$ pytest tests/ui    # run slow test
$ pytest tests       # run all tests

To compare a captured image from a part of your page with the reference image, use the assert_solara_snapshot fixture. For example, assert_solara_snapshot(button_sel.screenshot()) will take a screenshot of the button and compare it to the reference image. If the images are different, the test will fail.

For local development, you can use the --solara-update-snapshots flag to update the reference images. This will overwrite the existing reference images with the new ones generated during the test run. However, you should carefully review the changes before committing them to your repository to ensure the updates are accurate and expected.

Continuous Integration Recommendations

When a test fails, the output will be placed in a directory structure similar to what would be put in the solara_snapshots_directory directory but under the test-results directory in the root of your project (unless changed by passing --output=someotherdirectory to pytest).

In CI, we recommend downloading this directory using, for example, GitHub Actions:

- name: Download test results
  uses: actions/download-artifact@v2
  with:
    name: myproject-test-results
    path: test-results

After inspecting and approving the screenshots, you can copy them to the solara_snapshots_directory directory and commit them to your repository. This way, you ensure that the reference images are up-to-date and accurate for future tests.

Note about the Playwright

Visual testing with solara is based on Playwright for Python, which provides a page fixture. However, this fixture will make a new page for each test, which is not what we want. Therefore, we provide a page_session fixture that will reuse the same page for all tests. This is important because it will make the tests faster.

By following these recommendations and guidelines, you can efficiently test your Solara applications and ensure a smooth developer experience.