Testing and Deployment

End-to-End Testing

---

End-to-end (E2E) testing is used to assess that the application works as a whole, meaning that the application has all the components it would when deployed.

In E2E tests, we simulate user actions in the application to reach specific goals, mimicking the interactions that a user would perform. For web applications, this involves opening up a page in a browser, navigating in the page, perhaps filling in a form, submitting the form, and checking that the data is shown as expected.

As systems often have multiple use cases, multiple end-to-end tests are used.

At this point, you should have the walking skeleton with playwright at your disposal, and the contents of the e2e-tests folder should be similar to the following.

.
├── tests
│   └── example.spec.js
├── Dockerfile
├── package.json
└── playwright.config.js

If you do not have the walking skeleton, or you have not worked through the chapter on end-to-end testing with Playwright, set the walking skeleton up by following the instructions on setting up a walking skeleton.

Before proceeding, you may clear the contents of the folder tests in the e2e-tests folder, after which there are no end-to-end tests in the project.

First Playwright tests

Let’s start by creating a test that opens up the Aalto University page at https://www.aalto.fi/en and checks that the page title is “Aalto University”. Create a file aalto.spec.js in the tests folder (under e2e-tests) and place the following content in the file.

import { expect, test } from "@playwright/test";

test("Aalto site has title Aalto University.", async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  expect(await page.title()).toBe("Aalto University");
});

Save the file and run the command docker compose run --rm --entrypoint=npx e2e-tests playwright test in the root folder of the project. The command runs the tests in the e2e-tests folder using Playwright. When the tests are run, the output is similar to the following.

Running 1 test using 1 worker

  ✓  1 [chromium] › aalto.spec.js:3:1 › Aalto site has title Aalto University. (2.2s)

The test passes, indicating that the page title is “Aalto University”.

Next, change the content of aalto.spec.js file to match the following, and run the tests again.

import { expect, test } from "@playwright/test";

test("Aalto site has title Aalto University.", async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  expect(await page.title()).toBe("Aalto University");
});

test("Clicking Apply to Aalto and verifying page title", async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  await page.getByRole("button", { name: "Allow all" }).click();
  await page.getByRole("link", { name: "Apply to Aalto" }).click();
  expect(await page.title()).toBe("Study at Aalto | Aalto University");
});

When the tests are run, we see that now both tests pass.

  ✓  1 [chromium] › aalto.spec.js:3:1 › Aalto site has title Aalto University. (1.8s)
  ✓  2 [chromium] › aalto.spec.js:8:1 › Clicking Apply to Aalto and verifying page title (3.2s)

Playwright test structure

Playwright test files are placed in the e2e-tests/tests folder or its subfolders. Each test file has a suffix .spec.js that indicates that it contains Playwright tests. Each test file begins with the following line (or a variation).

import { expect, test } from "@playwright/test";

This line imports the expect and test functions from the @playwright/test package. The test function is used to define a test, and the expect function is used to define assertions in the test.

The test function is typically given two parameters. The first parameter is a string that is a brief textual description of the purpose of the test, and the second parameter is an asynchronous function that contains the test code. As an example, the following test has “Aalto site has title Aalto University.” as the description of the test, while the function navigates to the Aalto University page and checks that the title is as expected.

test("Aalto site has title Aalto University.", async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  expect(await page.title()).toBe("Aalto University");
});

When Playwright tests are run, they are run in a browser, defined in the Playwright configuration file playwright.config.js.

In our case, when setting up the walking skeleton, we set the browser as Chromium, which is downloaded into the e2e-tests Docker container. Although we use Chromium, it is possible to run the tests in multiple browsers.

Setting up tests

In the above example, where we navigated to the Aalto University page, interaction with the page beyond checking the title required dealing with a cookie consent dialog.

If the web application has functionality that needs to be taken care of before proceeding with tests, we can extract the functionality to a beforeEach hook that is run before each test. In our case, the hook would contain navigating to the page and clicking the “Allow all” button, which would look as follows.

test.beforeEach(async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  await page.getByRole("button", { name: "Allow all" }).click();
});

Now, we could modify the tests so that they would not have to contain the code for navigating to Aalto main page or clicking the “Allow all” button. As a whole, the aalto.spec.js test file would now look as follows.

import { expect, test } from "@playwright/test";

test.beforeEach(async ({ page }) => {
  await page.goto("https://www.aalto.fi/en");
  await page.getByRole("button", { name: "Allow all" }).click();
});

test("Aalto site has title Aalto University.", async ({ page }) => {
  expect(await page.title()).toBe("Aalto University");
});

test("Clicking Apply to Aalto and verifying page title", async ({ page }) => {
  await page.getByRole("link", { name: "Apply to Aalto" }).click();
  expect(await page.title()).toBe("Study at Aalto | Aalto University");
});

When we run the tests, both tests continue to pass.

  ✓  1 [chromium] › aalto.spec.js:8:5 › Aalto site has title Aalto University. (1.4s)
  ✓  2 [chromium] › aalto.spec.js:12:5 › Clicking Apply to Aalto and verifying page title (1.9s)

The beforeEach hook is especially useful for testing applications that have a lot of setup that needs to be done before going into testing the key functionality. Such setup could include registering and logging in, dealing with dialogs that pop up, or setting up the application state in a specific way.

Interaction and locators

Interaction with the web application is done using the page object. The page object provides the functionality that is needed to interact with a browser tab and the content that is shown in the tab. The functionality of the page object include navigating with the goto method, and a range of methods used to locate elements on the page, such as getByRole, getByLabel, getByText, and a more generic locator method.

The methods used to locate elements on the page return a locator object that can be further used to interact with the elements. The locator provides the same locators as the page object, such as getByLabel, a filter method for narrowing down on the elements, and methods to interact with the elements, such as fill and click.

As an example, we could open up the chat at https://dad-etc-websocket-example.deno.dev/, find the input field of type text, type in content to the input field, and then send the message.

await page.goto("https://dad-etc-websocket-example.deno.dev/");
const randomMessage = `Hello ${Math.floor(10000 + Math.random() * 90000)}`;
await page.locator("input[type=text]").fill(randomMessage);
await page.getByRole("button", { name: "Send" }).click();

If the input field would have a label, we could use the getByLabel method to find the input field. Unfortunately, this is not the case in the chat application.

Then, to verify whether the message was sent, we could look for the message in the chat. The message could be found by looking for a listitem that has the text of the message. Here, first finding the listitem elements by their role, then filtering them based on the text that they have, and finally checking that a relevant element is found is done as follows.

await expect(page.getByRole("listitem").filter({ hasText: randomMessage }))
    .toBeVisible();

Put together, the test for sending a message to the chat would look as follows. Save the content below to a file called dad.spec.js and place it to the tests folder.

import { expect, test } from "@playwright/test";

test("Sending a message on chat adds the message to the list of messages.", async ({ page }) => {
  await page.goto("https://dad-etc-websocket-example.deno.dev/");
  const randomMessage = `Hello ${Math.floor(10000 + Math.random() * 90000)}`;
  await page.locator("input[type=text]").fill(randomMessage);
  await page.getByRole("button", { name: "Send" }).click();
  await expect(page.getByRole("listitem").filter({ hasText: randomMessage }))
    .toBeVisible();
});

We can run individual test files by adding the name of the test file to the end of the command used for running tests. As an example, to run the tests in dad.spec.js, we would run the following command.

docker compose run --rm --entrypoint=npx e2e-tests playwright test tests/dad.spec.js

When we run the test with the above command, it fails.

  ✘  1 [chromium] › dad.spec.js:7:5 › Sending a message on chat adds the message to the list of messages (5.6s)

A note on hydration

Earlier on in the course, we discussed Rendering and Hydration, phrasing hydration as the process of transforming HTML into an interactive web page by attaching event listeners and reactivating JavaScript components associated with the content.

When a browser loads a client-side application, it first loads the HTML, after which the page is hydrated. At the same time, our tests are run in the browser as fast as they can, and they do not e.g. wait for hydration to happen. This means that, as the input element and the button are already present on the site before dynamic functionality is attached to them, filling the input field and pressing the button does not have the desired effect — the text is not sent to the chat server.

To fix this, we want to wait for the hydration before running the tests. An simple workaround is to add a wait to the tests. The page object has a method waitForTimeout that can be used to wait for a specific amount of time.

As an example, we could wait for a second before filling in the input field and pressing the button, changing the test to match the following.

import { expect, test } from "@playwright/test";

test("Sending a message on chat adds the message to the list of messages.", async ({ page }) => {
  await page.goto("https://dad-etc-websocket-example.deno.dev/");
  await page.waitForTimeout(1000);

  const randomMessage = `Hello ${Math.floor(10000 + Math.random() * 90000)}`;
  await page.locator("input[type=text]").fill(randomMessage);
  await page.getByRole("button", { name: "Send" }).click();
  await expect(page.getByRole("listitem").filter({ hasText: randomMessage }))
    .toBeVisible();
});

Now, the test passes.

✓  1 [chromium] › dad.spec.js:7:5 › Sending a message on chat adds the message to the list of messages (2.0s)

Although we used the waitForTimeout method to wait for a second, waiting for a fixed amount of time is not considered a good practice. A better approach would be to programmatically add an element to the application, and then in the tests, wait for the element to appear on the page. As we do not have control over the chat application, adding an element is not feasible.

Loading Exercise...

Waiting for hydration in Svelte

If we have control over the application, such as when building the application, adding information about hydration to Playwright tests is straightforward. We can modify the +layout.svelte file to add a class to the body of the application when the page is loaded. This would be done, for example, as follows.

<script>
  let { children } = $props();
  $effect(() => {
    document.body.classList.add("e2e-ready");
  });
</script>

<main>
  {@render children()}
</main>

With the above, the class “e2e-ready” is added to the class attribute of the body element of the application when the page is loaded. If we would have an application that has the above layout, we could write tests that wait for the class to be added to the body element before proceeding with the tests.

The following example shows how to do this with the locator — the locator looks for the class e2e-ready, and uses the method waitFor to wait for it becoming available. Then, the test proceeds to verify that the page has a heading with exactly the test “Todos” (and not, e.g., “Existing todos”).

import { expect, test } from "@playwright/test";

test("The page has a heading Todos.", async ({ page }) => {
  await page.goto("/");
  await page.locator(".e2e-ready").waitFor();
  await expect(page.getByRole("heading", { name: "Todos", exact: true }))
    .toBeVisible();
});

The above also shows how we would test our own applications. Due to how we setup the walking skeleton, our local application that runs at http://localhost:5173 application is available as the base URL /. When we navigate to the base URL, we are taken to the application, and we can then wait for the class e2e-ready to be added to the body element before proceeding with the tests.

Loading Exercise...

Playwright Test for VSCode

As an extra hint for tooling, Playwright has a plugin for VSCode that makes it easier to write tests. To install the plugin, look for “Playwright Test for VSCode” in VSCode Extensions.

Fig 1. Playwright Test for VSCode

Once the plugin is installed (as shown in Figure 1), you can install Playwright on your local computer by running the command “Install Playwright” in VSCode Command Palette (View -> Command Palette). When installing, select the browser you wish to use for testing, and select JavaScript as the language.

Installing the plugin requires Node.js to be installed on your computer. If Node.js is not installed, you can install it from nodejs.org.

Once the plugin is installed, and Playwright and a relevant browser installed, you can click on the “Testing” icon on the left hand side of VSCode to open up test explorer. The test explorer shows the tests that are available in the project, and allows running the tests, as shown in Figure 2.

Fig 2. Test Explorer

The plugin has two features that are especially useful. First, it is possible to view the browser as the tests are being run, which makes it easier to understand what is happening. To do this, select the checkbox “Show browser” in the left lower corner of the test explorer. When the “Show browser” checkbox is selected, a browser window will open up when the tests are run.

Second, the plugin has a tool for recording tests. Clicking the button “Record new” in the test explorer opens up a browser window, and you can interact with the browser as you would normally do. The actions that you take in the browser are recorded as Playwright test interactions, which you can then use for your own tests.

The automatically recorded tests are not always perfect, or correct, and you should always review the recorded tests before using them. In addition, you also need often add the expectations to the tests.