Testing and Deployment

Test Coverage

---

Test coverage refers to the proportion of execution paths covered by the tests, measured, for example, by the number of lines of code covered. Assessing test coverage consists of two steps: (1) running tests and extracting test coverage information, and (2) analyzing the test coverage information. In this part, we will look at how to extract test coverage information from Deno tests and how to analyze the test coverage information.

Extracting test coverage information

Let’s look at extracting test coverage using the example from the previous part. In the example, the app.js was as follows.

import { Hono } from "jsr:@hono/hono@4.6.5";

const app = new Hono();

app.get("/", (c) => c.json({ message: "Hello World!" }));

app.post("/", async (c) => {
  const body = await c.req.json();
  body.message = "Hello World!";
  return c.json(body);
});

export default app;

And the app_test.js was as follows.

import { assertEquals } from "jsr:@std/assert@1.0.8";
import app from "../src/app.js";

Deno.test("GET / returns { message: 'Hello World!' }", async () => {
  let request = new Request("http://localhost/");
  let res = await app.request(request);
  let body = await res.json();
  assertEquals(body, { message: "Hello World!" });
});

Deno.test("POST / returns JSON object with additional message property", async () => {
  let request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Hello!" }),
  });

  let res = await app.request(request);
  let body = await res.json();
  assertEquals(body, { message: "Hello World!", data: "Hello!" });
});

Test coverage information is extracted from tests by using the --coverage flag in the test command. The --coverage flag is given a folder as a parameter, which is used as the location for the test coverage data. In addition, the flag --clean is used, as it removes possible earlier generated test coverage data.

When we run the command deno test --clean --coverage=cov, the output is similar to the earlier example.

running 2 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (3ms)
POST / returns JSON object with additional message property ... ok (0ms)

ok | 2 passed | 0 failed (10ms)

Now, however, the root folder of the project contains a folder called cov.

Analyzing test coverage information

The extracted coverage information can be analyzed using the command deno coverage, which is followed by the folder that contains the coverage files. When we run the command deno coverage cov, i.e. asking for a code coverage report based on the files in the folder cov, we see the following output.

-----------------------------
File    | Branch % | Line % |
-----------------------------
 app.js |    100.0 |  100.0 |
-----------------------------
 All files |    100.0 |  100.0 |
-----------------------------

The output above states that all of the lines in the file app.js were covered by the tests.

If there are lines that the tests do not cover, the output would indicate this. To demonstrate this, modify app.js as follows.

import { Hono } from "jsr:@hono/hono@4.6.5";

const app = new Hono();

app.get("/", (c) => c.json({ message: "Hello World!" }));

app.post("/", async (c) => {
  const body = await c.req.json();

  if (body.data === "Oops!") {
    body.message = "Hello Oops!";
  } else {
    body.message = "Hello World!";
  }

  return c.json(body);
});

export default app;

Now, when you run the tests and and extract the test coverage information, we observe that while the tests pass, the test coverage is no longer at 100%.

deno test --clean --coverage=cov
running 2 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (5ms)
POST / returns JSON object with additional message property ... ok (0ms)

ok | 2 passed | 0 failed (24ms)

deno coverage cov
-----------------------------
File    | Branch % | Line % |
-----------------------------
 app.js |      0.0 |   85.0 |
-----------------------------
 All files |      0.0 |   85.0 |
-----------------------------

Test coverage caveats

When assessing test coverage, Deno checks which paths in code are executed when running tests and tries to look for paths that are not executed. High test coverage does not imply that the functionality in those paths is be appropriately tested, but only that those paths were executed during testing.

As an example, the following tests do not have any assertions at all, meaning they only check if the application avoids crashing, without validating the functionality.

import { assertEquals } from "jsr:@std/assert@1.0.8";
import app from "../src/app.js";

Deno.test("GET / returns { message: 'Hello World!' }", async () => {
  let request = new Request("http://localhost/");
  await app.request(request);
});

Deno.test("POST / returns JSON object with additional message property", async () => {
  let request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Hello!" }),
  });

  await app.request(request);
});

Deno.test('POST / with { data: "Oops!" } returns JSON object with additional message property', async () => {
  let request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Oops!" }),
  });

  await app.request(request);
});

When we extract the test coverage for the above tests, the test coverage is at 100%.

-----------------------------
File    | Branch % | Line % |
-----------------------------
 app.js |    100.0 |  100.0 |
-----------------------------
 All files |    100.0 |  100.0 |
-----------------------------

The above example illustrates the need for responsibility when writing tests. When writing tests, it is your responsibility to ensure that they evaluate the functionality of the program — aiming for just coverage can be misleading.

Loading Exercise...