Testing and Deployment

Test Coverage

Learning Objectives

  • You know of the concept of test coverage.
  • You know how to extract test coverage of Deno tests.

Test coverage refers to the proportion of execution paths covered by the tests, measured, for example, by the number of lines of code covered. Assessing test coverage consists of two steps: (1) running tests and extracting test coverage information, and (2) analyzing the test coverage information. In this part, we will look at how to extract test coverage information from Deno tests and how to analyze the test coverage information.

Extracting test coverage information

Let’s look at extracting test coverage using the example from the previous part. In the example, the app.js was as follows.

import { Hono } from "@hono/hono";
import { helloService } from "./services/helloService.js";

const app = new Hono();

app.get("/", (c) => c.json({ message: "Hello World!" }));

app.get("/hello", (c) => c.json({ message: helloService.getHello() }));

app.post("/", async (c) => {
  const body = await c.req.json();
  body.message = "Hello World!";
  return c.json(body);
});

export default app;

And the app_test.js was as follows.

import { assertEquals } from "@std/assert";
import { stub } from "@std/testing/mock";
import app from "@src/app.js";
import { helloService } from "@src/services/helloService.js";

Deno.test("GET / returns { message: 'Hello World!' }", async () => {
  const request = new Request("http://localhost/");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!" });
});

Deno.test("POST / returns JSON object with additional message property", async () => {
  const request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Hello!" }),
  });

  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!", data: "Hello!" });
});

Deno.test("GET /hello returns message from helloService", async () => {
  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello service world!" });
});

// new test
Deno.test("GET /hello returns stubbed hello", async () => {
  using getHelloStub = stub(helloService, "getHello", () => "Stubbed Hello!");

  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Stubbed Hello!" });
});

Test coverage information is extracted from tests by using the --coverage flag in the test command. When we run the command deno test --coverage, the initial output is similar to the earlier example. The output also has information about test coverage and about files with coverage reports.

$ deno test --coverage
running 4 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (9ms)
POST / returns JSON object with additional message property ... ok (1ms)
GET /hello returns message from helloService ... ok (0ms)
GET /hello returns stubbed hello ... ok (1ms)
running 3 tests from ./tests/services/helloService_test.js
Function getHello returns 'Hello service world!' ... ok (0ms)
Message set using setHello returned from getHello ... ok (0ms)
Stubbed getHello ... ok (0ms)

ok | 7 passed | 0 failed (298ms)

| File                     | Branch % | Line % |
| ------------------------ | -------- | ------ |
| app.js                   |    100.0 |  100.0 |
| services/helloService.js |    100.0 |  100.0 |
| All files                |    100.0 |  100.0 |
Lcov coverage report has been generated at file:///path-to-project/coverage/lcov.info
HTML coverage report has been generated at file:///path-to-project/coverage/html/index.html

By default, test coverage reports are placed into a folder called coverage.

Loading Exercise...

Analyzing test coverage information

The coverage information from the tests shows branch and line coverage per tested file.

| File                     | Branch % | Line % |
| ------------------------ | -------- | ------ |
| app.js                   |    100.0 |  100.0 |
| services/helloService.js |    100.0 |  100.0 |
| All files                |    100.0 |  100.0 |

Branch coverage shows to what extent possible outcomes (branches, defined e.g. with if statements) are executed during testing, while line coverage shows to what extent the lines of code of the application are tested. In our case, the coverage for both is 100%, which is good — with some caveats that we’ll discuss soon.

If there are lines or branches that the tests do not cover, the output would indicate this. To demonstrate this, modify app.js as follows.

import { Hono } from "@hono/hono";
import { helloService } from "./services/helloService.js";

const app = new Hono();

app.get("/", (c) => c.json({ message: "Hello World!" }));

app.get("/hello", (c) => c.json({ message: helloService.getHello() }));

app.post("/", async (c) => {
  const body = await c.req.json();

  if (body.data === "Oops!") {
    body.message = "Hello Oops!";
  } else {
    body.message = "Hello World!";
  }

  return c.json(body);
});

export default app;

Now, when you run the tests and and extract the test coverage information, we observe that while the tests pass, the coverage is no longer at 100%.

$ deno test --coverage
running 4 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (8ms)
POST / returns JSON object with additional message property ... ok (1ms)
GET /hello returns message from helloService ... ok (0ms)
GET /hello returns stubbed hello ... ok (0ms)
running 3 tests from ./tests/services/helloService_test.js
Function getHello returns 'Hello service world!' ... ok (1ms)
Message set using setHello returned from getHello ... ok (0ms)
Stubbed getHello ... ok (0ms)

ok | 7 passed | 0 failed (268ms)

| File                     | Branch % | Line % |
| ------------------------ | -------- | ------ |
| app.js                   |      0.0 |   87.0 |
| services/helloService.js |    100.0 |  100.0 |
| All files                |      0.0 |   91.4 |

The HTML files that are generated as a part of calculating test coverage also point out explicitly what has not been covered. The files can be found from a folder called html, which is under the folder coverage.

To improve the test coverage, we need to implement the tests that check the alternative pathway. Let’s add a test to app_test.js that sends “Oops!” as the value for the data property to the server, and checks that the message is “Hello Oops!”.

import { assertEquals } from "@std/assert";
import { stub } from "@std/testing/mock";
import app from "@src/app.js";
import { helloService } from "@src/services/helloService.js";

Deno.test("GET / returns { message: 'Hello World!' }", async () => {
  const request = new Request("http://localhost/");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!" });
});

Deno.test("POST / returns JSON object with additional message property", async () => {
  const request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Hello!" }),
  });

  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!", data: "Hello!" });
});

Deno.test("GET /hello returns message from helloService", async () => {
  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello service world!" });
});

Deno.test("GET /hello returns stubbed hello", async () => {
  using getHelloStub = stub(helloService, "getHello", () => "Stubbed Hello!");

  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Stubbed Hello!" });
});

// new test
Deno.test("POST / with Oops! in data has Hello Oops!", async () => {
  const request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Oops!" }),
  });

  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello Oops!", data: "Oops!" });
});

Now, when we run the tests, we are again at a good coverage.

$ deno test --clean --coverage
running 5 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (25ms)
POST / returns JSON object with additional message property ... ok (3ms)
GET /hello returns message from helloService ... ok (2ms)
GET /hello returns stubbed hello ... ok (7ms)
POST / with Oops! in data has Hello Oops! ... ok (1ms)
running 3 tests from ./tests/services/helloService_test.js
Function getHello returns 'Hello service world!' ... ok (1ms)
Message set using setHello returned from getHello ... ok (0ms)
Stubbed getHello ... ok (0ms)

ok | 8 passed | 0 failed (519ms)

| File                     | Branch % | Line % |
| ------------------------ | -------- | ------ |
| app.js                   |    100.0 |  100.0 |
| services/helloService.js |    100.0 |  100.0 |
| All files                |    100.0 |  100.0 |
Loading Exercise...

Test coverage caveats

When assessing test coverage, Deno checks which paths in code are executed when running tests and tries to look for paths that are not executed. High test coverage does not imply that the functionality in those paths is be appropriately tested, but only that those paths were executed during testing.

As an example, let’s modify the test that we just created, commenting out the assertEquals call.

import { assertEquals } from "@std/assert";
import { stub } from "@std/testing/mock";
import app from "@src/app.js";
import { helloService } from "@src/services/helloService.js";

Deno.test("GET / returns { message: 'Hello World!' }", async () => {
  const request = new Request("http://localhost/");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!" });
});

Deno.test("POST / returns JSON object with additional message property", async () => {
  const request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Hello!" }),
  });

  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello World!", data: "Hello!" });
});

Deno.test("GET /hello returns message from helloService", async () => {
  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Hello service world!" });
});

Deno.test("GET /hello returns stubbed hello", async () => {
  using getHelloStub = stub(helloService, "getHello", () => "Stubbed Hello!");

  const request = new Request("http://localhost/hello");
  const res = await app.request(request);
  const body = await res.json();
  assertEquals(body, { message: "Stubbed Hello!" });
});

// new test
Deno.test("POST / with Oops! in data has Hello Oops!", async () => {
  const request = new Request("http://localhost/", {
    method: "POST",
    body: JSON.stringify({ data: "Oops!" }),
  });

  const res = await app.request(request);
  const body = await res.json();
  // assertEquals(body, { message: "Hello Oops!", data: "Oops!" });
});

Now, when the tests are run, the assertEquals function is not called in the newly added test, and no actual check for correctness is being made.

Despite the change, when we run the tests, the test coverage is still high.

$ deno test --clean --coverage
running 5 tests from ./tests/app_test.js
GET / returns { message: 'Hello World!' } ... ok (9ms)
POST / returns JSON object with additional message property ... ok (1ms)
GET /hello returns message from helloService ... ok (0ms)
GET /hello returns stubbed hello ... ok (0ms)
POST / with Oops! in data has Hello Oops! ... ok (0ms)
running 3 tests from ./tests/services/helloService_test.js
Function getHello returns 'Hello service world!' ... ok (0ms)
Message set using setHello returned from getHello ... ok (0ms)
Stubbed getHello ... ok (0ms)

ok | 8 passed | 0 failed (275ms)

| File                     | Branch % | Line % |
| ------------------------ | -------- | ------ |
| app.js                   |    100.0 |  100.0 |
| services/helloService.js |    100.0 |  100.0 |
| All files                |    100.0 |  100.0 |

The above example illustrates the need for responsibility when writing tests. When writing tests, it is your responsibility to ensure that they evaluate the functionality of the program — aiming for just coverage can be misleading.

Loading Exercise...

Summary

In summary:

  • Test coverage measures the proportion of code execution paths covered by tests, typically measured by lines and branches of code executed during testing.
  • Coverage is extracted by running deno test --coverage, which outputs test coverage information and collects coverage data into a directory.
  • Branch coverage shows what extent of possible code paths (like if statements) are executed, while line coverage shows what extent of code lines are executed.
  • High test coverage doesn’t guarantee quality — it only means code paths were executed, not that they were properly tested with assertions.
Loading Exercise...
← Integration and HTTP Testing
End-to-End Testing →