HTTP Testing

Learning objectives

Knows how to write tests that test the HTTP interface of an oak application.

One of the approaches for testing web applications is testing the HTTP interfaces that the applications provide. Effectively, this means making queries to specific paths using specific methods and verifying that the responses to those queries are as expected. Here, we look into testing HTTP interfaces of an application written using the oak framework. For testing, we use SuperOak, which is a test framework for oak.

SuperOak

Superoak provides a domain specific language (DSL) for writing tests. In each test case, we wrap the oak application with superoak, which results in an object that has specific methods for testing (i.e. the DSL). The example below shows how superoak is used to test whether a GET request to the root path of an application provides the correct response. In this case, we expect that the response is the string 'Hello world!'.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = ({ response }) => {
  response.body = "Hello world";
};

router.get("/", hello);

app.use(router.routes());

Deno.test("GET request to / should return 'Hello world!'", async () => {
  const testClient = await superoak(app);
  await testClient.get("/").expect("Hello world!");
});

We assume that the above code is within a file called app.js. Similar to previously, the tests run executed with Deno's test utilities. In the example below, we run the tests. Note that, when using oak and superoak, we need to allow network access with the --allow-net flag.

deno test --allow-net app.js
running 1 tests
test GET request to / should return 'Hello world!' ... FAILED (29ms)

failures:

GET request to / should return 'Hello world!'
Error: expected 'Hello world!' response body, got 'Hello world'
    at error (test.ts:553:15)
    at Test.#assertBody (test.ts:454:16)
    at Test.#assertFunction (test.ts:533:13)
    at Test.#assert (test.ts:399:35)
    at test.ts:374:23
    at close (close.ts:47:52)
    at test.ts:359:22
    at Test.Request.callback (https://dev.jspm.io/npm:superagent@5.3.1/lib/client.dew.js:670:5)
    at Test.<anonymous> (https://dev.jspm.io/npm:superagent@5.3.1/lib/client.dew.js:490:14)
    at Test.Emitter.emit (https://dev.jspm.io/npm:component-emitter@1.3.0/index.dew.js:148:22)

failures:

        GET request to / should return 'Hello world!'

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out (29ms)

As we can see, the test fails. When we look at the error in more detail, we see what the issue is.

GET request to / should return 'Hello world!'
Error: expected 'Hello world!' response body, got 'Hello world'

When running the test GET request to / should return 'Hello world!', the response body created by the server as a response to the request contained the string Hello world, but the tests expected that the request body would have been Hello world!.

In the example below, the issue has been fixed.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = ({ response }) => {
  response.body = "Hello world!";
};

router.get("/", hello);

app.use(router.routes());

Deno.test("GET request to / should return 'Hello world!'", async () => {
  const testClient = await superoak(app);
  await testClient.get("/").expect("Hello world!");
});

Now, when we run the test again, we observe that all tests pass.

deno test --allow-net app.js
running 1 tests
test GET request to / should return 'Hello world!' ... ok (61ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (62ms)

Note that when we ran the test, we didn't launch the server using app.listen. If we would run the tests on an application that has the app.listen call, the tests would be executed, but the application would remain running -- the test runner itself would not be able to determine when to shut down the application.

Similar to the previous example, in practice, when testing oak applications, the application code and tests are separated from each others. In the example below, we have split the previous app.js file into two files, app.js and app_test.js. The example below shows the file app.js -- at the end of the file, we export the application for testing purposes.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";

const app = new Application();
const router = new Router();

const hello = ({ response }) => {
  response.body = "Hello world!";
};

router.get("/", hello);

app.use(router.routes());

export { app };

The file below shows app_test.js that contains the tests. In the file, we import both the superoak testing library and the application. This is followed by the test definition.

import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";
import { app } from "./app.js";

Deno.test("GET request to / should return 'Hello world!'", async () => {
  const testClient = await superoak(app);
  await testClient.get("/").expect("Hello world!");
});

Running the tests would now be done using either a command that runs the tests from a specific file, as shown below.

deno test --allow-net app_test.js
Check file:///path-to-file/$deno$test.ts
running 1 tests
test GET request to / should return 'Hello world!' ... ok (24ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (24ms)

Or, we could also run the tests without passing the file as a parameter. In this case, as seen previously, Deno would look for files that end with _test.js and run the tests within them.

deno test --allow-net
Check file:///path-to-file/$deno$test.ts
running 1 tests
test GET request to / should return 'Hello world!' ... ok (20ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (20ms)

Multiple expectations

The method expect is used to test whether the response from the server contains expected values. It can be used in multiple ways. If it is given an integer as a parameter, the test verifies that the status code of the response matches the parameter. If it is given a JavaScript object, it verifies that the response contains a JSON object that matches the JavaScript object. If it is given two parameters, then it can be used to check e.g. response headers. In addition, regular expressions can also be used to match the response content.

Furthermore, expect calls can be chained. The following example demonstrates a test that verifies that the status code of the response is 200, the content matches a specific JavaScript object, and that the content-type header contains the text application/json. A corresponding application is also given.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = ({ response }) => {
  response.body = { message: "Hello world!" };
};

router.get("/", hello);

app.use(router.routes());

Deno.test("GET request to / should return 'Hello world!'", async () => {
  const testClient = await superoak(app);
  await testClient.get("/")
    .expect(200)
    .expect("Content-Type", new RegExp("application/json"))
    .expect({ message: "Hello world!" });
});

Running the above with Deno's testing facility provides the following output -- the tests pass.

deno test --allow-net app.js
Check file:///path-to-file/$deno$test.ts
running 1 tests
test GET request to / should return 'Hello world!' ... ok (37ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (37ms)

Note that while method chaining is allowed, one cannot reuse an existing test client. The following example would fail as the same test client object is used twice.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = ({ response }) => {
  response.body = { message: "Hello world!" };
};

router.get("/", hello);

app.use(router.routes());

Deno.test("GET request to / should return 'Hello world!'", async () => {
  const testClient = await superoak(app);
  await testClient.get("/")
    .expect(200)
    .expect("Content-Type", new RegExp("application/json"))
    .expect({ message: "Hello world!" });

  await testClient.get("/")
    .expect(200)
    .expect("Content-Type", new RegExp("application/json"))
    .expect({ message: "Hello world!" });
});

deno test --allow-net app.js
running 1 test from file:///path-to-file
test GET request to / should return 'Hello world!' ... FAILED (25ms)

failures:

GET request to / should return 'Hello world!'
Error: Request has been terminated
Possible causes: the network is offline, Origin is not allowed by Access-Control-Allow-Origin, the page is being unloaded, etc.
    at Test.Request.crossDomainError (https://jspm.dev/npm:superagent@6.1.0!cjs:674:13)
    at XMLHttpRequestSham.xhr.onreadystatechange (https://jspm.dev/npm:superagent@6.1.0!cjs:777:19)
    at XMLHttpRequestSham.xhrReceive (https://deno.land/x/superdeno@4.3.0/src/xhrSham.js:115:29)
    at https://deno.land/x/superdeno@4.3.0/src/xhrSham.js:62:21
    at Object.xhr.onreadystatechange (https://deno.land/x/superdeno@4.3.0/src/xhrSham.js:211:7)
    at XMLHttpRequestSham.xhrSend (https://deno.land/x/superdeno@4.3.0/src/xhrSham.js:335:20)
    at async XMLHttpRequestSham.send (https://deno.land/x/superdeno@4.3.0/src/xhrSham.js:61:7)

failures:

        GET request to / should return 'Hello world!'

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out (148ms)

Sending data to server

Sending data to the server is also possible. In the following example, we post a name attribute with the value 'Jane' to the server. The test then expects that the response from the server is 'Hello Jane!'. When posting data, we use the post method, for which we give a path. Then, this is followed by a method send, for which we define the data that we wish to send to the server. After this, we again look into the response with the expect method.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = async ({ request, response }) => {
  const body = request.body();
  const params = await body.value;

  const name = params.get("name");

  response.body = `Hello ${name}!`;
};

router.get("/", hello);

app.use(router.routes());

Deno.test("POST to / with 'name=Jane' should return 'Hello Jane!'", async () => {
  const testClient = await superoak(app);
  await testClient.post("/")
    .send("name=Jane")
    .expect("Hello Jane!");
});

When we run the test, we notice that something is wrong.

deno test --allow-net app.js
running 1 tests
test POST to / with 'name=Jane' should return 'Hello Jane!' ... FAILED (17ms)

failures:

POST to / with 'name=Jane' should return 'Hello Jane!'
Error: expected 'Hello Jane!' response body, got ''
    at error (test.ts:553:15)
    at Test.#assertBody (test.ts:454:16)
    at Test.#assertFunction (test.ts:533:13)
    at Test.#assert (test.ts:399:35)
    at test.ts:374:23
    at close (close.ts:47:52)
    at test.ts:359:22
    at Test.Request.callback (https://dev.jspm.io/npm:superagent@5.3.1/lib/client.dew.js:670:5)
    at Test.<anonymous> (https://dev.jspm.io/npm:superagent@5.3.1/lib/client.dew.js:488:14)
    at Test.Emitter.emit (https://dev.jspm.io/npm:component-emitter@1.3.0/index.dew.js:148:22)

failures:

	POST to / with 'name=Jane' should return 'Hello Jane!'

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out (17ms)

The server has returned an empty response. What is going on?

We should have tested the application locally before resorting into running the tests. When trying out the application locally, it is easy to see that the POST request to the application does not work -- the application returns 404 as a response to the request. In practice, this means that the route was not found.

Looking into the application in detail, we notice that the route was mapped using a get method. In the following example, we have changed the mapping to post.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";

const app = new Application();
const router = new Router();

const hello = async ({ request, response }) => {
  const body = request.body();
  const params = await body.value;

  const name = params.get("name");

  response.body = `Hello ${name}!`;
};

router.post("/", hello);

app.use(router.routes());

Deno.test("POST to / with 'name=Jane' should return 'Hello Jane!'", async () => {
  const testClient = await superoak(app);
  await testClient.post("/")
    .send("name=Jane")
    .expect("Hello Jane!");
});

Now, when we run the tests, all tests pass.

deno test --allow-net app.js
running 1 tests
test POST to / with 'name=Jane' should return 'Hello Jane!' ... ok (24ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (24ms)

The above example also demonstrates the need for manual testing. If we would solely rely on the outputs of the automated tests, not trying out the application ourselves, figuring out what the issue is could be, at times, quite challenging.

Headers and request content

The following example demonstrates an application that sets a cookie and uses data from cookies to select an appropriate response.

Testing such an application can be done in two steps. First, we make a request, where we set a cookie. This is done using the set method, which is used to set header values. Then, we verify the response, and assign the response to a variable. After this, we study content of the response, and verify that the response headers contain a specific cookie using Deno's testing library.

import { Application, Router } from "https://deno.land/x/oak@v12.6.1/mod.ts";
import { superoak } from "https://deno.land/x/superoak@4.7.0/mod.ts";
import { assertMatch } from "https://deno.land/std@0.222.1/testing/asserts.ts";

const app = new Application();
const router = new Router();

const hello = async ({ cookies, response }) => {
  let name = "";
  if (cookies.get("name")) {
    name = cookies.get("name");
  }

  cookies.set("name", "secret");
  response.body = `Hello ${name}!`;
};

router.get("/", hello);

app.use(router.routes());

Deno.test("GET to / with Cookie 'name=Jane' should return 'Hello Jane!' and set a cookie 'name=secret'", async () => {
  const testClient = await superoak(app);
  const response = await testClient.get("/")
    .set("Cookie", "name=Jane")
    .send()
    .expect("Hello Jane!");

  console.log(response);

  const cookie = response.headers["set-cookie"];
  assertMatch(cookie, new RegExp("name=secret"));
});

When we run the above test, the output is as follows. In particular, as the response object is logged, the output demonstrates the content that the responses have.

deno test --allow-net app.js
running 1 tests
test GET to / with Cookie 'name=Jane' should return 'Hello Jane!' and set a cookie 'name=secret' ... Response {
  req: Test {
    _query: [],
    method: "GET",
    url: "http://127.0.0.1:24099/",
    header: { Cookie: "name=Jane" },
    _header: { cookie: "name=Jane" },
    _callbacks: { "$end": [ [Function] ], "$abort": [ [Function] ] },
    _maxRedirects: 0,
    app: Server { close: [Function], listener: { addr: [Object] } },
    _data: undefined,
    _endCalled: true,
    _callback: [AsyncFunction],
    xhr: XMLHttpRequestSham {
      id: "1",
      origin: "http://127.0.0.1:24099",
      onreadystatechange: [Function],
      readyState: 4,
      responseText: "Hello Jane!",
      responseType: "text",
      response: "Hello Jane!",
      status: 200,
      statusCode: 200,
      statusText: "OK",
      aborted: false,
      options: {
        requestHeaders: [Object],
        method: "GET",
        url: "http://127.0.0.1:24099/",
        username: undefined,
        password: undefined,
        requestBody: null
      },
      controller: AbortController {},
      getAllResponseHeaders: [Function],
      getResponseHeader: [Function]
    },
    _fullfilledPromise: Promise { [Circular] }
  },
  xhr: XMLHttpRequestSham {
    id: "1",
    origin: "http://127.0.0.1:24099",
    onreadystatechange: [Function],
    readyState: 4,
    responseText: "Hello Jane!",
    responseType: "text",
    response: "Hello Jane!",
    status: 200,
    statusCode: 200,
    statusText: "OK",
    aborted: false,
    options: {
      requestHeaders: { cookie: "name=Jane" },
      method: "GET",
      url: "http://127.0.0.1:24099/",
      username: undefined,
      password: undefined,
      requestBody: null
    },
    controller: AbortController {},
    getAllResponseHeaders: [Function],
    getResponseHeader: [Function]
  },
  text: "Hello Jane!",
  statusText: "OK",
  statusCode: 200,
  status: 200,
  statusType: 2,
  info: false,
  ok: true,
  redirect: false,
  clientError: false,
  serverError: false,
  error: false,
  created: false,
  accepted: false,
  noContent: false,
  badRequest: false,
  unauthorized: false,
  notAcceptable: false,
  forbidden: false,
  notFound: false,
  unprocessableEntity: false,
  headers: {
    "content-length": "11",
    "set-cookie": "name=secret; path=/; httponly",
    "content-type": "text/plain; charset=utf-8"
  },
  header: {
    "content-length": "11",
    "set-cookie": "name=secret; path=/; httponly",
    "content-type": "text/plain; charset=utf-8"
  },
  type: "text/plain",
  charset: "utf-8",
  links: {},
  body: null
}
ok (24ms)

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (25ms)

All of the variables in the response can, in practice, be studied and verified.