Measuring Performance

Learning objectives

Knows how to measure the server-side performance of a web application.
Learns to use k6 for measuring the performance of a web application.

There exists a wide variety of tools for measuring the server-side performance of web applications. These include Autocannon, Benny, Deno bench, Gatling, JMeter, k6, wrk, and wrk2.

Here, we briefly look into using k6 to test the performance of the JSON item API discussed in Web Applications with Deno part of the Web Software Development Rehearsal. The application as a whole is as follows.

const items = [];

const handleGetRoot = async (request) => {
  return new Response("Hello world at root!");
};

const handleGetItem = async (request, urlPatternResult) => {
  const id = urlPatternResult.pathname.groups.id;
  // trying to respond with an item at index id
  return Response.json(items[id]);
};

const handleGetItems = async (request) => {
  return Response.json(items);
};

const handlePostItems = async (request) => {
  const item = await request.json();
  items.push(item);
  return new Response("OK", { status: 200 });
};

const urlMapping = [
  {
    method: "GET",
    pattern: new URLPattern({ pathname: "/items/:id" }),
    fn: handleGetItem,
  },
  {
    method: "GET",
    pattern: new URLPattern({ pathname: "/items" }),
    fn: handleGetItems,
  },
  {
    method: "POST",
    pattern: new URLPattern({ pathname: "/items" }),
    fn: handlePostItems,
  },
  {
    method: "GET",
    pattern: new URLPattern({ pathname: "/" }),
    fn: handleGetRoot,
  },
];

const handleRequest = async (request) => {
  const mapping = urlMapping.find(
    (um) => um.method === request.method && um.pattern.test(request.url)
  );

  if (!mapping) {
    return new Response("Not found", { status: 404 });
  }

  const mappingResult = mapping.pattern.exec(request.url);
  return await mapping.fn(request, mappingResult);
}

const portConfig = { port: 7777 };

Deno.serve(portConfig, handleRequest);

To run the application, place it in a file called app.js and launch it with the command deno run --allow-net --unstable app.js.

Installing k6

To continue, install k6 following the guidelines at https://k6.io/docs/get-started/installation/. Once you have installed k6, running the command k6 on the command line produces an output similar to the following.

k6

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

Usage:
  k6 [command]

Available Commands:
  archive     Create an archive
  cloud       Run a test on the cloud
  completion  Generate the autocompletion script for the specified shell
  convert     Convert a HAR file to a k6 script
  help        Help about any command
  inspect     Inspect a script or archive
  login       Authenticate with a service
  pause       Pause a running test
  resume      Resume a paused test
  run         Start a load test
  scale       Scale a running test
  stats       Show test metrics
  status      Show test status
  version     Show application version

Flags:
  -a, --address string      address for the REST API server (default "localhost:6565")
  -c, --config string       JSON config file (default "/path-to-config/k6/config.json")
  -h, --help                help for k6
      --log-format string   log output format
      --log-output string   change the output for k6 logs, possible values are stderr,stdout,none,loki[=host:port],file[=./path.fileformat] (default "stderr")
      --no-color            disable colored output
  -q, --quiet               disable progress updates
  -v, --verbose             enable verbose logging

Use "k6 [command] --help" for more information about a command.

Testing a GET request

With k6, we use JavaScript to write the performance tests. A simple test that would measure how the web server responds to a GET request would be as follows. In the following, we assume that the server is running on localhost at port 7777.

import http from "k6/http";

export default function () {
  http.get("http://localhost:7777");
}

Save the above code to a file called simple-performance-test.js. Now, using k6, you can run the test with the command k6 run simple-performance-test.js, given that the server is running. When we run the test, the output is as follows.

k6 run simple-performance-test.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: simple-performance-test.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)


running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  00m00.0s/10m0s  1/1 iters, 1 per VU

     data_received..................: 136 B 49 kB/s
     data_sent......................: 80 B  29 kB/s
     http_req_blocked...............: avg=338.53µs min=338.53µs med=338.53µs max=338.53µs p(90)=338.53µs p(95)=338.53µs
     http_req_connecting............: avg=154.27µs min=154.27µs med=154.27µs max=154.27µs p(90)=154.27µs p(95)=154.27µs
     http_req_duration..............: avg=891.96µs min=891.96µs med=891.96µs max=891.96µs p(90)=891.96µs p(95)=891.96µs
       { expected_response:true }...: avg=891.96µs min=891.96µs med=891.96µs max=891.96µs p(90)=891.96µs p(95)=891.96µs
     http_req_failed................: 0.00% ✓ 0         ✗ 1
     http_req_receiving.............: avg=70.81µs  min=70.81µs  med=70.81µs  max=70.81µs  p(90)=70.81µs  p(95)=70.81µs
     http_req_sending...............: avg=78.22µs  min=78.22µs  med=78.22µs  max=78.22µs  p(90)=78.22µs  p(95)=78.22µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=742.92µs min=742.92µs med=742.92µs max=742.92µs p(90)=742.92µs p(95)=742.92µs
     http_reqs......................: 1     362.31359/s
     iteration_duration.............: avg=1.46ms   min=1.46ms   med=1.46ms   max=1.46ms   p(90)=1.46ms   p(95)=1.46ms
     iterations.....................: 1     362.31359/s

The above output outlines a number of metrics. From the metrics, we'll focus on http_req_duration, http_req_failed, and http_reqs for now. The metric http_req_duration outlines the total time for the request, http_req_failed shows the proportion and number of failed requests, and http_reqs shows the total number of requests. With the above test, we make only a single request to the server.

For the request, the duration was 891.96µs, i.e. 891.96 microseconds or 0.891 milliseconds. The request completed successfully.

Providing options to k6

Options for k6 can be added to the test configuration. In the following configuration, we state that we wish to run the test for 5 seconds, repeatedly sending requests with 10 concurrent users.

import http from "k6/http";

export const options = {
  duration: "5s",
  vus: 10,
};

export default function () {
  http.get("http://localhost:7777");
}

Now, running the test again provides a different view on the performance of the application.

k6 run simple-performance-test.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: simple-performance-test.js
     output: -

  scenarios: (100.00%) 1 scenario, 10 max VUs, 35s max duration (incl. graceful stop):
           * default: 10 looping VUs for 5s (gracefulStop: 30s)


running (05.0s), 00/10 VUs, 107358 complete and 0 interrupted iterations
default ✓ [======================================] 10 VUs  5s

     data_received..................: 15 MB  2.9 MB/s
     data_sent......................: 8.6 MB 1.7 MB/s
     http_req_blocked...............: avg=1.63µs   min=595ns   med=1.43µs   max=735.3µs  p(90)=1.88µs   p(95)=2.32µs
     http_req_connecting............: avg=6ns      min=0s      med=0s       max=142.87µs p(90)=0s       p(95)=0s
     http_req_duration..............: avg=426.12µs min=85.01µs med=388.3µs  max=5.85ms   p(90)=525.87µs p(95)=634.65µs
       { expected_response:true }...: avg=426.12µs min=85.01µs med=388.3µs  max=5.85ms   p(90)=525.87µs p(95)=634.65µs
     http_req_failed................: 0.00%  ✓ 0            ✗ 107358
     http_req_receiving.............: avg=17.59µs  min=5.08µs  med=14.63µs  max=5.17ms   p(90)=20.97µs  p(95)=26.94µs
     http_req_sending...............: avg=6.84µs   min=2.5µs   med=6.09µs   max=1.88ms   p(90)=8.06µs   p(95)=10.06µs
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=401.69µs min=71.31µs med=366.11µs max=5.68ms   p(90)=496.93µs p(95)=599.48µs
     http_reqs......................: 107358 21467.342318/s
     iteration_duration.............: avg=459.57µs min=101.8µs med=419.1µs  max=7.2ms    p(90)=565.33µs p(95)=684.7µs
     iterations.....................: 107358 21467.342318/s
     vus............................: 10     min=10         max=10
     vus_max........................: 10     min=10         max=10

Here, we can see that the average request duration was 426.12µs, there were no failed requests, and during the 5 second test, the server responded to a total of 107358 requests (approx 21467.3 per second). When considering the performance of an application, it is meaningful to not only consider its average performance, but to consider its performance in 95th percentile. Here, the result p(95)=634.65µs at the end of the row http_req_duration highlights that at the 95th percentile, the average request duration was 634.65µs.

There is a full documentation of the k6 options available at https://k6.io/docs/using-k6/k6-options/reference/. If we would wish to focus on particular statistics -- say average and 99th percentile, we would adjust the configuration as follows.

import http from "k6/http";

export const options = {
  duration: "5s",
  vus: 10,
  summaryTrendStats: ["avg", "p(99)"]
};

export default function () {
  http.get("http://localhost:7777");
}

The configuration can be further adjusted to focus only on specific

Testing more complex requests

The http module that the test imports provides a variety of functions that can be used for writing the test. We can, for example, write a test that visits the main page, visits the lists of items, and posts an item to the list.

import http from "k6/http";

export const options = {
  duration: "5s",
  vus: 10,
  summaryTrendStats: ["avg", "p(99)"],
};

export default function () {
  http.get("http://localhost:7777");
  http.get("http://localhost:7777/items");
  http.post(
    "http://localhost:7777/items",
    JSON.stringify({ name: "hamburger" })
  );
}

The output for the above test is as follows.

k6 run simple-performance-test.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: simple-performance-test.js
     output: -

  scenarios: (100.00%) 1 scenario, 10 max VUs, 35s max duration (incl. graceful stop):
           * default: 10 looping VUs for 5s (gracefulStop: 30s)


running (05.0s), 00/10 VUs, 6371 complete and 0 interrupted iterations
default ✓ [======================================] 10 VUs  5s

     data_received..................: 428 MB 85 MB/s
     data_sent......................: 1.9 MB 370 kB/s
     http_req_blocked...............: avg=2.02µs p(99)=7.75µs
     http_req_connecting............: avg=107ns  p(99)=0s
     http_req_duration..............: avg=2.57ms p(99)=10.11ms
       { expected_response:true }...: avg=2.57ms p(99)=10.11ms
     http_req_failed................: 0.00%  ✓ 0           ✗ 19113
     http_req_receiving.............: avg=33.2µs p(99)=146.88µs
     http_req_sending...............: avg=8.71µs p(99)=30.06µs
     http_req_tls_handshaking.......: avg=0s     p(99)=0s
     http_req_waiting...............: avg=2.53ms p(99)=10.03ms
     http_reqs......................: 19113  3812.560429/s
     iteration_duration.............: avg=7.85ms p(99)=16.45ms
     iterations.....................: 6371   1270.853476/s
     vus............................: 10     min=10        max=10
     vus_max........................: 10     min=10        max=10

As we can see from the output, the average request duration was 2.57ms, there were no failed requests, and during the 5 second test, the server responded to a total of 19113 requests (approx 3812.6 per second). This example also illustrates the utility of the percentiles in the test outputs -- in our case, the request duration for the 99th percentile was 10.1 milliseconds.