Scalability & Delivery

Measuring Performance & Load


Learning Objectives

  • You can define a workload and interpret recorded latency, throughput and error evidence.
  • You can check that a measured request completed the intended work.
  • You can distinguish a local observation from a production-capacity claim.

A page that works for one person may behave differently under concurrent use. Start with a question about a specific operation: for example, “How does this public listing read respond to five concurrent readers over a fixed synthetic dataset?” Record the source revision, data size, machine and workload so a later run can answer the same question.

Successful responses, latency and throughput

Latency is the time an operation takes. Throughput is completed work per unit of time. The error rate shows how much requested work did not succeed. Count a request as successful only when it returns the intended result. An HTTP 500 or an incomplete collection is a failure, however quickly it arrives.

A load runner reports these metrics. A p95 latency is a boundary at or below which approximately 95% of the observed requests fall. It is not the slowest request or a guarantee for another workload.

A semantic check asks whether the response contains the intended result, beyond its status code. For a public collection, that includes the expected records and their public fields. Try expressing that rule as a check before collecting timings.

Response success check

0 / 30 points

Complete isSuccessfulRead(observation, expectedRows) in successful-read.ts.

observation.status contains the HTTP status, and observation.body contains the already parsed response body. A JSON parse failure is represented by null. Return true exactly when all of these conditions hold:

  • HTTP status is 200.
  • expectedRows is a positive safe integer.
  • The body is an array containing exactly expectedRows records.
  • Every record is a non-null object with a positive safe-integer id, a string title, and status active or closed.

A positive safe integer is a whole number from 1 through Number.MAX_SAFE_INTEGER. Return false when any condition fails.

Leave the observation and its records unchanged. Allow additional record fields. Check the whole collection: a malformed final record matters as much as the first one.

Choose Submit solution. Status and count checks are worth 15 points; public-record validation is worth 15. Both groups include a valid response, so always returning false is incomplete.

The check keeps an incomplete collection from counting as a successful read. A load runner can now collect timings alongside that success result while several readers request the collection.

Virtual users and load-test thresholds

k6 is a load runner that can collect these observations. The following options keep an example workload small:

export const options = {
vus: 5,
duration: "20s",
thresholds: {
checks: ["rate==1"],
http_req_failed: ["rate==0"],
},
};

vus is the number of virtual users: simulated readers that repeatedly run the request script. The two thresholds turn recorded failures into a failed run: checks requires every semantic check to pass, and http_req_failed requires no failed HTTP requests. Without a threshold, a failed check would only record a metric. For a latency threshold, first establish a baseline; a useful bound depends on the machine and workload.

A fixture’s stored count and the endpoint’s result limit have different roles. If 20,000 stored listings include 200 public records and the endpoint returns at most 100, a complete response contains 100 records. If only 30 were public, it would contain 30. Keep the expected count and public-field checks alongside the HTTP status check.

Before reading the timings, predict the response count and decide which failures would make the run unsuitable for comparison. Use the fixture and thresholds in the next questions.

Before-and-after performance comparisons

The following illustrative results compare the same 20-second, five-reader workload:

ObservationBeforeAfter one implementation change
Successful requests / all requests398 / 398440 / 440
Successful requests per second19.922.0
Median latency40 ms24 ms
p95 latency82 ms52 ms
Returned rows per request100100

Both successful throughput and the observed latency distribution improved in this example. The identical row count is necessary but insufficient: fields, ordering and visibility rules must also match. Returning private data or omitting required fields would change the operation.

Warm-up and the measurement environment

The first requests may include initialization work. Later requests may benefit from caches. State whether the observation is cold or warm. A short warm-up followed by repeated runs is a practical way to compare warm behavior.

On one laptop, the API and load generator share resources; a database-backed application would add its database to that competition. Background tasks can affect the result. Keep the fixture and workload fixed, report variation, and avoid translating a local p95 into production capacity.

Record these conditions in each summary to check that the runs are comparable:

FieldCondition it identifies
contractIdThe promised response fields, visibility and ordering.
fixtureIdThe same stored data and its distribution.
environmentIdThe machine and service configuration.
workloadIdThe request mix and concurrency being exercised.
successRuleIdThe checks that decide whether a request completed the intended work.
warmWhether initialization/warm-up has already occurred.

The source revision may differ: that is the change being investigated. Actual elapsed duration may also differ, so normalize completed work by each run’s duration rather than comparing raw counts.

First decide whether two observations describe comparable work.

Two load observations

0 / 5 points

Run A uses 100,000 rows and counts only successful reads. Run B uses 10 rows on another machine and counts fast HTTP 500 responses as completions. Run B’s median latency is half of Run A’s. Which conclusion is justified?

Now compare two parsed summaries. Their p95 values are already present; calculate successful throughput from each run’s elapsed duration, then decide whether changed conditions prevent a useful comparison.

Compare recorded runs

0 / 40 points

Complete compareRuns(before, after) in compare-runs.ts. Keep the supplied interfaces and exported function.

The two summaries are already parsed and valid. In each summary, elapsedSeconds and requests are positive, successfulRequests is between zero and requests, and p95Ms is nonnegative. The identity strings are nonempty, and warm is a Boolean. Use the supplied valid inputs and p95 values.

Return the supplied Comparison shape:

Field Required value
before, after A Metrics object for the corresponding run, as described below.
changedConditions Every unequal field among contractId, fixtureId, environmentId, workloadId, successRuleId and warm, in any order.
comparable Whether none of those conditions changed.
differences null when conditions differ; otherwise an object with after-minus-before values for successfulPerSecond and p95Ms.

Each Metrics object contains:

Field Required value
successfulPerSecond Successful requests divided by that run’s elapsed seconds.
errorFraction Unsuccessful requests divided by attempted requests.
p95Ms The run’s supplied p95Ms, unchanged.

A changed sourceRevision is permitted. workloadId identifies the operation, readers and nominal configuration. Actual elapsed time is used to calculate each run’s rate, rather than compared as a condition.

For example, 80 successful requests out of 100 over 20 seconds means 4 successful requests/second and an error fraction of 0.2. Equal timing does not make different fixtures comparable.

Choose Submit solution. Metrics are worth 25 points; conditions and differences are worth 15.

Compare successful work per second alongside latency and the recorded conditions.

Repeat runs under the same conditions before attributing a difference to the change. Then connect the observation to a mechanism: did the query examine fewer rows or make fewer database requests? Report the observed variation, including unchanged timings.

A deterministic concurrency test answers a different question. Part 3 checked a precise overlapping-request invariant; a sampled load run cannot replace it. Correct locking may also serialize competing bids on the same listing, so contention can increase latency while preserving correctness.

The optional walkthrough contains all the code and commands to collect a bounded local run and see how a semantic check affects its result.

Optional: observe a bounded local k6 run

Use Bash on Linux, macOS or WSL with Docker running in Linux-container mode. The commands connect the supplied API and load runner through an isolated Docker network. Docker needs network access to obtain the two pinned images on first use. The official k6 installation guide also documents native installation if you later want to run it without Docker.

Create a fresh directory, change into it, and create these two files. The API is a deliberately small in-memory HTTP fixture: 20,000 stored listings, 200 public, and the first 100 public records in ascending ID order. Its measured duration describes this fixture’s HTTP/filter/serialization path.

Save as performance-api.ts:

const listings = Array.from({ length: 20_000 }, (_, index) => {
const id = index + 1;
return {
id,
title: `Listing ${id}`,
status: id <= 19_800 ? "draft" : id % 2 === 0 ? "active" : "closed",
sellerEmail: "synthetic@example.test",
};
});
Deno.serve({ hostname: "0.0.0.0", port: 8000 }, (request) => {
const url = new URL(request.url);
if (request.method !== "GET" || url.pathname !== "/api/listings") {
return new Response("Not found", { status: 404 });
}
const result = listings
.filter((row) => row.status !== "draft")
.slice(0, 100)
.map(({ id, title, status }) => ({ id, title, status }));
return Response.json(result);
});

Save as load.js. The script checks the collection’s size and ordering as well as each record’s public fields. The successful_reads counter counts requests that passed the combined check.

Run this file with k6, which provides the k6, k6/http and k6/metrics imports directly.

import http from "k6/http";
import { check, sleep } from "k6";
import { Counter } from "k6/metrics";
const successfulReads = new Counter("successful_reads");
export const options = {
vus: 5,
duration: "20s",
thresholds: {
checks: ["rate==1"],
http_req_failed: ["rate==0"],
},
};
const successfulRead = (response) => {
if (response.status !== 200) {
return false;
}
let rows;
try {
rows = response.json();
} catch {
return false;
}
return (
Array.isArray(rows) &&
rows.length === 100 &&
rows.every((row, index) => {
const id = 19_801 + index;
return (
row !== null &&
typeof row === "object" &&
Object.keys(row).sort().join(",") === "id,status,title" &&
row.id === id &&
row.title === `Listing ${id}` &&
row.status === (id % 2 === 0 ? "active" : "closed")
);
})
);
};
const run = () => {
const response = http.get("http://wsd-performance-api:8000/api/listings", {
timeout: "5s",
redirects: 0,
});
const ok = check(response, { "complete public read": successfulRead });
successfulReads.add(ok ? 1 : 0);
sleep(0.2);
};
export default run;

Each virtual reader waits for its response and the short pause before starting another request, so five virtual users do not set a fixed request rate. There is no machine-independent latency threshold here. The semantic and HTTP thresholds decide whether the intended work succeeded. k6 checks record outcomes; the associated thresholds make failures produce a nonzero exit status.

In the first terminal, from the directory containing performance-api.ts, start the API:

Terminal window
docker network create wsd-performance
docker run --rm --name wsd-performance-api --network wsd-performance \
-v "$PWD/performance-api.ts:/app/performance-api.ts:ro" \
denoland/deno:2.9.6 run --allow-net=0.0.0.0:8000 /app/performance-api.ts

Wait for the API’s listening message. Leave it running. In a second terminal, change into the same directory containing load.js. Both containers join wsd-performance; its DNS resolves the API container name used by the script. localhost inside the k6 container would refer to k6 itself.

Warm-up separates initial startup work from later observations. Run one short warm-up, followed by the identical bounded measurement command twice:

Terminal window
docker run --rm -i --network wsd-performance grafana/k6:2.2.0 \
run --vus 1 --duration 5s - <load.js
docker run --rm -i --network wsd-performance grafana/k6:2.2.0 run - <load.js
docker run --rm -i --network wsd-performance grafana/k6:2.2.0 run - <load.js

The commands feed the local script through standard input, as documented in Running k6. The two measured runs use five readers for 20 seconds. Record the warm-up separately from those measurements.

Start with the summary’s http_reqs, http_req_failed, check results and successful_reads. With the intact fixture, every semantic check and request should succeed. The custom counter’s rate reports successful reads per second. Then read the http_req_duration median and p95 alongside those outcomes.

If the run fails, use the observed outcome to decide where to investigate. Connection or name-resolution errors point to the API/network setup; a 200 response with failed checks means the returned representation did not satisfy the rule. Keep the failures visible and the contract checks fixed while diagnosing the cause.

Predict a fast but incomplete response

Try one small fault while keeping the load script unchanged. In performance-api.ts, the API currently takes the first 100 public records. What would the run report if it returned only 90, still with HTTP status 200?

In the first terminal, stop the API with Ctrl+C. In performance-api.ts, change only .slice(0, 100) to .slice(0, 90). Leave load.js unchanged, including its expected 100 rows. Restart the API using the existing network:

Terminal window
docker run --rm --name wsd-performance-api --network wsd-performance \
-v "$PWD/performance-api.ts:/app/performance-api.ts:ro" \
denoland/deno:2.9.6 run --allow-net=0.0.0.0:8000 /app/performance-api.ts

Wait for its listening message. In the second terminal, from the directory containing load.js, run the same bounded measurement:

Terminal window
docker run --rm -i --network wsd-performance grafana/k6:2.2.0 run - <load.js

The requests still receive HTTP 200, so http_req_failed stays at zero. The semantic checks fail because each collection is incomplete, and successful_reads stays at zero. The failed checks threshold makes k6 exit with a nonzero status. That failure is the intended result of this experiment.

Restore the example before finishing. Stop the API with Ctrl+C, change .slice(0, 90) back to .slice(0, 100), and restart it with the API command above. After its listening message, repeat the same measurement command in the second terminal. Every semantic check and request should succeed again. The semantic check distinguishes complete and incomplete responses even though both return HTTP 200.

For an optimization comparison, retain the same fixture, semantic check, pacing, virtual users, duration and environment, then warm and repeat the measured runs after the implementation change. Variation or no improvement is useful evidence. These measurements cover the fixture’s HTTP handling, filtering and serialization.

Stop the API with Ctrl+C in its terminal, then remove only this example’s network:

Terminal window
docker network rm wsd-performance

Check Your Understanding

  1. Why must success be checked alongside request duration?
  2. What does p95 summarize, and what does it not guarantee?
  3. Why should cold and warm observations be distinguished?
  4. Which conditions must remain comparable between runs?
  5. How does a concurrency correctness test differ from a load test?