Client-side caching

Learning objectives

Knows the key HTTP headers used for caching.
Understands how headers like Last-Modified and ETag are used.
Knows about potential security concerns with the use of HTTP headers such as ETag for caching.
Knows of the Web Cache API.

Client-side caching refers to temporarily storing data on the client (e.g. the browser). Let's consider a simple HTTP request and response. When a browser sends a request to a web server, asking for a page, the concrete HTTP request could be as follows.

GET /index.html HTTP/1.1
(headers)

When the server receives the request, it processes it, and creates a response. The response is then sent back to the client. For the above request for a page called index.html, the concrete HTTP response could be as follows.

HTTP/1.1 200 OK
(headers)

<html>
  <head>
    <link rel="stylesheet" href="/styles/styles.css">
  </head>
<body>
  <img src="/images/retro-sax-guy.gif" />
</body>
</html>

The HTML document returned as a part of the response has two resources that the client retrieves in subsequent requests. The resource at path /styles/styles.css likely contains a style definition for the page, while the resource at path /images/retro-sax-guy.gif likely contains an image.

Retrieving the resources happens in the same way as the initial HTTP request that asked for a resource. As an example, the request asking for the resource at path /styles/styles.css could be as follows.

GET /styles/styles.css HTTP/1.1
(headers)

The response would similarly follow the format of HTTP responses, and contain the data for the resource at /styles/styles.css on the server.

In a naive world, whenever the browser loads the page /index.html from the server, the browser would also retrieve the resources /styles/styles.css and /images/retro-sax-guy.gif from the server. This is not always necessary, however, as it might be that the contents of the resources do not change.

HTTP headers for caching

Information about caching (and cachability) of resources is provided in the HTTP headers. We'll briefly visit three of them, Cache-Control, Last-Modified, and ETag. For more in-depth information on the topic, refer to the Internet Standards RFC9111 document.

Cache-Control

The header Cache-Control is used to enable caching for a resource and, optionally, to set an age for the resource. As an example, when asking for a resource at /images/retro-sax-guy.gif, the HTTP request could be as follows.

GET /images/retro-sax-guy.gif HTTP/1.1
(headers)

While the response, with Cache-Control header, could be as follows.

HTTP/1.1 200 OK
Cache-Control: private, max-age: 86400

... data ...

The max-age option in the header is given in seconds. Now, based on the response, the resource /images/retro-sax-guy.gif could be cached by the browser, and the browser would not need to retrieve the resource during the next 24 hours (24*60*60 = 86400).

That is, when the users accesses the page, the browser loads the page and the resources, adding the image to the cache. If the user loads the page again within the 24 hours, the browser would not even attempt to retrieve the image.

Last-Modified (and If-Modified-Since)

The Last-Modified header provides additional information about the resource being retrieved. When the Last-Modified header is used, the server responds with information on when the requested resource was last changed.

Again, when retrieving the resource at /images/retro-sax-guy.gif, the request could be as follows.

GET /images/retro-sax-guy.gif HTTP/1.1
(headers)

When the Last-Modified header is in use, the response would be as follows -- in the example below, we've also included the Cache-Control header, as they are typically used jointly.

HTTP/1.1 200 OK
Cache-Control: private, max-age: 86400
Last-Modified: Mon, 31 Oct 2022 09:45:00 GMT

... data ...

Now, based on the response, the browser would know not to even attempt to retrieve the resource in the next 24 hours. After that, when attempting to retrieve the resource, the browser sends information about the known last modified date of the resource, which was previously received as a part of the response from the server. The header that contains the last modified date in the request is If-Modified-Since.

That is, after the 24 hour no-retrieve break indicated by Cache-Control, the browser would again start retrieving the resource. This time, however, the request would have information about the resource, which the server then can use to decide whether the data has been modified. A request would look as follows.

GET /images/retro-sax-guy.gif HTTP/1.1
If-Modified-Since: Mon, 31 Oct 2022 09:45:00 GMT
(further headers)

Now, when the server receives such a request, it can compare the date in the If-Modified-Since header with the actual modification date of the resource. If the resource has been modified since, the response would contain the data and a new Last-Modified header, as follows.

HTTP/1.1 200 OK
Cache-Control: private, max-age: 86400
Last-Modified: Fri, 13 Jan 2023 09:45:00 GMT

... data ...

On the other hand, if the resource would not have been modified, there is no need to send the data again. In this case, the server could return a response with the status code 304, indicating that the resource has not been modified.

HTTP/1.1 304 Not Modified
(headers)

ETag (and If-None-Matches)

The ETag header provides an unique identifier matching the resource that is being retrieved. It is generated on the server to correspond to the resource, and sent to the client. When using the ETag header, the initial HTTP request is again similar to the previous HTTP requests.

GET /images/retro-sax-guy.gif HTTP/1.1
(headers)

With a server supporting the ETag header, the response could be as follows. The unique identifier-from-server would be a string representation of the content of the requested resource (e.g. MD5 or CSC32C checksum of the contents of the resource).

HTTP/1.1 200 OK
Cache-Control: private, max-age: 86400
ETag: "unique-identifier-from-server"

... data ...

Now, when the client retrieves the resource for the next time (after the initial 24 hour delay determined in Cache-Control), the request for the resource would have a header If-None-Match that would contain the value previously received from the server.

GET /images/retro-sax-guy.gif HTTP/1.1
If-None-Match: unique-identifier-from-server

Similarly to the Last-Modified header, when the server receives a request with If-None-Match, it can compare the value in the If-None-Match header with the actual ETag of the resource. If the value of If-None-Match and the ETag do not match, the response would contain the data and a new ETag header, as follows.

HTTP/1.1 200 OK
Cache-Control: private, max-age: 86400
ETag: "another-unique-identifier-from-server"

... data ...

If the resource would not have been modified, there would be no need to send the data again. The server would again indicate this with the status code 304.

HTTP/1.1 304 Not Modified
(headers)

When compared with the Last-Modified header, the benefits of the ETag header include granularity. If a resource changes twice within a second, the use of the Last-Modified header could lead to stale cache, as the granularity of the format is in seconds. For ETag, the value would be changed whenever the resource would change.

Brief note about tracking

Both Last-Modified and ETag can also be used to track users (although due to the granularity of Last-Modified, ETag better suits this purpose). When a browser requests a resource, the server can e.g. an unique ETag per user-resource -pair, and use subsequent requests to follow the movement of individual users. This could effectively be used to bypass cookies (or sessions).

The use of a variety of headers for tracking users has been identified in research studies (see e.g. Flash Cookies and Privacy II: Now with HTML5 and ETag Respawning). Such use has also led to lawsuits (see e.g. Privacy suit filed over use of ETags).

Support from servers

When working with web applications, developers rarely have to implement the headers used for caching themselves, as servers come with such functionality out of the box. As a simple example, let's consider the following file structure.

tree --dirsfirst
.
├── static
│   └── styles.css
└── index.html

The file index.html would be as follows.

<html>
  <head>
    <link rel="stylesheet" href="/styles/styles.css">
  </head>
<body>
  <p>Hello world!</p>
</body>
</html>

And the styles.css within the folder styles would be as follows.

body {
  font-family: "Comic Sans MS", "Comic Sans", cursive;
}

Using Deno's file server functionality at https://deno.land/std@0.222.1/http/file_server.ts, we can start a simple file server as follows.

deno run --allow-net --allow-read "https://deno.land/std@0.222.1/http/file_server.ts"
Listening on http://localhost:4507/

The server was launched at port 4507. When we query make a query to the server, asking for the file index.html, we see the following response.

curl -v http://localhost:4507/index.html
// ..
> GET /index.html HTTP/1.1
// ..
< HTTP/1.1 200 OK
(headers..)
< etag: f3e86a6a
< last-modified: Fri, 13 Jan 2023 10:00:00 GMT
(more headers)

(data)

As we can see, the server responds with the resource, including also the ETag and Last-Modified headers to the response.

Now, when we ask for the resource again, but include the header If-None-Match, we see that the server responds with the status code 304.

curl -v -H "If-None-Match: f3e86a6a" http://localhost:4507/index.html
// ..
> GET /index.html HTTP/1.1
> If-None-Match: f3e86a6a
// ..
>
< HTTP/1.1 304 Not Modified
// ..

Within a Deno application, the same functionality would be achieved by using the serveFile function.

Caching and load balancers

Load balancers can also be used for caching. As an example, NGINX documentation provides guidelines on how to enable Etag. For this, however, the static file content should be available for the load balancer, as otherwise there could be issues with stale cache. We'll look into this briefly when looking into building production images. We'll also look further into caching when discussing Content Delivery Networks.

Web Cache API

The header-based caching discussed above works nicely with static resources such as HTML documents, stylesheets, images, and JavaScript. With modern web applications, the role of dynamic content has increased, and there exists a need to be able to control what is being cached and how in a more fine-grained fashion.

The Web Cache API allows a more fine-grained (programmatic) view into creating and managing a cache of resources. The API provides methods for adding, updating, and deleting resources from the cache, as well as for checking the status of a cache and the resources it contains. It is also documented as a part of the RFC9111. While the Cache API is mostly available in browsers (i.e. on the client), similar functionality is implemented on runtimes such as Deno as well (see documentation on Cache).

We'll look into this in more detail when discussing client-side development.