Data & Data Scalability

Caching

---

Caching

Caching refers to storing data in a temporary location that provides faster access to the data than the original data location. It is used to increase the efficiency of retrieving data that has already been retrieved once. As an example, an image on a web page could be stored in the browser (on the client) and read from there on subsequent web page accesses. Similarly, data requested from a database could be stored on the web server, and read from there on subsequent requests.

There are multiple benefits to the use of caching in web applications, including (1) improved performance — caching can speed up the retrieval of the data, resulting in a faster web application; (2) reduced server load — caching can reduce the number of requests made to the server, reducing the load on the server; and (3) increased scalability — through reducing server load, caching can help increase the scalability of a web application. In addition, caching can be used to enable offline access to data, and depending on the cost model of the hosting solution, caching may also lead to reduced hosting costs.

There are, however, also downsides to caching.

“There are only two hard problems in Computer Science: cache invalidation and naming things.” — Phil Karlton (see also TwoHardThings)

Loading Exercise...

Downsides to caching include it being hard to implement properly. If cache invalidation is not properly implemented, the use of caching can lead to stale data, meaning that the data is no longer accurate or up-to-date. Imagine stale data, for example, in a web shop where the price of a product has changed, but the old price is still shown due to the use of caching.

Caching also adds (some) complexity to the application, and may require introducing additional services. The use of caching also leads to an increased need of resources, as the cached data needs to be stored. Finally, if not properly secured, caching of data may also lead to security risks.

Caching can be implemented in multiple locations, including the browser, the server, and the application, proxy servers, load balancers, and so on. Here, we’ll focus on server-side caching, which refers to temporarily storing requested data on the server. This allows retrieving the data quickly from the cache instead of having to regenerate the data or to retrieve it from a database, which may take longer to do.

Loading Exercise...

There are multiple types of server-side caching, including in-memory caching, disk caching, and distributed caching. We’ll first look into in-memory caching, after which we’ll briefly discuss distributed caching and the use of Redis as a cache server.

In-memory caching

In-memory caching refers to storing information — not surprisingly — into the memory of the server. This allows very fast access to the cached data, as the server can quickly retrieve the data from memory without having to read it from disk (or to retrieve it from another service such as a database). In-memory caching is often used for frequently accessed data, such as database queries or results of expensive computations. As the data is stored in the memory, the amount of data that can be stored is limited; further, if and when the server is restarted, the data stored in the memory will be lost.

Let’s take a peek at naive in-memory caching. To start with, we have an application where a request takes some time to complete. This could be mimicked as follows — in the following example, one of the routes of our application simply waits for a second before returning a response.

app.get(
  "/",
  async (c) => {
    await new Promise((resolve) => setTimeout(resolve, 1000));
    return c.json({ message: "Hello world!" });
  },
);

When we launch the server and make a few requests to it, we notice that it takes a moment before we receive a response. In the following example, we’ve used curl’s ability to check the time, printing the total time to make the request at the end of the response. Each request takes a bit more than a second, which is what we would expect.

curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (1.038508s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (1.004617s)%

A simple in-memory caching approach could use a data structure for storing responses related to requests. When caching information, there’s a need for a cache key, which is used to identify the requested resource. Whenever a resource is requested, we first check the existence of a corresponding response from the data structure — if it exists, we return it. If the response does not exist, after the response has been created, we store the response to the cache for future use.

Hono Cache Middleware

Hono comes with a cache middleware that we can use as a cache in our application. To use the cache middleware, we need to import it, and then add it to a specific route. In the following example, we’ve added the cache middleware to the root route of our application.

// ...
import { cache } from "@hono/hono/cache";

// ...

app.get(
  "/",
  cache({
    cacheName: "hello-cache",
    wait: true,
  }),
);

app.get(
  "/",
  async (c) => {
    await new Promise((resolve) => setTimeout(resolve, 1000));
    return c.json({ message: "Hello world!" });
  },
);
// ...

Now, when we restart the server, and make a few requests to it, we notice that the first request takes a moment before we receive a response, but the subsequent requests are quite a bit faster. This is due to the cache middleware, which stores the response of the first request and returns it on subsequent requests.

curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (1.071413s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (0.005319s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (0.003138s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (0.003910s)%

As we can see from above, after the response has been cached, the request duration drops considerably. There are a plethora of downsides to the naive approach, however, including (1) running out of memory and (2) having a stale cache. These stem from the simplicity of the approach: there is no cache eviction policy and once the items have been cached, they remain in the cache.

The cache middleware uses, by default, the url as an extra key to the cache. This means that each individual url will be cached separately, if we we choose to apply the cache on a specific path with a wildcard. The following example demonstrates this with a path /hello/:name, where the name is a parameter.

app.get(
  "/hello/*",
  cache({
    cacheName: "hello-cache",
    wait: true,
  }),
);

app.get(
  "/hello/:name",
  async (c) => {
    await new Promise((resolve) => setTimeout(resolve, 1000));
    return c.json({ message: `Hello ${c.req.param("name")}!` });
  },
);

curl -w ' (%{time_total}s)' localhost:8000/hello/jane
{"message":"Hello jane!"} (1.070402s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/jane
{"message":"Hello jane!"} (0.005049s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/joe
{"message":"Hello joe!"} (1.026255s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/joe
{"message":"Hello joe!"} (0.003875s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/jane
{"message":"Hello jane!"} (0.003713s)%

Cache eviction

Hono’s cache middleware uses the Web Cache API for caching. The Web Cache API is available in Deno through a global variable called caches. To fully clear a cache, we can use the delete method of the cache object. In the following example, we’ve added a route to our application that clears the cache named hello-cache.

app.post(
  "/", async (c) => {
    await caches.delete("hello-cache");
    return c.json({ message: "Cache cleared!" });
  },
);

Now, on a POST request to the root route, the cache is cleared. After the cache has been cleared, the next request to the root route will take a moment before we receive a response, as the cache has been cleared.

curl -w ' (%{time_total}s)' -X POST localhost:8000
{"message":"Cache cleared!"} (0.033259s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (1.051636s)%
curl -w ' (%{time_total}s)' -X POST localhost:8000
{"message":"Cache cleared!"} (0.002609s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (1.025586s)%
curl -w ' (%{time_total}s)' localhost:8000
{"message":"Hello world!"} (0.003991s)%

The caches object also has methods for opening a cache based on a key, which returns a promise that resolves to a cache object. The cache object has methods for deleting cached content based on a key. In the following example, we’ve added a route to our application that clears the cache based on the url of the request.

app.post(
  "/hello/:name",
  async (c) => {
    const cache = await caches.open("hello-cache");
    if (cache) {
      await cache.delete(c.req.url);
    }
    return c.json({ message: "Cache cleared!" });
  },
);

Loading Exercise...

Distributed Caching and Redis

When web applications are scaled, the use of in-memory caching can become problematic, as the cache on one server may not reflect the state of the application on another server. As an example, one server might receive a request to adjust the data, leading to the eviction of the cache on that server. On another server, the cache would still be present, leading to stale data being returned.

It would be possible to implement cache eviction messages between the servers, but this would add complexity to the application. This is where distributed caching comes into play. Distributed caching refers to storing the cache in a location that is accessible by all servers in the application.

Here, we’ll briefly look into the use of Redis, which is an in-memory data store that can be used, among other things, as a (distributed) cache service.

To read more about the use of Redis in a production environment, check out Stack Overflow: How We Do App Caching - 2019 Edition by Nick Craver

Let’s dive straight in and add Redis to the compose.yaml file of our application. Redis offers official Docker images, which we can use. Let’s use the image version 7.4.2, i.e. the one tagged with redis:7.4.2. By default, Redis launches on port 6379, so we need to expose that to the Docker network. In addition, we wish to add a configuration file for the service — let’s create a folder called redis to the walking skeleton, and place a file called redis.conf to the folder.

The contents of redis.conf is as follows — we’re stating that we wish to use Redis as an in-memory cache with 10 megabytes of space, and that the cache eviction strategy should be least recently used (LRU). For further information on the Redis configuration, see the Redis configuration file example provided in Redis documentation.

maxmemory 10mb
maxmemory-policy allkeys-lru
save ""
appendonly no

To tell Redis about the location of the configuration file, we need to also add a command that is used to launch the Redis server — the command is followed by the configuration file location. In full, the configuration of our redis service is as follows.

  redis:
    image: redis:7.4.2
    command: redis-server /usr/local/etc/redis/redis.conf
    volumes:
      - ./redis/redis.conf:/usr/local/etc/redis/redis.conf

Now, when we run docker compose up --build, we see that redis is being initialized and that after a while, it is ready to accept connections.

redis-1                | 1:C 15 Jan 2025 12:37:10.929 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-1                | 1:C 15 Jan 2025 12:37:10.929 * Redis version=7.4.2, bits=64, commit=00000000, modified=0, pid=1, just started
...
redis-1                | 1:M 15 Jan 2025 12:37:10.930 * Ready to accept connections tcp

Using Redis in Deno

To use Redis as a cache in our application, we need a driver for it. Here, the ioredis driver from npm works well. To use it, let’s add it to deno.json in the server folder. After the modification, the deno.json should look as follows.

{
  "imports": {
    "@hono/hono": "jsr:@hono/hono@4.6.5",
    "postgres": "https://deno.land/x/postgresjs@v3.4.4/mod.js",
    "ioredis": "npm:ioredis@5.4.2"
  }
}

Now, modify app.js to import the redis driver, create an instance of the driver (we use the port 6379 and the address “redis”), and use the driver to set and get values from the cache. In the following example, we’ve created a route “/redis-test” that reads in a value from the cache, increments it, and stores it back to the cache.

// ...
import { Redis } from "ioredis";
// ...

const redis = new Redis(6379, "redis");
// ...

app.get("/redis-test", async (c) => {
  let count = await redis.get("test");
  if (!count) {
    count = 0;
  } else {
    count = Number(count);
  }

  count++;

  await redis.set("test", count);
  return c.json({ count });
});
// ...

With the above, we’ve created a simple route that increments a value in the cache. When we make a few requests to the route, we notice that the value is incremented on each request.

curl localhost:8000/redis-test
{"count":1}%
curl localhost:8000/redis-test
{"count":2}%
curl localhost:8000/redis-test
{"count":3}%
curl localhost:8000/redis-test
{"count":4}%

Redis as a cache

To use Redis as a cache in our application, we would need to create a middleware that checks whether there exists a corresponding result for the request in the cache cache, and if yes return the result. Otherwise, the request would be processed normally, and the result would be stored to the cache for subsequent requests.

One implementation could look as follows — there’s a bit of trickery to cloning requests, which are not discussed here in detail.

const redisCacheMiddleware = async (c, next) => {
  const cachedResponse = await redis.get(c.req.url);
  if (cachedResponse) {
    const res = JSON.parse(cachedResponse);
    return Response.json(res.json, res);
  }

  await next();

  if (!c.res.ok) {
    return;
  }

  const clonedResponse = c.res.clone();

  const res = {
    status: clonedResponse.status,
    statusText: clonedResponse.statusText,
    headers: Object.fromEntries(clonedResponse.headers),
    json: await clonedResponse.json(),
  };

  await redis.set(c.req.url, JSON.stringify(res));
};

Now, we could replace our earlier cache middleware with the above middleware, and the cache would be stored in Redis.

app.get(
  "/hello/*",
  redisCacheMiddleware,
);

app.get(
  "/hello/:name",
  async (c) => {
    await new Promise((resolve) => setTimeout(resolve, 1000));
    return c.json({ message: `Hello ${c.req.param("name")}!` });
  },
);

With the above in place and the redis server up and running, we can make a few requests to the application. Again, the first request will take a moment, but the subsequent requests will be much faster.

curl -w ' (%{time_total}s)' localhost:8000/hello/cache
{"message":"Hello cache!"} (1.007271s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/more-cache
{"message":"Hello more-cache!"} (1.005267s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/more-cache
{"message":"Hello more-cache!"} (0.002939s)%
curl -w ' (%{time_total}s)' localhost:8000/hello/cache
{"message":"Hello cache!"} (0.001959s)%

You might notice that the difference between the version that uses an external redis cache and the version that uses the in-memory cache is not that big. This is due to the fact that the data is stored in memory in both cases, and the retrieval of the data does not require reading it from a disk. Network traffic is also quite fast, especially when on the same server and docker network.

In a scenario, where the redis cache would be on a separate server, or where the data would be stored on a disk, the difference would be larger.

Loading Exercise...