Server-side caching

Server-side caching refers to temporarily storing requested data on the server. This allows retrieving the data quickly from the cache instead of having to regenerate the data or to retrieve it from a database, which may take longer to do. There are mutliple types of server-side caching, including in-memory caching, disk caching, and distributed caching. Here, we'll briefly look into in-memory caching and distributed caching.

Naive in-memory caching

In-memory caching refers to storing information -- not surprisingly -- into the memory of the server. This allows very fast access to the cached data, as the server can quickly retrieve the data from memory without having to read it from disk (or to retrieve it from another service such as a database). In-memory caching is often used for frequently accessed data, such as database queries or results of expensive computations. As the data is stored in the memory, the amount of data that can be stored is limited; further, if and when the server is restarted, the data stored in the memory will be lost.

Let's take a peek at naive in-memory caching. To start with, we have an application where a request takes some time to complete. This could be mimicked as follows -- in the following example, our application simply waits for a second before returning a response.

const handleRequest = async (request) => {
  await new Promise(resolve => setTimeout(resolve, 1000));
  return new Response("Phew, it took quite some time!");
};

const portConfig = { port: 7777, hostname: "0.0.0.0" };
Deno.serve(portConfig, handleRequest);

When we launch the server and make a few requests to it, we notice that it takes a moment before we receive a response. In the following example, we've used curl's ability to check the time, printing the total time to make the request at the end of the response. Each request takes a bit more than a second, which is what we would expect.

curl -w ' (%{time_total}s)' localhost:7777
Phew, it took quite some time! (1,006220s)%
curl -w ' (%{time_total}s)' localhost:7777
Phew, it took quite some time! (1,005657s)%

A simple in-memory caching approach could use just a map for storing responses related to requests. When caching information, there's a need for a cache key, which is used to identify the requested resource. In our case, we could use the request method and the requested url. Whenever a resource is requested, we first check the existence of a corresponding response from the map -- if it exists, we clone the response and return it. If the response does not exist, after the response has been created, we create a clone of it and store it to the map.

Such an implementation could look as follows. Note that instead of serving requests with the handleRequest function, we are now responding to requests with the handleRequestWithCache function, which wraps the handleRequest function and provides the caching functionality.

const cache = new Map();

const handleRequest = async (request) => {
  await new Promise(resolve => setTimeout(resolve, 1000));
  return new Response("Phew, it took quite some time!");
};

const handleRequestWithCache = async (request) => {
  const key = `${request.method}-${request.url}`;
  if (cache.has(key)) {
    return cache.get(key).clone();
  }

  const response = await handleRequest(request);
  cache.set(key, response.clone());
  return response;
}

const portConfig = { port: 7777, hostname: "0.0.0.0" };
Deno.serve(portConfig, handleRequestWithCache);

Now, the first request still takes some time as the in-memory cache doesn't yet contain a response. The subsequent requests, however, are quite a bit faster.

curl -w ' (%{time_total}s)' localhost:7777
Phew, it took quite some time! (1,012933s)%
curl -w ' (%{time_total}s)' localhost:7777
Phew, it took quite some time! (0,005435s)%
curl -w ' (%{time_total}s)' localhost:7777
Phew, it took quite some time! (0,005376s)%

In effect, after the response has been cached, the request duration drops by approximately one second. This is to be expected, as the function handleRequest contained a call that waited for one second. There are a plethora of downsides to the naive approach, however, including (1) running out of memory and (2) having a stale cache. These stem from the simplicity of the approach: there is no cache eviction policy and once the items have been cached, they remain in the cache.

Caching database queries

Let's continue with our items-api and add simple functionality for caching database queries. Our project structure at the moment is as follows.

tree --dirsfirst
├── app-cache
│   └── ...
├── flyway
│   └── sql
│       └── V1___initial_schema.sql
├── items-api
│   ├── app.js
│   ├── deps.js
│   └── Dockerfile
├── nginx
│   └── nginx.conf
├── docker-compose.yml
└── project.env

Separating concerns

As the functionality in our application grows, let's introduce a new file called itemService.js and place it under a folder called services. After this change, the project structure is as follows.

tree --dirsfirst
├── app-cache
│   └── ...
├── flyway
│   └── sql
│       └── V1___initial_schema.sql
├── items-api
│   ├── services
|   |   └── itemService.js
│   ├── app.js
│   ├── deps.js
│   └── Dockerfile
├── nginx
│   └── nginx.conf
├── docker-compose.yml
└── project.env

The file itemService.js will contain the functionality responsible for the queries to the items table in the database. Presently, there are three database-specific functionalities in the application -- retrieving a single item, retrieving all the items, and adding an item. We'll introduce individual functions for each of these, and export them from the item service for others to use. In practice, this would look as follows.

import { postgres } from "../deps.js";

const sql = postgres({});

const getItem = async (id) => {
  const items = await sql`SELECT * FROM items WHERE id = ${id}`;
  return items[0];
};

const getItems = async () => {
  return await sql`SELECT * FROM items`;
};

const addItem = async (name) => {
  await sql`INSERT INTO items (name) VALUES (${name})`;
};

export { getItem, getItems, addItem };

To include the functions from itemService.js to our application, we need to modify our app.js. Effectively, we'll remove the database queries from app.js, and replace them with the functions from the item service. After the change, the key parts in app.js look as follows.

import * as itemService from "./services/itemService.js";

// ..

const handleGetItem = async (request, urlPatternResult) => {
  const id = urlPatternResult.pathname.groups.id;
  return Response.json(await itemService.getItem(id));
};

const handleGetItems = async (request) => {
  return Response.json(await itemService.getItems());
};

const handlePostItems = async (request) => {
  const item = await request.json();
  await itemService.addItem(item.name);
  return new Response("OK", { status: 200 });
};

// ..

Now, we have clear separation of the database functionality, which makes caching easier.

Caching itemService

To cache the calls to the itemservice, we want to wrap each of the functions with caching functionality. As this would become rather verbose if we would do this one by one, we can do a trick with the Proxy object, which allows wrapping functionality.

By default, Proxy allows us proxying calls for retrieving a property (e.g. a method) and for calling a function but not both at the same time, which we wish to do if we wish to proxy calls like itemService.getItems(). To achieve both, we need to handle retrieving a property, and within that, also handle calling it. This can be done as follows.

Note that it is not important to understand the following function.

const cache = new Map();

const cacheMethodCalls = (object, methodsToFlushCacheWith = []) => {
  const handler = {
    get: (module, methodName) => {
      const method = module[methodName];
      return async (...methodArgs) => {
        if (methodsToFlushCacheWith.includes(methodName)) {
          cache.clear();
          return await method.apply(this, methodArgs);
        }

        const cacheKey = `${methodName}-${JSON.stringify(methodArgs)}`;

        if (!cache.has(cacheKey)) {
          cache.set(cacheKey, await method.apply(this, methodArgs));
        }

        return cache.get(cacheKey);
      };
    },
  };

  return new Proxy(object, handler);
};

The above function cacheMethodCalls is given two parameters: (1) an object whose methods should be cached and (2) list of method names that the cache should be flushed with. In effect, the function returns a proxied version of the object, where the proxy caches the method call results, and on subsequent calls, returns those from the cache. A simple flushing mechanism is in place -- whenever a method in the list methodsToFlushCacheWith is called, the cache is cleared.

If you're interested in digging deeper into the above function, check out e.g. the chapter on Metaprogramming with proxies in the book Exploring ES6.

As the function cacheMethodCalls is an utility function, it is meaningful to place it in a separate location. For now, we'll create a folder util and a file cacheUtil.js into it, and place the function there. To accommodate the move, we'll also have to add an export to the end.

// ...
export { cacheMethodCalls };

The above function allows us to wrap our itemService, resulting in an object with caching. With this in place, the key changed parts in app.js would be as follows.

import * as itemService from "./services/itemService.js";
import { cacheMethodCalls } from "./util/cacheUtil.js";

// ..

const cachedItemService = cacheMethodCalls(itemService, ["addItem"]);

const handleGetItem = async (request, urlPatternResult) => {
  const id = urlPatternResult.pathname.groups.id;
  return Response.json(await cachedItemService.getItem(id));
};

const handleGetItems = async (request) => {
  return Response.json(await cachedItemService.getItems());
};

const handlePostItems = async (request) => {
  // assuming that the request has a json object and that
  // the json object has a property name
  const item = await request.json();
  await cachedItemService.addItem(item.name);
  return new Response("OK", { status: 200 });
};
// ..

Let's next test our application. First, we retrieve the items, then we add an item, and then we retrieve the items a couple of times.

curl localhost:7800/items
[]%
curl -X POST -d '{"name": "bread"}' localhost:7800/items
OK%
curl localhost:7800/items
[]%
curl localhost:7800/items
[{"id":1,"name":"bread"}]%
curl localhost:7800/items
[]%

It seems that the cache is working, but we face another issue. As there are two servers (two replicas), the caches are local to those machines. Thus, the outputs from the servers are not in sync.

Caching and multiple servers

When the number of servers increases, the use of in-memory within-server caching can become problematic, as the cache on one server may not reflect the state of the application on another server. This is where distributed caching comes into play. When using distributed caching, we typically have a service that is responsible for caching.

Here, we'll briefly look into the use of Redis, which is an in-memory data store that can be used, among other things, as a (distributed) cache service.

To read more about the use of Redis in a production enviroment, check out Stack Overflow: How We Do App Caching - 2019 Edition by Nick Craver

Let's dive straight in and add Redis to the docker-compose.yml file of our application. Redis offers official Docker images, which we can use. Let's use the latest image, i.e. the one tagged with redis:latest. By default, Redis launches on port 6379, so we need to expose that to the Docker network. In addition, we wish to add a configuration file for the service -- let's use a similar approach that we did with nginx and create a folder for the configuration. We'll call the folder redis and place a file called redis.conf in the folder.

The contents of the configuration file is as follows -- we're essentially stating that we wish to use Redis as an in-memory cache with 5 megabytes of space, and that the cache eviction strategy should be least recently used (LRU). For further information on the Redis configuration, see the Redis configuration file example provided in Redis documentation.

maxmemory 5mb
maxmemory-policy allkeys-lru
save ""
appendonly no

To tell Redis about the location of the configuration file, we need to also add a command that is used to launch the Redis server -- the command is followed by the configuration file location. In full, the configuration of our redis service is as follows.

  redis:
    image: redis:latest
    command: redis-server /usr/local/etc/redis/redis.conf
    volumes:
      - ./redis/redis.conf:/usr/local/etc/redis/redis.conf
    expose:
      - 6379

In addition, as we wish to use the Redis service as a cache, we add it as a dependency of our items-api service. The configuration of items-api looks now as follows.

  items-api:
    build: items-api
    image: items-api
    restart: "no"
    volumes:
      - ./items-api/:/app
      - ./app-cache/:/app-cache
    expose:
      - 7777
    depends_on:
      - database
      - flyway
      - redis
    env_file:
      - project.env
    deploy:
      replicas: 2

When we start our project using docker compose up, despite some Redis warnings in the log that we can ignore, we see that all of our services start up.

docker ps
CONTAINER ID   IMAGE           COMMAND                  CREATED & STATUS   PORTS
d685aa179e71   nginx:latest    "/docker-entrypoint.…"   ... Up             80/tcp, 0.0.0.0:7800->7800/tcp, :::7800->7800/tcp
cc2140213e69   items-api       "/tini -- docker-ent…"   ... Up             7777/tcp
fefa5eb87b16   items-api       "/tini -- docker-ent…"   ... Up             7777/tcp
de7915cd8571   postgres:14.1   "docker-entrypoint.s…"   ... Up             5432/tcp
f8ed587315c8   redis:latest    "docker-entrypoint.s…"   ... Up             6379/tcp

To use Redis in our application, we need a driver. Here, the deno-redis driver works nicely. Let's add it to the deps.js, which now looks as follows -- the key function that we need is connect, which is used to form a connection to Redis.

import postgres from "https://deno.land/x/postgresjs@v3.4.4/mod.js";
export { postgres };
export { connect } from "https://deno.land/x/redis@v0.31.0/mod.ts";

When creating a connection, we gain access to a client, which can be used to manipulate data on the Redis server. By default, Redis uses no password, and it is visible only within the network of our application. As the name of the service is redis and the port is 6379, forming the connection is easy.

import { connect } from "./deps.js";

const redis = await connect({
  hostname: "redis",
  port: 6379,
});

The commands available in the Redis client are outlined in the documentation on RedisCommands.

As we already have the functionality for caching method calls, the main thing we need to change is what we use as a cache. Instead of using a map, we rely on redis. To accommodate for this, we need to adjust our cacheMethodCalls a bit, including transforming the data into a string and back when interacting with Redis.

import { connect } from "../deps.js";

const redis = await connect({
  hostname: "redis",
  port: 6379,
});

const cacheMethodCalls = (object, methodsToFlushCacheWith = []) => {
  const handler = {
    get: (module, methodName) => {
      const method = module[methodName];
      return async (...methodArgs) => {
        if (methodsToFlushCacheWith.includes(methodName)) {
          await redis.flushdb()
          return await method.apply(this, methodArgs);
        }

        const cacheKey = `${methodName}-${JSON.stringify(methodArgs)}`;
        const cacheResult = await redis.get(cacheKey);
        if (!cacheResult) {
          const result = await method.apply(this, methodArgs);
          await redis.set(cacheKey, JSON.stringify(result));
          return result;
        }

        return JSON.parse(cacheResult);
      };
    },
  };

  return new Proxy(object, handler);
};

export { cacheMethodCalls };

With this in place, our application now works as expected and both items-api replicas have an up-to-date cache.

curl localhost:7800/items
[]%
curl -X POST -d '{"name": "bread"}' localhost:7800/items
OK%
curl localhost:7800/items
[{"id":1,"name":"bread"}]%
curl localhost:7800/items
[{"id":1,"name":"bread"}]%
curl localhost:7800/items
[{"id":1,"name":"bread"}]%

At the end, the structure of our project is as follows.

tree --dirsfirst
.
├── flyway
│   └── sql
│       └── V1___initial_schema.sql
├── items-api
│   ├── services
│   │   └── itemService.js
│   ├── util
│   │   └── cacheUtil.js
│   ├── app.js
│   ├── deps.js
│   └── Dockerfile
├── nginx
│   └── nginx.conf
├── redis
│   └── redis.conf
├── docker-compose.yml
└── project.env