Items API and load balancing

Let's continue with our Items API. We were left off in a situation, where the docker-compose.yml contained a configuration that allowed launching the items-api service using two replicas, each with their own port. The configuration for items-api was as follows.

# ..
  items-api:
    build: items-api
    image: items-api
    restart: "no"
    volumes:
      - ./items-api/:/app
      - ./app-cache/:/app-cache
    ports:
      - "7777-7778:7777"
    depends_on:
      - database
      - flyway
    env_file:
      - project.env
    deploy:
      replicas: 2
# ..

To achieve a situation where incoming requests are directed to one of the items-api instances, we need a load balancer. There exists a wide variety of load balancing solutions, including NGINX, HAProxy, and Envoy. Here, we'll take a peek at NGINX.

Starting with NGINX

To get started with NGINX, we add it as a service -- called nginx -- to the docker-compose.yml file.

NGINX offers an official Docker image, nginx:latest, which we can use. The NGINX Docker image requires a configuration file, which we will include to the container as an image. We'll create a folder called nginx and place the configuration file (called nginx.conf) to the folder -- we'll do this in a moment. In addition, we'll define items-api as a dependency for the NGINX service, which means that the NGINX service will wait for the items-api to start up before launching. Finally, we'll expose a port from nginx, mapping the internal port 7800 to the public port 7800.

In docker-compose.yml, this looks as follows.

# ..
  nginx:
    image: nginx:latest
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - items-api
    ports:
      - 7800:7800
# ..

For the NGINX configuration, which we'll place in the file nginx.conf within the folder nginx, we use the following.

worker_processes 1;

events {
  worker_connections 1024;
}

http {

  upstream items-api {
    server items-api:7777;
  }

  server {
    listen 7800;

    location / {
      proxy_pass http://items-api;
    }
  }
}

The configuration describes the use of one worker process for handling incoming connections, where the worker could have up to 1024 concurrent open connections. It then proceeds to describe how to handle http traffic (within the http block). There, we first label the service items-api running on port 7777 as items-api, and then define a server that listens for incoming connections on the port 7800. In the server, for all incoming requests (location /), we delegate (proxy_pass) them to service earlier labeled as items-api.

With the configuration in place, the folder structure of the whole application is as follows.

tree --dirsfirst
├── app-cache
│   └── ...
├── flyway
│   └── sql
│       └── V1___initial_schema.sql
├── items-api
│   ├── app.js
│   ├── deps.js
│   └── Dockerfile
├── nginx
│   └── nginx.conf
├── docker-compose.yml
└── project.env

Now, when we launch our application using docker compose up, we see the nginx service also starting up. When we try out making requests to the server at the port 7800, where the service nginx is running, we see that it responds with data from the items-api.

curl localhost:7800/items
[{"id":1,"name":"water"},{"id":2,"name":"bread"}]%
curl localhost:7800
Hello world at root!%

Balancing requests?

Our current nginx service clearly works in that it retrieves content from the items-api. But, does it actually use both replicas of items-api? To test this out, let's add an UUID to the server that is generated when the server is launched, and adjust the handleGetRoot method to return the identifier.

// ..
const SERVER_ID = crypto.randomUUID();

const handleGetRoot = async (request) => {
  return new Response(`Hello from ${ SERVER_ID }`);
};
// ..

Now, when we try out accessing our server through the load balancer, the responses look as follows.

curl localhost:7800
Hello from 3dbc9c93-0a6c-4da1-9605-47d87da3f69c%
curl localhost:7800
Hello from fe829d07-6b0c-4849-9944-a7f40fa0dc51%
curl localhost:7800
Hello from 3dbc9c93-0a6c-4da1-9605-47d87da3f69c%
curl localhost:7800
Hello from fe829d07-6b0c-4849-9944-a7f40fa0dc51%
curl localhost:7800
Hello from 3dbc9c93-0a6c-4da1-9605-47d87da3f69c%

It seems that the load balancing is working and the requests are distributed across the servers.

Why does this work, given that in our docker-compose.yml file the items-api uses the port range 7777-7778, while the NGINX configuration only relies on the port 7777?

Internal and external ports

To understand what is going on, we need to have a rough idea of how Docker networks work. When we launch an application using docker compose up, we are actually creating a network of services. The services have names and ports that are internal to the network, which can be opened up to the world if we choose to do so. The names in the internal network correspond to names of our services in the docker-compose.yml file; as an example, items-api is identified as items-api in the internal network.

Docker takes internally care of replicas, balancing traffic. That is, if a request is made to items-api within the network, Docker directs the request to one of the replicas of items-api. When we declare in the NGINX configuration that traffic to / should be delegated to the service items-api in the NGINX configuration, the actual mapping of items-api to a container is handled by Docker.

Hiding Items API

With information about how the Docker network works, we can hide the items-api port from the world, leading to a situation where the port 7777 (or 7778) no longer responds to requests from outside. As we still want that the port 7777 is available within the Docker network, we use the configuration option expose to expose the port from the container (while omitting publishing the port for all).

With this adjustment, the configuration of the nginx and the items-api service is as follows.

# ..
  nginx:
    image: nginx:latest
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - items-api
    ports:
      - 7800:7800

  items-api:
    build: items-api
    image: items-api
    restart: "no"
    volumes:
      - ./items-api/:/app
      - ./app-cache/:/app-cache
    expose:
      - 7777
    depends_on:
      - database
      - flyway
    env_file:
      - project.env
    deploy:
      replicas: 2
# ..

Now, when we launch our application using docker compose up, we observe that there's nothing available at the port 7777, while 7800 -- where NGINX is running -- continues to respond to requests.

curl localhost:7800
Hello from 7803f43c-97af-4e0e-8597-8d5f63a359fe%
curl localhost:7800
Hello from d66c52c7-b0e9-47c9-911e-e53c63d0d4ca%
curl localhost:7800
Hello from 7803f43c-97af-4e0e-8597-8d5f63a359fe%
curl localhost:7777
curl: (7) Failed to connect to localhost port 7777: Connection refused