Resource Limits
Learning Objectives
- You can calculate a connection budget across application instances.
- You can distinguish shared database state from process-local controls.
- You can distinguish a connection-budget calculation from evidence of successful throughput.
Running more API instances changes the application’s combined resource use. Before adding eight copies, calculate their database connection demand and identify which controls each process applies separately.
Database connection budgets
Each running copy of the API is an instance, also called a replica. Its connection pool is a group of reusable database connections owned by that process. Suppose each instance can open a pool of ten connections. Eight instances can therefore request up to eighty connections before accounting for migrations, monitoring, and other services.
Figure 1 shows why the database must accommodate the combined pools. Each pool belongs to one API process; requests may wait when its connections are busy.
Requests may wait for a free connection even when the API process has CPU available. Budget the combined pools against the database’s available connections, leaving room for operational work.
Before adding instances, compare their proposed combined pools with the application allowance. A measured improvement on one instance leaves that shared allowance unchanged.
A proposed deployment pool
0 / 5 points
A one-instance experiment reports a lower p95 with a larger pool. The proposed deployed pools would request 56 application connections, but only 48 remain after the operational reserve. Which decision is justified?
A uniform integer allowance
After reserving operational capacity, divide the remaining connections among all replicas. If 41 connections remain for six replicas, a uniform pool of six uses 36 and leaves five unassigned. A uniform pool of seven would request 42 and exceed the allowance. Round down to a whole number of connections per replica.
Keep the available budget separate from the proposed request. The budget comes from the database limit after its reserve; the request combines every proposed pool. A proposal fits when its total does not exceed the available budget. Any unused remainder can stay unused. Use these quantities to write the budget calculation for the supplied valid inputs.
Connection plan
0 / 15 points
Complete connectionPlan(limit, reserve, replicas, poolPerReplica) in connection-budget.ts, keeping its exported name and four returned fields.
Inputs are valid bounded integers. limit, replicas and poolPerReplica are positive; reserve is between zero and limit inclusive.
| Output | Meaning |
|---|---|
available |
Database connections left for applications after the operational reserve. |
requested |
The total number of database connections requested by all proposed replica pools at their maximum sizes. |
fits |
Whether that combined connection total fits the application allowance; equality counts as fitting. |
maxPoolPerReplica |
The largest whole-number pool that every replica could have within the allowance. Leave an indivisible remainder unused. |
For example, connectionPlan(50, 7, 3, 14) gives { available: 43, requested: 42, fits: true, maxPoolPerReplica: 14 }. When the reserve consumes the limit, no application connections remain.
Choose Submit solution. Combined-budget calculations are worth 5 points; fit and the integer allowance are worth 10.
This calculation tells us whether the proposed pools fit before opening the connections.
Shared database state and process-local controls
Two API instances can read the same PostgreSQL session rows and application data. A fresh authoritative session read can therefore observe a revocation written by either instance.
Part 5’s attempt counters live in the API process’s memory. A capacity gate can also limit how many expensive password checks that process runs at once. Each additional instance therefore has its own attempt counters and hashing-capacity allowance. If one instance has reached its allowance, another can still have capacity. Construct these controls once per application instance and state their scope; adding copies changes the combined allowance.
A session revoked through one instance and an attempt counter incremented in that instance therefore have different paths to the other instance. Trace where each value is stored before deciding which change the other instance can observe.
Two replica states
0 / 5 points
Two API replicas share PostgreSQL session rows but have separate in-memory attempt counters. A session is revoked in the database, and replica A exhausts its local attempt allowance. What can replica B rely on?
The connection plan tells us whether the proposed pools fit the database’s allowance. To learn how much successful work those instances can complete, we still need measurements for their workload. Next we will make the application’s checks repeatable through continuous integration, then follow a verified change into a release.
Check Your Understanding
- Why must a connection budget include every replica and operational work?
- Why does a larger pool not establish greater successful throughput?
- Which state is shared by the database, and which supplied controls remain local to a process?
- What would you need to observe before claiming that adding replicas increased successful throughput?