Why Your Django App Runs Out of Postgres Connections

The symptom is always the same. Everything is fine, traffic rises slightly, and then the site starts throwing:

FATAL: sorry, too many clients already

The instinct is to raise max_connections in postgresql.conf, restart, and move on. That works for about a month, and it is almost always the wrong lever. Understanding why requires knowing one thing about how Django talks to Postgres.

Django holds a connection per process, not per request

Django's database layer is not pooled. Each worker process opens its own connection on first query and — depending on CONN_MAX_AGE — keeps it. That connection belongs to that process whether it is running a query or sitting idle waiting for the next request.

So your connection count is not a function of traffic. It is a function of process count, and you can calculate it exactly:

gunicorn --workers 5 --threads 4    # 5 processes x 4 threads = 20
celery  --concurrency 8            # 8 more
celery beat                        # 1
a management command someone ran   # 1
                                   # -------------------------------
                                   # 30 connections, at zero traffic

Now run that on three app servers and you are at 90 permanent connections before a single visitor arrives. Postgres ships with max_connections = 100.

Why raising max_connections hurts

A Postgres connection is a forked backend process, not a lightweight handle. Each one costs memory even when idle, and each one has its own work_mem allowance — which is per operation, not per connection, so a single query doing three sorts can allocate three times that figure.

More expensive than the memory is the contention. Postgres was not designed for thousands of backends; past a few hundred, the lock manager and the process scheduler start costing you more than the extra concurrency buys. Throughput goes down while your connection count goes up, which is a deeply confusing thing to debug if you believe more connections means more capacity.

The real fix is to stop equating "an application process exists" with "a database backend must exist".

PgBouncer in transaction mode

PgBouncer sits between Django and Postgres. Applications connect to it, and it maintains a much smaller pool of real Postgres connections that it hands out on demand.

The mode matters enormously:

In transaction mode, an idle Django worker holds no Postgres backend at all. Thirty application processes that are mostly idle might need six real connections. That is the entire trick.

; pgbouncer.ini
[databases]
appdb = host=127.0.0.1 port=5432 dbname=appdb

[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction

; real Postgres connections per (database, user) pair
default_pool_size = 20
; clients allowed to queue in front of that pool
max_client_conn = 300
; emergency headroom when the pool is saturated
reserve_pool_size = 5
reserve_pool_timeout = 3

The Django settings that must change

This is where people get bitten. Point Django at port 6432 and the job is not done:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "appdb",
        "HOST": "127.0.0.1",
        "PORT": "6432",          # pgbouncer, not postgres
        # Critical: persistent connections and transaction pooling
        # are the same optimisation. Doing both means Django pins
        # a pooled connection forever and you are back where you started.
        "CONN_MAX_AGE": 0,
        "OPTIONS": {
            # psycopg3 uses server-side prepared statements by default.
            # Those are bound to a backend that transaction pooling will
            # hand to someone else. Disable them.
            "prepare_threshold": None,
        },
    }
}

The CONN_MAX_AGE = 0 line looks wrong to people who have read the Django performance docs, and it is the single most common misconfiguration I find. Both settings solve the same problem. Enabling both means each Django process grabs a PgBouncer client slot and never releases it, which reproduces the original exhaustion one layer higher up.

What transaction pooling takes away

Transaction pooling means you get a different backend for each transaction, so anything that stores state on a connection stops working:

• LISTEN/NOTIFY — the listener needs a persistent backend. Give it a separate direct connection on port 5432.
• Session-level advisory locks — pg_advisory_lock() outlives the transaction and will be released on a connection you no longer hold. Use pg_advisory_xact_lock(), which is transaction-scoped.
• SET statements outside a transaction — including per-tenant search_path tricks. If you do schema-per-tenant this way, read the implications carefully before switching.
• Cursors held open across transactions — including some uses of Django's .iterator() with server-side cursors. Set DISABLE_SERVER_SIDE_CURSORS = True.

Long-running migrations are fine, since they run in a transaction. The thing to watch for is code that assumed "same connection" without saying so.

Checking it actually worked

PgBouncer exposes an admin console over the same port. This is the first place to look when something feels wrong:

psql -p 6432 -U pgbouncer pgbouncer

-- one row per pool: cl_active clients, sv_active servers,
-- and maxwait, which is the number that matters
SHOW POOLS;

-- per-server request rates and mean query time
SHOW STATS;

maxwait is the diagnostic. It is how long the oldest queued client has been waiting for a server connection. If it is consistently above zero, your default_pool_size is genuinely too small for the workload — that is the point at which adding capacity is the right answer, and now you can size it from evidence rather than from a stack trace.

On the Postgres side, the count you want to watch is simply:

SELECT count(*), state FROM pg_stat_activity GROUP BY state;

Before pooling, this is dominated by idle. After, idle should be small and roughly equal to your pool size. That shift — from dozens of idle backends to a handful of busy ones — is the whole benefit, and it usually arrives without touching a line of application code.