n8n Queue Mode in Docker Compose: Main, Workers, Webhooks, and Runners

This post is adapted from Chapter 15 of the free book The Ultimate Guide to Deploying n8n Community Edition (2nd edition, September 2026, covering n8n 2.38). Grab the full PDF or EPUB, download the free self-hosting starter excerpt, or join the House of Loops community to swap workflows with other self-hosters.

Overview

In a single-instance stack, one n8n container does everything: it serves the editor, listens for webhooks, fires timers, and runs every execution. That is the right shape until a slow workflow blocks the editor, or a burst of webhooks queues up behind a long-running job.

Queue mode splits those jobs across processes. A main instance keeps the editor, the API, and the triggers. One or more worker instances do nothing but execute workflows. Redis sits between them as the message broker. Postgres stays where it was, holding the workflows, credentials, and execution data that both sides read and write.

You get four things from this:

• Horizontal scaling. Add workers to add throughput; remove them when the load drops.
• A responsive editor. Heavy executions no longer compete with the UI for main's event loop.
• Fault isolation. A worker that dies takes its in-flight executions with it, not your webhook endpoint.
• Separate sizing. Give workers CPU and memory. Main needs far less of both.

You will build a five-service stack — postgres, redis, n8n (main), n8n-worker, and a task runner sidecar for each n8n process — then test it, scale it, add an optional webhook processor, and update it without losing executions.

Prerequisites

• A working n8n stack on Postgres. Queue mode over SQLite is not supported. See our Docker Compose guide if you need that stack first.
• Docker Engine with Compose v2. Every command is docker compose, with a space.
• At least 4 GB RAM. Redis is small; two n8n processes plus two runner sidecars are not.
• A pinned n8n version — 2.38.1 throughout this guide. Every n8n process must run the same one.

1. How an Execution Travels

Queue mode changes where work happens, not what happens. The main instance still decides an execution should occur. It just refuses to run it.

Step by step:

1. Main handles a timer, a poller, or an incoming webhook and creates an execution record.
2. It pushes the execution ID onto a Bull queue in Redis — the ID, not the workflow.
3. The next available worker picks the ID off the queue.
4. The worker reads the workflow and its credentials from Postgres, decrypts the credentials with the shared encryption key, and runs the workflow.
5. The worker writes results back to Postgres and posts a completion message to Redis.
6. Redis notifies main, which updates the UI and, for a webhook, returns the response to the caller still holding the connection open.

Two consequences drive most of the configuration below. Redis carries references, not data, except for webhook responses on their way back. And every n8n process needs the same database and the same encryption key, because the worker, not main, decrypts your credentials.

2. Three Rules You Cannot Break

Get these wrong and the stack starts, looks healthy, and fails at the first execution.

The encryption key must be identical on every n8n container. N8N_ENCRYPTION_KEY encrypts credentials at rest in Postgres. A worker with a different key reads the row, fails to decrypt it, and the execution errors with an unhelpful message about credentials. Generate the key once with openssl rand -hex 32, put it in .env, and reference it from main, every worker, and any webhook processor. If you have been letting n8n generate the key, read it out of /home/node/.n8n/config on your existing instance first, and back it up with your database dumps.

Filesystem binary data mode is not supported in queue mode. N8N_DEFAULT_BINARY_DATA_MODE=filesystem writes binary payloads to a local disk the other containers cannot see. Use database, which stores payloads in Postgres and is the queue-mode default, or s3 if your workflows move files big enough that you do not want them in your primary database. The old in-memory default mode was removed in n8n 2.0.

Each n8n process needs its own task runner sidecar. Since 2.0, every Code node runs in an external task runner. The n8n process hosts a broker on port 5679 and the n8nio/runners container connects to it. A runner talks to one broker only, so a worker cannot borrow main's runner. One sidecar per n8n process, each pointed at its own service name, all sharing N8N_RUNNERS_AUTH_TOKEN, all on the same version tag as n8n. Our task runners guide covers this pattern in depth.

3. The .env File

# Generate with: openssl rand -hex 32
N8N_ENCRYPTION_KEY=replace-with-64-hex-characters
N8N_RUNNERS_AUTH_TOKEN=replace-with-a-long-random-string

POSTGRES_USER=n8n
POSTGRES_PASSWORD=replace-me
POSTGRES_DB=n8n

N8N_HOST=n8n.example.com
GENERIC_TIMEZONE=America/Los_Angeles

Nothing here is new except that these values now reach four containers instead of two. Keep the file at mode 0600 and out of version control.

4. The Compose File

services:
  postgres:
    image: postgres:18
    restart: unless-stopped
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - postgres_data:/var/lib/postgresql
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: ['redis-server', '--appendonly', 'yes']
    volumes:
      - redis_data:/data
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 10s
      timeout: 5s
      retries: 5

  n8n:
    image: n8nio/n8n:2.38.1
    restart: unless-stopped
    ports:
      - '127.0.0.1:5678:5678'
    environment:
      N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY}
      DB_TYPE: postgresdb
      DB_POSTGRESDB_HOST: postgres
      DB_POSTGRESDB_PORT: 5432
      DB_POSTGRESDB_DATABASE: ${POSTGRES_DB}
      DB_POSTGRESDB_USER: ${POSTGRES_USER}
      DB_POSTGRESDB_PASSWORD: ${POSTGRES_PASSWORD}
      EXECUTIONS_MODE: queue
      QUEUE_BULL_REDIS_HOST: redis
      QUEUE_BULL_REDIS_PORT: 6379
      OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS: 'true'
      N8N_DEFAULT_BINARY_DATA_MODE: database
      N8N_HOST: ${N8N_HOST}
      N8N_PROTOCOL: https
      WEBHOOK_URL: https://${N8N_HOST}/
      N8N_EDITOR_BASE_URL: https://${N8N_HOST}/
      N8N_PROXY_HOPS: 1
      GENERIC_TIMEZONE: ${GENERIC_TIMEZONE}
      TZ: ${GENERIC_TIMEZONE}
      N8N_RUNNERS_MODE: external
      N8N_RUNNERS_BROKER_LISTEN_ADDRESS: 0.0.0.0
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_NATIVE_PYTHON_RUNNER: 'true'
      EXECUTIONS_DATA_PRUNE: 'true'
      EXECUTIONS_DATA_MAX_AGE: 336
      N8N_GRACEFUL_SHUTDOWN_TIMEOUT: 60
      N8N_DIAGNOSTICS_ENABLED: 'false'
    volumes:
      - n8n_data:/home/node/.n8n
      - n8n_files:/home/node/.n8n-files
    healthcheck:
      test:
        [
          'CMD-SHELL',
          'node -e "require(''http'').get(''http://127.0.0.1:5678/healthz'',r=>process.exit(r.statusCode===200?0:1)).on(''error'',()=>process.exit(1))"',
        ]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

  runners:
    image: n8nio/runners:2.38.1
    restart: unless-stopped
    environment:
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n:5679
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_RUNNERS_AUTO_SHUTDOWN_TIMEOUT: 15
    depends_on:
      - n8n

  n8n-worker:
    image: n8nio/n8n:2.38.1
    restart: unless-stopped
    command: worker --concurrency=10
    environment:
      N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY}
      DB_TYPE: postgresdb
      DB_POSTGRESDB_HOST: postgres
      DB_POSTGRESDB_PORT: 5432
      DB_POSTGRESDB_DATABASE: ${POSTGRES_DB}
      DB_POSTGRESDB_USER: ${POSTGRES_USER}
      DB_POSTGRESDB_PASSWORD: ${POSTGRES_PASSWORD}
      EXECUTIONS_MODE: queue
      QUEUE_BULL_REDIS_HOST: redis
      QUEUE_BULL_REDIS_PORT: 6379
      QUEUE_HEALTH_CHECK_ACTIVE: 'true'
      N8N_DEFAULT_BINARY_DATA_MODE: database
      N8N_WEBHOOK_RESPONSE_RELAY_OFFLOAD_ENABLED: 'true'
      WEBHOOK_URL: https://${N8N_HOST}/
      GENERIC_TIMEZONE: ${GENERIC_TIMEZONE}
      TZ: ${GENERIC_TIMEZONE}
      N8N_RUNNERS_MODE: external
      N8N_RUNNERS_BROKER_LISTEN_ADDRESS: 0.0.0.0
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_NATIVE_PYTHON_RUNNER: 'true'
      N8N_GRACEFUL_SHUTDOWN_TIMEOUT: 60
      N8N_DIAGNOSTICS_ENABLED: 'false'
    volumes:
      - n8n_files:/home/node/.n8n-files
    healthcheck:
      test:
        [
          'CMD-SHELL',
          'node -e "require(''http'').get(''http://127.0.0.1:5678/healthz/readiness'',r=>process.exit(r.statusCode===200?0:1)).on(''error'',()=>process.exit(1))"',
        ]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

  worker-runners:
    image: n8nio/runners:2.38.1
    restart: unless-stopped
    environment:
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n-worker:5679
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_RUNNERS_AUTO_SHUTDOWN_TIMEOUT: 15
    depends_on:
      - n8n-worker

volumes:
  postgres_data:
  redis_data:
  n8n_data:
  n8n_files:

What changed, and why

• EXECUTIONS_MODE: queue on main and every worker. A worker started without it silently runs as a regular instance and never touches the queue.
• QUEUE_BULL_REDIS_HOST / QUEUE_BULL_REDIS_PORT are the only Redis variables n8n reads. Anything else Redis-shaped you have picked up from an older guide is doing nothing; delete it. Add QUEUE_BULL_REDIS_PASSWORD if your Redis requires one.
• OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS: "true" sends the executions you start by clicking Test workflow to a worker too. Without it, main still runs those itself — exactly the load you moved off main.
• N8N_DEFAULT_BINARY_DATA_MODE: database on every process, for the reason in section 2.
• No published ports on the worker. Workers take work from Redis. Nothing needs to reach them from outside; the healthcheck runs inside the container.
• QUEUE_HEALTH_CHECK_ACTIVE: "true" turns on the worker's own HTTP server at /healthz and /healthz/readiness. Readiness is the useful one: it reports whether the worker's Postgres and Redis connections are live. It listens on 5678; change QUEUE_HEALTH_CHECK_PORT if that collides.
• The healthcheck uses node, not curl. The n8n image does not ship curl, so a curl-based healthcheck fails permanently and produces a restart loop that looks like an n8n problem.
• The worker has no /home/node/.n8n volume. It does not need one: the encryption key comes from the environment. Sharing one n8n_data volume across processes invites settings-file permission errors and two processes writing one event log.
• N8N_RESTRICT_FILE_ACCESS_TO applies on every n8n container, so main and every worker mount the same n8n_files volume at /home/node/.n8n-files, matching the reference stack.
• No version: key, no container_name. version: is obsolete in Compose v2, and container_name makes the scaling in section 7 impossible.

5. Launch and Verify

docker compose up -d
docker compose ps

Wait for Postgres and Redis to report healthy, then n8n and the worker. Confirm the worker joined the pool — you want a line saying it has started and is waiting for jobs, with its concurrency:

docker compose logs n8n-worker | grep -i "waiting for"

Then confirm both runner sidecars registered:

docker compose logs runners worker-runners | grep -i "registered\|broker"

If a sidecar reports a connection refused on port 5679, the n8n process it targets is missing N8N_RUNNERS_BROKER_LISTEN_ADDRESS: 0.0.0.0. The broker binds to localhost otherwise and nothing outside the container can reach it.

6. Testing the Queue

The point of this test is to prove executions land on the worker, not on main.

1. Build a workflow with a Manual Trigger, a Code node that prints something identifiable, and a Wait node set to 30 seconds.
2. Publish it with a Webhook trigger, or just run it manually — with OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS on, a manual run goes to the queue too.
3. Watch both sides at once:

docker compose logs -f n8n

docker compose logs -f n8n-worker

Main should log that it enqueued the job; the worker should log starting and finishing it. If main logs the execution running, EXECUTIONS_MODE is missing on main, or the offload variable is.

Then push it. Fire the webhook fifteen times and watch the worker interleave them up to its concurrency limit:

for i in $(seq 1 15); do
  curl -s -o /dev/null -X POST "https://${N8N_HOST}/webhook/your-path" &
done
wait

Finally, confirm the queue drains. With everything idle:

docker compose exec redis redis-cli --scan --pattern 'bull:*' | head

A backlog that never shrinks means you need more workers, or your workers are blocked on something — usually the Postgres connection pool.

7. Scaling Workers

The obvious command works:

docker compose up -d --scale n8n-worker=3

Three worker containers now pull from the same queue. But look at what you did not scale: worker-runners is still one container pointed at http://n8n-worker:5679. That name now round-robins across three containers, and the sidecar holds one connection to whichever it resolved first. Two of your three workers have no task runner, and every Code node they pick up fails.

Scaling the sidecar does not fix it. Compose has no concept of pairing sidecar n with worker n; three runners pointed at one service name may all land on the same worker. That pairing is a scheduler's job, which is why a Kubernetes-based queue-mode deployment can express it and Compose cannot.

On Compose, scale by writing the workers out explicitly. A YAML anchor keeps the duplication to the two lines that actually differ, and the shared worker environment moves into a worker.env file:

x-worker: &worker
  image: n8nio/n8n:2.38.1
  restart: unless-stopped
  command: worker --concurrency=10
  env_file: worker.env
  depends_on:
    postgres:
      condition: service_healthy
    redis:
      condition: service_healthy

x-worker-runner: &worker-runner
  image: n8nio/runners:2.38.1
  restart: unless-stopped
  env_file: runner.env

services:
  n8n-worker-1:
    <<: *worker

  worker-1-runners:
    <<: *worker-runner
    environment:
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n-worker-1:5679
    depends_on:
      - n8n-worker-1

  n8n-worker-2:
    <<: *worker

  worker-2-runners:
    <<: *worker-runner
    environment:
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n-worker-2:5679
    depends_on:
      - n8n-worker-2

worker.env and runner.env carry everything the workers and sidecars share, so the only per-service line left is the broker URI. It is more YAML than --scale, but each worker gets exactly one runner and you can restart one worker without disturbing the others.

--scale is still fine if no workflow uses a Code node, or if you accept N8N_RUNNERS_MODE=internal on the workers. Internal mode runs Code in a child process of the worker itself; n8n does not recommend it for production, because it drops the process isolation that makes an untrusted Code node safe to run.

On concurrency. --concurrency=10 is the default and a reasonable start; n8n recommends 5 or higher. Do not chase throughput with many workers at low concurrency: each worker holds its own Postgres connections, and twenty workers at concurrency 2 exhaust the pool long before ten at concurrency 10 do.

8. Adding a Webhook Processor

Add n8n-webhook when webhook ingestion is the bottleneck — thousands of inbound requests per minute, where accepting and enqueuing the request is itself the cost. If your load is heavy workflows arriving at a modest rate, more workers help and a webhook processor does not.

n8n-webhook:
  image: n8nio/n8n:2.38.1
  restart: unless-stopped
  command: webhook
  env_file: worker.env
  environment:
    N8N_HOST: ${N8N_HOST}
    N8N_PROTOCOL: https
    WEBHOOK_URL: https://${N8N_HOST}/
    N8N_PROXY_HOPS: 1
  depends_on:
    postgres:
      condition: service_healthy
    redis:
      condition: service_healthy

It needs the same encryption key, database, Redis, and EXECUTIONS_MODE=queue, and it listens on 5678 like main. Your reverse proxy — see our self-hosting guide for the Nginx setup — routes by path: /webhook/* and /webhook-waiting/* to the processors, everything else to main. Keep /webhook-test/* on main; that is where manual test URLs resolve.

Do not put main in the webhook pool. It will pick up production traffic and slow the editor down, which is the problem you were solving. Once the processors are in place, set N8N_DISABLE_PRODUCTION_MAIN_PROCESS=true on main so it stops registering production webhooks at all.

A webhook processor does not execute workflows, so it needs no runner sidecar.

9. Large Webhook Responses

In queue mode a Respond to Webhook node runs on a worker, but the HTTP client is still connected to main. The response travels back through Redis inside a queue message, and Redis holds several copies of that message in flight.

N8N_WEBHOOK_RESPONSE_RELAY_SIZE_MAX caps the message at 64 MiB by default. Budget roughly 1.5 times the cap in Redis memory per in-flight response. Above the cap, the node fails.

Since n8n 2.34 you can offload instead of failing. The worker writes the body to binary data storage, the queue message carries a reference, and main streams the body to the client, then deletes it:

N8N_WEBHOOK_RESPONSE_RELAY_OFFLOAD_ENABLED: 'true'
N8N_WEBHOOK_RESPONSE_RELAY_SIZE_MAX: 64

Set the offload variable on workers only, and only after every main and webhook process is on 2.34 or later — an older main returns the storage reference to the client instead of the body.

Offloading needs storage every process can read. database mode works and is what this stack uses, but main loads the whole body into memory before sending and the bytes pass through your primary database. If you routinely return tens of megabytes, switch N8N_DEFAULT_BINARY_DATA_MODE to s3, which streams a chunk at a time. Filesystem mode is out here for the same reason it is out everywhere in queue mode.

10. Graceful Shutdown and Rolling Updates

When a worker gets SIGTERM it stops accepting new jobs and finishes what it has. N8N_GRACEFUL_SHUTDOWN_TIMEOUT sets how many seconds it waits before killing the process; the default is 30. Set it above your longest normal execution — 60 in the compose file above — and give Docker room to honour it:

docker compose stop --timeout 90 n8n-worker

If Docker's stop timeout is shorter than n8n's, Docker sends SIGKILL first, the graceful path never runs, and half-finished executions are marked as crashed.

QUEUE_WORKER_TIMEOUT is the deprecated old name for this setting. QUEUE_WORKER_MAX_STALLED_COUNT was removed in 2.0 and does nothing.

To update, change the pinned tag — never a floating tag here, since a partial rollout across versions is exactly what you are avoiding:

sed -i 's/2\.38\.1/2\.39\.0/g' compose.yml
docker compose pull

Then roll, workers first:

docker compose up -d --no-deps n8n-worker-1 worker-1-runners
docker compose logs -f --tail 20 n8n-worker-1
docker compose up -d --no-deps n8n-worker-2 worker-2-runners
docker compose up -d --no-deps n8n n8n-webhook runners

The queue absorbs the gap: while a worker restarts, jobs sit in Redis and the remaining workers pick them up. Main is the only service with visible downtime, and only for the few seconds it takes to restart. Read the release notes before a major version bump — some releases want main started first so it can run migrations.

11. Persistence and Backups

Postgres is the thing that matters — see our Docker Compose guide for the backup and restore commands. Back up the encryption key with it — a dump you cannot decrypt is not a backup.

Redis holds only in-flight queue state. With --appendonly yes a restart replays jobs that were enqueued but not finished, which is worth having; a backup routine is not. Lose Redis entirely and you lose the mid-flight executions and nothing else. Rebuild it empty and let the triggers fire again.

12. Redis High Availability

One Redis container is a single point of failure for execution dispatch. Two ways out, both a step up in cost.

Sentinel watches a primary and promotes a replica when it fails:

redis-sentinel:
  image: redis:7-alpine
  restart: unless-stopped
  command: ['redis-sentinel', '/etc/redis/sentinel.conf']
  volumes:
    - ./sentinel.conf:/etc/redis/sentinel.conf:ro

Run three sentinels across three hosts for a quorum that means anything. Three on one host is a rehearsal, not high availability.

Redis Cluster shards keys across nodes. Point n8n at them directly:

QUEUE_BULL_REDIS_CLUSTER_NODES: redis-1:6379,redis-2:6379,redis-3:6379

With this set, n8n creates a cluster client and ignores QUEUE_BULL_REDIS_HOST and QUEUE_BULL_REDIS_PORT. Add QUEUE_BULL_REDIS_PASSWORD and QUEUE_BULL_REDIS_TLS: "true" if Redis is off-host. At that point a managed Redis is usually cheaper.

13. Troubleshooting Common Issues

Issue	Cause	Solution
Executions stay "waiting" forever	Nothing is consuming the queue	Check docker compose logs n8n-worker for a startup line; confirm EXECUTIONS_MODE: queue on the worker, not only on main
Worker starts but runs nothing	It started as a regular instance	Add EXECUTIONS_MODE: queue to the worker and recreate it
Credential errors on the worker only	N8N_ENCRYPTION_KEY differs between processes	Set the same key from .env on every n8n container, then recreate all of them
Code nodes fail on some workers	One sidecar shared across scaled workers	Use the per-worker layout in section 7: one sidecar each, own broker URI
Runner logs "connection refused" on 5679	Broker bound to localhost	Set N8N_RUNNERS_BROKER_LISTEN_ADDRESS: 0.0.0.0 on the process the sidecar targets
Binary data missing or unreadable	N8N_DEFAULT_BINARY_DATA_MODE: filesystem	Switch every process to database or s3
"Response is too large to be sent back from the worker"	Over N8N_WEBHOOK_RESPONSE_RELAY_SIZE_MAX, offload off	Set N8N_WEBHOOK_RESPONSE_RELAY_OFFLOAD_ENABLED: "true" on the workers, or raise the limit
Restart loop, healthcheck never passes	Healthcheck uses curl, absent from the image	Use the node one-liner from section 4, or wget --spider
Executions marked crashed after a deploy	Docker's stop timeout is shorter than n8n's	Raise --timeout on docker compose stop, or lower N8N_GRACEFUL_SHUTDOWN_TIMEOUT
Slower as worker count grows	Postgres connection pool exhausted	Fewer workers at higher --concurrency; raise max_connections
Backlog grows and never drains	Not enough worker capacity	Add workers or raise concurrency, then re-measure

Further reading

• Enable queue mode: https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/enable-queue-mode
• Queue mode environment variables: https://docs.n8n.io/deploy/host-n8n/configure-n8n/basic-configuration/use-environment-variables/queue-mode
• Set up task runners: https://docs.n8n.io/deploy/host-n8n/configure-n8n/set-up-task-runners
• Handle binary data: https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/handle-binary-data
• Install with Docker Compose: https://docs.n8n.io/deploy/host-n8n/install-options/install-using-docker-compose
• n8n 2.0 breaking changes: https://docs.n8n.io/changelog/v20-breaking-changes

---

Got a queue-mode stack that will not cooperate? Post it in the House of Loops community and get eyes on it fast, or join the free weekly workflow list and get one delivered to your inbox every week.