traefik: add log rotation for access.log / traefik.log #23

Merged
AVB merged 9 commits from fix/traefik-log-rotation into main 2026-09-12 00:29:28 -07:00
Owner

Problem

traefik/traefik.yaml runs Traefik with --accesslog.filePath and
--log.filePath pointing at /traefik/logs/{access,traefik}.log
(--log.level=DEBUG). Neither file has any rotation — Traefik has no
native rotate-on-size or reopen-on-signal support, and Docker's
json-file log-driver max-size/max-file only caps stdout, not files
written directly via these flags.

Found access.log at ~15.5GB, unrotated, on /volume1/docker-root
(CephFS, 82-84% used) while investigating an unrelated Uptime Kuma
WebSocket-flapping incident. Correction/update: that incident's actual
root cause turned out to be Uptime Kuma monitor misconfigurations (a
Postgres monitor with a null-reference bug, plus several monitors failing
TLS validation against self-signed/internal-IP certs) — confirmed resolved
by the operator independently of this change. This log rotation fix is
not causally tied to that incident; it's still worth merging as
standalone disk/IO hygiene given the file size and shared-storage pressure.

Why this needed its own design (not just "add a logrotate conf")

traefik_reverse-proxy runs Swarm mode: global — one Traefik
instance on each of docker-1/2/3, all three writing to the same
physical file
via the shared CephFS mount /volume1/docker/traefik -> /traefik. That rules out standard signal-based rotation (no reopen
handler in Traefik, and no clean way to signal 3 independent per-node
containers in lockstep) and rules out running logrotate on just one node
(silent SPOF if that node is down).

Solution:

  1. copytruncate — all three Traefik processes keep appending to the
    same inode, no signaling required.
  2. Cron on all three nodes, coordinated via a shared flock + shared
    logrotate state file (both on the same CephFS mount) — whichever node's
    cron fires first does the rotation; no single node is a SPOF.
  3. Size-triggered (250M), not calendar-triggered — this file can grow
    fast under bursts, so a daily rule alone wouldn't bound it well.

Full rationale in traefik/logrotate/README.md.

What's included

  • traefik/logrotate/traefik-logs.conf — logrotate config
  • traefik/logrotate/traefik-logrotate.sh — cron-invoked wrapper (flock + shared state)
  • traefik/logrotate/install.sh — one-time per-node installer
  • traefik/logrotate/README.md — full rationale, install steps, and manual bootstrap instructions for shrinking the existing 15.5GB file (deliberately not automated — needs disk-headroom awareness given /volume1/docker-root is at 82%+)

What's NOT included (by design)

  • No change to traefik.yaml / no stack redeploy — this is host-level
    cron/logrotate config that stack-deploy.sh has no reach into (it only
    touches Swarm services). Zero risk to the running traefik stack.
  • No change to --log.level=DEBUG — flagged in the README as related
    follow-up debt, but lowering it means a real modification + redeploy of
    a high-blast-radius stack, so it's out of scope here.
  • No automatic shrink of the existing 15.5GB file — see manual bootstrap
    steps in the README; needs a disk-space check first (~14G free at last
    check, copytruncate needs roughly the source file's size in headroom).

Rollout (manual, once per node, not via stack-deploy.sh)

ssh root@<192.168.4.31|.32|.33>
cd /volume1/docker/compose-files
bash deploy/git-guard.sh
sudo bash traefik/logrotate/install.sh

Risk

Low — no Swarm service touched, no traefik.yaml change, additive-only
host config. Worst case if misconfigured: rotation doesn't happen (status
quo), not a routing outage.

## Problem `traefik/traefik.yaml` runs Traefik with `--accesslog.filePath` and `--log.filePath` pointing at `/traefik/logs/{access,traefik}.log` (`--log.level=DEBUG`). Neither file has any rotation — Traefik has no native rotate-on-size or reopen-on-signal support, and Docker's `json-file` log-driver `max-size`/`max-file` only caps stdout, not files written directly via these flags. Found `access.log` at **~15.5GB, unrotated**, on `/volume1/docker-root` (CephFS, 82-84% used) while investigating an unrelated Uptime Kuma WebSocket-flapping incident. **Correction/update:** that incident's actual root cause turned out to be Uptime Kuma monitor misconfigurations (a Postgres monitor with a null-reference bug, plus several monitors failing TLS validation against self-signed/internal-IP certs) — confirmed resolved by the operator independently of this change. This log rotation fix is *not* causally tied to that incident; it's still worth merging as standalone disk/IO hygiene given the file size and shared-storage pressure. ## Why this needed its own design (not just "add a logrotate conf") `traefik_reverse-proxy` runs Swarm **`mode: global`** — one Traefik instance on **each** of docker-1/2/3, all three writing to the **same physical file** via the shared CephFS mount `/volume1/docker/traefik -> /traefik`. That rules out standard signal-based rotation (no reopen handler in Traefik, and no clean way to signal 3 independent per-node containers in lockstep) and rules out running logrotate on just one node (silent SPOF if that node is down). Solution: 1. **`copytruncate`** — all three Traefik processes keep appending to the same inode, no signaling required. 2. **Cron on all three nodes**, coordinated via a shared `flock` + shared logrotate state file (both on the same CephFS mount) — whichever node's cron fires first does the rotation; no single node is a SPOF. 3. **Size-triggered (250M)**, not calendar-triggered — this file can grow fast under bursts, so a `daily` rule alone wouldn't bound it well. Full rationale in `traefik/logrotate/README.md`. ## What's included - `traefik/logrotate/traefik-logs.conf` — logrotate config - `traefik/logrotate/traefik-logrotate.sh` — cron-invoked wrapper (flock + shared state) - `traefik/logrotate/install.sh` — one-time per-node installer - `traefik/logrotate/README.md` — full rationale, install steps, and manual bootstrap instructions for shrinking the *existing* 15.5GB file (deliberately **not** automated — needs disk-headroom awareness given `/volume1/docker-root` is at 82%+) ## What's NOT included (by design) - No change to `traefik.yaml` / no stack redeploy — this is host-level cron/logrotate config that `stack-deploy.sh` has no reach into (it only touches Swarm services). Zero risk to the running `traefik` stack. - No change to `--log.level=DEBUG` — flagged in the README as related follow-up debt, but lowering it means a real modification + redeploy of a high-blast-radius stack, so it's out of scope here. - No automatic shrink of the existing 15.5GB file — see manual bootstrap steps in the README; needs a disk-space check first (~14G free at last check, `copytruncate` needs roughly the source file's size in headroom). ## Rollout (manual, once per node, not via stack-deploy.sh) ```bash ssh root@<192.168.4.31|.32|.33> cd /volume1/docker/compose-files bash deploy/git-guard.sh sudo bash traefik/logrotate/install.sh ``` ## Risk Low — no Swarm service touched, no `traefik.yaml` change, additive-only host config. Worst case if misconfigured: rotation doesn't happen (status quo), not a routing outage.
Bot added 9 commits 2026-09-12 00:22:15 -07:00
Traefik has no built-in log rotation and no SIGUSR1/reopen handling.
--accesslog.filePath and --log.filePath (traefik.yaml) write directly
to /volume1/docker/traefik/logs, which had grown to 15.5GB unrotated,
contributing to disk pressure (docker-2 CephFS at 82%) and adding
write latency to every request through Traefik.
reverse-proxy runs Swarm mode:global (one instance per node), all
writing to the same CephFS file. This wrapper uses a shared flock +
shared logrotate state file (both also on the CephFS mount) so cron
on docker-1/2/3 can run independently without racing or double-rotating.
AVB merged commit c3c12ef0f9 into main 2026-09-12 00:29:28 -07:00
AVB deleted branch fix/traefik-log-rotation 2026-09-12 00:29:29 -07:00
Sign in to join this conversation.