Files
compose-files/postgresql
admin dbabc69c2a
ci/woodpecker/push/deploy Pipeline was successful
cutover: surface stderr in Phase 3's canary-propagation check (diagnostic fix)
Phase 3's "Canary row did not propagate to <replica> within 5s" failure
has now recurred twice (2026 run #4, root-caused as the Phase 2 lag-check
bug; and the run immediately after the Ceph IOPS fix, cause unconfirmed)
with genuinely healthy Phase 1/2 beforehand both times. The per-attempt
SELECT against the replica was discarding stderr entirely (2>/dev/null),
so a real connection/auth error and a genuine multi-second replication
delay were indistinguishable in the log — both just showed "row not
found".

Each of the 5 propagation-check attempts now captures stderr to
/tmp/cutover_phase3_attempt_<N>.stderr (mirrors the existing Phase 8
pattern) and echoes result+stderr into the log per-attempt. The final
trigger_rollback() message on failure includes the last non-empty
stderr seen directly in the FAIL line. The initial canary write's
stderr is also no longer discarded. No behavior change to timing/retry
counts — purely additive diagnostics.

See ADR-0001 note, Session Update 10 (to be added).
2026-08-08 13:07:53 -07:00
..