⚠️ Hotfix — I (the assistant) caused this regression
The AI secrets migration PR (#4) was a full-file rewrite of deploy.yml. In reproducing the file, one $ was mechanically dropped from every$${VAR} occurrence in the file — not just the new AI_* additions, but every pre-existing secret reference too (SWARM_MANAGER_IP, IMMICH_*, GIT_*, POSTGRESQL_*, VAULTWARDEN_*, ENTERTAINMENT_*, etc.).
Per this file's own header comment (written after a previous, unrelated incident with the same root cause): Woodpecker pre-processes the whole YAML text and blanks any single-dollar ${VAR} braced reference at compile time unless it's one of Woodpecker's own CI_* metadata vars — secrets aren't in that map. That's exactly the "SWARM_MANAGER_IP secret is empty" failure you hit immediately on the first push after #4 merged.
Impact
provision-secrets, deploy, and verify steps all failed at their first if [ -z ... ] guard and exited before any ssh/scp/rsync ran.
No live secret, service, or deployed stack was touched. Confirmed by checking ai/ai.env on docker-2 (unchanged, old values intact) and docker stack ps ai (all 7 services still running from before this ever merged).
This was a CI-only outage: nothing could auto-provision or auto-deploy via Woodpecker until fixed.
Fix
Restored $${VAR} for every secret-backed reference throughout the file. Verified line-by-line against the previous (pre-#4) version. ${CI_PIPELINE_FILES} / ${CI_COMMIT_MESSAGE} correctly remain single-dollar — those are Woodpecker compile-time metadata, not secrets, and were never part of the bug.
Also added a note to the file's header comment documenting this specific incident, plus a process lesson: grep for the literal string $${ and diff the count against the previous version before ever committing a full-file rewrite of this pipeline — a mechanical/global find-replace error like this is exactly the kind of thing a line-count diff would have caught before merge.
Verification after merge
Push should succeed through provision-secrets this time (all 8 ai_* secrets are already confirmed present in Woodpecker). Watch for:
provision-secrets no longer erroring on SWARM_MANAGER_IP secret is empty
ai) case logs [OK] ai/ai.env secrets updated
No unexpected diffs/restarts on unrelated stacks (vaultwarden, immich, etc.) — their secret values haven't changed, only the escaping that lets the pipeline read them correctly again.
## ⚠️ Hotfix — I (the assistant) caused this regression
The AI secrets migration PR (#4) was a full-file rewrite of `deploy.yml`. In reproducing the file, one `$` was mechanically dropped from **every** `$${VAR}` occurrence in the file — not just the new `AI_*` additions, but every pre-existing secret reference too (`SWARM_MANAGER_IP`, `IMMICH_*`, `GIT_*`, `POSTGRESQL_*`, `VAULTWARDEN_*`, `ENTERTAINMENT_*`, etc.).
Per this file's own header comment (written after a previous, unrelated incident with the same root cause): Woodpecker pre-processes the whole YAML text and blanks any single-dollar `${VAR}` braced reference at compile time unless it's one of Woodpecker's own `CI_*` metadata vars — secrets aren't in that map. That's exactly the "SWARM_MANAGER_IP secret is empty" failure you hit immediately on the first push after #4 merged.
## Impact
- `provision-secrets`, `deploy`, and `verify` steps all failed at their first `if [ -z ... ]` guard and exited before any `ssh`/`scp`/`rsync` ran.
- **No live secret, service, or deployed stack was touched.** Confirmed by checking `ai/ai.env` on docker-2 (unchanged, old values intact) and `docker stack ps ai` (all 7 services still running from before this ever merged).
- This was a **CI-only outage**: nothing could auto-provision or auto-deploy via Woodpecker until fixed.
## Fix
Restored `$${VAR}` for every secret-backed reference throughout the file. Verified line-by-line against the previous (pre-#4) version. `${CI_PIPELINE_FILES}` / `${CI_COMMIT_MESSAGE}` correctly remain single-dollar — those are Woodpecker compile-time metadata, not secrets, and were never part of the bug.
Also added a note to the file's header comment documenting this specific incident, plus a process lesson: **grep for the literal string `$${` and diff the count against the previous version before ever committing a full-file rewrite of this pipeline** — a mechanical/global find-replace error like this is exactly the kind of thing a line-count diff would have caught before merge.
## Verification after merge
Push should succeed through `provision-secrets` this time (all 8 `ai_*` secrets are already confirmed present in Woodpecker). Watch for:
- `provision-secrets` no longer erroring on `SWARM_MANAGER_IP secret is empty`
- `ai)` case logs `[OK] ai/ai.env secrets updated`
- No unexpected diffs/restarts on unrelated stacks (vaultwarden, immich, etc.) — their secret values haven't changed, only the escaping that lets the pipeline read them correctly again.
The AI secrets migration PR (#4) was a full-file rewrite of deploy.yml.
That rewrite mechanically dropped one $ from EVERY $${VAR} occurrence in
the file, not just the new AI additions -- silently reverting all
pre-existing secret references (SWARM_MANAGER_IP, IMMICH_*, GIT_*,
POSTGRESQL_*, VAULTWARDEN_*, ENTERTAINMENT_*, etc.) to single-dollar
form. Per this file's own header comment, Woodpecker blanks single-dollar
braced refs at compile time since secrets aren't in that variable map --
this is the exact "SWARM_MANAGER_IP secret is empty" failure mode
documented above, and it fired immediately on the first push after #4
merged.
Impact: provision-secrets/deploy/verify all failed at their first
if-empty guard and exited before any ssh/scp/rsync ran. No live secret,
service, or deployed stack was touched -- this was a CI-only outage.
Fix: restored $${VAR} for every secret-backed reference throughout the
file. CI_PIPELINE_FILES / CI_COMMIT_MESSAGE stay single-dollar (correct
-- those are Woodpecker compile-time metadata, not secrets). The \$FILE
/ \$TMP backslash-escaping inside the ai) case's remote SSH command is
unrelated and was already correct (it protects those local-to-remote
vars from expanding before the SSH payload is sent).
This is a straight revert-of-the-regression -- no new secrets, no logic
changes beyond restoring the escaping.
AVB
merged commit 27df7cf58f into main2026-08-25 23:34:08 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
⚠️ Hotfix — I (the assistant) caused this regression
The AI secrets migration PR (#4) was a full-file rewrite of
deploy.yml. In reproducing the file, one$was mechanically dropped from every$${VAR}occurrence in the file — not just the newAI_*additions, but every pre-existing secret reference too (SWARM_MANAGER_IP,IMMICH_*,GIT_*,POSTGRESQL_*,VAULTWARDEN_*,ENTERTAINMENT_*, etc.).Per this file's own header comment (written after a previous, unrelated incident with the same root cause): Woodpecker pre-processes the whole YAML text and blanks any single-dollar
${VAR}braced reference at compile time unless it's one of Woodpecker's ownCI_*metadata vars — secrets aren't in that map. That's exactly the "SWARM_MANAGER_IP secret is empty" failure you hit immediately on the first push after #4 merged.Impact
provision-secrets,deploy, andverifysteps all failed at their firstif [ -z ... ]guard and exited before anyssh/scp/rsyncran.ai/ai.envon docker-2 (unchanged, old values intact) anddocker stack ps ai(all 7 services still running from before this ever merged).Fix
Restored
$${VAR}for every secret-backed reference throughout the file. Verified line-by-line against the previous (pre-#4) version.${CI_PIPELINE_FILES}/${CI_COMMIT_MESSAGE}correctly remain single-dollar — those are Woodpecker compile-time metadata, not secrets, and were never part of the bug.Also added a note to the file's header comment documenting this specific incident, plus a process lesson: grep for the literal string
$${and diff the count against the previous version before ever committing a full-file rewrite of this pipeline — a mechanical/global find-replace error like this is exactly the kind of thing a line-count diff would have caught before merge.Verification after merge
Push should succeed through
provision-secretsthis time (all 8ai_*secrets are already confirmed present in Woodpecker). Watch for:provision-secretsno longer erroring onSWARM_MANAGER_IP secret is emptyai)case logs[OK] ai/ai.env secrets updated