Commit Graph
19 Commits
Author SHA1 Message Date
admin 6af2633936 fix(ai): write AI_-prefixed AWS key names to ai.env, not plain names
ai.yaml's litellm service (as of commit 37ed671a, "Change AWS keys to
use Woodpecker Secrets") references ${AI_AWS_ACCESS_KEY_ID} /
${AI_AWS_SECRET_ACCESS_KEY} and renders them into the container as
plain AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY. The ai) provisioning
case added in the earlier secrets-migration PR wrote the plain
(unprefixed) names into ai.env instead, which would leave
${AI_AWS_ACCESS_KEY_ID} unresolved at compose-render time (renders
empty) -- silently breaking Bedrock auth in litellm on the next ai
stack deploy.

Fixed both the grep -vE exclusion pattern and the two printf lines to
use the AI_-prefixed names. All other migrated vars in ai.yaml use
plain names and are unaffected.

No other changes in this file.
2026-08-25 23:42:39 -07:00
admin 5188aef250 hotfix: restore $${VAR} double-dollar escaping for all secret-backed vars
The AI secrets migration PR (#4) was a full-file rewrite of deploy.yml.
That rewrite mechanically dropped one $ from EVERY $${VAR} occurrence in
the file, not just the new AI additions -- silently reverting all
pre-existing secret references (SWARM_MANAGER_IP, IMMICH_*, GIT_*,
POSTGRESQL_*, VAULTWARDEN_*, ENTERTAINMENT_*, etc.) to single-dollar
form. Per this file's own header comment, Woodpecker blanks single-dollar
braced refs at compile time since secrets aren't in that variable map --
this is the exact "SWARM_MANAGER_IP secret is empty" failure mode
documented above, and it fired immediately on the first push after #4
merged.

Impact: provision-secrets/deploy/verify all failed at their first
if-empty guard and exited before any ssh/scp/rsync ran. No live secret,
service, or deployed stack was touched -- this was a CI-only outage.

Fix: restored $${VAR} for every secret-backed reference throughout the
file. CI_PIPELINE_FILES / CI_COMMIT_MESSAGE stay single-dollar (correct
-- those are Woodpecker compile-time metadata, not secrets). The \$FILE
/ \$TMP backslash-escaping inside the ai) case's remote SSH command is
unrelated and was already correct (it protects those local-to-remote
vars from expanding before the SSH payload is sent).

This is a straight revert-of-the-regression -- no new secrets, no logic
changes beyond restoring the escaping.
2026-08-25 23:27:47 -07:00
admin 2959721d3c ai: migrate AWS/LiteLLM/OpenWebUI/OAuth secrets from ai.env to Woodpecker secrets
Adds 8 new from_secret-backed env vars to provision-secrets and rewrites
the `ai)` case to do a targeted update of only those 8 keys in the
remote ai/ai.env via grep -v + printf (no sed, safe for values containing
/, $, &, etc). All other lines in ai.env (MCPO_API_KEY, OAUTH_CLIENT_ID,
WEBUI_URL, etc.) are left completely untouched -- MCPO_API_KEY migration
is deferred to a follow-up per plan, and this change never reads or
writes that value.

New secrets required in Woodpecker (Settings -> Secrets) before merge:
  ai_aws_access_key_id
  ai_aws_secret_access_key
  ai_litellm_master_key
  ai_litellm_salt_key
  ai_litellm_db_password
  ai_webui_secret_key
  ai_open_webui_database_url
  ai_oauth_client_secret
2026-08-25 22:40:05 -07:00
AVB 129badada4 Add secrets (and add postgresql) back so they are seen as utility folders and do deploy
ci/woodpecker/push/deploy Pipeline was successful
2026-07-28 08:36:56 -07:00
AVB e1f569b3ec Add secrets folder to utility folders list so it isn't auto-deployed as a stack
ci/woodpecker/push/deploy Pipeline was successful
2026-07-28 08:25:35 -07:00
AVB b5767ec7d9 revert 8719c398fd
ci/woodpecker/push/deploy Pipeline was successful
revert Update .woodpecker/deploy.yml
2026-07-28 08:24:29 -07:00
AVB 8719c398fd Update .woodpecker/deploy.yml
ci/woodpecker/push/deploy Pipeline was successful
2026-07-28 08:22:57 -07:00
admin 0f26d31a02 postgresql: add replication + patroni Woodpecker secrets to provision-secrets step (ADR-0001 Phase 1)
ci/woodpecker/push/deploy Pipeline was canceled
2026-07-28 07:20:35 -07:00
admin e0a6b7d233 deploy.yml: fix pipeline compile error 'missing closing brace' -- the documentation comment itself contained a literal dollar-brace sequence with invalid contents (dollar-brace CI_...), which Woodpecker's substitution engine parses even inside comments. Rewrote header to describe the rules in words without any literal dollar-brace sequences. Also removed backtick-quoted dollar-brace remnants from the vaultwarden case comment.
ci/woodpecker/push/deploy Pipeline was successful
2026-07-27 13:35:13 -07:00
admin 6757b0dfe1 deploy.yml: THE ACTUAL ROOT CAUSE FIX. Woodpecker substitutes ${VAR} at pipeline-compile time from its CI metadata map, where secrets do not exist -- so every braced reference to a secret-backed env var (SWARM_MANAGER_IP, all provisioning values) was silently blanked before the shell ever ran. Proven by debug output showing eval "val=\$$v" arriving as val=\$v ($$ unescaped to $) and lengths matching var NAMES. Fix: escape all runtime secret refs as $${VAR}; keep ${CI_PIPELINE_FILES} braced (compile-time is correct for CI metadata). Removes flawed debug step; adds header documenting the substitution rules. 2026-07-27 12:50:21 -07:00
admin 7a81bc73bd deploy.yml: add temporary debug-secrets-all step declaring the identical 31-secret environment block as provision-secrets, printing byte length of every var (never values). Runs as its own step so it completes even when provision-secrets fails, telling us in one run whether all secrets are empty, only some, or only swarm_manager_ip.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 12:39:53 -07:00
admin 332708ad41 deploy.yml: rename env var alias 3DPRINT_DB_PASSWORD -> PRINT3D_DB_PASSWORD in provision-secrets. Shell/env var identifiers cannot start with a digit (POSIX); this invalid key was likely poisoning the entire environment block for this step, explaining why SWARM_MANAGER_IP read empty specifically in provision-secrets/deploy/verify (which all declare 20-30 secrets) while working fine in the isolated 2-secret debug step. The underlying Woodpecker secret name (3dprint_db_password) is unchanged -- only the YAML alias and its $-reference were renamed.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 12:12:39 -07:00
admin 01333893f1 deploy.yml: remove temporary debug-secrets step. Secret injection confirmed working (SSH_KEY=516 bytes, SWARM_MANAGER_IP=12 bytes) after repo was switched back to public. Root cause of the entire multi-day secret-provisioning saga: (1) CI_COMMIT_CHANGED_FILES renamed to CI_PIPELINE_FILES in Woodpecker 3.16, (2) --no-interpolate flag was misplaced, (3) dotfolders matched as bogus stack names, (4) private-repo flip broke Woodpecker's secret injection entirely. All four now fixed.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 11:57:02 -07:00
admin d89cfbf055 deploy.yml: add temporary debug-secrets step (prints byte length of SSH_KEY and SWARM_MANAGER_IP only, never values) to determine whether secret injection is failing repo-wide or just for swarm_manager_ip. Repo has been switched back to public.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 11:54:54 -07:00
admin c511b2b607 deploy.yml: fix two bugs surfaced now that CI_PIPELINE_FILES parsing actually works: (1) --no-interpolate was placed before the config subcommand instead of after, causing 'unknown flag' on any real folder-stack validate; (2) folder-detection regex matched dotfolders like .woodpecker as bogus stack names, causing SSH/SCP attempts with empty SWARM_MANAGER_IP. Also added explicit empty-var guards for SWARM_MANAGER_IP in provision-secrets/deploy/verify so future secret misconfig fails fast with a clear message instead of a cryptic ssh error.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 11:03:35 -07:00
admin 206f6ff103 deploy.yml: fix root cause - CI_COMMIT_CHANGED_FILES does not exist in Woodpecker 3.16; the correct variable is CI_PIPELINE_FILES (JSON array format, e.g. [\"a/b.yaml\",\"c.yaml\"]). Updated all 5 usages (validate, provision-secrets x2, deploy, verify, notify-success) to read CI_PIPELINE_FILES and strip [ ] \" before splitting on comma. Removes temporary debug-env step.
ci/woodpecker/push/deploy Pipeline failed
2026-07-27 10:51:37 -07:00
admin 2b6286a1c9 deploy.yml: add temporary debug-env step to diagnose CI_COMMIT_CHANGED_FILES not populating despite Woodpecker UI showing correct changed files. No logic changes to existing steps -- remove once root cause is found.
ci/woodpecker/push/deploy Pipeline was successful
2026-07-27 10:31:10 -07:00
admin 6ff6260b2f deploy.yml: fix vaultwarden secret provisioning (was bypassing shared helper, silently failing on rm-while-in-use). Provision under versioned name vaultwarden_database_url_v2; compose-side remap happens in a follow-up commit.
ci/woodpecker/push/deploy Pipeline was successful
2026-07-27 08:50:16 -07:00
admin 270489b207 fix(ci): move .woodpecker.yml -> .woodpecker/deploy.yml
Root cause of Woodpecker push pipelines silently no-op'ing since
2026-07-18: Woodpecker's config resolution is
  .woodpecker/*.{yaml,yml} -> .woodpecker.yaml -> .woodpecker.yml
Once .woodpecker/renovate.yml existed, the root .woodpecker.yml was
entirely ignored. Every push webhook hit renovate.yml's `when: event:
cron` filter, matched zero steps, and Woodpecker logged
"ignoring hook: 'when' filters filtered out all steps" instead of
running deploy.

Moving the push pipeline into .woodpecker/deploy.yml restores it as
an independent workflow alongside renovate.yml -- each file's own
`when:` filter now applies correctly (push -> deploy.yml, cron ->
renovate.yml).
2026-07-25 22:00:49 -07:00