2026-09-03 incident: the hand-maintained ai) provisioning case (grep‑v + printf line surgery on ai/ai.env) dropped/duplicated env keys during manual edits, breaking every ai-stack service. Manual host-side recovery forked ai.yaml/ai.env, which then (correctly) tripped git-guard's dirty-tree + secret-scan and blocked all subsequent ai deploys. Separately: connection-string structure (e.g. the LiteLLM postgres URL shape) was visible in deploy.yml plaintext, and adding a secret required editing fragile shell heredocs.
Note: main's current ai) case is still broken (line ~354: printf 'DATABASE_URL%s\n' is missing its =; POSTGRES_PASSWORD and WEBUI_SECRET_KEY are each printed twice; one line commented out). Do not push any ai/** change to main before this merges.
New architecture (data-driven, migrated one stack per PR — ai first)
File
Role
secrets/secrets-map.yaml
Data only — no code, no values. Maps stack → env template + Docker-secret names → CI var names
ai/ai.env.template
Authoritative full env file. Non-secret config as literals; secrets as placeholders. Rendered whole every run — keys can never silently go missing again
deploy/provision-stack.py
Generic renderer/provisioner. Hard-fails listing NAMES of missing/empty vars; hard-fails on unresolved placeholders; ships env + Docker secrets via ssh stdin only (values never on a command line, unlike create-secrets.sh); never prints a value
.woodpecker/deploy.yml
ai) case is now one line: python3 deploy/provision-stack.py ai. No value structure remains in this file for ai. All other cases/steps byte-for-byte unchanged from the pre-incident baseline
Adding a future secret = Woodpecker UI + 2-line from_secret declaration (Woodpecker v3 mandates explicit declaration; no expose-all exists) + 1 line in template/manifest. Zero shell edits.
Prerequisites before merging (blocking)
Create Woodpecker secret ai_mcpo_api_key in compose-files — value = the current MCPO_API_KEY from the host ai.env. The template now owns this key; provision will hard-fail (by design) if it's absent.
Confirm these hold full, correct values (the template consumes them as complete opaque values): ai_litellm_database_url (entire postgres URL — structure no longer lives in git), ai_litellm_postgres_password, ai_oauth_client_secret, ai_webui_secret_key, ai_open_webui_database_url, ai_aws_access_key_id/secret, ai_litellm_master_key/salt_key.
These hand-added Woodpecker secrets become unused after merge (config values are now template literals — you can delete them at leisure): litellm_database_migrations, litellm_modify_params, ai_aws_region_name, ai_open_web_ui_enable_oauth_signup, _oauth_merge_accounts_by_email, _oauth_provider_name, _openid_provider_url, _oauth_client_id, _oauth_scopes, _openid_redirect_uri, ai_litellm_db_password.
Host-state cleanup already done (by agent, on docker-2)
ai/ai.env.work + ai/ai.yaml.save moved to /root/ai-stack-backup-20260903/ (mode 600, outside the repo tree); current ai.env snapshotted there too
ai/ai.yaml restored to git HEAD (git checkout -- ai/ai.yaml) — working tree now clean, git-guard will pass
Running services untouched (all 6 ai_* services 1/1 / 2/2 at time of writing)
What happens on merge
changed-files → ai + secrets + deploy: provision-stack.py ai renders complete ai.env (fixing the duplicate-key drift in the current host file) + re-verifies flowagent secrets → stack-deploy.sh ai redeploys. Any missing secret aborts before deploy with named vars.
Reviewer checklist
ai_mcpo_api_key created in Woodpecker
Template literals sanity-check (region, OAuth URLs/ID/scopes, WEBUI_URL) against the known-good host ai.env
deploy.yml diff: confirm only the header comment, env block, apk line, and ai) case changed; all legacy $${ escapes intact
After merge: docker service ls --filter name=ai_ all up; docker service logs ai_mcpo --tail 50 shows flowagent az login outcome
Follow-ups (separate work)
Rotation list now 9 credentials: original 6 + LITELLM_MASTER_KEY, LITELLM_SALT_KEY, LiteLLM DB password (exposed in chat transcript during incident recovery, 2026-09-03)
Migrate remaining stacks (git, immich, n8n, vaultwarden, postgresql, entertainment, ...) to the manifest, one PR each
## Why
2026-09-03 incident: the hand-maintained `ai)` provisioning case (grep‑v + printf line surgery on `ai/ai.env`) dropped/duplicated env keys during manual edits, breaking every ai-stack service. Manual host-side recovery forked `ai.yaml`/`ai.env`, which then (correctly) tripped git-guard's dirty-tree + secret-scan and blocked all subsequent ai deploys. Separately: connection-string *structure* (e.g. the LiteLLM postgres URL shape) was visible in `deploy.yml` plaintext, and adding a secret required editing fragile shell heredocs.
**Note: main's current `ai)` case is still broken** (line ~354: `printf 'DATABASE_URL%s\n'` is missing its `=`; `POSTGRES_PASSWORD` and `WEBUI_SECRET_KEY` are each printed twice; one line commented out). Do not push any `ai/**` change to main before this merges.
## New architecture (data-driven, migrated one stack per PR — ai first)
| File | Role |
|---|---|
| `secrets/secrets-map.yaml` | **Data only** — no code, no values. Maps stack → env template + Docker-secret names → CI var names |
| `ai/ai.env.template` | **Authoritative full env file.** Non-secret config as literals; secrets as placeholders. Rendered whole every run — keys can never silently go missing again |
| `deploy/provision-stack.py` | Generic renderer/provisioner. Hard-fails listing NAMES of missing/empty vars; hard-fails on unresolved placeholders; ships env + Docker secrets via **ssh stdin only** (values never on a command line, unlike create-secrets.sh); never prints a value |
| `.woodpecker/deploy.yml` | `ai)` case is now **one line**: `python3 deploy/provision-stack.py ai`. No value structure remains in this file for ai. All other cases/steps byte-for-byte unchanged from the pre-incident baseline |
Adding a future secret = Woodpecker UI + 2-line `from_secret` declaration (Woodpecker v3 mandates explicit declaration; no expose-all exists) + 1 line in template/manifest. Zero shell edits.
## Prerequisites before merging (blocking)
1. **Create Woodpecker secret `ai_mcpo_api_key`** in compose-files — value = the current `MCPO_API_KEY` from the host `ai.env`. The template now owns this key; provision will hard-fail (by design) if it's absent.
2. Confirm these hold **full, correct values** (the template consumes them as complete opaque values): `ai_litellm_database_url` (entire postgres URL — structure no longer lives in git), `ai_litellm_postgres_password`, `ai_oauth_client_secret`, `ai_webui_secret_key`, `ai_open_webui_database_url`, `ai_aws_access_key_id/secret`, `ai_litellm_master_key/salt_key`.
3. These hand-added Woodpecker secrets become **unused** after merge (config values are now template literals — you can delete them at leisure): `litellm_database_migrations`, `litellm_modify_params`, `ai_aws_region_name`, `ai_open_web_ui_enable_oauth_signup`, `_oauth_merge_accounts_by_email`, `_oauth_provider_name`, `_openid_provider_url`, `_oauth_client_id`, `_oauth_scopes`, `_openid_redirect_uri`, `ai_litellm_db_password`.
## Host-state cleanup already done (by agent, on docker-2)
- `ai/ai.env.work` + `ai/ai.yaml.save` moved to `/root/ai-stack-backup-20260903/` (mode 600, outside the repo tree); current `ai.env` snapshotted there too
- `ai/ai.yaml` restored to git HEAD (`git checkout -- ai/ai.yaml`) — **working tree now clean**, git-guard will pass
- Running services untouched (all 6 ai_* services 1/1 / 2/2 at time of writing)
## What happens on merge
changed-files → `ai` + `secrets` + `deploy`: `provision-stack.py ai` renders complete `ai.env` (fixing the duplicate-key drift in the current host file) + re-verifies flowagent secrets → `stack-deploy.sh ai` redeploys. Any missing secret aborts *before* deploy with named vars.
## Reviewer checklist
- [ ] `ai_mcpo_api_key` created in Woodpecker
- [ ] Template literals sanity-check (region, OAuth URLs/ID/scopes, WEBUI_URL) against the known-good host ai.env
- [ ] deploy.yml diff: confirm only the header comment, env block, apk line, and `ai)` case changed; all legacy `$${` escapes intact
- [ ] After merge: `docker service ls --filter name=ai_` all up; `docker service logs ai_mcpo --tail 50` shows flowagent az login outcome
## Follow-ups (separate work)
- Rotation list now **9 credentials**: original 6 + `LITELLM_MASTER_KEY`, `LITELLM_SALT_KEY`, LiteLLM DB password (exposed in chat transcript during incident recovery, 2026-09-03)
- Migrate remaining stacks (git, immich, n8n, vaultwarden, postgresql, entertainment, ...) to the manifest, one PR each
Part 1/4 of the provisioning architecture redesign after the 2026-09-03
ai.env incident (line-surgery provisioning dropped keys; broken services;
manual host-side recovery forked ai.yaml/ai.env).
Design: plain-data manifest consumed by deploy/provision-stack.py.
No code, no shell, no secret values, no value structure (e.g. no
connection-string shapes) live in this file or in deploy.yml anymore.
Part 2/4 of the provisioning redesign. Key properties:
- Complete key list for ai.env in one reviewable place. Rendered whole
every run by provision-stack.py — the "grep -v + printf line surgery"
that dropped MCPO_API_KEY/AWS_REGION_NAME/OAUTH_* keys (2026-09-03
incident) is gone for this stack.
- Non-secret config (region, OAuth endpoints/IDs/scopes, WEBUI_URL,
LiteLLM booleans) as literals, values taken from the verified-working
host ai.env. Secrets as placeholders resolved from Woodpecker secrets;
renderer fails hard on any missing/empty placeholder.
- LiteLLM DATABASE_URL comes from ai_litellm_database_url as a complete
opaque value — no connection-string structure in git (addresses the
plaintext-structure concern in deploy.yml).
- Drops legacy plain AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY keys —
ai.yaml only references the AI_-prefixed names.
- NEW Woodpecker secret required before merging: ai_mcpo_api_key
(value = current MCPO_API_KEY from host ai.env).
Part 3/4 of the provisioning redesign. One script for every migrated
stack; no per-stack shell code.
- Renders the stack's full env file from its .env.template (whole file,
every run — no line surgery), fails hard listing NAMES of any
missing/empty vars, fails hard on unresolved placeholders.
- Ships the rendered file over ssh stdin with write-temp + chmod 600 +
atomic mv. Secret values never appear on any command line (an
improvement over create-secrets.sh, which passes values as remote
shell arguments).
- Creates/rotates Docker Swarm secrets from the manifest's
docker_secrets map, values via ssh stdin, same sha256-checksum-label
skip-if-unchanged convention as create-secrets.sh.
- Never prints a secret value — names and counts only.
- Stacks absent from the manifest exit 0 (legacy case-entries keep
handling them), enabling one-stack-at-a-time migration.
Part 4/4 of the provisioning redesign (matched with secrets-map.yaml,
ai/ai.env.template, deploy/provision-stack.py in this same PR).
Changes, all scoped to the provision-secrets step:
- ai) case: the 54-line grep-v+printf heredoc (which had accumulated a
missing '=' on the DATABASE_URL printf, duplicate POSTGRES_PASSWORD and
WEBUI_SECRET_KEY printfs, and a commented-out line from manual edits —
the 2026-09-03 outage) is replaced by:
python3 deploy/provision-stack.py ai
No secret value structure (connection-string shapes etc.) remains in
this file for the ai stack.
- environment block: pruned the AI_* declarations the template no longer
needs (region/oauth-config/litellm-boolean values are now literals in
ai/ai.env.template): AI_LITELLM_DB_PASSWORD, AI_LITELLM_DATABASE_
MIGRATIONS, AI_LITELLM_MODIFY_PARAMS, AI_AWS_REGION_NAME, and the
seven AI_OPEN_WEB_UI_* config entries. Added AI_MCPO_API_KEY
(from_secret: ai_mcpo_api_key — NEW Woodpecker secret, must exist
before merge). Kept the 9 real ai secrets + 3 flowagent_* entries.
- apk line gains python3 py3-yaml for the provisioner.
- Header comment: documented the 2026-09-03 incident + migration model.
Everything else — validate, all other stacks' case entries, deploy,
verify, notify steps — is byte-for-byte identical to main.
Dollar-escape audit done per the header's own lesson: all remaining
double-dollar-brace references belong to unmigrated legacy cases and are
unchanged; the new ai) case contains no dollar sequences at all.
Key names now match exactly what current ai/ai.yaml references:
AI_OPEN_WEBUI_* (was WEBUI_*/OAUTH_*), AI_LITELLM_* (was LITELLM_*/
DATABASE_URL/POSTGRES_PASSWORD), AI_AWS_REGION_NAME (was AWS_REGION_NAME).
MCPO_API_KEY stays unprefixed (ai.yaml references it unprefixed for both
mcpo and mcpo-critical).
Placeholder names match the CI env vars declared in deploy.yml's
provision-secrets block, including the normalized
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET (fixing main's WEB_UI/WEBUI typo that
currently renders an EMPTY OAuth client secret into ai.env).
Rebase complete (2026-09-07) — branch rebuilt against current main after the AI_<SERVICE>_* renames landed there.
ai/ai.env.template: keys renamed to exactly what current ai.yaml references (AI_OPEN_WEBUI_*, AI_LITELLM_*, AI_AWS_REGION_NAME; MCPO_API_KEY stays unprefixed — ai.yaml consumes it unprefixed for both mcpo services).
.woodpecker/deploy.yml: reconstructed from current main + two scoped changes (ai case → one-line provision-stack.py ai call; env block normalized/pruned + AI_MCPO_API_KEY added).
⚠️ This rebase found live bugs in main's current ai) heredoc — the exact drift class this PR eliminates:
AI_LITELLM_POSTGRES_PASSWORD printed twice (first copy sourced from the retired AI_LITELLM_DB_PASSWORD var — whichever secret is stale wins depending on consumer parsing order)
Stray legacy AI_WEBUI_SECRET_KEY key that current ai.yaml never reads
printf ... AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET referencing a shell var while the env block declares AI_OPEN_WEB_UI_... (WEB_UI) → undefined var → rendered ai.env currently has an EMPTY OAuth client secret. If Authentik SSO login to Open WebUI breaks on the next ai redeploy from main, this is why. Merging this PR fixes it.
Pre-merge checklist (updated):
Woodpecker secret ai_mcpo_api_key exists (value = current MCPO_API_KEY from host ai.env)
ai_litellm_database_url, ai_litellm_postgres_password, ai_oauth_client_secret, ai_webui_secret_key, ai_open_webui_database_url hold full correct values
After merge: provision-stack.py output shows complete render (names only), all 6 ai_* services healthy, Authentik SSO login to ai.bryanmail.net works
Now-unused Woodpecker secrets after merge (deletable at leisure): litellm_database_migrations, litellm_modify_params, ai_aws_region_name, ai_litellm_db_password, and the seven ai_open_web_ui_* config secrets.
**Rebase complete (2026-09-07)** — branch rebuilt against current main after the `AI_<SERVICE>_*` renames landed there.
- `ai/ai.env.template`: keys renamed to exactly what current `ai.yaml` references (`AI_OPEN_WEBUI_*`, `AI_LITELLM_*`, `AI_AWS_REGION_NAME`; `MCPO_API_KEY` stays unprefixed — ai.yaml consumes it unprefixed for both mcpo services).
- `.woodpecker/deploy.yml`: reconstructed from current main + two scoped changes (ai case → one-line `provision-stack.py ai` call; env block normalized/pruned + `AI_MCPO_API_KEY` added).
**⚠️ This rebase found live bugs in main's current `ai)` heredoc** — the exact drift class this PR eliminates:
1. `AI_LITELLM_POSTGRES_PASSWORD` printed **twice** (first copy sourced from the retired `AI_LITELLM_DB_PASSWORD` var — whichever secret is stale wins depending on consumer parsing order)
2. Stray legacy `AI_WEBUI_SECRET_KEY` key that current ai.yaml never reads
3. `printf ... AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET` referencing a shell var while the env block declares `AI_OPEN_WEB_UI_...` (WEB_UI) → **undefined var → rendered ai.env currently has an EMPTY OAuth client secret**. If Authentik SSO login to Open WebUI breaks on the next ai redeploy from main, this is why. Merging this PR fixes it.
**Pre-merge checklist (updated):**
- [ ] Woodpecker secret `ai_mcpo_api_key` exists (value = current MCPO_API_KEY from host ai.env)
- [ ] `ai_litellm_database_url`, `ai_litellm_postgres_password`, `ai_oauth_client_secret`, `ai_webui_secret_key`, `ai_open_webui_database_url` hold full correct values
- [ ] After merge: provision-stack.py output shows complete render (names only), all 6 ai_* services healthy, Authentik SSO login to ai.bryanmail.net works
Now-unused Woodpecker secrets after merge (deletable at leisure): `litellm_database_migrations`, `litellm_modify_params`, `ai_aws_region_name`, `ai_litellm_db_password`, and the seven `ai_open_web_ui_*` config secrets.
Rebase of the provisioning-v2 branch's deploy.yml onto current main.
Reconstructed from main line-by-line, then two scoped changes:
1. ai) case -> `python3 deploy/provision-stack.py ai` (one line). Retires
main's current heredoc, which has active drift found during this
rebase: AI_LITELLM_POSTGRES_PASSWORD printed TWICE (first copy sourced
from the retired AI_LITELLM_DB_PASSWORD var), a stray legacy
AI_WEBUI_SECRET_KEY key current ai.yaml never reads, and printf
referencing AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET while the env block
declares AI_OPEN_WEB_UI_OAUTH_CLIENT_SECRET (WEB_UI) — undefined shell
var at runtime, so rendered ai.env currently carries an EMPTY OAuth
client secret.
2. env block ai section: normalized to the exact template placeholder
names (AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET <- ai_oauth_client_secret,
AI_OPEN_WEBUI_SECRET_KEY <- ai_webui_secret_key), added AI_MCPO_API_KEY
(NEW Woodpecker secret required pre-merge), pruned now-template-literal
vars (AI_AWS_REGION_NAME, AI_LITELLM_MODIFY_PARAMS/_DATABASE_MIGRATIONS,
AI_LITELLM_DB_PASSWORD, the seven ai_open_web_ui_* config secrets).
apk line gains python3 py3-yaml. All other steps and cases byte-match
current main. Dollar-escape audit done per header lesson.
Closed — superseded by PR #16 (secrets-provisioning-v3). This branch's 2026-09-03 merge base predated main's heavy hand-edits (AI_* renames, Proxmox parameterization), making it unmergeable despite manual content reconciliation. #16 carries identical file content on a branch cut from current main and merges clean. All review discussion/checklists carried over to #16's description.
Closed — superseded by **PR #16** (`secrets-provisioning-v3`). This branch's 2026-09-03 merge base predated main's heavy hand-edits (AI_* renames, Proxmox parameterization), making it unmergeable despite manual content reconciliation. #16 carries identical file content on a branch cut from current main and merges clean. All review discussion/checklists carried over to #16's description.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Why
2026-09-03 incident: the hand-maintained
ai)provisioning case (grep‑v + printf line surgery onai/ai.env) dropped/duplicated env keys during manual edits, breaking every ai-stack service. Manual host-side recovery forkedai.yaml/ai.env, which then (correctly) tripped git-guard's dirty-tree + secret-scan and blocked all subsequent ai deploys. Separately: connection-string structure (e.g. the LiteLLM postgres URL shape) was visible indeploy.ymlplaintext, and adding a secret required editing fragile shell heredocs.Note: main's current
ai)case is still broken (line ~354:printf 'DATABASE_URL%s\n'is missing its=;POSTGRES_PASSWORDandWEBUI_SECRET_KEYare each printed twice; one line commented out). Do not push anyai/**change to main before this merges.New architecture (data-driven, migrated one stack per PR — ai first)
secrets/secrets-map.yamlai/ai.env.templatedeploy/provision-stack.py.woodpecker/deploy.ymlai)case is now one line:python3 deploy/provision-stack.py ai. No value structure remains in this file for ai. All other cases/steps byte-for-byte unchanged from the pre-incident baselineAdding a future secret = Woodpecker UI + 2-line
from_secretdeclaration (Woodpecker v3 mandates explicit declaration; no expose-all exists) + 1 line in template/manifest. Zero shell edits.Prerequisites before merging (blocking)
ai_mcpo_api_keyin compose-files — value = the currentMCPO_API_KEYfrom the hostai.env. The template now owns this key; provision will hard-fail (by design) if it's absent.ai_litellm_database_url(entire postgres URL — structure no longer lives in git),ai_litellm_postgres_password,ai_oauth_client_secret,ai_webui_secret_key,ai_open_webui_database_url,ai_aws_access_key_id/secret,ai_litellm_master_key/salt_key.litellm_database_migrations,litellm_modify_params,ai_aws_region_name,ai_open_web_ui_enable_oauth_signup,_oauth_merge_accounts_by_email,_oauth_provider_name,_openid_provider_url,_oauth_client_id,_oauth_scopes,_openid_redirect_uri,ai_litellm_db_password.Host-state cleanup already done (by agent, on docker-2)
ai/ai.env.work+ai/ai.yaml.savemoved to/root/ai-stack-backup-20260903/(mode 600, outside the repo tree); currentai.envsnapshotted there tooai/ai.yamlrestored to git HEAD (git checkout -- ai/ai.yaml) — working tree now clean, git-guard will passWhat happens on merge
changed-files →
ai+secrets+deploy:provision-stack.py airenders completeai.env(fixing the duplicate-key drift in the current host file) + re-verifies flowagent secrets →stack-deploy.sh airedeploys. Any missing secret aborts before deploy with named vars.Reviewer checklist
ai_mcpo_api_keycreated in Woodpeckerai)case changed; all legacy$${escapes intactdocker service ls --filter name=ai_all up;docker service logs ai_mcpo --tail 50shows flowagent az login outcomeFollow-ups (separate work)
LITELLM_MASTER_KEY,LITELLM_SALT_KEY, LiteLLM DB password (exposed in chat transcript during incident recovery, 2026-09-03)Part 4/4 of the provisioning redesign (matched with secrets-map.yaml, ai/ai.env.template, deploy/provision-stack.py in this same PR). Changes, all scoped to the provision-secrets step: - ai) case: the 54-line grep-v+printf heredoc (which had accumulated a missing '=' on the DATABASE_URL printf, duplicate POSTGRES_PASSWORD and WEBUI_SECRET_KEY printfs, and a commented-out line from manual edits — the 2026-09-03 outage) is replaced by: python3 deploy/provision-stack.py ai No secret value structure (connection-string shapes etc.) remains in this file for the ai stack. - environment block: pruned the AI_* declarations the template no longer needs (region/oauth-config/litellm-boolean values are now literals in ai/ai.env.template): AI_LITELLM_DB_PASSWORD, AI_LITELLM_DATABASE_ MIGRATIONS, AI_LITELLM_MODIFY_PARAMS, AI_AWS_REGION_NAME, and the seven AI_OPEN_WEB_UI_* config entries. Added AI_MCPO_API_KEY (from_secret: ai_mcpo_api_key — NEW Woodpecker secret, must exist before merge). Kept the 9 real ai secrets + 3 flowagent_* entries. - apk line gains python3 py3-yaml for the provisioner. - Header comment: documented the 2026-09-03 incident + migration model. Everything else — validate, all other stacks' case entries, deploy, verify, notify steps — is byte-for-byte identical to main. Dollar-escape audit done per the header's own lesson: all remaining double-dollar-brace references belong to unmigrated legacy cases and are unchanged; the new ai) case contains no dollar sequences at all.Rebase complete (2026-09-07) — branch rebuilt against current main after the
AI_<SERVICE>_*renames landed there.ai/ai.env.template: keys renamed to exactly what currentai.yamlreferences (AI_OPEN_WEBUI_*,AI_LITELLM_*,AI_AWS_REGION_NAME;MCPO_API_KEYstays unprefixed — ai.yaml consumes it unprefixed for both mcpo services)..woodpecker/deploy.yml: reconstructed from current main + two scoped changes (ai case → one-lineprovision-stack.py aicall; env block normalized/pruned +AI_MCPO_API_KEYadded).⚠️ This rebase found live bugs in main's current
ai)heredoc — the exact drift class this PR eliminates:AI_LITELLM_POSTGRES_PASSWORDprinted twice (first copy sourced from the retiredAI_LITELLM_DB_PASSWORDvar — whichever secret is stale wins depending on consumer parsing order)AI_WEBUI_SECRET_KEYkey that current ai.yaml never readsprintf ... AI_OPEN_WEBUI_OAUTH_CLIENT_SECRETreferencing a shell var while the env block declaresAI_OPEN_WEB_UI_...(WEB_UI) → undefined var → rendered ai.env currently has an EMPTY OAuth client secret. If Authentik SSO login to Open WebUI breaks on the next ai redeploy from main, this is why. Merging this PR fixes it.Pre-merge checklist (updated):
ai_mcpo_api_keyexists (value = current MCPO_API_KEY from host ai.env)ai_litellm_database_url,ai_litellm_postgres_password,ai_oauth_client_secret,ai_webui_secret_key,ai_open_webui_database_urlhold full correct valuesNow-unused Woodpecker secrets after merge (deletable at leisure):
litellm_database_migrations,litellm_modify_params,ai_aws_region_name,ai_litellm_db_password, and the sevenai_open_web_ui_*config secrets.Closed — superseded by PR #16 (
secrets-provisioning-v3). This branch's 2026-09-03 merge base predated main's heavy hand-edits (AI_* renames, Proxmox parameterization), making it unmergeable despite manual content reconciliation. #16 carries identical file content on a branch cut from current main and merges clean. All review discussion/checklists carried over to #16's description.Pull request closed