Author SHA1 Message Date
Bot cb10c2e22d provision-secrets: rebuild ai) migration on current main (fixes live env-name drift incl. WEB_UI/WEBUI OAuth secret mismatch)
Rebase of the provisioning-v2 branch's deploy.yml onto current main.
Reconstructed from main line-by-line, then two scoped changes:

1. ai) case -> `python3 deploy/provision-stack.py ai` (one line). Retires
   main's current heredoc, which has active drift found during this
   rebase: AI_LITELLM_POSTGRES_PASSWORD printed TWICE (first copy sourced
   from the retired AI_LITELLM_DB_PASSWORD var), a stray legacy
   AI_WEBUI_SECRET_KEY key current ai.yaml never reads, and printf
   referencing AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET while the env block
   declares AI_OPEN_WEB_UI_OAUTH_CLIENT_SECRET (WEB_UI) — undefined shell
   var at runtime, so rendered ai.env currently carries an EMPTY OAuth
   client secret.

2. env block ai section: normalized to the exact template placeholder
   names (AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET <- ai_oauth_client_secret,
   AI_OPEN_WEBUI_SECRET_KEY <- ai_webui_secret_key), added AI_MCPO_API_KEY
   (NEW Woodpecker secret required pre-merge), pruned now-template-literal
   vars (AI_AWS_REGION_NAME, AI_LITELLM_MODIFY_PARAMS/_DATABASE_MIGRATIONS,
   AI_LITELLM_DB_PASSWORD, the seven ai_open_web_ui_* config secrets).

apk line gains python3 py3-yaml. All other steps and cases byte-match
current main. Dollar-escape audit done per header lesson.
2026-09-06 22:52:38 -07:00
Bot 24cc4e6655 ai.env.template: rebase onto main's AI_<SERVICE>_* naming (2026-09-06 renames)
Key names now match exactly what current ai/ai.yaml references:
AI_OPEN_WEBUI_* (was WEBUI_*/OAUTH_*), AI_LITELLM_* (was LITELLM_*/
DATABASE_URL/POSTGRES_PASSWORD), AI_AWS_REGION_NAME (was AWS_REGION_NAME).
MCPO_API_KEY stays unprefixed (ai.yaml references it unprefixed for both
mcpo and mcpo-critical).

Placeholder names match the CI env vars declared in deploy.yml's
provision-secrets block, including the normalized
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET (fixing main's WEB_UI/WEBUI typo that
currently renders an EMPTY OAuth client secret into ai.env).
2026-09-06 22:46:16 -07:00
Bot 88b3e46f77 provision-secrets: replace broken ai) line-surgery with one-line call to provision-stack.py
Part 4/4 of the provisioning redesign (matched with secrets-map.yaml,
ai/ai.env.template, deploy/provision-stack.py in this same PR).

Changes, all scoped to the provision-secrets step:
- ai) case: the 54-line grep-v+printf heredoc (which had accumulated a
  missing '=' on the DATABASE_URL printf, duplicate POSTGRES_PASSWORD and
  WEBUI_SECRET_KEY printfs, and a commented-out line from manual edits —
  the 2026-09-03 outage) is replaced by:
      python3 deploy/provision-stack.py ai
  No secret value structure (connection-string shapes etc.) remains in
  this file for the ai stack.
- environment block: pruned the AI_* declarations the template no longer
  needs (region/oauth-config/litellm-boolean values are now literals in
  ai/ai.env.template): AI_LITELLM_DB_PASSWORD, AI_LITELLM_DATABASE_
  MIGRATIONS, AI_LITELLM_MODIFY_PARAMS, AI_AWS_REGION_NAME, and the
  seven AI_OPEN_WEB_UI_* config entries. Added AI_MCPO_API_KEY
  (from_secret: ai_mcpo_api_key — NEW Woodpecker secret, must exist
  before merge). Kept the 9 real ai secrets + 3 flowagent_* entries.
- apk line gains python3 py3-yaml for the provisioner.
- Header comment: documented the 2026-09-03 incident + migration model.

Everything else — validate, all other stacks' case entries, deploy,
verify, notify steps — is byte-for-byte identical to main.
Dollar-escape audit done per the header's own lesson: all remaining
double-dollar-brace references belong to unmigrated legacy cases and are
unchanged; the new ai) case contains no dollar sequences at all.
2026-09-03 22:49:03 -07:00
Bot a91974caa1 Add deploy/provision-stack.py — generic manifest-driven provisioner
Part 3/4 of the provisioning redesign. One script for every migrated
stack; no per-stack shell code.

- Renders the stack's full env file from its .env.template (whole file,
  every run — no line surgery), fails hard listing NAMES of any
  missing/empty vars, fails hard on unresolved placeholders.
- Ships the rendered file over ssh stdin with write-temp + chmod 600 +
  atomic mv. Secret values never appear on any command line (an
  improvement over create-secrets.sh, which passes values as remote
  shell arguments).
- Creates/rotates Docker Swarm secrets from the manifest's
  docker_secrets map, values via ssh stdin, same sha256-checksum-label
  skip-if-unchanged convention as create-secrets.sh.
- Never prints a secret value — names and counts only.
- Stacks absent from the manifest exit 0 (legacy case-entries keep
  handling them), enabling one-stack-at-a-time migration.
2026-09-03 22:43:48 -07:00
Bot 8ae2ddbe97 Add ai/ai.env.template — full-file authoritative env template for the ai stack
Part 2/4 of the provisioning redesign. Key properties:

- Complete key list for ai.env in one reviewable place. Rendered whole
  every run by provision-stack.py — the "grep -v + printf line surgery"
  that dropped MCPO_API_KEY/AWS_REGION_NAME/OAUTH_* keys (2026-09-03
  incident) is gone for this stack.
- Non-secret config (region, OAuth endpoints/IDs/scopes, WEBUI_URL,
  LiteLLM booleans) as literals, values taken from the verified-working
  host ai.env. Secrets as placeholders resolved from Woodpecker secrets;
  renderer fails hard on any missing/empty placeholder.
- LiteLLM DATABASE_URL comes from ai_litellm_database_url as a complete
  opaque value — no connection-string structure in git (addresses the
  plaintext-structure concern in deploy.yml).
- Drops legacy plain AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY keys —
  ai.yaml only references the AI_-prefixed names.
- NEW Woodpecker secret required before merging: ai_mcpo_api_key
  (value = current MCPO_API_KEY from host ai.env).
2026-09-03 22:40:11 -07:00
Bot 0869cd319d Add secrets/secrets-map.yaml — data-only provisioning manifest (ai stack first)
Part 1/4 of the provisioning architecture redesign after the 2026-09-03
ai.env incident (line-surgery provisioning dropped keys; broken services;
manual host-side recovery forked ai.yaml/ai.env).

Design: plain-data manifest consumed by deploy/provision-stack.py.
No code, no shell, no secret values, no value structure (e.g. no
connection-string shapes) live in this file or in deploy.yml anymore.
2026-09-03 22:40:10 -07:00
Bot 355ccada8c Update ai/ai.yaml
ci/woodpecker/push/deploy Pipeline failed
ci/woodpecker/cron/renovate Pipeline was successful
2026-09-03 00:15:38 -07:00
Bot 096ebd6de5 Update .woodpecker/deploy.yml
ci/woodpecker/push/deploy Pipeline was successful
2026-09-02 23:28:25 -07:00
Bot 980c54f84e Update .woodpecker/deploy.yml 2026-09-02 23:27:19 -07:00
Bot 324a06ed2d Update .woodpecker/deploy.yml 2026-09-02 23:26:30 -07:00
Bot cee47822ec Update .woodpecker/deploy.yml 2026-09-02 23:25:39 -07:00
Bot 68365746a6 Update .woodpecker/deploy.yml 2026-09-02 23:24:37 -07:00
Bot eecfcf9367 Update .woodpecker/deploy.yml 2026-09-02 23:23:57 -07:00
Bot d01c763354 Update .woodpecker/deploy.yml
Redeploy after adding secret:
secret "ai_open_web_ui_openid_provider_url" not found
2026-09-02 23:17:48 -07:00
Bot 0baee47b67 Fixed duplicate env: "AI_OPEN_WEB_UI_ENABLE_OAUTH_SIG"
yaml: construct errors: line 190: mapping key "AI_OPEN_WEB_UI_ENABLE_OAUTH_SIGNUP" already defined at line 186
2026-09-02 23:02:51 -07:00
Bot 0435def8a0 Update .woodpecker/deploy.yml
Added Open WebUI and LiteLLM secrets
2026-09-02 23:01:44 -07:00
Bot ded255decc Upgrade to open-webui:0.11.3
ci/woodpecker/push/deploy Pipeline was successful
open-webui:0.11.1 > open-webui:0.11.3
2026-09-02 22:26:15 -07:00
Bot ac091999b4 Merge pull request 'FlowAgent MCP cutover: point ai.yaml's mcpo at flowagent-mcpo image + wire Azure secrets (matched pair)' (#14) from flowagent-ai-cutover into main
ci/woodpecker/push/deploy Pipeline failed
Reviewed-on: #14
https://ai.bryanmail.net/c/9f4c84c7-7aa1-4b24-a27e-98be6a5e737e
2026-09-02 21:43:18 -07:00
Bot a15e0100df provision-secrets: add flowagent_* case-entry to the ai) stack (matched pair with ai.yaml's mcpo cutover)
Surgical addition to the existing case "ai)" block only — every other
case, comment, and line in this ~26KB pipeline file is byte-for-byte
unchanged from main. Given this file's documented incident history from
prior full-file rewrites (see header comment), this was hand-verified
against the current main content line-by-line before writing, not
generated fresh.

Adds:
  - 3 new environment: entries (FLOWAGENT_AZURE_CLIENT_ID/TENANT_ID/
    CLIENT_SECRET) via from_secret, alongside the existing ai_* secrets
  - Inside the existing `ai)` case, after the ai.env rewrite block: three
    create_or_update_secret calls provisioning flowagent_azure_client_id,
    flowagent_azure_tenant_id, flowagent_azure_client_secret as native
    Docker Swarm secrets (Pattern C, _FILE convention) — NOT written into
    ai/ai.env, since mcpo's flowagent entry reads them via
    /run/secrets/flowagent_azure_* per flowagent/entrypoint.sh, not env vars
  - A short comment explaining why these three are Pattern C secrets
    inside an otherwise Pattern-B (host .env) stack case

This is the matched-pair companion to the ai.yaml commit in this same PR,
per the FlowAgent progress note's explicit sequencing requirement (both
land together, reviewed, not auto-merged to main).
2026-09-02 16:53:10 -07:00
Bot bcfb24b16d ai.yaml: cut mcpo over to the confirmed-working flowagent-mcpo image + wire flowagent Docker secrets (never touches mcpo-critical)
Image pinned to git.bryanmail.net/homelab/flowagent-mcpo:916164714429 —
the exact sha-tagged image built and smoke-tested clean in mcp-config
pipeline #22 (build-and-push-flowagent + smoke-test-flowagent-image both
success). NOT :latest, for reproducible deploys.

Adds a secrets: block to the mcpo service only, referencing the 3
flowagent_azure_* Docker Swarm secrets (Pattern C, _FILE convention,
consumed by flowagent/entrypoint.sh via /run/secrets/flowagent_azure_*).
Declares those 3 secrets as external at top level, mirroring the existing
vaultwarden_admin_token_v2 pattern in vaultwarden.yaml.

Companion secret-provisioning change lands in the same PR as a separate
commit in .woodpecker/deploy.yml's provision-secrets step (case "ai)").

mcpo-critical (lines defining Proxmox access) is completely untouched —
still stock ghcr.io/open-webui/mcpo:main, no secrets: block, unchanged.

Per FlowAgent MCP progress note: Azure AD App Registration is reported
created with real secrets already in Woodpecker (both mcp-config and
compose-files repos) as of this PR. This is a reviewed PR against a
feature branch, NOT auto-merged to main — deploy only happens after
manual review/merge, per user's explicit request for a manual-revert
safety net on this high-blast-radius shared stack (LiteLLM, Open WebUI,
n8n also live here).
2026-09-02 16:51:31 -07:00
Bot 42d31e38e4 git stack: switch provision-secrets to Pattern B .env rewrite (test phase) + add git to bootstrap-tier guard
ci/woodpecker/push/deploy Pipeline was successful
ci/woodpecker/cron/renovate Pipeline was successful
Context: git hosts the source of truth for every other stack's compose
files, so in a disaster-recovery scenario it must be restorable from a
flat git.yaml + git.env backup alone, with zero dependency on a running
Swarm's Docker secret store (native Docker secrets can't be resolved
until Swarm already exists, which is the circular dependency git.yaml
would otherwise create). Path 2 (documented decision): Pattern B for
git specifically, same mechanism the ai stack already uses.

provision-secrets / git) case:
  - Previously created native Docker secrets (git_db_password,
    git_runner_token, git_mcp_access_token) via create-secrets.sh —
    leftover from an earlier, abandoned Pattern C attempt.
  - Now rewrites only the secret-bearing lines (GITEA__database__PASSWD,
    GITEA_RUNNER_REGISTRATION_TOKEN, GITEA_MCP_ACCESS_TOKEN) via
    grep -v + printf, mirroring the ai) case exactly. Also strips the
    legacy GITEA_ACCESS_TOKEN key name so the test file converges on the
    git.env.example-documented key.
  - TEST PHASE: target is git.env.pipelinetest, NOT git.env. The real
    git.env is never opened for writing by this step. First run seeds
    the test file from the real git.env (to carry over all non-secret
    lines), then only the 3 secret lines are refreshed on every push.
  - Real cutover (pointing git.yaml/stack-deploy at the generated file,
    then retiring git.env.pipelinetest) is a deliberate follow-up step
    after manually diffing the rendered output.

deploy / bootstrap-tier guard:
  - Added git to the traefik|woodpecker|postgresql|secrets guard list.
    git.yaml changes now sync to the host but require a manual
    `stack-deploy.sh git` run, same as the other foundational stacks —
    prevents an auto-deploy of a bad git.yaml change from taking down
    Gitea before a human can look at it (Gitea itself is what every
    other pipeline needs to trigger a fix).
2026-09-02 00:02:36 -07:00
AVB 16a01a9ea5 Merge pull request 'Remove orphaned mcpo/ mirror (migrated to homelab/mcp-config)' (#13) from remove-orphaned-mcpo-mirror into main
ci/woodpecker/push/deploy Pipeline failed
ci/woodpecker/cron/renovate Pipeline was successful
Reviewed-on: #13
2026-08-29 20:57:58 -07:00
admin b6b23a7970 Remove mcpo/woodpecker-mcp.mjs — migrated unchanged to homelab/mcp-config 2026-08-29 20:46:01 -07:00
admin 150b262db3 Remove orphaned mcpo/config.json mirror — migrated to homelab/mcp-config
This file was never actually deployed by this repo's pipeline (ai.yaml
bind-mounts /volume1/docker/mcpo/config.json on the host directly; nothing
here ever synced it there — see homelab/mcp-config README for full writeup).
It also contained several real plaintext credentials (Proxmox token,
Authentik JWT/token, UniFi password, Gitea tokens) that have since been
rotated and templated in the new repo.

NOTE: deleting this from HEAD does not remove it from compose-files' git
history. The rotated credentials must be treated as permanently compromised
regardless of this deletion.
2026-08-29 20:46:00 -07:00
10 changed files with 380 additions and 553 deletions
+73 -44
View File
@@ -48,6 +48,17 @@ when:
# already unconditionally rsynced at the top of the deploy step regardless
# of which stacks changed, so it's safe to exclude it from the stack list
# everywhere folders are detected below.
#
# 2026-09-03/07 INCIDENT + REDESIGN: hand-maintained grep -v + printf
# line-surgery cases repeatedly drifted (missing '=', duplicated keys,
# mismatched env var names rendering EMPTY secrets) and broke live
# services. Root cause: no authoritative key list and shell heredocs
# hostile to hand-editing. Stacks migrate one at a time to a data-driven
# model: secrets/secrets-map.yaml (data only) + per-stack .env.template
# (authoritative FULL file) + deploy/provision-stack.py (whole-file
# render, hard failure naming any missing value). Migrated stacks call
# the script; unmigrated stacks keep legacy case entries until their own
# PR. See the "Secrets & Deployment Architecture — Global Direction" note.
# ─────────────────────────────────────────────────────────────────────────────
steps:
@@ -163,6 +174,11 @@ steps:
from_secret: entertainment_sparky_encryption_key
ENTERTAINMENT_BETTER_AUTH_SECRET:
from_secret: entertainment_better_auth_secret
# ── ai stack (manifest-driven — secrets/secrets-map.yaml +
# ai/ai.env.template + deploy/provision-stack.py). Env var names
# below match the template placeholders EXACTLY; this block is the
# only per-secret touchpoint left in this file for migrated stacks
# (Woodpecker v3 requires explicit from_secret declarations). ──
AI_AWS_ACCESS_KEY_ID:
from_secret: ai_aws_access_key_id
AI_AWS_SECRET_ACCESS_KEY:
@@ -171,16 +187,26 @@ steps:
from_secret: ai_litellm_master_key
AI_LITELLM_SALT_KEY:
from_secret: ai_litellm_salt_key
AI_LITELLM_DB_PASSWORD:
from_secret: ai_litellm_db_password
AI_WEBUI_SECRET_KEY:
AI_LITELLM_DATABASE_URL:
from_secret: ai_litellm_database_url
AI_LITELLM_POSTGRES_PASSWORD:
from_secret: ai_litellm_postgres_password
AI_OPEN_WEBUI_SECRET_KEY:
from_secret: ai_webui_secret_key
AI_OPEN_WEBUI_DATABASE_URL:
from_secret: ai_open_webui_database_url
AI_OAUTH_CLIENT_SECRET:
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET:
from_secret: ai_oauth_client_secret
AI_MCPO_API_KEY:
from_secret: ai_mcpo_api_key
FLOWAGENT_AZURE_CLIENT_ID:
from_secret: flowagent_azure_client_id
FLOWAGENT_AZURE_TENANT_ID:
from_secret: flowagent_azure_tenant_id
FLOWAGENT_AZURE_CLIENT_SECRET:
from_secret: flowagent_azure_client_secret
commands:
- apk add --no-cache openssh-client
- apk add --no-cache openssh-client python3 py3-yaml
- mkdir -p ~/.ssh
- echo "$SSH_KEY" | base64 -d > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
@@ -221,10 +247,38 @@ steps:
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'gamma_auth_token' '$${GAMMA_AUTH_TOKEN}'";;
git)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'git_db_password' '$${GIT_DB_PASSWORD}'
create_or_update_secret 'git_runner_token' '$${GIT_RUNNER_TOKEN}'
create_or_update_secret 'git_mcp_access_token' '$${GIT_MCP_ACCESS_TOKEN}'";;
# PATTERN B, DELIBERATE (see decision notes) — git hosts the source
# of truth for every other stack's compose files, so it must be
# restorable from a flat git.yaml + git.env backup alone, with zero
# dependency on a running Swarm's Docker secret store. Native Docker
# secrets (Pattern C) can't satisfy that: they only exist inside an
# already-running Swarm, which is exactly the circular dependency
# this stack can't have. Mirrors the retired ai) case's grep -v +
# printf rewrite-in-place approach, never sed (values may contain
# slash, dollar sign, ampersand).
#
# TEST PHASE: target is git.env.pipelinetest, NOT the real git.env.
# The real file is never opened for writing by this step. First run
# seeds the test file from the real git.env (carries over all
# non-secret lines untouched); every push after that only refreshes
# the 3 secret lines below. Also strips the legacy GITEA_ACCESS_TOKEN
# key name so the test file converges on the git.env.example-
# documented GITEA_MCP_ACCESS_TOKEN key. Cutover to the real file —
# and pointing git.yaml/stack-deploy at it — is a deliberate,
# separate follow-up after manually diffing this render.
# (Candidate for the secrets-map.yaml/provision-stack.py migration
# in its own PR; kept legacy for now.)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "FILE=/volume1/docker/compose-files/git.env.pipelinetest
TMP=\$FILE.tmp.\$\$
[ -f \$FILE ] || cp /volume1/docker/compose-files/git.env \$FILE
grep -vE '^(GITEA__database__PASSWD|GITEA_RUNNER_REGISTRATION_TOKEN|GITEA_MCP_ACCESS_TOKEN|GITEA_ACCESS_TOKEN)=' \$FILE > \$TMP 2>/dev/null || touch \$TMP
{ cat \$TMP
printf 'GITEA__database__PASSWD=%s\n' '$${GIT_DB_PASSWORD}'
printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' '$${GIT_RUNNER_TOKEN}'
printf 'GITEA_MCP_ACCESS_TOKEN=%s\n' '$${GIT_MCP_ACCESS_TOKEN}'
} > \$FILE
rm -f \$TMP
echo ' [OK] git.env.pipelinetest updated - real git.env untouched'";;
homeassistant)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'homeassistant_frigate_rtsp_password' '$${HOMEASSISTANT_FRIGATE_RTSP_PASSWORD}'
@@ -261,40 +315,15 @@ steps:
create_or_update_secret 'vaultwarden_admin_token' '$${VAULTWARDEN_ADMIN_TOKEN}'
create_or_update_secret 'vaultwarden_database_url_v2' '$${VAULTWARDEN_DATABASE_URL}'";;
ai)
# NOTE: ai stack uses Pattern B (host .env, not native Docker secrets) —
# LiteLLM/boto3 and Open WebUI don't support the _FILE convention for these
# vars. Instead of Docker secrets, we regenerate ONLY the migrated lines in
# the remote ai/ai.env in place via grep -v + printf (never sed, since values
# may contain slash, dollar sign, ampersand). All other lines — including
# MCPO_API_KEY, OAUTH_CLIENT_ID, WEBUI_URL, and other non-secret config — are
# left completely untouched. MCPO_API_KEY migration is deferred to a
# follow-up; this step never reads or writes it.
#
# IMPORTANT: ai.yaml's litellm service references ${AI_AWS_ACCESS_KEY_ID} /
# ${AI_AWS_SECRET_ACCESS_KEY} (AI_-prefixed) and renders them into the
# container as plain AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (see commit
# 37ed671a, "Change AWS keys to use Woodpecker Secrets"). So ai.env must be
# written with the AI_-prefixed key names, NOT the plain ones — writing
# plain AWS_ACCESS_KEY_ID here would leave ${AI_AWS_ACCESS_KEY_ID}
# unresolved at compose-render time (empty), silently breaking Bedrock auth.
# All other migrated vars in ai.yaml use plain (unprefixed) names, so only
# these two lines need the AI_ prefix.
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "FILE=/volume1/docker/compose-files/ai/ai.env
TMP=\$FILE.tmp.\$\$
grep -vE '^(AI_AWS_ACCESS_KEY_ID|AI_AWS_SECRET_ACCESS_KEY|LITELLM_MASTER_KEY|LITELLM_SALT_KEY|POSTGRES_PASSWORD|DATABASE_URL|WEBUI_SECRET_KEY|OPEN_WEBUI_DATABASE_URL|OAUTH_CLIENT_SECRET)=' \$FILE > \$TMP 2>/dev/null || touch \$TMP
{ cat \$TMP
printf 'AI_AWS_ACCESS_KEY_ID=%s\n' '$${AI_AWS_ACCESS_KEY_ID}'
printf 'AI_AWS_SECRET_ACCESS_KEY=%s\n' '$${AI_AWS_SECRET_ACCESS_KEY}'
printf 'LITELLM_MASTER_KEY=%s\n' '$${AI_LITELLM_MASTER_KEY}'
printf 'LITELLM_SALT_KEY=%s\n' '$${AI_LITELLM_SALT_KEY}'
printf 'POSTGRES_PASSWORD=%s\n' '$${AI_LITELLM_DB_PASSWORD}'
printf 'DATABASE_URL=postgresql://LiteLLM:%s@postgresql:5432/litellm\n' '$${AI_LITELLM_DB_PASSWORD}'
printf 'WEBUI_SECRET_KEY=%s\n' '$${AI_WEBUI_SECRET_KEY}'
printf 'OPEN_WEBUI_DATABASE_URL=%s\n' '$${AI_OPEN_WEBUI_DATABASE_URL}'
printf 'OAUTH_CLIENT_SECRET=%s\n' '$${AI_OAUTH_CLIENT_SECRET}'
} > \$FILE
rm -f \$TMP
echo ' [OK] ai/ai.env secrets updated'";;
# MIGRATED (2026-09-07) to manifest-driven provisioning after the
# line-surgery approach repeatedly drifted (dropped keys 2026-09-03;
# duplicate AI_LITELLM_POSTGRES_PASSWORD + stray legacy keys +
# WEB_UI/WEBUI env-name mismatch rendering an EMPTY OAuth client
# secret, found 2026-09-07). All logic lives in
# deploy/provision-stack.py; the authoritative key list lives in
# ai/ai.env.template; the mapping lives in secrets/secrets-map.yaml.
# This case is intentionally one line.
python3 deploy/provision-stack.py ai;;
entertainment)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'entertainment_discord_token' '$${ENTERTAINMENT_DISCORD_TOKEN}'
@@ -352,7 +381,7 @@ steps:
# Bootstrap-tier guard: file synced to host, deploy is MANUAL
case "$STACK" in
traefik|woodpecker|postgresql|secrets)
traefik|woodpecker|postgresql|secrets|git)
echo " [BOOTSTRAP] $STACK: file synced. Deploy is MANUAL."
echo " Run: bash /volume1/docker/compose-files/deploy/stack-deploy.sh $STACK"
continue ;;
+55
View File
@@ -0,0 +1,55 @@
# ─────────────────────────────────────────────────────────────────────────────
# ai.env.template — AUTHORITATIVE template for ai/ai.env (rendered by
# deploy/provision-stack.py per secrets/secrets-map.yaml).
#
# - This file IS the complete key list for ai.env. The whole file is
# rendered on every provisioning run — no line surgery, so a key can
# never silently go missing again (root cause of the 2026-09-03 outage).
# - Key names match EXACTLY what ai/ai.yaml references (AI_<SERVICE>_*
# naming adopted on main 2026-09-06).
# - Non-secret config lives here as LITERAL values (visible, reviewable).
# - Secret values are dollar-brace placeholders resolved from the CI env
# (Woodpecker from_secret vars) at provisioning time. provision-stack.py
# FAILS HARD if any placeholder is missing/empty.
# - The rendered ai/ai.env exists only on the host (gitignored).
# - Rendered by provision-stack.py, NOT Woodpecker's yaml preprocessor —
# single-dollar placeholders are safe here (deploy.yml's double-dollar
# rule does NOT apply to this file).
#
# Consumed by ai/ai.yaml. DOMAIN_NAME comes from deploy/global.env, not here.
# ─────────────────────────────────────────────────────────────────────────────
# ── LiteLLM (non-secret config) ──────────────────────────────────────────────
AI_AWS_REGION_NAME=us-east-2
AI_LITELLM_MODIFY_PARAMS=False
AI_LITELLM_DATABASE_MIGRATIONS=True
# ── LiteLLM (secrets) ────────────────────────────────────────────────────────
AI_AWS_ACCESS_KEY_ID=${AI_AWS_ACCESS_KEY_ID}
AI_AWS_SECRET_ACCESS_KEY=${AI_AWS_SECRET_ACCESS_KEY}
AI_LITELLM_MASTER_KEY=${AI_LITELLM_MASTER_KEY}
AI_LITELLM_SALT_KEY=${AI_LITELLM_SALT_KEY}
# Full connection URL is itself a secret (ai_litellm_database_url) — the
# URL structure never appears in git.
AI_LITELLM_DATABASE_URL=${AI_LITELLM_DATABASE_URL}
AI_LITELLM_POSTGRES_PASSWORD=${AI_LITELLM_POSTGRES_PASSWORD}
# ── Open WebUI (non-secret config) ───────────────────────────────────────────
AI_OPEN_WEBUI_URL=https://ai.bryanmail.net
AI_OPEN_WEBUI_ENABLE_OAUTH_SIGNUP=true
AI_OPEN_WEBUI_OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true
AI_OPEN_WEBUI_OAUTH_PROVIDER_NAME=Authentik
AI_OPEN_WEBUI_OPENID_PROVIDER_URL=https://auth.bryanmail.net/application/o/open-web-ui/.well-known/openid-configuration
# OAuth client ID is a public identifier by OAuth2 design (it is sent to the
# browser); the client SECRET below is the protected credential.
AI_OPEN_WEBUI_OAUTH_CLIENT_ID=hVmhi1dS3TnG2cUw5QwLOx5FDLSWtnQUdZyeB5zK
AI_OPEN_WEBUI_OAUTH_SCOPES=openid email profile
AI_OPEN_WEBUI_OPENID_REDIRECT_URI=https://ai.bryanmail.net/oauth/oidc/callback
# ── Open WebUI (secrets) ─────────────────────────────────────────────────────
AI_OPEN_WEBUI_SECRET_KEY=${AI_OPEN_WEBUI_SECRET_KEY}
AI_OPEN_WEBUI_DATABASE_URL=${AI_OPEN_WEBUI_DATABASE_URL}
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET=${AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET}
# ── mcpo / mcpo-critical (secrets) ───────────────────────────────────────────
MCPO_API_KEY=${AI_MCPO_API_KEY}
+16 -2
View File
@@ -1,6 +1,6 @@
services:
open-webui:
image: ghcr.io/open-webui/open-webui:0.11.1
image: ghcr.io/open-webui/open-webui:0.11.3 #
entrypoint:
- /bin/bash
- /app/tools/startup.sh
@@ -127,7 +127,7 @@ services:
labels:
- traefik.enable=false
mcpo:
image: ghcr.io/open-webui/mcpo:main
image: git.bryanmail.net/homelab/flowagent-mcpo:916164714429
command:
- --config
- /app/config/config.json
@@ -141,6 +141,13 @@ services:
- /volume1/docker/mcpo/data:/mcpo_data
- /volume1/docker/cronicle/ssh_keys:/app/ssh_keys:ro
- /volume1/docker/mcpo/uv-cache:/app/uv-cache
secrets:
- source: flowagent_azure_client_id
target: flowagent_azure_client_id
- source: flowagent_azure_tenant_id
target: flowagent_azure_tenant_id
- source: flowagent_azure_client_secret
target: flowagent_azure_client_secret
networks:
- traefik_backend
deploy:
@@ -173,6 +180,13 @@ services:
- traefik.http.middlewares.n8n.headers.STSPreload=true
- traefik.http.routers.n8n.middlewares=forwardAuth-authentik@file, crowdsec@file
- traefik.swarm.network=traefik_backend
secrets:
flowagent_azure_client_id:
external: true
flowagent_azure_tenant_id:
external: true
flowagent_azure_client_secret:
external: true
networks:
traefik_backend:
external: true
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""provision-stack.py — manifest-driven env-file rendering + Docker secret
provisioning for one stack.
Usage (from the CI workspace root, inside the provision-secrets step):
python3 deploy/provision-stack.py <stack>
Reads secrets/secrets-map.yaml (data only — no code, no values) and, for the
named stack:
1. env_template -> renders the COMPLETE env file. Placeholders of the form
dollar-brace VARNAME are resolved from this process's environment (the
Woodpecker from_secret-backed vars). The whole file is rendered every
run; nothing is line-edited in place, so keys can never silently go
missing (root cause of the 2026-09-03 ai.env incident).
2. env_dest -> ships the rendered file to the Swarm manager over ssh stdin
(write-to-temp + atomic mv, mode 600). The rendered file never touches
the CI workspace disk under the repo (no chance of being committed) and
never appears on a command line.
3. docker_secrets -> for each swarm-secret-name -> ENV_VAR mapping, creates
or rotates the Docker secret. Values are passed via ssh stdin only.
Rotation uses the same sha256-checksum-label convention as
deploy/create-secrets.sh (unchanged secrets are skipped silently).
Safety properties:
- FAILS HARD (non-zero) if any referenced env var is missing or empty, and
lists the missing NAMES. A partial/broken render can never ship.
- FAILS HARD if any unresolved placeholder remains after rendering.
- NEVER prints a secret value — names and counts only.
- Requires SWARM_MANAGER_IP in the environment and a usable ssh identity
(both already set up by the provision-secrets step).
Stacks not present in the manifest exit 0 with a notice, so this script is
safe to call unconditionally; legacy case-entries in deploy.yml keep handling
unmigrated stacks.
"""
import hashlib
import os
import re
import subprocess
import sys
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MANIFEST_PATH = os.path.join(REPO_ROOT, "secrets", "secrets-map.yaml")
REMOTE_BASE = "/volume1/docker/compose-files"
PLACEHOLDER_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
SSH_OPTS = ["-o", "StrictHostKeyChecking=no"]
def die(msg: str) -> None:
print(f"ERROR: {msg}", file=sys.stderr)
sys.exit(1)
def load_manifest() -> dict:
try:
import yaml # py3-yaml, installed by the provision-secrets step
except ImportError:
die("PyYAML not available — provision-secrets step must apk add py3-yaml")
if not os.path.isfile(MANIFEST_PATH):
die(f"manifest not found: {MANIFEST_PATH}")
with open(MANIFEST_PATH, "r", encoding="utf-8") as fh:
data = yaml.safe_load(fh) or {}
stacks = data.get("stacks")
if not isinstance(stacks, dict):
die("manifest has no 'stacks:' mapping")
return stacks
def ssh_target() -> str:
ip = os.environ.get("SWARM_MANAGER_IP", "").strip()
if not ip:
die("SWARM_MANAGER_IP is empty — check Woodpecker repo secrets")
return f"root@{ip}"
def ssh_run(target: str, remote_cmd: str, stdin_data: bytes | None = None,
check: bool = True) -> subprocess.CompletedProcess:
proc = subprocess.run(
["ssh", *SSH_OPTS, target, remote_cmd],
input=stdin_data, capture_output=True,
)
if check and proc.returncode != 0:
# stderr may be verbose but must never contain our secret values —
# we only ever send values via stdin, never embed them in remote_cmd.
die(f"remote command failed (rc={proc.returncode}): {remote_cmd}\n"
f"{proc.stderr.decode(errors='replace').strip()}")
return proc
def render_template(template_path: str) -> str:
if not os.path.isfile(template_path):
die(f"env_template not found: {template_path}")
with open(template_path, "r", encoding="utf-8") as fh:
raw = fh.read()
referenced = sorted(set(PLACEHOLDER_RE.findall(raw)))
missing = [v for v in referenced
if not os.environ.get(v, "").strip()]
if missing:
die("template references vars that are MISSING or EMPTY in the CI "
"environment (add them via from_secret in "
".woodpecker/deploy.yml provision-secrets, and as Woodpecker "
"secrets):\n " + "\n ".join(missing))
rendered = PLACEHOLDER_RE.sub(lambda m: os.environ[m.group(1)], raw)
# Belt-and-braces: nothing placeholder-shaped may survive the render.
leftover = sorted(set(PLACEHOLDER_RE.findall(rendered)))
if leftover:
die("unresolved placeholders remain after rendering: "
+ ", ".join(leftover))
print(f" [render] {template_path}: {len(referenced)} secret placeholder(s) "
f"resolved: {', '.join(referenced)}")
return rendered
def ship_env_file(target: str, rendered: str, dest_rel: str) -> None:
dest = f"{REMOTE_BASE}/{dest_rel}"
tmp = f"{dest}.provision-tmp"
# Value travels over ssh stdin; never on a command line; atomic mv.
ssh_run(target,
f"umask 077 && cat > {tmp} && chmod 600 {tmp} && mv {tmp} {dest}",
stdin_data=rendered.encode())
keys = [ln.split("=", 1)[0] for ln in rendered.splitlines()
if "=" in ln and not ln.lstrip().startswith("#") and ln.strip()]
print(f" [env] shipped {dest} ({len(keys)} keys): {', '.join(keys)}")
def provision_docker_secret(target: str, name: str, env_var: str) -> None:
value = os.environ.get(env_var, "")
if not value.strip():
die(f"docker secret '{name}': env var {env_var} is missing/empty")
new_hash = hashlib.sha256(value.encode()).hexdigest()
probe = ssh_run(
target,
f"docker secret inspect {name} "
"--format '{{index .Spec.Labels \"checksum\"}}' 2>/dev/null || true",
check=True)
old_hash = probe.stdout.decode().strip()
if old_hash == new_hash:
print(f" [skip] docker secret {name} (unchanged)")
return
if old_hash:
rm = ssh_run(target, f"docker secret rm {name}", check=False)
if rm.returncode != 0:
die(f"docker secret {name}: value changed but removal failed — "
"it is probably referenced by a running service. Provision "
"under a versioned name (see vaultwarden_database_url_v2 "
"precedent) or scale the service down first.")
action = "update"
else:
action = "create"
ssh_run(target,
f"docker secret create --label checksum={new_hash} "
f"--label managed-by=woodpecker {name} -",
stdin_data=value.encode())
print(f" [{action}] docker secret {name} (value via stdin)")
def main() -> None:
if len(sys.argv) != 2:
die("usage: provision-stack.py <stack>")
stack = sys.argv[1]
stacks = load_manifest()
cfg = stacks.get(stack)
if cfg is None:
print(f" [info] stack '{stack}' not in secrets-map.yaml — "
"legacy provisioning (deploy.yml case-entry) applies. Nothing to do.")
return
target = ssh_target()
print(f"==> provision-stack: {stack}")
template_rel = cfg.get("env_template")
dest_rel = cfg.get("env_dest")
if template_rel and not dest_rel:
die("env_template set but env_dest missing in manifest")
if template_rel:
rendered = render_template(os.path.join(REPO_ROOT, template_rel))
ship_env_file(target, rendered, dest_rel)
for name, env_var in (cfg.get("docker_secrets") or {}).items():
provision_docker_secret(target, name, env_var)
print(f"==> provision-stack: {stack} done")
if __name__ == "__main__":
main()
-186
View File
@@ -1,186 +0,0 @@
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-filesystem",
"/mcpo_data/filesystem"
]
},
"memory": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-memory"
]
},
"proxmox-nuck7-1": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.11",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.11",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-1",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"proxmox-nuck7-2": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.12",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.12",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-2",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"proxmox-nuck7-3": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.13",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.13",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-3",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"homeassistant": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://home.bryanmail.net/mcp_server/sse",
"--header",
"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIxNzhmYzI0NjA2Y2I0ZTg4ODI1N2VmMzdiZTNhY2E5YSIsImlhdCI6MTc3NDY4MzAxNiwiZXhwIjoyMDkwMDQzMDE2fQ.32kY2LVHzKZHWLc96T6z2P-8beNTnp2DHRUf2UEie2w"
]
},
"teams": {
"command": "npx",
"args": [
"-y",
"@floriscornel/teams-mcp@latest"
],
"env": {
"TEAMS_MCP_READ_ONLY": "true",
"HOME": "/app/teams-mcp-auth"
}
},
"ms365": {
"command": "npx",
"args": [
"-y",
"@softeria/ms-365-mcp-server@0.129.0",
"--preset",
"personal",
"--read-only",
"--discovery"
],
"env": {
"HOME": "/app/teams-mcp-auth",
"MS365_MCP_CLIENT_ID": "14d82eec-204b-4c2f-b7e8-296a70dab67e",
"MS365_MCP_TOKEN_CACHE_PATH": "/app/teams-mcp-auth/.teams-mcp-token-cache.json",
"SILENT": "true"
}
},
"unifi-network": {
"command": "uvx",
"args": [
"unifi-network-mcp@latest"
],
"env": {
"UNIFI_HOST": "192.168.4.1",
"UNIFI_USERNAME": "unifi-mcp",
"UNIFI_PASSWORD": "3dHOEOMygTYeX3",
"UNIFI_PORT": "443",
"UNIFI_VERIFY_SSL": "false",
"UV_CACHE_DIR": "/app/uv-cache"
}
},
"authentik": {
"command": "uvx",
"args": [
"authentik-diag-mcp",
"--base-url",
"https://auth.bryanmail.net",
"--token",
"LAm3lBTumOmsU8AiFQM2FmCZyoj8bTSR0FQnAcBy1QnTMEyU4oWozwdxTUap"
]
},
"gitea": {
"command": "/mcpo_data/gitea-mcp",
"args": [
"-t",
"stdio"
],
"env": {
"GITEA_HOST": "https://git.bryanmail.net",
"GITEA_ACCESS_TOKEN": "5aa3a554c001b1dbe5215cb8cb388112801930e9"
}
},
"gitea-admin": {
"command": "/mcpo_data/gitea-mcp",
"args": [
"-t",
"stdio"
],
"env": {
"GITEA_HOST": "https://git.bryanmail.net",
"GITEA_ACCESS_TOKEN": "289225b0b8f1827242191874b2408db76af06321"
}
},
"powerautomate": {
"command": "npx",
"args": [
"-y",
"powerautomate-mcp@latest",
"--stdio"
],
"env": {
"HOME": "/app/powerautomate-auth",
"PA_MCP_CLIENT_ID": "84b431ed-ef5d-48a0-b0a1-878cfdb71453",
"PA_MCP_TENANT_ID": "0f6cf991-c449-480a-a71b-83003ce6edc1",
"PA_CONFIG_PATH": "/app/powerautomate-auth/config.json"
}
}
}
}
-21
View File
@@ -1,21 +0,0 @@
FROM ghcr.io/open-webui/mcpo:main
# Azure CLI — required for non-interactive service-principal auth
# (`az login --service-principal`) performed by entrypoint.sh at container
# start. FlowAgent's auth model is Azure CLI + MSAL; see
# https://github.com/microsoft/power-platform-skills/blob/main/plugins/power-automate/references/connection-patterns.md
RUN apk add --no-cache py3-pip curl \
&& pip install --no-cache-dir --break-system-packages azure-cli
# FlowAgent self-contained MCP bundle (stdio transport, all tools inlined,
# Node 18+ only — no npm install / remote host needed at runtime).
# Pinned to a ref (commit SHA or tag), NOT `main`, for reproducible builds.
# Bump deliberately via PR when upstream ships updates:
# https://github.com/microsoft/power-platform-skills/tree/main/plugins/power-automate/server
ARG FLOWAGENT_REF=main
RUN mkdir -p /app/flowagent \
&& curl -fsSL "https://raw.githubusercontent.com/microsoft/power-platform-skills/${FLOWAGENT_REF}/plugins/power-automate/server/mcp.mjs" \
-o /app/flowagent/mcp.mjs
COPY entrypoint.sh /app/flowagent/entrypoint.sh
RUN chmod +x /app/flowagent/entrypoint.sh
-30
View File
@@ -1,30 +0,0 @@
#!/bin/sh
# FlowAgent MCP entrypoint — non-interactive Azure service-principal login.
#
# Expects three env vars pointing at Docker secret files (Pattern C,
# _FILE convention, provisioned by Woodpecker — never hand-typed):
# FLOWAGENT_AZURE_CLIENT_ID_FILE
# FLOWAGENT_AZURE_TENANT_ID_FILE
# FLOWAGENT_AZURE_CLIENT_SECRET_FILE
#
# On success, execs into the real command (node /app/flowagent/mcp.mjs),
# replacing this shell so mcpo's stdio pipe talks directly to the MCP process.
set -eu
: "${FLOWAGENT_AZURE_CLIENT_ID_FILE:?FLOWAGENT_AZURE_CLIENT_ID_FILE not set}"
: "${FLOWAGENT_AZURE_TENANT_ID_FILE:?FLOWAGENT_AZURE_TENANT_ID_FILE not set}"
: "${FLOWAGENT_AZURE_CLIENT_SECRET_FILE:?FLOWAGENT_AZURE_CLIENT_SECRET_FILE not set}"
CLIENT_ID="$(cat "$FLOWAGENT_AZURE_CLIENT_ID_FILE")"
TENANT_ID="$(cat "$FLOWAGENT_AZURE_TENANT_ID_FILE")"
CLIENT_SECRET="$(cat "$FLOWAGENT_AZURE_CLIENT_SECRET_FILE")"
az login --service-principal \
-u "$CLIENT_ID" \
-p "$CLIENT_SECRET" \
--tenant "$TENANT_ID" \
--output none
unset CLIENT_SECRET
exec "$@"
-217
View File
@@ -1,217 +0,0 @@
#!/usr/bin/env node
// Minimal MCP server (stdio, JSON-RPC 2.0) for Woodpecker CI.
// Zero dependencies — plain Node 18+ (built-in fetch). Config via env vars.
//
// Configuration (env):
// WOODPECKER_URL e.g. https://your-woodpecker.example.com
// WOODPECKER_TOKEN Woodpecker Personal Access Token (JWT)
//
// Tools:
// woodpecker_list_repos
// woodpecker_list_pipelines { repo_id, limit? }
// woodpecker_get_pipeline { repo_id, number } -> status + workflow/step tree
// woodpecker_pipeline_logs { repo_id, number, step_id, tail? } -> decoded step logs
import { Buffer } from "node:buffer";
const BASE = (process.env.WOODPECKER_URL || "").replace(/\/+$/, "");
const TOKEN = process.env.WOODPECKER_TOKEN || "";
function log(...a) {
// Diagnostics go to stderr so they never corrupt the stdout JSON-RPC stream.
process.stderr.write("[woodpecker-mcp] " + a.join(" ") + "\n");
}
async function api(path) {
if (!BASE || !TOKEN) throw new Error("Missing WOODPECKER_URL or WOODPECKER_TOKEN in environment");
const res = await fetch(`${BASE}/api${path}`, {
headers: { Authorization: `Bearer ${TOKEN}`, Accept: "application/json" },
});
const body = await res.text();
if (!res.ok) throw new Error(`HTTP ${res.status} ${path}: ${body.slice(0, 300)}`);
return body ? JSON.parse(body) : null;
}
// --- tool response formatting ---
function summarizePipeline(p) {
const lines = [];
lines.push(`pipeline #${p.number} (id=${p.id}) status=${p.status} event=${p.event} branch=${p.branch}`);
lines.push(`commit=${(p.commit || "").slice(0, 12)} author=${p.author}`);
if (p.message) lines.push(`message: ${p.message.split("\n")[0]}`);
for (const wf of p.workflows || []) {
lines.push(` WORKFLOW "${wf.name}" (pid=${wf.pid}) state=${wf.state}${wf.error ? ` error=${wf.error}` : ""}`);
for (const c of wf.children || []) {
lines.push(
` step "${c.name}" (id=${c.id}, pid=${c.pid}) ${c.state} exit=${c.exit_code ?? "-"} type=${c.type}`
);
}
}
return lines.join("\n");
}
function decodeLogs(entries) {
if (!Array.isArray(entries)) return String(entries);
return entries
.map((e) => {
const d = e?.data;
if (d == null) return "";
try {
return Buffer.from(d, "base64").toString("utf-8");
} catch {
return String(d);
}
})
.join("");
}
const TOOLS = [
{
name: "woodpecker_list_repos",
description: "List repositories the token can access (id, full name, default branch).",
inputSchema: { type: "object", properties: {}, additionalProperties: false },
handler: async () => {
const repos = await api(`/user/repos`);
return (repos || [])
.map((r) => `id=${r.id} ${r.full_name} default_branch=${r.default_branch}`)
.join("\n") || "(no repositories)";
},
},
{
name: "woodpecker_list_pipelines",
description: "Recent pipelines for a repository. Params: repo_id (number), limit (number, default 20).",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number", description: "Woodpecker repository ID" },
limit: { type: "number", description: "How many pipelines to return (default 20)" },
},
required: ["repo_id"],
additionalProperties: false,
},
handler: async ({ repo_id, limit }) => {
const list = await api(`/repos/${repo_id}/pipelines?perPage=${limit || 20}`);
return (list || [])
.map(
(p) =>
`#${p.number} ${p.status.padEnd(8)} ${p.event.padEnd(12)} ${p.branch} ${(p.commit || "").slice(0, 8)} ${(p.message || "").split("\n")[0]}`
)
.join("\n") || "(no pipelines)";
},
},
{
name: "woodpecker_get_pipeline",
description: "Pipeline details: status plus the workflow/step tree (with step ids for fetching logs). Params: repo_id, number.",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number" },
number: { type: "number", description: "Pipeline number (as shown in the UI)" },
},
required: ["repo_id", "number"],
additionalProperties: false,
},
handler: async ({ repo_id, number }) => {
const p = await api(`/repos/${repo_id}/pipelines/${number}`);
return summarizePipeline(p);
},
},
{
name: "woodpecker_pipeline_logs",
description: "Decoded logs for a single step. Params: repo_id, number (pipeline), step_id (from woodpecker_get_pipeline). Optional tail (last N lines).",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number" },
number: { type: "number" },
step_id: { type: "number", description: "Step id from woodpecker_get_pipeline" },
tail: { type: "number", description: "Return only the last N lines (optional)" },
},
required: ["repo_id", "number", "step_id"],
additionalProperties: false,
},
handler: async ({ repo_id, number, step_id, tail }) => {
const entries = await api(`/repos/${repo_id}/logs/${number}/${step_id}`);
let txt = decodeLogs(entries);
if (tail && tail > 0) {
txt = txt.split("\n").slice(-tail).join("\n");
}
return txt || "(no logs)";
},
},
];
// --- JSON-RPC over stdio loop ---
function send(msg) {
process.stdout.write(JSON.stringify(msg) + "\n");
}
function reply(id, result) {
send({ jsonrpc: "2.0", id, result });
}
function replyError(id, code, message) {
send({ jsonrpc: "2.0", id, error: { code, message } });
}
async function handle(req) {
const { id, method, params } = req;
if (method === "initialize") {
reply(id, {
protocolVersion: params?.protocolVersion || "2024-11-05",
capabilities: { tools: {} },
serverInfo: { name: "woodpecker-mcp", version: "1.0.0" },
});
return;
}
if (method === "notifications/initialized" || method === "notifications/cancelled") {
return; // notifications carry no response
}
if (method === "ping") {
reply(id, {});
return;
}
if (method === "tools/list") {
reply(id, {
tools: TOOLS.map((t) => ({ name: t.name, description: t.description, inputSchema: t.inputSchema })),
});
return;
}
if (method === "tools/call") {
const tool = TOOLS.find((t) => t.name === params?.name);
if (!tool) {
replyError(id, -32602, `Unknown tool: ${params?.name}`);
return;
}
try {
const text = await tool.handler(params.arguments || {});
reply(id, { content: [{ type: "text", text }] });
} catch (e) {
reply(id, { content: [{ type: "text", text: `Error: ${e.message}` }], isError: true });
}
return;
}
if (id !== undefined) replyError(id, -32601, `Unsupported method: ${method}`);
}
let buf = "";
process.stdin.setEncoding("utf-8");
process.stdin.on("data", (chunk) => {
buf += chunk;
let nl;
while ((nl = buf.indexOf("\n")) >= 0) {
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
let req;
try {
req = JSON.parse(line);
} catch {
continue;
}
handle(req).catch((e) => log("handler error:", e.message));
}
});
process.stdin.on("end", () => process.exit(0));
log("ready", BASE ? `(${BASE})` : "(WOODPECKER_URL is not set!)");
-53
View File
@@ -1,53 +0,0 @@
# flowagent (Power Automate MCP, via mcpo service) — Secrets Reference
# Source: mcpo/flowagent — built into a custom image, consumed by the
# "mcpo" service in ai.yaml (NEVER mcpo-critical).
#
# Add SECRET values to Woodpecker at:
# https://woodpecker.bryanmail.net
# homelab/compose-files → Settings → Secrets
#
# Prerequisite (manual, outside GitOps — Azure Portal):
# 1. Entra ID → App registrations → New registration
# Name: flowagent-mcp-homelab, single tenant
# 2. API permissions → add Power Automate / Flow Service application
# permissions + Dynamics CRM user_impersonation (Dataverse access)
# 3. Grant admin consent
# 4. Certificates & secrets → new client secret → copy value immediately
# 5. Power Platform Admin Center → target environment → S2S apps →
# register the application user for this app ID (required for
# Dataverse/environment access by a service principal)
#
# NOTE: the OLD "powerautomate" mcpo/config.json entry (npm package
# powerautomate-mcp, client_id 84b431ed-..., tenant_id 0f6cf991-...) was
# never onboarded to this secrets pipeline and is being replaced by this.
# Do not reuse those IDs unless you've independently confirmed in Azure
# Portal that the old App Registration still exists, still has valid
# permissions/consent, and you intend to reuse it — otherwise register new.
# ── SECRETS (add to Woodpecker) ──────────────────────────────────────────
# Woodpecker secret name: flowagent_azure_client_id
# Used for: Azure AD App Registration client ID
# Env var in entrypoint: FLOWAGENT_AZURE_CLIENT_ID_FILE (Docker secret _FILE)
flowagent_azure_client_id=
# Woodpecker secret name: flowagent_azure_tenant_id
# Used for: Azure AD tenant ID
# Env var in entrypoint: FLOWAGENT_AZURE_TENANT_ID_FILE (Docker secret _FILE)
flowagent_azure_tenant_id=
# Woodpecker secret name: flowagent_azure_client_secret
# Used for: Azure AD App Registration client secret (rotate if leaked)
# Env var in entrypoint: FLOWAGENT_AZURE_CLIENT_SECRET_FILE (Docker secret _FILE)
flowagent_azure_client_secret=
# ── Woodpecker provision-secrets case entry ──────────────────────────────
# Add this to the provision-secrets step in .woodpecker/deploy.yml
# (separate follow-up PR — this file only documents it):
#
# ai)
# ... existing ai-stack secret provisioning ...
# create_or_update_secret "flowagent_azure_client_id" "$FLOWAGENT_AZURE_CLIENT_ID"
# create_or_update_secret "flowagent_azure_tenant_id" "$FLOWAGENT_AZURE_TENANT_ID"
# create_or_update_secret "flowagent_azure_client_secret" "$FLOWAGENT_AZURE_CLIENT_SECRET"
# ;;
+38
View File
@@ -0,0 +1,38 @@
# ─────────────────────────────────────────────────────────────────────────────
# secrets-map.yaml — DATA-ONLY manifest for deploy/provision-stack.py
#
# RULES:
# - This file contains NO code, NO shell, NO secret values — only names.
# - Each stack entry declares:
# env_template: repo path of the FULL env-file template (tracked).
# The template is authoritative: the COMPLETE env file is
# rendered from it on every provisioning run. Nothing is
# line-edited in place, so keys can never silently go
# missing.
# env_dest: host path (relative to /volume1/docker/compose-files/)
# the rendered env file is shipped to. Rendered file
# exists ONLY on the host — never committed to git.
# docker_secrets: map of docker-swarm-secret-name -> CI ENV VAR NAME
# (Pattern C). The env var must be declared via
# from_secret: in .woodpecker/deploy.yml's
# provision-secrets step (Woodpecker v3 requires explicit
# per-secret declaration; there is no expose-all).
#
# ADDING A NEW SECRET (3 small steps, no shell edits):
# 1. Add the secret value in Woodpecker UI (repo Settings -> Secrets).
# 2. Declare it in .woodpecker/deploy.yml provision-secrets environment:
# block (from_secret) — mechanical two-line addition.
# 3. Reference it here (docker_secrets:) and/or in the stack's
# .env.template as a dollar-brace placeholder.
#
# Stacks not listed here fall through to deploy.yml's legacy case-entries
# untouched. Migration is deliberately one stack per PR.
# ─────────────────────────────────────────────────────────────────────────────
stacks:
ai:
env_template: ai/ai.env.template
env_dest: ai/ai.env
docker_secrets:
flowagent_azure_client_id: FLOWAGENT_AZURE_CLIENT_ID
flowagent_azure_tenant_id: FLOWAGENT_AZURE_TENANT_ID
flowagent_azure_client_secret: FLOWAGENT_AZURE_CLIENT_SECRET