Author SHA1 Message Date
admin f7c8ebc05e Add flowagent secrets reference doc (placeholders only, no real values)
Documents the Pattern C secrets needed once mcpo/config.json + ai.yaml are
updated in a follow-up PR to actually wire FlowAgent into the mcpo service.
2026-08-29 20:31:11 -07:00
admin 2ca0643439 Add FlowAgent entrypoint: non-interactive az login via Pattern C secrets
Reads the three _FILE-convention secrets (mounted by Swarm from Docker secrets
provisioned by Woodpecker — see secrets/flowagent.secrets.example), performs
`az login --service-principal`, then execs into whatever command mcpo invokes
(node /app/flowagent/mcp.mjs). No secret value is ever written to disk outside
the ephemeral Docker secret mount, logged, or baked into the image.
2026-08-29 20:31:10 -07:00
admin 5ae478272f Add FlowAgent MCP image (Dockerfile + entrypoint) — build only, not wired to any stack yet
Part 1/3 of FlowAgent (Power Automate MCP) integration into the mcpo service
(NOT mcpo-critical). This PR only adds the image build artifacts; it does not
modify ai.yaml, mcpo/config.json, or the Woodpecker deploy pipeline, so no live
service is affected by merging this alone.

Background: mcpo/config.json previously had a dead/abandoned "powerautomate"
entry (npm package powerautomate-mcp, never onboarded to the secrets pipeline)
that will be replaced by this in a follow-up PR. This build produces a
self-contained image with:
  - azure-cli, for non-interactive `az login --service-principal` at container
    start (see entrypoint.sh)
  - FlowAgent's self-contained MCP engine (server/mcp.mjs from
    microsoft/power-platform-skills — official MIT-licensed Microsoft repo),
    pinned via FLOWAGENT_REF build arg rather than tracking `main`, so builds
    stay reproducible until deliberately bumped.

Requires (added in follow-up PRs, not yet live):
  - Woodpecker secrets: flowagent_azure_client_id, flowagent_azure_tenant_id,
    flowagent_azure_client_secret (Pattern C, Docker secrets via _FILE)
  - Azure AD App Registration with Power Automate + Dataverse permissions,
    admin-consented (manual, outside GitOps — see secrets/flowagent.secrets.example)
2026-08-29 20:31:09 -07:00
16 changed files with 670 additions and 818 deletions
+53 -130
View File
@@ -48,38 +48,6 @@ when:
# already unconditionally rsynced at the top of the deploy step regardless # already unconditionally rsynced at the top of the deploy step regardless
# of which stacks changed, so it's safe to exclude it from the stack list # of which stacks changed, so it's safe to exclude it from the stack list
# everywhere folders are detected below. # everywhere folders are detected below.
#
# 2026-09-03/07 INCIDENT + REDESIGN: hand-maintained grep -v + printf
# line-surgery cases repeatedly drifted (missing '=', duplicated keys,
# mismatched env var names rendering EMPTY secrets) and broke live
# services. Root cause: no authoritative key list and shell heredocs
# hostile to hand-editing. Stacks migrate one at a time to a data-driven
# model: secrets/secrets-map.yaml (data only) + per-stack .env.template
# (authoritative FULL file) + deploy/provision-stack.py (whole-file
# render, hard failure naming any missing value). Migrated stacks call
# the script; unmigrated stacks keep legacy case entries until their own
# PR. See the "Secrets & Deployment Architecture — Global Direction" note.
#
# 2026-09-08 FIX: secrets/ is a tooling/docs folder (secrets-map.yaml +
# *.secrets.example), not a stack — but folder-detection treated it as one
# the first time a commit touched it (PR #16). deploy survived only because
# 'secrets' sits in the bootstrap-tier skip list; verify had no guard and
# died on `docker stack ps secrets` failing under errexit (assignment from
# a failing command substitution aborts the step). Fixed by excluding
# secrets/ alongside deploy/ in ALL folder-detection sites, and by
# tolerating a failing stack-ps in verify (|| true) so a genuinely missing
# stack produces the designed WARNING instead of killing the step. This
# hazard was first flagged in July (PR #3, closed unmerged).
#
# 2026-09-12 MIGRATION: traefik moved from Pattern B (secrets hand-typed
# into the host-only traefik.env, no CI involvement at all) to the same
# manifest-driven provisioning as ai) below. Root cause of the migration:
# traefik.env had been accidentally committed to git with a literal
# "***REDACTED***" placeholder as KEEPALIVED_PASSWORD; every git-guard
# resync/checkout silently restored that broken value, which diverged from
# keepalived-backup's stale-but-correct in-memory value and produced a
# continuous VRRP auth failure ("received an invalid passwd!") and VIP
# instability. The manifest-driven path never commits the rendered file.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
steps: steps:
@@ -97,9 +65,9 @@ steps:
FLAT=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.ya?ml$' || true) FLAT=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.ya?ml$' || true)
# Folder: any file under a subfolder (e.g. immich/immich.yml). # Folder: any file under a subfolder (e.g. immich/immich.yml).
# Exclude dotfolders (.woodpecker, .git, .gitea, etc.) and the # Exclude dotfolders (.woodpecker, .git, .gitea, etc.) and deploy/
# non-stack tooling folders deploy/ and secrets/ (see notes above). # (shared tooling, not a stack — see note above).
FOLDERS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -vE '^(deploy|secrets)$' | sort -u || true) FOLDERS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -v '^deploy$' | sort -u || true)
[ -z "$FLAT" ] && [ -z "$FOLDERS" ] && echo "No stacks changed" && exit 0 [ -z "$FLAT" ] && [ -z "$FOLDERS" ] && echo "No stacks changed" && exit 0
@@ -195,11 +163,6 @@ steps:
from_secret: entertainment_sparky_encryption_key from_secret: entertainment_sparky_encryption_key
ENTERTAINMENT_BETTER_AUTH_SECRET: ENTERTAINMENT_BETTER_AUTH_SECRET:
from_secret: entertainment_better_auth_secret from_secret: entertainment_better_auth_secret
# ── ai stack (manifest-driven — secrets/secrets-map.yaml +
# ai/ai.env.template + deploy/provision-stack.py). Env var names
# below match the template placeholders EXACTLY; this block is the
# only per-secret touchpoint left in this file for migrated stacks
# (Woodpecker v3 requires explicit from_secret declarations). ──
AI_AWS_ACCESS_KEY_ID: AI_AWS_ACCESS_KEY_ID:
from_secret: ai_aws_access_key_id from_secret: ai_aws_access_key_id
AI_AWS_SECRET_ACCESS_KEY: AI_AWS_SECRET_ACCESS_KEY:
@@ -208,31 +171,16 @@ steps:
from_secret: ai_litellm_master_key from_secret: ai_litellm_master_key
AI_LITELLM_SALT_KEY: AI_LITELLM_SALT_KEY:
from_secret: ai_litellm_salt_key from_secret: ai_litellm_salt_key
AI_LITELLM_DATABASE_URL: AI_LITELLM_DB_PASSWORD:
from_secret: ai_litellm_database_url from_secret: ai_litellm_db_password
AI_LITELLM_POSTGRES_PASSWORD: AI_WEBUI_SECRET_KEY:
from_secret: ai_litellm_postgres_password
AI_OPEN_WEBUI_SECRET_KEY:
from_secret: ai_webui_secret_key from_secret: ai_webui_secret_key
AI_OPEN_WEBUI_DATABASE_URL: AI_OPEN_WEBUI_DATABASE_URL:
from_secret: ai_open_webui_database_url from_secret: ai_open_webui_database_url
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET: AI_OAUTH_CLIENT_SECRET:
from_secret: ai_oauth_client_secret from_secret: ai_oauth_client_secret
AI_MCPO_API_KEY:
from_secret: ai_mcpo_api_key
FLOWAGENT_AZURE_CLIENT_ID:
from_secret: flowagent_azure_client_id
FLOWAGENT_AZURE_TENANT_ID:
from_secret: flowagent_azure_tenant_id
FLOWAGENT_AZURE_CLIENT_SECRET:
from_secret: flowagent_azure_client_secret
# ── traefik stack (manifest-driven — secrets/secrets-map.yaml +
# traefik/traefik.env.template + deploy/provision-stack.py).
# Migrated 2026-09-12; see header note above for root cause. ──
TRAEFIK_KEEPALIVED_PASSWORD:
from_secret: traefik_keepalived_password
commands: commands:
- apk add --no-cache openssh-client python3 py3-yaml - apk add --no-cache openssh-client
- mkdir -p ~/.ssh - mkdir -p ~/.ssh
- echo "$SSH_KEY" | base64 -d > ~/.ssh/id_rsa - echo "$SSH_KEY" | base64 -d > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa - chmod 600 ~/.ssh/id_rsa
@@ -245,32 +193,21 @@ steps:
- | - |
CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n') CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n')
FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true) FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true)
FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -vE '^(deploy|secrets)$' | sort -u || true) FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -v '^deploy$' | sort -u || true)
ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u) ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u)
[ -z "$ALL_STACKS" ] && echo "No stacks changed, skipping" && exit 0 [ -z "$ALL_STACKS" ] && echo "No stacks changed, skipping" && exit 0
- scp -o StrictHostKeyChecking=no deploy/create-secrets.sh root@$${SWARM_MANAGER_IP}:/tmp/cs.sh - scp -o StrictHostKeyChecking=no deploy/create-secrets.sh root@$${SWARM_MANAGER_IP}:/tmp/cs.sh
- | - |
CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n') CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n')
FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true) FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true)
FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -vE '^(deploy|secrets)$' | sort -u || true) FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -v '^deploy$' | sort -u || true)
ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u) ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u)
for STACK in $ALL_STACKS; do for STACK in $ALL_STACKS; do
echo "Provisioning: $STACK" echo "Provisioning: $STACK"
case "$STACK" in case "$STACK" in
maintenance|media|unifi|guacamole|security|auth|meshcentral|ddm) maintenance|media|unifi|guacamole|security|auth|traefik|meshcentral|ddm)
echo " No Docker secrets for $STACK — secrets in host .env";; echo " No Docker secrets for $STACK — secrets in host .env";;
traefik)
# MIGRATED (2026-09-12) to manifest-driven provisioning after
# discovering traefik.env was committed to git with a literal
# "***REDACTED***" placeholder as KEEPALIVED_PASSWORD — every
# git-guard resync/checkout restored the broken value, causing
# VRRP auth mismatch between keepalived-master/-backup and VIP
# instability. All logic lives in deploy/provision-stack.py;
# the authoritative key list lives in
# traefik/traefik.env.template; the mapping lives in
# secrets/secrets-map.yaml. This case is intentionally one line.
python3 deploy/provision-stack.py traefik;;
immich) immich)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'immich_db_password' '$${IMMICH_DB_PASSWORD}' create_or_update_secret 'immich_db_password' '$${IMMICH_DB_PASSWORD}'
@@ -284,38 +221,10 @@ steps:
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'gamma_auth_token' '$${GAMMA_AUTH_TOKEN}'";; create_or_update_secret 'gamma_auth_token' '$${GAMMA_AUTH_TOKEN}'";;
git) git)
# PATTERN B, DELIBERATE (see decision notes) — git hosts the source ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
# of truth for every other stack's compose files, so it must be create_or_update_secret 'git_db_password' '$${GIT_DB_PASSWORD}'
# restorable from a flat git.yaml + git.env backup alone, with zero create_or_update_secret 'git_runner_token' '$${GIT_RUNNER_TOKEN}'
# dependency on a running Swarm's Docker secret store. Native Docker create_or_update_secret 'git_mcp_access_token' '$${GIT_MCP_ACCESS_TOKEN}'";;
# secrets (Pattern C) can't satisfy that: they only exist inside an
# already-running Swarm, which is exactly the circular dependency
# this stack can't have. Mirrors the retired ai) case's grep -v +
# printf rewrite-in-place approach, never sed (values may contain
# slash, dollar sign, ampersand).
#
# TEST PHASE: target is git.env.pipelinetest, NOT the real git.env.
# The real file is never opened for writing by this step. First run
# seeds the test file from the real git.env (carries over all
# non-secret lines untouched); every push after that only refreshes
# the 3 secret lines below. Also strips the legacy GITEA_ACCESS_TOKEN
# key name so the test file converges on the git.env.example-
# documented GITEA_MCP_ACCESS_TOKEN key. Cutover to the real file —
# and pointing git.yaml/stack-deploy at it — is a deliberate,
# separate follow-up after manually diffing this render.
# (Candidate for the secrets-map.yaml/provision-stack.py migration
# in its own PR; kept legacy for now.)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "FILE=/volume1/docker/compose-files/git.env.pipelinetest
TMP=\$FILE.tmp.\$\$
[ -f \$FILE ] || cp /volume1/docker/compose-files/git.env \$FILE
grep -vE '^(GITEA__database__PASSWD|GITEA_RUNNER_REGISTRATION_TOKEN|GITEA_MCP_ACCESS_TOKEN|GITEA_ACCESS_TOKEN)=' \$FILE > \$TMP 2>/dev/null || touch \$TMP
{ cat \$TMP
printf 'GITEA__database__PASSWD=%s\n' '$${GIT_DB_PASSWORD}'
printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' '$${GIT_RUNNER_TOKEN}'
printf 'GITEA_MCP_ACCESS_TOKEN=%s\n' '$${GIT_MCP_ACCESS_TOKEN}'
} > \$FILE
rm -f \$TMP
echo ' [OK] git.env.pipelinetest updated - real git.env untouched'";;
homeassistant) homeassistant)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'homeassistant_frigate_rtsp_password' '$${HOMEASSISTANT_FRIGATE_RTSP_PASSWORD}' create_or_update_secret 'homeassistant_frigate_rtsp_password' '$${HOMEASSISTANT_FRIGATE_RTSP_PASSWORD}'
@@ -352,15 +261,40 @@ steps:
create_or_update_secret 'vaultwarden_admin_token' '$${VAULTWARDEN_ADMIN_TOKEN}' create_or_update_secret 'vaultwarden_admin_token' '$${VAULTWARDEN_ADMIN_TOKEN}'
create_or_update_secret 'vaultwarden_database_url_v2' '$${VAULTWARDEN_DATABASE_URL}'";; create_or_update_secret 'vaultwarden_database_url_v2' '$${VAULTWARDEN_DATABASE_URL}'";;
ai) ai)
# MIGRATED (2026-09-07) to manifest-driven provisioning after the # NOTE: ai stack uses Pattern B (host .env, not native Docker secrets) —
# line-surgery approach repeatedly drifted (dropped keys 2026-09-03; # LiteLLM/boto3 and Open WebUI don't support the _FILE convention for these
# duplicate AI_LITELLM_POSTGRES_PASSWORD + stray legacy keys + # vars. Instead of Docker secrets, we regenerate ONLY the migrated lines in
# WEB_UI/WEBUI env-name mismatch rendering an EMPTY OAuth client # the remote ai/ai.env in place via grep -v + printf (never sed, since values
# secret, found 2026-09-07). All logic lives in # may contain slash, dollar sign, ampersand). All other lines — including
# deploy/provision-stack.py; the authoritative key list lives in # MCPO_API_KEY, OAUTH_CLIENT_ID, WEBUI_URL, and other non-secret config — are
# ai/ai.env.template; the mapping lives in secrets/secrets-map.yaml. # left completely untouched. MCPO_API_KEY migration is deferred to a
# This case is intentionally one line. # follow-up; this step never reads or writes it.
python3 deploy/provision-stack.py ai;; #
# IMPORTANT: ai.yaml's litellm service references ${AI_AWS_ACCESS_KEY_ID} /
# ${AI_AWS_SECRET_ACCESS_KEY} (AI_-prefixed) and renders them into the
# container as plain AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (see commit
# 37ed671a, "Change AWS keys to use Woodpecker Secrets"). So ai.env must be
# written with the AI_-prefixed key names, NOT the plain ones — writing
# plain AWS_ACCESS_KEY_ID here would leave ${AI_AWS_ACCESS_KEY_ID}
# unresolved at compose-render time (empty), silently breaking Bedrock auth.
# All other migrated vars in ai.yaml use plain (unprefixed) names, so only
# these two lines need the AI_ prefix.
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "FILE=/volume1/docker/compose-files/ai/ai.env
TMP=\$FILE.tmp.\$\$
grep -vE '^(AI_AWS_ACCESS_KEY_ID|AI_AWS_SECRET_ACCESS_KEY|LITELLM_MASTER_KEY|LITELLM_SALT_KEY|POSTGRES_PASSWORD|DATABASE_URL|WEBUI_SECRET_KEY|OPEN_WEBUI_DATABASE_URL|OAUTH_CLIENT_SECRET)=' \$FILE > \$TMP 2>/dev/null || touch \$TMP
{ cat \$TMP
printf 'AI_AWS_ACCESS_KEY_ID=%s\n' '$${AI_AWS_ACCESS_KEY_ID}'
printf 'AI_AWS_SECRET_ACCESS_KEY=%s\n' '$${AI_AWS_SECRET_ACCESS_KEY}'
printf 'LITELLM_MASTER_KEY=%s\n' '$${AI_LITELLM_MASTER_KEY}'
printf 'LITELLM_SALT_KEY=%s\n' '$${AI_LITELLM_SALT_KEY}'
printf 'POSTGRES_PASSWORD=%s\n' '$${AI_LITELLM_DB_PASSWORD}'
printf 'DATABASE_URL=postgresql://LiteLLM:%s@postgresql:5432/litellm\n' '$${AI_LITELLM_DB_PASSWORD}'
printf 'WEBUI_SECRET_KEY=%s\n' '$${AI_WEBUI_SECRET_KEY}'
printf 'OPEN_WEBUI_DATABASE_URL=%s\n' '$${AI_OPEN_WEBUI_DATABASE_URL}'
printf 'OAUTH_CLIENT_SECRET=%s\n' '$${AI_OAUTH_CLIENT_SECRET}'
} > \$FILE
rm -f \$TMP
echo ' [OK] ai/ai.env secrets updated'";;
entertainment) entertainment)
ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} "source /tmp/cs.sh
create_or_update_secret 'entertainment_discord_token' '$${ENTERTAINMENT_DISCORD_TOKEN}' create_or_update_secret 'entertainment_discord_token' '$${ENTERTAINMENT_DISCORD_TOKEN}'
@@ -397,7 +331,7 @@ steps:
- | - |
CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n') CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n')
FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true) FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true)
FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -vE '^(deploy|secrets)$' | sort -u || true) FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -v '^deploy$' | sort -u || true)
ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u) ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u)
[ -z "$ALL_STACKS" ] && echo "No stacks changed" && exit 0 [ -z "$ALL_STACKS" ] && echo "No stacks changed" && exit 0
@@ -405,13 +339,6 @@ steps:
rsync -av -e "ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa" \ rsync -av -e "ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa" \
deploy/ root@$${SWARM_MANAGER_IP}:/volume1/docker/compose-files/deploy/ deploy/ root@$${SWARM_MANAGER_IP}:/volume1/docker/compose-files/deploy/
# Sync secrets/ tooling (manifest + examples) alongside deploy/ —
# provision-stack.py reads secrets/secrets-map.yaml from the CI
# checkout, but the host mirror should stay complete for emergency
# manual provisioning runs.
rsync -av -e "ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa" \
secrets/ root@$${SWARM_MANAGER_IP}:/volume1/docker/compose-files/secrets/
for STACK in $ALL_STACKS; do for STACK in $ALL_STACKS; do
echo "--- Deploying: $STACK ---" echo "--- Deploying: $STACK ---"
# Sync files to host first (always, even for bootstrap stacks) # Sync files to host first (always, even for bootstrap stacks)
@@ -425,7 +352,7 @@ steps:
# Bootstrap-tier guard: file synced to host, deploy is MANUAL # Bootstrap-tier guard: file synced to host, deploy is MANUAL
case "$STACK" in case "$STACK" in
traefik|woodpecker|postgresql|secrets|git) traefik|woodpecker|postgresql|secrets)
echo " [BOOTSTRAP] $STACK: file synced. Deploy is MANUAL." echo " [BOOTSTRAP] $STACK: file synced. Deploy is MANUAL."
echo " Run: bash /volume1/docker/compose-files/deploy/stack-deploy.sh $STACK" echo " Run: bash /volume1/docker/compose-files/deploy/stack-deploy.sh $STACK"
continue ;; continue ;;
@@ -458,7 +385,7 @@ steps:
- | - |
CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n') CHANGED_FILES=$(echo "${CI_PIPELINE_FILES}" | tr -d '[]"' | tr ',' '\n')
FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true) FLAT_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/]+\.yaml$' | sed 's/\.yaml$//' || true)
FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -vE '^(deploy|secrets)$' | sort -u || true) FOLDER_STACKS=$(echo "$CHANGED_FILES" | grep -E '^[^/.][^/]*/' | cut -d/ -f1 | grep -v '^deploy$' | sort -u || true)
ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u) ALL_STACKS=$(printf '%s\n%s' "$FLAT_STACKS" "$FOLDER_STACKS" | grep -v '^$' | sort -u)
[ -z "$ALL_STACKS" ] && exit 0 [ -z "$ALL_STACKS" ] && exit 0
@@ -469,10 +396,6 @@ steps:
# "nothing found in stack" output on ordinary deploys (e.g. vaultwarden, # "nothing found in stack" output on ordinary deploys (e.g. vaultwarden,
# 2026-08-25). Retry with backoff instead of a single fixed sleep, and # 2026-08-25). Retry with backoff instead of a single fixed sleep, and
# only warn (don't fail the pipeline) if tasks never show up. # only warn (don't fail the pipeline) if tasks never show up.
# 2026-09-08: `|| true` inside the command substitution is REQUIRED —
# this step runs under errexit, and an assignment from a failing
# command substitution (e.g. `docker stack ps` on a stack that doesn't
# exist) kills the whole step before the WARNING path can run.
ATTEMPTS=6 ATTEMPTS=6
DELAY=5 DELAY=5
for STACK in $ALL_STACKS; do for STACK in $ALL_STACKS; do
@@ -481,7 +404,7 @@ steps:
while [ "$i" -le "$ATTEMPTS" ]; do while [ "$i" -le "$ATTEMPTS" ]; do
OUTPUT=$(ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} \ OUTPUT=$(ssh -o StrictHostKeyChecking=no root@$${SWARM_MANAGER_IP} \
"docker stack ps $STACK --filter desired-state=running \ "docker stack ps $STACK --filter desired-state=running \
--format ' {{.Name}} {{.CurrentState}}'" 2>/dev/null || true) --format ' {{.Name}} {{.CurrentState}}'" 2>/dev/null)
if [ -n "$OUTPUT" ]; then if [ -n "$OUTPUT" ]; then
echo "$OUTPUT" echo "$OUTPUT"
break break
-55
View File
@@ -1,55 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────────
# ai.env.template — AUTHORITATIVE template for ai/ai.env (rendered by
# deploy/provision-stack.py per secrets/secrets-map.yaml).
#
# - This file IS the complete key list for ai.env. The whole file is
# rendered on every provisioning run — no line surgery, so a key can
# never silently go missing again (root cause of the 2026-09-03 outage).
# - Key names match EXACTLY what ai/ai.yaml references (AI_<SERVICE>_*
# naming adopted on main 2026-09-06).
# - Non-secret config lives here as LITERAL values (visible, reviewable).
# - Secret values are dollar-brace placeholders resolved from the CI env
# (Woodpecker from_secret vars) at provisioning time. provision-stack.py
# FAILS HARD if any placeholder is missing/empty.
# - The rendered ai/ai.env exists only on the host (gitignored).
# - Rendered by provision-stack.py, NOT Woodpecker's yaml preprocessor —
# single-dollar placeholders are safe here (deploy.yml's double-dollar
# rule does NOT apply to this file).
#
# Consumed by ai/ai.yaml. DOMAIN_NAME comes from deploy/global.env, not here.
# ─────────────────────────────────────────────────────────────────────────────
# ── LiteLLM (non-secret config) ──────────────────────────────────────────────
AI_AWS_REGION_NAME=us-east-2
AI_LITELLM_MODIFY_PARAMS=False
AI_LITELLM_DATABASE_MIGRATIONS=True
# ── LiteLLM (secrets) ────────────────────────────────────────────────────────
AI_AWS_ACCESS_KEY_ID=${AI_AWS_ACCESS_KEY_ID}
AI_AWS_SECRET_ACCESS_KEY=${AI_AWS_SECRET_ACCESS_KEY}
AI_LITELLM_MASTER_KEY=${AI_LITELLM_MASTER_KEY}
AI_LITELLM_SALT_KEY=${AI_LITELLM_SALT_KEY}
# Full connection URL is itself a secret (ai_litellm_database_url) — the
# URL structure never appears in git.
AI_LITELLM_DATABASE_URL=${AI_LITELLM_DATABASE_URL}
AI_LITELLM_POSTGRES_PASSWORD=${AI_LITELLM_POSTGRES_PASSWORD}
# ── Open WebUI (non-secret config) ───────────────────────────────────────────
AI_OPEN_WEBUI_URL=https://ai.bryanmail.net
AI_OPEN_WEBUI_ENABLE_OAUTH_SIGNUP=true
AI_OPEN_WEBUI_OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true
AI_OPEN_WEBUI_OAUTH_PROVIDER_NAME=Authentik
AI_OPEN_WEBUI_OPENID_PROVIDER_URL=https://auth.bryanmail.net/application/o/open-web-ui/.well-known/openid-configuration
# OAuth client ID is a public identifier by OAuth2 design (it is sent to the
# browser); the client SECRET below is the protected credential.
AI_OPEN_WEBUI_OAUTH_CLIENT_ID=hVmhi1dS3TnG2cUw5QwLOx5FDLSWtnQUdZyeB5zK
AI_OPEN_WEBUI_OAUTH_SCOPES=openid email profile
AI_OPEN_WEBUI_OPENID_REDIRECT_URI=https://ai.bryanmail.net/oauth/oidc/callback
# ── Open WebUI (secrets) ─────────────────────────────────────────────────────
AI_OPEN_WEBUI_SECRET_KEY=${AI_OPEN_WEBUI_SECRET_KEY}
AI_OPEN_WEBUI_DATABASE_URL=${AI_OPEN_WEBUI_DATABASE_URL}
AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET=${AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET}
# ── mcpo / mcpo-critical (secrets) ───────────────────────────────────────────
MCPO_API_KEY=${AI_MCPO_API_KEY}
+20 -35
View File
@@ -1,6 +1,6 @@
services: services:
open-webui: open-webui:
image: ghcr.io/open-webui/open-webui:0.11.3 # image: ghcr.io/open-webui/open-webui:0.11.1
entrypoint: entrypoint:
- /bin/bash - /bin/bash
- /app/tools/startup.sh - /app/tools/startup.sh
@@ -16,20 +16,20 @@ services:
start_period: 60s start_period: 60s
environment: environment:
- OLLAMA_BASE_URL=http://ollama-intel-arc:11434 - OLLAMA_BASE_URL=http://ollama-intel-arc:11434
- WEBUI_SECRET_KEY=${AI_OPEN_WEBUI_SECRET_KEY} - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY}
- WEBUI_DB_HOST=postgresql - WEBUI_DB_HOST=postgresql
- DATABASE_URL=${AI_OPEN_WEBUI_DATABASE_URL} - DATABASE_URL=${OPEN_WEBUI_DATABASE_URL}
- ENABLE_TOOL_SERVER_CALLING=true - ENABLE_TOOL_SERVER_CALLING=true
- TOOL_SERVER_CALLING=true - TOOL_SERVER_CALLING=true
- WEBUI_URL=${AI_OPEN_WEBUI_URL} - WEBUI_URL=${WEBUI_URL}
- ENABLE_OAUTH_SIGNUP=${AI_OPEN_WEBUI_ENABLE_OAUTH_SIGNUP} - ENABLE_OAUTH_SIGNUP=${ENABLE_OAUTH_SIGNUP}
- OAUTH_MERGE_ACCOUNTS_BY_EMAIL=${AI_OPEN_WEBUI_OAUTH_MERGE_ACCOUNTS_BY_EMAIL} - OAUTH_MERGE_ACCOUNTS_BY_EMAIL=${OAUTH_MERGE_ACCOUNTS_BY_EMAIL}
- OAUTH_PROVIDER_NAME=${AI_OPEN_WEBUI_OAUTH_PROVIDER_NAME} - OAUTH_PROVIDER_NAME=${OAUTH_PROVIDER_NAME}
- OPENID_PROVIDER_URL=${AI_OPEN_WEBUI_OPENID_PROVIDER_URL} - OPENID_PROVIDER_URL=${OPENID_PROVIDER_URL}
- OAUTH_CLIENT_ID=${AI_OPEN_WEBUI_OAUTH_CLIENT_ID} - OAUTH_CLIENT_ID=${OAUTH_CLIENT_ID}
- OAUTH_CLIENT_SECRET=${AI_OPEN_WEBUI_OAUTH_CLIENT_SECRET} - OAUTH_CLIENT_SECRET=${OAUTH_CLIENT_SECRET}
- OAUTH_SCOPES=${AI_OPEN_WEBUI_OAUTH_SCOPES} - OAUTH_SCOPES=${OAUTH_SCOPES}
- OPENID_REDIRECT_URI=${AI_OPEN_WEBUI_OPENID_REDIRECT_URI} - OPENID_REDIRECT_URI=${OPENID_REDIRECT_URI}
networks: networks:
- traefik_backend - traefik_backend
- postgresql_db-backend - postgresql_db-backend
@@ -62,13 +62,13 @@ services:
environment: environment:
- AWS_ACCESS_KEY_ID=${AI_AWS_ACCESS_KEY_ID} - AWS_ACCESS_KEY_ID=${AI_AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AI_AWS_SECRET_ACCESS_KEY} - AWS_SECRET_ACCESS_KEY=${AI_AWS_SECRET_ACCESS_KEY}
- AWS_REGION_NAME=${AI_AWS_REGION_NAME} - AWS_REGION_NAME=${AWS_REGION_NAME}
- LITELLM_MASTER_KEY=${AI_LITELLM_MASTER_KEY} - LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
- LITELLM_SALT_KEY=${AI_LITELLM_SALT_KEY} - LITELLM_SALT_KEY=${LITELLM_SALT_KEY}
- DATABASE_URL=${AI_LITELLM_DATABASE_URL} - DATABASE_URL=${DATABASE_URL}
- POSTGRES_PASSWORD=${AI_LITELLM_POSTGRES_PASSWORD} - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- LITELLM_MODIFY_PARAMS=${AI_LITELLM_MODIFY_PARAMS} - LITELLM_MODIFY_PARAMS=${LITELLM_MODIFY_PARAMS}
- LITELLM_DATABASE_MIGRATIONS=${AI_LITELLM_DATABASE_MIGRATIONS} - LITELLM_DATABASE_MIGRATIONS=${LITELLM_DATABASE_MIGRATIONS}
networks: networks:
- traefik_backend - traefik_backend
- postgresql_db-backend - postgresql_db-backend
@@ -127,7 +127,7 @@ services:
labels: labels:
- traefik.enable=false - traefik.enable=false
mcpo: mcpo:
image: git.bryanmail.net/homelab/flowagent-mcpo:c5b22618404a image: ghcr.io/open-webui/mcpo:main
command: command:
- --config - --config
- /app/config/config.json - /app/config/config.json
@@ -141,14 +141,6 @@ services:
- /volume1/docker/mcpo/data:/mcpo_data - /volume1/docker/mcpo/data:/mcpo_data
- /volume1/docker/cronicle/ssh_keys:/app/ssh_keys:ro - /volume1/docker/cronicle/ssh_keys:/app/ssh_keys:ro
- /volume1/docker/mcpo/uv-cache:/app/uv-cache - /volume1/docker/mcpo/uv-cache:/app/uv-cache
- /volume1/docker/mcpo/flowagent-auth:/app/flowagent-auth
secrets:
- source: flowagent_azure_client_id
target: flowagent_azure_client_id
- source: flowagent_azure_tenant_id
target: flowagent_azure_tenant_id
- source: flowagent_azure_client_secret
target: flowagent_azure_client_secret
networks: networks:
- traefik_backend - traefik_backend
deploy: deploy:
@@ -181,13 +173,6 @@ services:
- traefik.http.middlewares.n8n.headers.STSPreload=true - traefik.http.middlewares.n8n.headers.STSPreload=true
- traefik.http.routers.n8n.middlewares=forwardAuth-authentik@file, crowdsec@file - traefik.http.routers.n8n.middlewares=forwardAuth-authentik@file, crowdsec@file
- traefik.swarm.network=traefik_backend - traefik.swarm.network=traefik_backend
secrets:
flowagent_azure_client_id:
external: true
flowagent_azure_tenant_id:
external: true
flowagent_azure_client_secret:
external: true
networks: networks:
traefik_backend: traefik_backend:
external: true external: true
+68 -258
View File
@@ -5,20 +5,10 @@
# Behavior: # Behavior:
# - Clean + up to date -> pass silently # - Clean + up to date -> pass silently
# - Clean + behind (ff-only) -> auto `git pull --ff-only`, then pass # - Clean + behind (ff-only) -> auto `git pull --ff-only`, then pass
# - Clean + ahead only -> interactive: offer to push; non-interactive: BLOCK # - Ahead only (unpushed) -> interactive: offer to push; non-interactive: BLOCK
# (unpushed) # - Dirty tracked changes -> offer to commit + push right now
# - Unresolved merge conflict -> REFUSE immediately. Never auto-commits over # (auto in non-interactive/CI runs, after a
# markers present conflict markers. Prints remediation options.
# - Dirty + local in sync -> offer to commit + push right now
# with origin (auto in non-interactive/CI runs, after a
# secret-pattern scan of the staged diff) # secret-pattern scan of the staged diff)
# - Dirty + local STALE/ -> NEVER commit on top of a stale base. Stash
# diverged vs origin the dirty changes first, resync main with
# origin using the same behind/ahead/diverged
# rules as the clean-tree case, then reapply
# the stash and re-run. On any failure the
# stash is preserved and remediation options
# (with exact commands) are printed.
# - Diverged (local AND -> REFUSE. Never auto-resolves. Prints the # - Diverged (local AND -> REFUSE. Never auto-resolves. Prints the
# remote both moved) backup/stash/reset recovery steps and exits. # remote both moved) backup/stash/reset recovery steps and exits.
# #
@@ -46,246 +36,12 @@ git status --porcelain | grep -q . && DIRTY=1
SECRET_PATTERN='(-----BEGIN [A-Z]+ PRIVATE KEY-----|AKIA[0-9A-Z]{16}|xox[baprs]-[0-9a-zA-Z-]+|password[[:space:]]*[:=][[:space:]]*[^$ ]|api[_-]?key[[:space:]]*[:=][[:space:]]*[^$ ])' SECRET_PATTERN='(-----BEGIN [A-Z]+ PRIVATE KEY-----|AKIA[0-9A-Z]{16}|xox[baprs]-[0-9a-zA-Z-]+|password[[:space:]]*[:=][[:space:]]*[^$ ]|api[_-]?key[[:space:]]*[:=][[:space:]]*[^$ ])'
# resync_with_origin <local_sha> <remote_sha> <base_sha>
#
# Handles the behind/ahead/diverged cases against a CLEAN working tree.
# Shared by both the "tree was already clean" path and the new
# "dirty tree turned out to be stale, so we stashed first" path, so the
# two paths can never drift out of sync with each other.
#
# Returns 0 if it's now safe to deploy, 1 if it could not safely resolve
# (guidance already printed to stdout in that case).
resync_with_origin() {
local local_sha="$1" remote_sha="$2" base_sha="$3"
# ---- Case: fully in sync ----
if [ "$local_sha" = "$remote_sha" ]; then
echo "==> In sync with origin/main ($local_sha). OK to deploy."
return 0
fi
# ---- Case: behind only (fast-forwardable) ----
if [ "$local_sha" = "$base_sha" ]; then
echo "!! Local main is behind origin/main."
if [ "$INTERACTIVE" -eq 1 ]; then
read -rp "Fast-forward pull now? [y/N] " ans
else
ans="y"
echo "(non-interactive session — auto fast-forwarding)"
fi
if [[ "$ans" =~ ^[Yy]$ ]]; then
git pull --ff-only origin main
echo "==> Fast-forwarded to $(git rev-parse --short main). OK to deploy."
return 0
else
echo "Aborting - pull manually, then retry:"
echo " cd $DIR && git pull --ff-only origin main"
return 1
fi
fi
# ---- Case: ahead only (local commits not yet pushed) ----
if [ "$remote_sha" = "$base_sha" ]; then
echo "!! Local main is AHEAD of origin/main (unpushed commits):"
git log --oneline "origin/main..main"
echo
if [ "$INTERACTIVE" -eq 1 ]; then
read -rp "Push local commits to origin/main now? [y/N] " ans
else
ans="n"
echo "(non-interactive session — will NOT auto-push ahead commits; needs human review)"
fi
if [[ "$ans" =~ ^[Yy]$ ]]; then
git push origin main
echo "==> Pushed. OK to deploy."
return 0
else
echo "Aborting. Review with:"
echo " cd $DIR && git log origin/main..main"
echo "Then push manually when ready:"
echo " git push origin main"
return 1
fi
fi
# ---- Case: true divergence (both ahead and behind) — NEVER auto-fix ----
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
echo "!! DIVERGED: local main and origin/main have both moved independently."
echo "!!"
echo "!! Local-only commits:"
git log --oneline "$base_sha..main" | sed 's/^/!! /'
echo "!!"
echo "!! Remote-only commits:"
git log --oneline "$base_sha..origin/main" | sed 's/^/!! /'
echo "!!"
echo "!! This requires a human decision - git-guard will NOT auto-resolve this."
echo "!!"
echo "!! Before stashing anything new, check whether a git-guard safety stash"
echo "!! ALREADY exists from this same run (avoids confusing duplicate stashes):"
echo "!! git stash list"
echo "!!"
echo "!! Recommended recovery:"
echo "!! 1. tar backup: tar czf /volume1/docker/compose-files-backup-\$(date +%Y%m%d-%H%M%S).tar.gz -C /volume1/docker compose-files"
echo "!! 2. name the branch: git branch backup/pre-reset-\$(date +%Y%m%d)"
echo "!! 3. stash any NEW uncommitted state only if 'git stash list' above"
echo "!! didn't already show one for this run:"
echo "!! git stash push -u -m 'pre-reset-snapshot'"
echo "!! 4. reset to origin: git reset --hard origin/main"
echo "!! 5. selectively restore needed files from the stash/backup branch:"
echo "!! git stash list"
echo "!! git stash show -p stash@{N}"
echo "!! git stash apply stash@{N} # 'apply' keeps the stash as a backup; use 'pop' to also drop it"
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
return 1
}
# ---- Case: unresolved merge conflict already present ----
# Can happen if a PRIOR git-guard run's `git stash pop` conflicted and the
# resulting conflict markers were never resolved before the next deploy
# attempt. Must be checked BEFORE the dirty-tree commit flow below, because
# an unmerged path shows up as "dirty" too, and `git add -A` would silently
# stage the literal <<<<<<< / ======= / >>>>>>> markers into a real commit.
if git ls-files -u | grep -q .; then
echo "ERROR: unresolved merge conflict markers present in the working tree."
echo "Refusing to auto-commit over a conflict — this would push literal"
echo "<<<<<<< / ======= / >>>>>>> markers to origin/main."
echo
echo "Conflicted paths:"
git diff --name-only --diff-filter=U | sed 's/^/ /'
echo
echo "Remediation options:"
echo " A) Resolve the conflict by hand, then commit and push:"
echo " cd $DIR"
echo " git status # see conflicted paths"
echo " git diff # inspect the conflict markers"
echo " \$EDITOR <conflicted-file> # remove markers, keep correct content"
echo " git add <conflicted-file>"
echo " git commit -m 'resolve git-guard stash-pop conflict'"
echo " git push origin main"
echo " B) Discard the conflicted merge attempt entirely and start clean from"
echo " origin/main, then decide separately whether to re-apply anything"
echo " from a prior safety stash:"
echo " cd $DIR"
echo " git checkout -- ."
echo " git reset --hard origin/main"
echo " git stash list # look for a git-guard-safety-stash-* entry"
echo " git stash show -p stash@{N} # inspect before deciding"
echo " C) Once resolved (via A or B) and no longer needed, clean up the stash:"
echo " git stash drop stash@{N}"
exit 1
fi
# ---- Case: dirty tracked changes ---- # ---- Case: dirty tracked changes ----
if [ "$DIRTY" -eq 1 ]; then if [ "$DIRTY" -eq 1 ]; then
echo "!! WORKING TREE DIRTY — uncommitted changes detected:" echo "!! WORKING TREE DIRTY — uncommitted changes detected:"
git status --short git status --short
echo echo
# If local is ALSO stale/diverged from origin, committing right now would
# create a doomed commit on top of a base that's about to be rejected on
# push (this is exactly what caused a real incident: a stray on-disk edit
# sat on a checkout that was 4 commits behind, git-guard auto-committed
# anyway, then the push bounced). Stash first, resync safely using the
# same rules as the clean-tree path, then reapply.
if [ "$LOCAL" != "$REMOTE" ]; then
echo "!! Local main is ALSO stale/diverged from origin/main."
echo " Refusing to commit on top of a stale base — stashing the dirty"
echo " changes safely first, then resyncing with origin."
echo
STASH_MSG="git-guard-safety-stash-$(date -u +%Y%m%dT%H%M%SZ)"
if ! git stash push -u -m "$STASH_MSG"; then
echo "ERROR: 'git stash push' itself failed (disk full, permissions, or"
echo " some other git error). Your changes are still on disk,"
echo " uncommitted — nothing has been lost, but git-guard cannot"
echo " proceed safely until this is resolved."
echo
echo "Remediation options:"
echo " A) Check disk space and permissions, then retry the deploy:"
echo " df -h $DIR"
echo " ls -la $DIR"
echo " B) Identify and manually move aside whatever is blocking the stash,"
echo " then retry:"
echo " cd $DIR"
echo " git status --short # find the offending path(s)"
echo " mv <path> <path>.bak-\$(date +%s)"
echo " C) Inspect the raw git error above for specifics before proceeding."
exit 1
fi
echo "==> Stashed as: $STASH_MSG"
if resync_with_origin "$LOCAL" "$REMOTE" "$BASE"; then
echo "==> Resync succeeded. Reapplying stashed changes..."
if git stash pop; then
echo "==> Stash reapplied cleanly. Re-checking sync state..."
exec bash "$0" "$@"
else
echo "ERROR: 'git stash pop' did not complete successfully."
echo
if git ls-files -u | grep -q .; then
echo "This is a MERGE CONFLICT — your stashed changes were partially"
echo "applied and conflict markers (<<<<<<< / ======= / >>>>>>>) are now"
echo "in the working tree. The stash itself is still preserved as a backup."
echo
echo "Remediation options:"
echo " A) Resolve the conflict by hand, then commit and push:"
echo " cd $DIR"
echo " git status # see conflicted paths"
echo " git diff # inspect the markers"
echo " \$EDITOR <conflicted-file> # remove markers, keep correct content"
echo " git add <conflicted-file>"
echo " git commit -m 'resolve git-guard stash-pop conflict'"
echo " git push origin main"
echo " git stash list # confirm which entry is: $STASH_MSG"
echo " git stash drop stash@{N} # once confirmed no longer needed"
echo " B) Abandon this merge attempt and fall back to a clean, resynced"
echo " tree, then re-apply the change manually with full visibility:"
echo " cd $DIR"
echo " git checkout -- ."
echo " git reset --hard origin/main # now matches origin, no conflict"
echo " git stash list # find: $STASH_MSG"
echo " git stash show -p stash@{N} # review the content"
echo " git stash apply stash@{N} # 'apply' keeps the backup; use 'pop' to also drop it"
else
echo "This looks like an UNTRACKED-FILE COLLISION, not a merge conflict"
echo "(a file added upstream shares a path with an untracked file in your"
echo "stash). No conflict markers were written; the stash was NOT applied"
echo "and remains fully intact."
echo
echo "Remediation options:"
echo " A) Move the colliding upstream file aside, pop, then reconcile:"
echo " cd $DIR"
echo " git status --short # identify the colliding path"
echo " mv <path> <path>.upstream-\$(date +%s)"
echo " git stash pop"
echo " diff <path> <path>.upstream-* # reconcile manually, then remove the .upstream-* backup"
echo " B) Inspect the stash without applying, and hand-merge the needed"
echo " pieces instead:"
echo " git stash list # find N"
echo " git stash show -p stash@{N}"
fi
echo
echo "Your stash reference for this run: $STASH_MSG"
exit 1
fi
else
echo "ERROR: could not safely resync with origin/main."
echo "Your uncommitted changes are preserved in the stash: $STASH_MSG"
echo
echo "Remediation options:"
echo " A) Follow the manual recovery steps printed above (from the"
echo " behind/ahead/diverged case), THEN reapply your change:"
echo " cd $DIR"
echo " git stash list # find: $STASH_MSG"
echo " git stash apply stash@{N} # or 'pop' to also drop it once resynced"
echo " B) If the stashed change is no longer needed (e.g. it's already"
echo " represented in a since-merged PR), verify then drop it:"
echo " git stash show -p stash@{N}"
echo " git stash drop stash@{N}"
exit 1
fi
fi
if [ "$INTERACTIVE" -eq 1 ]; then if [ "$INTERACTIVE" -eq 1 ]; then
read -rp "Commit and push these changes to origin/main now? [y/N] " ans read -rp "Commit and push these changes to origin/main now? [y/N] " ans
else else
@@ -298,8 +54,7 @@ if [ "$DIRTY" -eq 1 ]; then
if git diff --cached | grep -Eiq "$SECRET_PATTERN"; then if git diff --cached | grep -Eiq "$SECRET_PATTERN"; then
echo "ERROR: possible secret detected in staged changes. Refusing to auto-commit." echo "ERROR: possible secret detected in staged changes. Refusing to auto-commit."
echo "Review manually:" echo "Review manually: git diff --cached"
echo " cd $DIR && git diff --cached"
git reset git reset
exit 1 exit 1
fi fi
@@ -308,25 +63,80 @@ if [ "$DIRTY" -eq 1 ]; then
if git push origin main; then if git push origin main; then
echo "==> Pushed. Re-checking sync state..." echo "==> Pushed. Re-checking sync state..."
exec bash "$0" "$@" exec "$0" "$@"
else else
echo "ERROR: push failed (likely diverged from origin). Aborting deploy." echo "ERROR: push failed (likely diverged from origin). Aborting deploy."
echo "Run:" echo "Run: cd $DIR && git status"
echo " cd $DIR && git status"
exit 1 exit 1
fi fi
else else
echo "Aborting deploy - commit or stash changes manually, then retry:" echo "Aborting deploy - commit or stash changes manually, then retry."
echo " cd $DIR"
echo " git add -A && git commit -m 'your message' && git push origin main"
echo " # or: git stash push -u -m 'manual-stash'"
exit 1 exit 1
fi fi
fi fi
# ---- Clean tree: resync with origin using the shared logic above ---- # ---- Case: fully in sync ----
if resync_with_origin "$LOCAL" "$REMOTE" "$BASE"; then if [ "$LOCAL" = "$REMOTE" ]; then
echo "==> In sync with origin/main ($LOCAL). OK to deploy."
exit 0
fi
# ---- Case: behind only (fast-forwardable) ----
if [ "$LOCAL" = "$BASE" ]; then
echo "!! Local main is behind origin/main."
if [ "$INTERACTIVE" -eq 1 ]; then
read -rp "Fast-forward pull now? [y/N] " ans
else
ans="y"
echo "(non-interactive session — auto fast-forwarding)"
fi
if [[ "$ans" =~ ^[Yy]$ ]]; then
git pull --ff-only origin main
echo "==> Fast-forwarded to $(git rev-parse --short main). OK to deploy."
exit 0 exit 0
else else
echo "Aborting deploy - pull manually, then retry."
exit 1 exit 1
fi fi
fi
# ---- Case: ahead only (local commits not yet pushed) ----
if [ "$REMOTE" = "$BASE" ]; then
echo "!! Local main is AHEAD of origin/main (unpushed commits):"
git log --oneline "origin/main..main"
echo
if [ "$INTERACTIVE" -eq 1 ]; then
read -rp "Push local commits to origin/main now? [y/N] " ans
else
ans="n"
echo "(non-interactive session — will NOT auto-push ahead commits; needs human review)"
fi
if [[ "$ans" =~ ^[Yy]$ ]]; then
git push origin main
echo "==> Pushed. OK to deploy."
exit 0
else
echo "Aborting deploy. Review with: git log origin/main..main"
exit 1
fi
fi
# ---- Case: true divergence (both ahead and behind) — NEVER auto-fix ----
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
echo "!! DIVERGED: local main and origin/main have both moved independently."
echo "!!"
echo "!! Local-only commits:"
git log --oneline "$BASE..main" | sed 's/^/!! /'
echo "!!"
echo "!! Remote-only commits:"
git log --oneline "$BASE..origin/main" | sed 's/^/!! /'
echo "!!"
echo "!! This requires a human decision - git-guard will NOT auto-resolve this."
echo "!! Recommended recovery:"
echo "!! 1. tar backup: tar czf /volume1/docker/compose-files-backup-\$(date +%Y%m%d-%H%M%S).tar.gz -C /volume1/docker compose-files"
echo "!! 2. name the branch: git branch backup/pre-reset-\$(date +%Y%m%d)"
echo "!! 3. stash all state: git stash push -u -m 'pre-reset-snapshot'"
echo "!! 4. reset to origin: git reset --hard origin/main"
echo "!! 5. selectively restore needed files from the stash/backup branch"
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
exit 1
-198
View File
@@ -1,198 +0,0 @@
#!/usr/bin/env python3
"""provision-stack.py — manifest-driven env-file rendering + Docker secret
provisioning for one stack.
Usage (from the CI workspace root, inside the provision-secrets step):
python3 deploy/provision-stack.py <stack>
Reads secrets/secrets-map.yaml (data only — no code, no values) and, for the
named stack:
1. env_template -> renders the COMPLETE env file. Placeholders of the form
dollar-brace VARNAME are resolved from this process's environment (the
Woodpecker from_secret-backed vars). The whole file is rendered every
run; nothing is line-edited in place, so keys can never silently go
missing (root cause of the 2026-09-03 ai.env incident).
2. env_dest -> ships the rendered file to the Swarm manager over ssh stdin
(write-to-temp + atomic mv, mode 600). The rendered file never touches
the CI workspace disk under the repo (no chance of being committed) and
never appears on a command line.
3. docker_secrets -> for each swarm-secret-name -> ENV_VAR mapping, creates
or rotates the Docker secret. Values are passed via ssh stdin only.
Rotation uses the same sha256-checksum-label convention as
deploy/create-secrets.sh (unchanged secrets are skipped silently).
Safety properties:
- FAILS HARD (non-zero) if any referenced env var is missing or empty, and
lists the missing NAMES. A partial/broken render can never ship.
- FAILS HARD if any unresolved placeholder remains after rendering.
- NEVER prints a secret value — names and counts only.
- Requires SWARM_MANAGER_IP in the environment and a usable ssh identity
(both already set up by the provision-secrets step).
Stacks not present in the manifest exit 0 with a notice, so this script is
safe to call unconditionally; legacy case-entries in deploy.yml keep handling
unmigrated stacks.
"""
import hashlib
import os
import re
import subprocess
import sys
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MANIFEST_PATH = os.path.join(REPO_ROOT, "secrets", "secrets-map.yaml")
REMOTE_BASE = "/volume1/docker/compose-files"
PLACEHOLDER_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
SSH_OPTS = ["-o", "StrictHostKeyChecking=no"]
def die(msg: str) -> None:
print(f"ERROR: {msg}", file=sys.stderr)
sys.exit(1)
def load_manifest() -> dict:
try:
import yaml # py3-yaml, installed by the provision-secrets step
except ImportError:
die("PyYAML not available — provision-secrets step must apk add py3-yaml")
if not os.path.isfile(MANIFEST_PATH):
die(f"manifest not found: {MANIFEST_PATH}")
with open(MANIFEST_PATH, "r", encoding="utf-8") as fh:
data = yaml.safe_load(fh) or {}
stacks = data.get("stacks")
if not isinstance(stacks, dict):
die("manifest has no 'stacks:' mapping")
return stacks
def ssh_target() -> str:
ip = os.environ.get("SWARM_MANAGER_IP", "").strip()
if not ip:
die("SWARM_MANAGER_IP is empty — check Woodpecker repo secrets")
return f"root@{ip}"
def ssh_run(target: str, remote_cmd: str, stdin_data: bytes | None = None,
check: bool = True) -> subprocess.CompletedProcess:
proc = subprocess.run(
["ssh", *SSH_OPTS, target, remote_cmd],
input=stdin_data, capture_output=True,
)
if check and proc.returncode != 0:
# stderr may be verbose but must never contain our secret values —
# we only ever send values via stdin, never embed them in remote_cmd.
die(f"remote command failed (rc={proc.returncode}): {remote_cmd}\n"
f"{proc.stderr.decode(errors='replace').strip()}")
return proc
def render_template(template_path: str) -> str:
if not os.path.isfile(template_path):
die(f"env_template not found: {template_path}")
with open(template_path, "r", encoding="utf-8") as fh:
raw = fh.read()
referenced = sorted(set(PLACEHOLDER_RE.findall(raw)))
missing = [v for v in referenced
if not os.environ.get(v, "").strip()]
if missing:
die("template references vars that are MISSING or EMPTY in the CI "
"environment (add them via from_secret in "
".woodpecker/deploy.yml provision-secrets, and as Woodpecker "
"secrets):\n " + "\n ".join(missing))
rendered = PLACEHOLDER_RE.sub(lambda m: os.environ[m.group(1)], raw)
# Belt-and-braces: nothing placeholder-shaped may survive the render.
leftover = sorted(set(PLACEHOLDER_RE.findall(rendered)))
if leftover:
die("unresolved placeholders remain after rendering: "
+ ", ".join(leftover))
print(f" [render] {template_path}: {len(referenced)} secret placeholder(s) "
f"resolved: {', '.join(referenced)}")
return rendered
def ship_env_file(target: str, rendered: str, dest_rel: str) -> None:
dest = f"{REMOTE_BASE}/{dest_rel}"
tmp = f"{dest}.provision-tmp"
# Value travels over ssh stdin; never on a command line; atomic mv.
ssh_run(target,
f"umask 077 && cat > {tmp} && chmod 600 {tmp} && mv {tmp} {dest}",
stdin_data=rendered.encode())
keys = [ln.split("=", 1)[0] for ln in rendered.splitlines()
if "=" in ln and not ln.lstrip().startswith("#") and ln.strip()]
print(f" [env] shipped {dest} ({len(keys)} keys): {', '.join(keys)}")
def provision_docker_secret(target: str, name: str, env_var: str) -> None:
value = os.environ.get(env_var, "")
if not value.strip():
die(f"docker secret '{name}': env var {env_var} is missing/empty")
new_hash = hashlib.sha256(value.encode()).hexdigest()
probe = ssh_run(
target,
f"docker secret inspect {name} "
"--format '{{index .Spec.Labels \"checksum\"}}' 2>/dev/null || true",
check=True)
old_hash = probe.stdout.decode().strip()
if old_hash == new_hash:
print(f" [skip] docker secret {name} (unchanged)")
return
if old_hash:
rm = ssh_run(target, f"docker secret rm {name}", check=False)
if rm.returncode != 0:
die(f"docker secret {name}: value changed but removal failed — "
"it is probably referenced by a running service. Provision "
"under a versioned name (see vaultwarden_database_url_v2 "
"precedent) or scale the service down first.")
action = "update"
else:
action = "create"
ssh_run(target,
f"docker secret create --label checksum={new_hash} "
f"--label managed-by=woodpecker {name} -",
stdin_data=value.encode())
print(f" [{action}] docker secret {name} (value via stdin)")
def main() -> None:
if len(sys.argv) != 2:
die("usage: provision-stack.py <stack>")
stack = sys.argv[1]
stacks = load_manifest()
cfg = stacks.get(stack)
if cfg is None:
print(f" [info] stack '{stack}' not in secrets-map.yaml — "
"legacy provisioning (deploy.yml case-entry) applies. Nothing to do.")
return
target = ssh_target()
print(f"==> provision-stack: {stack}")
template_rel = cfg.get("env_template")
dest_rel = cfg.get("env_dest")
if template_rel and not dest_rel:
die("env_template set but env_dest missing in manifest")
if template_rel:
rendered = render_template(os.path.join(REPO_ROOT, template_rel))
ship_env_file(target, rendered, dest_rel)
for name, env_var in (cfg.get("docker_secrets") or {}).items():
provision_docker_secret(target, name, env_var)
print(f"==> provision-stack: {stack} done")
if __name__ == "__main__":
main()
-9
View File
@@ -63,15 +63,6 @@ services:
FRIGATE_RTSP_PASSWORD: "${FRIGATE_RTSP_PASSWORD}" FRIGATE_RTSP_PASSWORD: "${FRIGATE_RTSP_PASSWORD}"
networks: networks:
- traefik_backend - traefik_backend
ports:
- target: 8555
published: 8555
protocol: udp
mode: ingress
- target: 8555
published: 8555
protocol: tcp
mode: ingress
deploy: deploy:
labels: labels:
- traefik.enable=true - traefik.enable=true
+186
View File
@@ -0,0 +1,186 @@
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-filesystem",
"/mcpo_data/filesystem"
]
},
"memory": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-memory"
]
},
"proxmox-nuck7-1": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.11",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.11",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-1",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"proxmox-nuck7-2": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.12",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.12",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-2",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"proxmox-nuck7-3": {
"command": "sh",
"args": [
"-c",
"LOG_LEVEL=silent npx -y --package @bldg-7/proxmox-mcp@1.2.1 --package pino-pretty proxmox-mcp 2>/dev/null"
],
"env": {
"PROXMOX_HOST": "192.168.4.13",
"PROXMOX_PORT": "8006",
"PROXMOX_USER": "MCP@pve",
"PROXMOX_TOKEN_NAME": "MCP",
"PROXMOX_TOKEN_VALUE": "2052990e-749f-43f6-be7f-c9ad206281cc",
"PROXMOX_SSL_MODE": "insecure",
"PROXMOX_ALLOW_ELEVATED": "true",
"PROXMOX_SSH_ENABLED": "true",
"PROXMOX_SSH_HOST": "192.168.4.13",
"PROXMOX_SSH_PORT": "22",
"PROXMOX_SSH_USER": "root",
"PROXMOX_SSH_KEY_PATH": "/app/ssh_keys/nuc-cluster",
"PROXMOX_SSH_NODE": "nuck7-3",
"PROXMOX_ALLOW_UNSAFE_COMMANDS": "true"
}
},
"homeassistant": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://home.bryanmail.net/mcp_server/sse",
"--header",
"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIxNzhmYzI0NjA2Y2I0ZTg4ODI1N2VmMzdiZTNhY2E5YSIsImlhdCI6MTc3NDY4MzAxNiwiZXhwIjoyMDkwMDQzMDE2fQ.32kY2LVHzKZHWLc96T6z2P-8beNTnp2DHRUf2UEie2w"
]
},
"teams": {
"command": "npx",
"args": [
"-y",
"@floriscornel/teams-mcp@latest"
],
"env": {
"TEAMS_MCP_READ_ONLY": "true",
"HOME": "/app/teams-mcp-auth"
}
},
"ms365": {
"command": "npx",
"args": [
"-y",
"@softeria/ms-365-mcp-server@0.129.0",
"--preset",
"personal",
"--read-only",
"--discovery"
],
"env": {
"HOME": "/app/teams-mcp-auth",
"MS365_MCP_CLIENT_ID": "14d82eec-204b-4c2f-b7e8-296a70dab67e",
"MS365_MCP_TOKEN_CACHE_PATH": "/app/teams-mcp-auth/.teams-mcp-token-cache.json",
"SILENT": "true"
}
},
"unifi-network": {
"command": "uvx",
"args": [
"unifi-network-mcp@latest"
],
"env": {
"UNIFI_HOST": "192.168.4.1",
"UNIFI_USERNAME": "unifi-mcp",
"UNIFI_PASSWORD": "3dHOEOMygTYeX3",
"UNIFI_PORT": "443",
"UNIFI_VERIFY_SSL": "false",
"UV_CACHE_DIR": "/app/uv-cache"
}
},
"authentik": {
"command": "uvx",
"args": [
"authentik-diag-mcp",
"--base-url",
"https://auth.bryanmail.net",
"--token",
"LAm3lBTumOmsU8AiFQM2FmCZyoj8bTSR0FQnAcBy1QnTMEyU4oWozwdxTUap"
]
},
"gitea": {
"command": "/mcpo_data/gitea-mcp",
"args": [
"-t",
"stdio"
],
"env": {
"GITEA_HOST": "https://git.bryanmail.net",
"GITEA_ACCESS_TOKEN": "5aa3a554c001b1dbe5215cb8cb388112801930e9"
}
},
"gitea-admin": {
"command": "/mcpo_data/gitea-mcp",
"args": [
"-t",
"stdio"
],
"env": {
"GITEA_HOST": "https://git.bryanmail.net",
"GITEA_ACCESS_TOKEN": "289225b0b8f1827242191874b2408db76af06321"
}
},
"powerautomate": {
"command": "npx",
"args": [
"-y",
"powerautomate-mcp@latest",
"--stdio"
],
"env": {
"HOME": "/app/powerautomate-auth",
"PA_MCP_CLIENT_ID": "84b431ed-ef5d-48a0-b0a1-878cfdb71453",
"PA_MCP_TENANT_ID": "0f6cf991-c449-480a-a71b-83003ce6edc1",
"PA_CONFIG_PATH": "/app/powerautomate-auth/config.json"
}
}
}
}
+21
View File
@@ -0,0 +1,21 @@
FROM ghcr.io/open-webui/mcpo:main
# Azure CLI — required for non-interactive service-principal auth
# (`az login --service-principal`) performed by entrypoint.sh at container
# start. FlowAgent's auth model is Azure CLI + MSAL; see
# https://github.com/microsoft/power-platform-skills/blob/main/plugins/power-automate/references/connection-patterns.md
RUN apk add --no-cache py3-pip curl \
&& pip install --no-cache-dir --break-system-packages azure-cli
# FlowAgent self-contained MCP bundle (stdio transport, all tools inlined,
# Node 18+ only — no npm install / remote host needed at runtime).
# Pinned to a ref (commit SHA or tag), NOT `main`, for reproducible builds.
# Bump deliberately via PR when upstream ships updates:
# https://github.com/microsoft/power-platform-skills/tree/main/plugins/power-automate/server
ARG FLOWAGENT_REF=main
RUN mkdir -p /app/flowagent \
&& curl -fsSL "https://raw.githubusercontent.com/microsoft/power-platform-skills/${FLOWAGENT_REF}/plugins/power-automate/server/mcp.mjs" \
-o /app/flowagent/mcp.mjs
COPY entrypoint.sh /app/flowagent/entrypoint.sh
RUN chmod +x /app/flowagent/entrypoint.sh
+30
View File
@@ -0,0 +1,30 @@
#!/bin/sh
# FlowAgent MCP entrypoint — non-interactive Azure service-principal login.
#
# Expects three env vars pointing at Docker secret files (Pattern C,
# _FILE convention, provisioned by Woodpecker — never hand-typed):
# FLOWAGENT_AZURE_CLIENT_ID_FILE
# FLOWAGENT_AZURE_TENANT_ID_FILE
# FLOWAGENT_AZURE_CLIENT_SECRET_FILE
#
# On success, execs into the real command (node /app/flowagent/mcp.mjs),
# replacing this shell so mcpo's stdio pipe talks directly to the MCP process.
set -eu
: "${FLOWAGENT_AZURE_CLIENT_ID_FILE:?FLOWAGENT_AZURE_CLIENT_ID_FILE not set}"
: "${FLOWAGENT_AZURE_TENANT_ID_FILE:?FLOWAGENT_AZURE_TENANT_ID_FILE not set}"
: "${FLOWAGENT_AZURE_CLIENT_SECRET_FILE:?FLOWAGENT_AZURE_CLIENT_SECRET_FILE not set}"
CLIENT_ID="$(cat "$FLOWAGENT_AZURE_CLIENT_ID_FILE")"
TENANT_ID="$(cat "$FLOWAGENT_AZURE_TENANT_ID_FILE")"
CLIENT_SECRET="$(cat "$FLOWAGENT_AZURE_CLIENT_SECRET_FILE")"
az login --service-principal \
-u "$CLIENT_ID" \
-p "$CLIENT_SECRET" \
--tenant "$TENANT_ID" \
--output none
unset CLIENT_SECRET
exec "$@"
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env node
// Minimal MCP server (stdio, JSON-RPC 2.0) for Woodpecker CI.
// Zero dependencies — plain Node 18+ (built-in fetch). Config via env vars.
//
// Configuration (env):
// WOODPECKER_URL e.g. https://your-woodpecker.example.com
// WOODPECKER_TOKEN Woodpecker Personal Access Token (JWT)
//
// Tools:
// woodpecker_list_repos
// woodpecker_list_pipelines { repo_id, limit? }
// woodpecker_get_pipeline { repo_id, number } -> status + workflow/step tree
// woodpecker_pipeline_logs { repo_id, number, step_id, tail? } -> decoded step logs
import { Buffer } from "node:buffer";
const BASE = (process.env.WOODPECKER_URL || "").replace(/\/+$/, "");
const TOKEN = process.env.WOODPECKER_TOKEN || "";
function log(...a) {
// Diagnostics go to stderr so they never corrupt the stdout JSON-RPC stream.
process.stderr.write("[woodpecker-mcp] " + a.join(" ") + "\n");
}
async function api(path) {
if (!BASE || !TOKEN) throw new Error("Missing WOODPECKER_URL or WOODPECKER_TOKEN in environment");
const res = await fetch(`${BASE}/api${path}`, {
headers: { Authorization: `Bearer ${TOKEN}`, Accept: "application/json" },
});
const body = await res.text();
if (!res.ok) throw new Error(`HTTP ${res.status} ${path}: ${body.slice(0, 300)}`);
return body ? JSON.parse(body) : null;
}
// --- tool response formatting ---
function summarizePipeline(p) {
const lines = [];
lines.push(`pipeline #${p.number} (id=${p.id}) status=${p.status} event=${p.event} branch=${p.branch}`);
lines.push(`commit=${(p.commit || "").slice(0, 12)} author=${p.author}`);
if (p.message) lines.push(`message: ${p.message.split("\n")[0]}`);
for (const wf of p.workflows || []) {
lines.push(` WORKFLOW "${wf.name}" (pid=${wf.pid}) state=${wf.state}${wf.error ? ` error=${wf.error}` : ""}`);
for (const c of wf.children || []) {
lines.push(
` step "${c.name}" (id=${c.id}, pid=${c.pid}) ${c.state} exit=${c.exit_code ?? "-"} type=${c.type}`
);
}
}
return lines.join("\n");
}
function decodeLogs(entries) {
if (!Array.isArray(entries)) return String(entries);
return entries
.map((e) => {
const d = e?.data;
if (d == null) return "";
try {
return Buffer.from(d, "base64").toString("utf-8");
} catch {
return String(d);
}
})
.join("");
}
const TOOLS = [
{
name: "woodpecker_list_repos",
description: "List repositories the token can access (id, full name, default branch).",
inputSchema: { type: "object", properties: {}, additionalProperties: false },
handler: async () => {
const repos = await api(`/user/repos`);
return (repos || [])
.map((r) => `id=${r.id} ${r.full_name} default_branch=${r.default_branch}`)
.join("\n") || "(no repositories)";
},
},
{
name: "woodpecker_list_pipelines",
description: "Recent pipelines for a repository. Params: repo_id (number), limit (number, default 20).",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number", description: "Woodpecker repository ID" },
limit: { type: "number", description: "How many pipelines to return (default 20)" },
},
required: ["repo_id"],
additionalProperties: false,
},
handler: async ({ repo_id, limit }) => {
const list = await api(`/repos/${repo_id}/pipelines?perPage=${limit || 20}`);
return (list || [])
.map(
(p) =>
`#${p.number} ${p.status.padEnd(8)} ${p.event.padEnd(12)} ${p.branch} ${(p.commit || "").slice(0, 8)} ${(p.message || "").split("\n")[0]}`
)
.join("\n") || "(no pipelines)";
},
},
{
name: "woodpecker_get_pipeline",
description: "Pipeline details: status plus the workflow/step tree (with step ids for fetching logs). Params: repo_id, number.",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number" },
number: { type: "number", description: "Pipeline number (as shown in the UI)" },
},
required: ["repo_id", "number"],
additionalProperties: false,
},
handler: async ({ repo_id, number }) => {
const p = await api(`/repos/${repo_id}/pipelines/${number}`);
return summarizePipeline(p);
},
},
{
name: "woodpecker_pipeline_logs",
description: "Decoded logs for a single step. Params: repo_id, number (pipeline), step_id (from woodpecker_get_pipeline). Optional tail (last N lines).",
inputSchema: {
type: "object",
properties: {
repo_id: { type: "number" },
number: { type: "number" },
step_id: { type: "number", description: "Step id from woodpecker_get_pipeline" },
tail: { type: "number", description: "Return only the last N lines (optional)" },
},
required: ["repo_id", "number", "step_id"],
additionalProperties: false,
},
handler: async ({ repo_id, number, step_id, tail }) => {
const entries = await api(`/repos/${repo_id}/logs/${number}/${step_id}`);
let txt = decodeLogs(entries);
if (tail && tail > 0) {
txt = txt.split("\n").slice(-tail).join("\n");
}
return txt || "(no logs)";
},
},
];
// --- JSON-RPC over stdio loop ---
function send(msg) {
process.stdout.write(JSON.stringify(msg) + "\n");
}
function reply(id, result) {
send({ jsonrpc: "2.0", id, result });
}
function replyError(id, code, message) {
send({ jsonrpc: "2.0", id, error: { code, message } });
}
async function handle(req) {
const { id, method, params } = req;
if (method === "initialize") {
reply(id, {
protocolVersion: params?.protocolVersion || "2024-11-05",
capabilities: { tools: {} },
serverInfo: { name: "woodpecker-mcp", version: "1.0.0" },
});
return;
}
if (method === "notifications/initialized" || method === "notifications/cancelled") {
return; // notifications carry no response
}
if (method === "ping") {
reply(id, {});
return;
}
if (method === "tools/list") {
reply(id, {
tools: TOOLS.map((t) => ({ name: t.name, description: t.description, inputSchema: t.inputSchema })),
});
return;
}
if (method === "tools/call") {
const tool = TOOLS.find((t) => t.name === params?.name);
if (!tool) {
replyError(id, -32602, `Unknown tool: ${params?.name}`);
return;
}
try {
const text = await tool.handler(params.arguments || {});
reply(id, { content: [{ type: "text", text }] });
} catch (e) {
reply(id, { content: [{ type: "text", text: `Error: ${e.message}` }], isError: true });
}
return;
}
if (id !== undefined) replyError(id, -32601, `Unsupported method: ${method}`);
}
let buf = "";
process.stdin.setEncoding("utf-8");
process.stdin.on("data", (chunk) => {
buf += chunk;
let nl;
while ((nl = buf.indexOf("\n")) >= 0) {
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
let req;
try {
req = JSON.parse(line);
} catch {
continue;
}
handle(req).catch((e) => log("handler error:", e.message));
}
});
process.stdin.on("end", () => process.exit(0));
log("ready", BASE ? `(${BASE})` : "(WOODPECKER_URL is not set!)");
+5 -18
View File
@@ -6,9 +6,9 @@ This directory documents the secrets required for each Docker Swarm stack.
``` ```
Secret values live in Woodpecker (encrypted) Secret values live in Woodpecker (encrypted)
pipeline reads them at deploy time pipeline reads them at deploy time
Docker Swarm secret store (encrypted Raft DB, replicated across all nodes) Docker Swarm secret store (encrypted Raft DB, replicated across all nodes)
mounted into containers at runtime mounted into containers at runtime
/run/secrets/<secret_name> /run/secrets/<secret_name>
``` ```
@@ -38,10 +38,7 @@ Use the stack's `.secrets.example` file as your checklist.
### Step 2 — Add the stack's case to `.woodpecker.yml` ### Step 2 — Add the stack's case to `.woodpecker.yml`
In the `provision-secrets` step, add a case for the stack that calls In the `provision-secrets` step, add a case for the stack that calls
`create_or_update_secret` for each secret. (Or, preferred for new `create_or_update_secret` for each secret.
migrations: add an entry to `secrets/secrets-map.yaml` + a
`<stack>.env.template` and call `deploy/provision-stack.py <stack>`
instead — see the `ai` and `traefik` entries for the current pattern.)
### Step 3 — Test by pushing a trivial change to the stack's yaml file ### Step 3 — Test by pushing a trivial change to the stack's yaml file
Watch the pipeline run: provision-secrets → validate → deploy → verify → notify. Watch the pipeline run: provision-secrets → validate → deploy → verify → notify.
@@ -74,20 +71,10 @@ Examples:
- **PostgreSQL is highest risk.** Its master password is used by nearly every - **PostgreSQL is highest risk.** Its master password is used by nearly every
other stack. Migrate it last. other stack. Migrate it last.
- **Never commit a rendered env file, even by accident.** `traefik/traefik.env`
was committed to git for a period (discovered/fixed 2026-09-12) with a
literal "***REDACTED***" placeholder as KEEPALIVED_PASSWORD, which was
silently restored every time the file was deleted or the checkout resynced
from git — causing a real VRRP auth outage. `.gitignore` blanket-excludes
`*.env`, but that rule does NOT retroactively untrack a file already
committed before the rule existed. If you ever see a stack's `.env` file
show up in `git status` as tracked, stop and untrack it (`git rm --cached`)
before doing anything else.
## Migration Status ## Migration Status
| Stack | Secrets in Woodpecker | Pipeline Step Added | .env Removed | | Stack | Secrets in Woodpecker | Pipeline Step Added | .env Removed |
|-------|------------------------|----------------------|---------------| |-------|----------------------|---------------------|--------------|
| 3dprint | ⏳ | ⏳ | ⏳ | | 3dprint | ⏳ | ⏳ | ⏳ |
| ai | ⏳ | ⏳ | ⏳ | | ai | ⏳ | ⏳ | ⏳ |
| auth | ⏳ | ⏳ | ⏳ | | auth | ⏳ | ⏳ | ⏳ |
@@ -104,7 +91,7 @@ Examples:
| postgresql | ⏳ | ⏳ | ⏳ | | postgresql | ⏳ | ⏳ | ⏳ |
| productivity | ⏳ | ⏳ | ⏳ | | productivity | ⏳ | ⏳ | ⏳ |
| security | ⏳ | ⏳ | ⏳ | | security | ⏳ | ⏳ | ⏳ |
| traefik | ✅ (manifest-driven, 2026-09-12) | ✅ (manifest-driven, 2026-09-12) | ⏳ | | traefik | ⏳ | ⏳ | ⏳ |
| unifi | ✅ N/A (no secrets) | ✅ N/A | ⏳ | | unifi | ✅ N/A (no secrets) | ✅ N/A | ⏳ |
| vaultwarden | ⏳ | ⏳ | ⏳ | | vaultwarden | ⏳ | ⏳ | ⏳ |
| woodpecker | ⏳ Manual only | ⏳ N/A | ⏳ | | woodpecker | ⏳ Manual only | ⏳ N/A | ⏳ |
+53
View File
@@ -0,0 +1,53 @@
# flowagent (Power Automate MCP, via mcpo service) — Secrets Reference
# Source: mcpo/flowagent — built into a custom image, consumed by the
# "mcpo" service in ai.yaml (NEVER mcpo-critical).
#
# Add SECRET values to Woodpecker at:
# https://woodpecker.bryanmail.net
# homelab/compose-files → Settings → Secrets
#
# Prerequisite (manual, outside GitOps — Azure Portal):
# 1. Entra ID → App registrations → New registration
# Name: flowagent-mcp-homelab, single tenant
# 2. API permissions → add Power Automate / Flow Service application
# permissions + Dynamics CRM user_impersonation (Dataverse access)
# 3. Grant admin consent
# 4. Certificates & secrets → new client secret → copy value immediately
# 5. Power Platform Admin Center → target environment → S2S apps →
# register the application user for this app ID (required for
# Dataverse/environment access by a service principal)
#
# NOTE: the OLD "powerautomate" mcpo/config.json entry (npm package
# powerautomate-mcp, client_id 84b431ed-..., tenant_id 0f6cf991-...) was
# never onboarded to this secrets pipeline and is being replaced by this.
# Do not reuse those IDs unless you've independently confirmed in Azure
# Portal that the old App Registration still exists, still has valid
# permissions/consent, and you intend to reuse it — otherwise register new.
# ── SECRETS (add to Woodpecker) ──────────────────────────────────────────
# Woodpecker secret name: flowagent_azure_client_id
# Used for: Azure AD App Registration client ID
# Env var in entrypoint: FLOWAGENT_AZURE_CLIENT_ID_FILE (Docker secret _FILE)
flowagent_azure_client_id=
# Woodpecker secret name: flowagent_azure_tenant_id
# Used for: Azure AD tenant ID
# Env var in entrypoint: FLOWAGENT_AZURE_TENANT_ID_FILE (Docker secret _FILE)
flowagent_azure_tenant_id=
# Woodpecker secret name: flowagent_azure_client_secret
# Used for: Azure AD App Registration client secret (rotate if leaked)
# Env var in entrypoint: FLOWAGENT_AZURE_CLIENT_SECRET_FILE (Docker secret _FILE)
flowagent_azure_client_secret=
# ── Woodpecker provision-secrets case entry ──────────────────────────────
# Add this to the provision-secrets step in .woodpecker/deploy.yml
# (separate follow-up PR — this file only documents it):
#
# ai)
# ... existing ai-stack secret provisioning ...
# create_or_update_secret "flowagent_azure_client_id" "$FLOWAGENT_AZURE_CLIENT_ID"
# create_or_update_secret "flowagent_azure_tenant_id" "$FLOWAGENT_AZURE_TENANT_ID"
# create_or_update_secret "flowagent_azure_client_secret" "$FLOWAGENT_AZURE_CLIENT_SECRET"
# ;;
-41
View File
@@ -1,41 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────
# secrets-map.yaml — DATA-ONLY manifest for deploy/provision-stack.py
#
# RULES:
# - This file contains NO code, NO shell, NO secret values — only names.
# - Each stack entry declares:
# env_template: repo path of the FULL env-file template (tracked).
# The template is authoritative: the COMPLETE env file is
# rendered from it on every provisioning run. Nothing is
# line-edited in place, so keys can never silently go
# missing.
# env_dest: host path (relative to /volume1/docker/compose-files/)
# the rendered env file is shipped to. Rendered file
# exists ONLY on the host — never committed to git.
# docker_secrets: map of docker-swarm-secret-name -> CI ENV VAR NAME
# (Pattern C). The env var must be declared via
# from_secret: in .woodpecker/deploy.yml's
# provision-secrets step (Woodpecker v3 requires explicit
# per-secret declaration; there is no expose-all).
#
# ADDING A NEW SECRET (3 small steps, no shell edits):
# 1. Add the secret value in Woodpecker UI (repo Settings -> Secrets).
# 2. Declare it in .woodpecker/deploy.yml provision-secrets environment:
# block (from_secret) — mechanical two-line addition.
# 3. Reference it here (docker_secrets:) and/or in the stack's
# .env.template as a dollar-brace placeholder.
#
# Stacks not listed here fall through to deploy.yml's legacy case-entries
# untouched. Migration is deliberately one stack per PR.
# ─────────────────────────────────────────────────────────────────────────
stacks:
ai:
env_template: ai/ai.env.template
env_dest: ai/ai.env
docker_secrets:
flowagent_azure_client_id: FLOWAGENT_AZURE_CLIENT_ID
flowagent_azure_tenant_id: FLOWAGENT_AZURE_TENANT_ID
flowagent_azure_client_secret: FLOWAGENT_AZURE_CLIENT_SECRET
traefik:
env_template: traefik/traefik.env.template
env_dest: traefik/traefik.env
+10 -30
View File
@@ -1,8 +1,5 @@
# traefik Stack — Secrets Reference # traefik Stack — Secrets Reference
# Source: traefik.env.template (rendered by deploy/provision-stack.py per # Source: traefik.env
# secrets/secrets-map.yaml — see secrets-map.yaml header for how
# this works). traefik/traefik.env is rendered fresh on every
# provisioning run and is NEVER committed to git.
# #
# Add SECRET values to Woodpecker at: # Add SECRET values to Woodpecker at:
# https://woodpecker.bryanmail.net # https://woodpecker.bryanmail.net
@@ -11,44 +8,27 @@
# ⚠️ HIGH RISK: Traefik is the entry point for all homelab services. # ⚠️ HIGH RISK: Traefik is the entry point for all homelab services.
# If this stack fails, nothing is reachable from outside. # If this stack fails, nothing is reachable from outside.
# Migrate carefully. The Keepalived VIP (192.168.4.30) depends on this stack. # Migrate carefully. The Keepalived VIP (192.168.4.30) depends on this stack.
#
# ⚠️ 2026-09-12 INCIDENT: traefik.env was previously committed to git with a
# literal "***REDACTED***" placeholder as KEEPALIVED_PASSWORD. Every
# git-guard resync/checkout restored that broken value onto disk,
# diverging from keepalived-backup's stale-but-correct in-memory value
# and causing continuous VRRP auth failures + VIP instability. This was
# the trigger for migrating this stack to Pattern C. If you ever see
# KEEPALIVED_PASSWORD as a literal placeholder-looking string on disk
# again, do NOT hand-edit it — check `git log traefik/` for a stray
# commit and fix the template in Gitea instead.
# ── SECRETS (add to Woodpecker) ───────────────────────────────────────── # ── SECRETS (add to Woodpecker) ───────────────────────────────────────────────
# Woodpecker secret name: traefik_keepalived_password # Woodpecker secret name: traefik_keepalived_password
# Used for: Keepalived VRRP authentication password # Used for: Keepalived VRRP authentication password
# Must match across all 3 nodes (docker-1, docker-2, docker-3) # Must match across all 3 nodes (docker-1, docker-2, docker-3)
# NOTE: classic VRRP simple-auth is silently # Env var in .env: KEEPALIVED_PASSWORD
# truncated to 8 chars by keepalived itself — keep
# the value <= 8 characters, or be aware only the
# first 8 are actually significant on the wire.
# Env var in .env.template: KEEPALIVED_PASSWORD (via TRAEFIK_KEEPALIVED_PASSWORD)
traefik_keepalived_password= traefik_keepalived_password=
# ── NON-SECRETS (safe in compose file or .env.template) ───────────────── # ── NON-SECRETS (safe in compose file or .env) ────────────────────────────────
# KEEPALIVED_VIRTUAL_IPS Python2BASH list of VIP addresses (192.168.4.30) — literal in template # KEEPALIVED_UNICAST_PEERS Python2BASH list of peer IPs
# KEEPALIVED_VIRTUAL_IPS Python2BASH list of VIP addresses (192.168.4.30)
# ACME_EMAIL Let's Encrypt certificate email # ACME_EMAIL Let's Encrypt certificate email
# TRUSTED_IPS Trusted proxy CIDR ranges # TRUSTED_IPS Trusted proxy CIDR ranges
# TRAEFIK_HOST Traefik dashboard hostname # TRAEFIK_HOST Traefik dashboard hostname
# SPEEDTEST_HOST Speedtest Traefik hostname # SPEEDTEST_HOST Speedtest Traefik hostname
# WHOAMI_HOST Whoami Traefik hostname # WHOAMI_HOST Whoami Traefik hostname
# ── Provisioning (manifest-driven, deploy/provision-stack.py) ─────────── # ── Woodpecker provision-secrets case entry ───────────────────────────────────
# #
# This stack is migrated — provisioning happens automatically via: # traefik)
# secrets/secrets-map.yaml (traefik: entry) # create_or_update_secret "traefik_keepalived_password" "$TRAEFIK_KEEPALIVED_PASSWORD"
# traefik/traefik.env.template (authoritative key list) # ;;
# deploy/provision-stack.py (renders + ships traefik/traefik.env)
#
# The .woodpecker/deploy.yml provision-secrets step calls this with a
# single line: `python3 deploy/provision-stack.py traefik`
+2
View File
@@ -0,0 +1,2 @@
KEEPALIVED_PASSWORD=***REDACTED***
KEEPALIVED_VIRTUAL_IPS="#PYTHON2BASH:['192.168.4.30']"
-39
View File
@@ -1,39 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────
# traefik.env.template — AUTHORITATIVE template for traefik/traefik.env
# (rendered by deploy/provision-stack.py per secrets/secrets-map.yaml)
#
# - This file IS the complete key list for traefik.env. The whole file is
# rendered on every provisioning run — no line surgery, so a key can
# never silently go missing.
# - Non-secret config lives here as LITERAL values (visible, reviewable).
# - Secret values are dollar-brace placeholders resolved from the CI env
# (Woodpecker from_secret vars) at provisioning time. provision-stack.py
# FAILS HARD if any placeholder is missing/empty.
# - The rendered traefik/traefik.env exists only on the host (gitignored).
# - Rendered by provision-stack.py, NOT Woodpecker's yaml preprocessor —
# single-dollar placeholders are safe here (deploy.yml's double-dollar
# rule does NOT apply to this file).
#
# Consumed by traefik/traefik.yaml (KEEPALIVED_PASSWORD, KEEPALIVED_VIRTUAL_IPS).
# DOMAIN comes from deploy/global.env, not here.
#
# ⚠️ HIGH RISK: Traefik is the entry point for all homelab services, and the
# Keepalived VIP (192.168.4.30) depends on this stack. KEEPALIVED_PASSWORD
# MUST be identical across keepalived-master and keepalived-backup — both
# consume this same rendered value — or VRRP auth fails and the VIP
# becomes unstable (this is exactly what the 2026-09-12 incident was:
# a committed literal "***REDACTED***" placeholder in git, restored every
# time the file was deleted/re-synced, diverging from -backup's stale but
# correct in-memory value).
#
# Classic VRRP simple-auth is silently truncated to 8 characters by
# keepalived itself. Set TRAEFIK_KEEPALIVED_PASSWORD in Woodpecker to a
# value 8 characters or fewer (or accept that only the first 8 chars are
# actually significant) so the effective negotiated value is unambiguous.
# ─────────────────────────────────────────────────────────────────────────
# ── Keepalived (non-secret config) ────────────────────────────────────────
KEEPALIVED_VIRTUAL_IPS="#PYTHON2BASH:['192.168.4.30']"
# ── Keepalived (secret) ────────────────────────────────────────────────────
KEEPALIVED_PASSWORD=${TRAEFIK_KEEPALIVED_PASSWORD}