Flags may appear before or after the stack name (stack-deploy.sh traefik -e
and stack-deploy.sh -e traefik both work) — the stack name is whatever
argument isn't a recognized flag. No change to the render/mount-guard/deploy
logic; this only changes how the git-guard.sh pre-flight is invoked. See
deploy/git-guard.sh header for what each flag actually does.
Adds two opt-in flags to break the circular dependency discovered during
the 2026-09-12 traefik/keepalived incident: Traefik down -> VIP/hostname
unreachable -> git-guard can't fetch -> stack-deploy.sh traefik blocked ->
can't redeploy traefik to fix Traefik.
-e/--emergency: probes git.bryanmail.net, then the VIP (192.168.4.30), then
each node's direct IP (.31/.32/.33) on Gitea's direct ingress port 3000
(bypasses Traefik/VIP entirely), switches origin to the first reachable one,
then runs the NORMAL sync logic against it (behind/ahead/diverged handling
unchanged — this only changes which host is used, never skips the safety
checks). Prints a reminder to restore the real origin URL afterward; never
persists the swap anywhere.
-f/--force: skips the sync check entirely, for the genuine last-resort case
where Gitea itself (not just routing) is unreachable. Loud warning banner.
Verified git ls-remote succeeds unauthenticated over plain http against
each candidate before writing this, so no credential-smuggling concern.
Found via live testing on docker-2 after merging #19: git-guard.sh is
tracked in git at mode 100644 (not executable). stack-deploy.sh correctly
invokes it as `bash deploy/git-guard.sh`, sidestepping the exec bit for the
first call -- but the script's own internal `exec "$0" "$@"` calls (used to
cleanly re-run after a successful push or stash-pop) rely on the kernel
executing the file directly, which requires +x. Since every fresh checkout
or fast-forward preserves the tracked 644 mode, this failed with
"Permission denied" (exit 126) the moment either self-re-invocation path
was actually exercised.
Confirmed via a live dry run: simulated the exact #18 incident shape (dirty
git-guard.sh on a HEAD 3 commits behind origin/main) on docker-2. The new
stash-first logic from #19 worked perfectly end-to-end -- detected
dirty+stale, stashed safely, fast-forwarded via resync_with_origin(), and
popped the stash cleanly -- but then hit this pre-existing bug on the
final `exec "$0" "$@"` re-invocation. This bug pre-dates #19 (the old
dirty-commit-then-push-success path had the identical pattern); #19 just
added a second trigger point that happened to surface it during testing.
Fix: `exec bash "$0" "$@"` explicitly invokes through the interpreter
instead of relying on the file's own execute bit -- correct regardless of
what git tracks the file's mode as.
Adds the unmerged-conflict guard flagged in PR review: before the dirty-tree
flow can run, check `git ls-files -u` and refuse to auto-commit if unresolved
merge conflict markers are present (e.g. left behind by a prior run's failed
`git stash pop`). Without this, a re-run's `git add -A` would silently stage
literal <<<<<<< / ======= / >>>>>>> markers into a real commit and push them
to origin/main.
Also expands every failure-path message (stash-push failure, stash-pop
conflict vs. untracked-file-collision, resync failure, true divergence,
manual-abort cases) to include concrete, copy-pasteable remediation command
sequences with multiple options (resolve-in-place vs. abandon-and-reapply vs.
drop-if-unneeded), rather than a single generic hint. The true-divergence
message now also tells the operator to check `git stash list` first, so a
stash created by the new dirty+stale path isn't confused with a second
manually-created one.
No change to the common dirty-but-current or clean-but-stale behavior.
Previously the dirty-tree branch committed unconditionally, then only
discovered staleness/divergence when the push was rejected -- leaving a
doomed auto-commit sitting on a stale base. Root-caused from a real
incident: a stray on-disk edit to ai/ai.yaml sat on a checkout that was
already 4 commits behind, git-guard auto-committed on top of it, then
the push was rejected (ahead 1, behind 4), aborting the deploy.
Fix: when DIRTY=1 and LOCAL != REMOTE, stash the dirty changes first,
resync main with origin using the exact same behind/ahead/diverged
rules as the clean-tree path (now shared via resync_with_origin()),
then reapply the stash and re-run. Never commits on top of a stale
base again. On any failure the stash is preserved and printed for
manual recovery -- changes are never silently lost.
Carried over from secrets-provisioning-v2 (PR #15), content identical.
Whole-file template render (hard-fail naming missing vars), env + Docker
secrets shipped via ssh stdin only, sha256-checksum skip-if-unchanged,
never prints a value. Stacks absent from the manifest exit 0 so legacy
case-entries keep handling them.
Root cause of the LITELLM_MASTER_KEY/LITELLM_SALT_KEY truncation
incident (2026-08-26): stack-deploy.sh's single-file deploy path is
`envsubst "$VARS" < stack.yaml | docker stack deploy -c - stack`.
envsubst embeds the raw secret value into the compose YAML text. If
that value contains a literal '$' followed by word chars, the
resulting YAML now contains what looks like a second variable
reference. `docker stack deploy -c -` runs Compose's own interpolation
pass on that text before creating the service, finds no such env var,
and silently substitutes empty string -- truncating the secret in the
running container with no error.
Confirmed: an 87-char LITELLM_MASTER_KEY arrived in the ai_litellm
container as 73 chars, silently, on a real deploy.
This is not specific to ai -- it affects every Pattern B stack (host
.env + envsubst, not native Docker secrets): maintenance, media,
unifi, guacamole, security, auth, traefik, meshcentral, ddm. Any of
them could have a '$'-containing value truncating right now without
detection, since the failure produces no warning.
Fix: escape every literal '$' as '$$' in export/export_merged (which
feed the `eval` that sets envsubst's actual source values), before
envsubst ever sees them. envsubst does not interpret '$' in replacement
text, so the doubled dollar survives envsubst intact; Compose's own
interpolation pass then consumes exactly one level of escaping,
landing on the correct single '$' with no leftover false variable
reference. vars/vars_merged (envsubst's allowlist string, unrelated to
values) are untouched.
NOT deployed/merged yet -- pending review. The currently-running
ai_litellm service still has the truncated keys and needs a fresh
`stack-deploy.sh ai` run after this merges to pick up the corrected
values.
- Detect <stack>/ folder vs flat <stack>.yaml automatically
- Main file must be named <stack>.yml or <stack>.yaml inside folder
- Extra *.yml/*.yaml files in folder merged via docker compose config
- Env file resolved from folder/<stack>.env or flat <stack>.env
- Flat path unchanged — zero regression for existing stacks
- envsubst applied after compose config merge in folder mode"
Bot
admin
admin