GRM-170: Fix stall-detection robustness bugs in runner healthcheck #277

Merged
kireto merged 2 commits from GRM-170-stall-diag-robustness into master 2026-09-18 11:19:16 +00:00
2 changed files with 10 additions and 2 deletions
Showing only changes of commit 8cd0d4dd57 - Show all commits
@@ -57,7 +57,11 @@ now_epoch=$(date +%s)
echo "=== stall diagnostics for $cname ($cid), age ${age_min}m ==="
echo "--- exec probe: TIMEOUT (>10s) ---"
echo "--- docker inspect ---"
timeout 15 docker inspect "$cid" 2>/dev/null
# Implements: REQ-3 — full inspect, but redact the Env block:
# job containers carry CI tokens in env vars; the bundle must
# not become a secret-material artifact.
timeout 15 docker inspect "$cid" 2>/dev/null \
| sed -E 's/("[^"]*(TOKEN|PASSWORD|SECRET|KEY)[^=]*=)[^",]*/\1<redacted>/Ig'
echo "--- docker top ---"
timeout 15 docker top "$cid" 2>/dev/null
echo "--- docker stats --no-stream ---"
+5 -1
View File
@@ -36,7 +36,11 @@ because GNU date rejects `+0000 UTC` together. The age gate then
compares real minutes.
REQ-3: Remove the `head -200` truncation on `docker inspect` output so
the full State block is captured in the diagnostics bundle.
the full State block is captured — but pipe through a `sed` filter that
redacts the value of any env entry whose name contains TOKEN, PASSWORD,
SECRET, or KEY. Job containers carry CI tokens in their Env block; the
diagnostics bundle must not become a secret-material artifact
(OBL-INFRA-548 S02).
REQ-4: Diagnostics-only constraint unchanged — no kills, no restarts.