oblachno-oss/grm
2
0
Fork 0
Code Issues 12 Pull Requests 1 Actions Packages Projects Releases 7 Wiki Activity

fix: reduce healthcheck interval to 1min and disk thresholds

Browse Source 
Runners are consistently running out of disk during parallel molecule
tests. The 2min healthcheck interval is too coarse — a single molecule
scenario can fill 10+ GB in 2 minutes. Reduce to 1min and lower disk
thresholds (60% warn, 70% critical) so pruning kicks in earlier.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Emil Simeonov
2026-08-10 09:04:24 +02:00
co-authored by Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
parent c192929e34
commit 3f22910a7d
1 changed files with 8 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ansible/roles/gitea_runner/defaults/main.yml
+8 -6
View File
@@ -36,18 +36,20 @@ gitea_runner_prune_label: "gitea-runner=true"
gitea_runner_service_restart_sec: "5"
# Health check configuration
# 2min interval — catches hung daemons before multiple CI jobs fail between checks.
# The previous 5min interval was too coarse: a stuck daemon could fail 3+ molecule
# jobs in the window between healthcheck runs.
gitea_runner_healthcheck_interval: "2min"
# 1min interval — catches hung daemons and disk exhaustion before multiple CI
# jobs fail between checks. The previous 2min interval was too coarse under
# parallel DinD load: a single molecule scenario can fill 10+ GB in 2 minutes,
# causing subsequent scenarios on the same runner to fail with "Failed to
# create temporary directory" (disk full).
gitea_runner_healthcheck_interval: "1min"
gitea_runner_healthcheck_boot_delay: "2min"
gitea_runner_healthcheck_disk_threshold: 70
gitea_runner_healthcheck_disk_threshold: 60
# When disk reaches this level, prune EVERYTHING (no until-filter) — the
# runner is dangerously full and the gentle until=1h prune isn't enough.
# This removes all stopped containers and unused images regardless of age.
# At 75%+, molecule containers fail with "container is not running" because
# overlay2 runs out of space under parallel DinD load.
gitea_runner_healthcheck_disk_critical: 75
gitea_runner_healthcheck_disk_critical: 70
gitea_runner_healthcheck_script_path: "{{ gitea_runner_config_dir }}/healthcheck.sh"
# Auto-recovery: when the healthcheck detects an unregistered runner, it