With fail-fast: true, Gitea cancels remaining matrix slots when one
fails. Combined with molecule_ci_guard's cross-runner cancellation,
a single DinD failure on a high-disk runner kills all 4 slots.
Setting fail-fast: false lets slots on healthy runners complete even
if slots on high-disk runners fail.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:21:06 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Reduce max-parallel from 6 to 4 and add aggressive Docker prune with
disk space gate. Runners at 76% disk after prune fail with "Failed to
create temporary directory" errors when overlay2 runs out of space
under parallel DinD load.
If disk usage >= 85% after prune, skip molecule tests for that slot
with a warning instead of letting containers die with cryptic ENOSPC.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pick up Gitea mirror support for Ansible collections and version
extraction fix.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Switch requirements.yml to type: url entries pointing to the Gitea
package registry. The devx setup.py downloads these with token auth
and installs offline, falling back to galaxy.ansible.com if the
mirror is unavailable.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The prune service and healthcheck were force-removing ALL containers
(except GITEA-ACTIONS-TASK), including molecule test containers that
CI jobs were actively using. This caused "No such container" errors
during molecule prepare/converge phases.
Filter by RunningFor field — only remove containers showing
"hour/day/week/month/year ago", excluding "minutes/seconds ago".
This prevents killing molecule containers from running CI jobs
while still cleaning up stale containers from crashed jobs.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
docker network prune -f removes ALL unused networks, including ones
that molecule tests are actively creating (e.g. 'traefik' network
created during molecule create phase before containers are attached).
This caused "network traefik not found" errors in CI molecule tests.
Add --filter "until=1h" to only prune networks older than 1 hour,
matching the container prune filter.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The healthcheck's `journalctl --user` command fails with "No journal
files were opened due to insufficient permissions" for runner users
that lack journal access. This caused the healthcheck to always report
"OK: runner healthy" even when all runners were unregistered — the
auto-recovery never triggered.
Replace journal-based detection with a Gitea API query: read the
runner's ID from the .runner file and verify it exists in
GET /api/v1/admin/actions/runners. This works regardless of journal
permissions.
Also add scripts/cleanup_stale_runners.py for bulk cleanup of stale
runner registrations (runners that haven't been online for a
configurable threshold).
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The prune service and healthcheck force-remove ALL containers via
"docker ps -aq | xargs -r docker rm -f", but this includes the CI
job container itself (named GITEA-ACTIONS-TASK-*). Removing it causes
"RWLayer of container is unexpectedly nil" errors and kills the
active CI job.
Exclude containers whose name starts with GITEA-ACTIONS-TASK from
the force-remove step. Use "docker ps -a --format" with name filtering
instead of "docker ps -aq".
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:07:38 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The docker-prune systemd service and healthcheck script used
"docker container prune -f" which only removes stopped containers.
Running containers from failed/interrupted molecule tests accumulated
indefinitely on runner hosts, consuming disk and memory, causing CI
test speed variance (12s → 39s for identical test suites).
Changes:
- docker-prune.service.j2: stop+rm ALL containers before system prune
- runner-healthcheck.sh.j2: force-remove all containers when disk
threshold is hit, use "system prune -af --volumes" for thorough cleanup
- template-content/verify.yml: update assertions to match new templates
The molecule_ci_guard.py in devx was also updated to run "molecule
destroy" on test failure/interruption, preventing containers from
being left running in the first place.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
gitea-admin
kireto
emo
emil