Three-part effort:
1. Rootless refactor: removes docker/binary modes, unifies to rootless Docker with per-runner system users
2. Auto-merge fix: fix status check context mismatch in branch protection, add retry/wait logic to auto_merge.py
3. Molecule platform matrix: add OS platform matrix to CI (ubuntu-2204, ubuntu-2404, debian-12, archlinux), distribute (scenario, platform) pairs across runners
4. Cross-runner molecule cancellation via Gitea API polling
5. Sequential molecule execution within each runner
6. Fix idempotence, systemd user bus, and Arch Linux package name issues
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:11:48 +00:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The deregister scenario's verify.yml was missing the runner_name var,
which is required because gitea_runner_data_dir depends on it via
defaults/main.yml. Without it, the verify phase fails with
"'runner_name' is undefined".
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:45:47 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Ansible evaluates environment blocks even when when conditions are
false. The deregister scenario sets skip_runner_registration: true
but the environment block still references gitea_runner_uid, causing
"variable is undefined" errors. Add default(0) filter to prevent
this.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:34:07 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Arch Linux pacman package for GPG is called 'gnupg', not 'gpg'.
The molecule prepare.yml was trying to install a non-existent 'gpg'
package, causing failures on the archlinux platform.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:22:20 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The "Enable lingering" task always reported changed=true, and the
"Download gitea_runner binary" task used force=true which always
re-downloads. Both caused molecule idempotence tests to fail.
- Check /var/lib/systemd/linger/<user> before enabling lingering
- Set force=false on get_url so binary is only downloaded if missing
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:13:19 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The "Restart gitea-runner" handler was not guarded by
docker_rootless_setup, causing failures in CI containers without a
systemd user bus. Also add failed_when: false to all lifecycle
side_effect.yml systemctl --user tasks.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:06:27 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The daemon-reload, service restart, and service check tasks in
service.yml, prune.yml, update_runner.yml, and integration_test.yml
were not guarded by docker_rootless_setup. In CI containers without
a systemd user bus, these tasks fail with "Failed to connect to bus".
Also fix the integration_test.yml validation task to not fail on
service status when docker_rootless_setup is false.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:00:07 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Parallel molecule execution within a single runner caused conflicts
(shared temp directories, Docker network collisions). Rewrote
molecule_ci_guard.py to run pairs sequentially while still polling
the Gitea API for cross-runner cancellation.
Each pair now gets its own subprocess with proper environment setup
(MOLECULE_PLATFORM_NAME/IMAGE/COMMAND), and output streams directly
to CI logs for debugging.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:53:13 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
run_molecule_parallel.py was capturing stdout/stderr, which hid the
actual molecule failure details from CI logs. Inherit the parent
stdout/stderr instead so failures are visible for debugging.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:44:16 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Gitea Actions does not implement fail-fast/max-parallel for matrix jobs,
so a failing runner does not stop the others. Added molecule_ci_guard.py
which polls the Gitea API in a background thread. If any other molecule
runner reports failure, the current runner kills its molecule subprocess
and exits early.
CI returns to a 3-runner matrix; each runner executes its assigned pairs
in parallel via run_molecule_parallel.py, guarded by molecule_ci_guard.py.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:37:52 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The initial wait loop used proc.wait(timeout=0.5) which could raise
subprocess.TimeoutExpired and crash the runner. Added a try/except and
increased timeout to 5s so the runner polls correctly without crashing.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:30:36 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Gitea Actions does not honor fail-fast/max-parallel for cancelling
other matrix runners when one fails. Use a single molecule job that
runs all (scenario, platform) pairs via run_molecule_parallel.py.
This gives true parallel execution + immediate termination on the
first failure, which is what we need to debug efficiently.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:25:49 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Added scripts/run_molecule_parallel.py to run a runner's assigned
(scenario, platform) pairs in parallel. If any subprocess fails, the
remaining ones are terminated with SIGTERM/SIGKILL and the runner
exits immediately. This gives fast feedback without continuing to run
tests that are guaranteed to fail for the same reason.
CI workflow now calls this script per matrix runner. Added fail-fast and
max-parallel for best-effort cancellation across runners.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:19:25 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Use a single molecule test job (no matrix) on ubuntu-2204 only. This
stops the workflow immediately when the first scenario fails instead of
wasting time running 3 parallel runners that all fail for the same
reason. Multi-platform will be restored once the base scenarios pass.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 23:08:10 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The validate.yml had an unconditional 'docker version' check, and
service.yml/prune.yml unconditionally enabled services that need
Docker running. Added when: docker_rootless_setup to:
- validate.yml: Verify rootless Docker connectivity
- service.yml: Enable and start gitea-runner service
- prune.yml: Enable and start docker-prune timer
Also made lifecycle side_effect tolerant of service start failure
since Docker daemon isn't available in molecule containers.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:50:25 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
With fail-fast: true and max-parallel: 1, runner 0 must complete
before runner 1 starts. If runner 0 fails, runners 1 and 2 are
cancelled. This gives immediate feedback on the first failure
instead of waiting for all 3 to fail in parallel.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:42:58 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Rootless Docker requires newuidmap/newgidmap kernel support which
doesn't work in nested Docker containers (Operation not permitted).
Added docker_rootless_setup variable (default true) to skip the
daemon startup steps. Set to false in all molecule converge playbooks
so tests verify package installation, user creation, service file
rendering, and config without requiring a working rootless daemon.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:37:31 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Three changes:
1. CI: add set -e and fail-fast: true to stop on first molecule failure
instead of continuing (all pairs fail for same reason anyway)
2. Docker APT repo: use copy module to write sources.list directly
instead of apt_repository module which wasn't picking up the repo
3. Fix arch mapping: ansible_facts returns x86_64, Docker repo needs amd64
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:31:08 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
ansible_facts['architecture'] returns x86_64 but Docker APT repo
expects amd64. Added docker_apt_arch mapping. Also added debug tasks
to show apt sources and apt-cache search results for docker-ce.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:24:02 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The apt_repository update_cache option wasn't reliably picking up the
new Docker APT repo. Split into separate apt update step. Also moved
the long repo string to a default variable to satisfy yaml line-length
lint rule.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:15:28 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The geerlingguy Docker containers don't include curl or gpg, which
are needed by the rootless Docker role to download and dearmor the
Docker APT repository GPG key. Added these prerequisites to the
molecule common prepare playbook.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 22:07:55 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The deb822_repository module isn't available in the CI Ansible
collection. Reverted to apt_repository but now properly dearmors
the GPG key using gpg --dearmor before referencing it in signed-by.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 21:46:33 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The apt_repository module with signed-by wasn't working because the
downloaded GPG key wasn't properly dearmored. The deb822_repository
module handles GPG key download and dearmoring automatically.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 21:35:19 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The rootless_docker.yml task was trying to apt install docker-ce
without first adding the Docker APT repository, causing package not
found errors on Debian/Ubuntu containers.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 21:23:54 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Rootless Docker requires loginctl enable-linger and systemctl --user,
which need systemd as PID 1 inside the container. Updated all platform
entries to use /lib/systemd/systemd (or /usr/lib/systemd/systemd for
Arch) as the container command instead of sleep infinity.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 21:12:43 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The regex only matched "N passed in X.XXs" but pytest can output
"N passed, M warnings in X.XXs". Updated regex to handle both.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 21:06:41 +02:00
Emil SimeonovandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Three major improvements:
1. Rootless Docker refactor: Removes docker/binary modes, unifies to
rootless Docker with per-runner system users. Each runner gets its
own rootless Docker daemon, systemd user service, and isolated
environment. Simplifies CLI (removes --mode option), Ansible role
(single code path), and molecule scenarios (removes binary scenario).
2. Auto-merge fix: Fixes status check context mismatch in branch
protection (was requiring "lint", "unit-tests", "molecule-tests" but
actual contexts are "CI / quality", "CI / molecule-tests*"). Adds
retry/wait logic to auto_merge.py that polls commit statuses for up
to 15 minutes before attempting merge, eliminating the chicken-and-egg
problem where auto-merge would fail because CI hadn't completed yet.
3. Molecule platform matrix: Adds OS platform matrix to CI — all 6
scenarios now run on all 4 supported OSes (ubuntu-2204, ubuntu-2404,
debian-12, archlinux) = 24 test pairs distributed across 3 parallel
runners. Updates distribute_molecule.py to distribute (scenario,
platform) pairs. Updates Makefile with molecule-all target for
local multi-platform testing.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
grm-ci-bot
emil