grm/docs/tech/testing-strategy.md

Testing Strategy

Unit Tests

make test-unit

Runs pytest with 100% coverage requirement.

From the Makefile:

• test-unit — pytest tests/unit/ -v --no-cov (unit tests without coverage)
• pytest-cov — pytest tests/unit/ -v --cov=src/gitea_runner_manager --cov=scripts --cov-report=term-missing --cov-fail-under=100 (unit tests with 100% coverage enforcement)

The coverage requirement is --cov-fail-under=100 — 100% test coverage is required.

Molecule Tests

make molecule      # Quick: all 6 scenarios on Ubuntu 22.04
make molecule-all  # Full: all 6 scenarios on all 4 supported OSes

Runs six scenarios:

• default — Rootless Docker runner installation
• multi-instance — Two isolated runner instances on the same host
• lifecycle — Stop, disable, re-enable, and start sequence
• template-content — Verify rendered systemd user service and prune templates
• deregister — Runner deregistration
• update — Runner binary update

All scenarios test idempotence (second run produces zero changes).

Platforms

4 platforms are tested: ubuntu-2204, ubuntu-2404, debian-12, archlinux.

The platform list is defined in scripts/distribute_molecule.py (single source of truth).

CI Test Distribution

CI runs all 6 scenarios × 4 platforms (24 test pairs) distributed across 3 parallel runners.

From .gitea/workflows/ci.yml, the molecule-tests job uses a matrix of runner-index: [0, 1, 2] and calls scripts/distribute_molecule.py --runner-index <index> --max-runners 3 to discover assigned test pairs, then runs scripts/molecule_ci_guard.py with those pairs.

Integration Tests

make test-integration

Tests the full CLI lifecycle commands end-to-end (mocked executor boundary).

From the Makefile:

• test-integration — pytest tests/integration/ -v --no-cov

Full Test Suite

make test-all      # Runs unit tests + linters + molecule

From the Makefile:

• test-all — pytest-cov + molecule (unit tests with coverage + all 6 molecule scenarios on Ubuntu 22.04)

Build & Test Commands Summary

From AGENTS.md:

make setup          # Create venv, install deps, set up hooks
make lint-all       # ruff + pyright + bandit + ansible-lint + checkmake
make pytest-cov     # Unit tests with 100% coverage enforcement
make test-unit      # Unit tests without coverage
make molecule       # All 6 scenarios on Ubuntu 22.04
make molecule-all   # All 6 scenarios on all 4 supported OSes
make test-all       # pytest-cov + molecule

Known Issues

• ansible-lint may warn about command-instead-of-module for systemctl --user calls — this is expected (systemd module doesn't support user services) and skipped in .ansible-lint
• Molecule Docker driver may print "Event loop is closed" warnings on interrupt — harmless