# Testing Strategy

GRM employs a multi-layered testing strategy: unit tests with 100% coverage enforcement, integration tests for the CLI lifecycle, and Molecule scenarios for Ansible role validation across multiple OS platforms.

## Unit Tests

```bash
make test-unit      # Without coverage
make pytest-cov     # With 100% coverage enforcement
```

Runs pytest with 100% coverage requirement.

From the `Makefile`:

- `test-unit` — `pytest tests/unit/ -v --no-cov` (unit tests without coverage)
- `pytest-cov` — `pytest tests/ -v --cov=src/grm --cov-report=term-missing --cov-fail-under=100` (unit tests with 100% coverage enforcement)

The coverage requirement is `--cov-fail-under=100` — 100% test coverage is required for all code in `src/grm/`. The CI validate job runs `make pytest-cov` on every PR, and the release step in the release-and-maintain job runs it again before tagging a release.

### Test speed verification

The CI validate job also runs `python -m devx.tools.check_test_speed --max-seconds 10` to verify that unit tests run fast (under 10 seconds total). This catches performance regressions early.

## Integration Tests

```bash
make test-integration
```

Tests the full CLI lifecycle commands end-to-end with a mocked executor boundary. This verifies that the CLI correctly parses arguments, resolves runners from the registry, constructs the right Ansible commands, and handles errors — all without actually connecting to remote hosts.

From the `Makefile`:

- `test-integration` — `pytest tests/integration/ -v --no-cov`

Integration tests are marked with `@pytest.mark.integration` and are not counted toward coverage.

## Molecule Tests

```bash
make molecule      # Quick: all 6 scenarios on Ubuntu 22.04
make molecule-all  # Full: all 6 scenarios on all 4 supported OSes
```

Molecule tests validate the Ansible role (`ansible/roles/gitea_runner/`) by creating Docker containers as test hosts, applying the role, and verifying the results. Each scenario tests a specific aspect of the role.

### Scenarios

Seven Molecule scenarios are defined under `ansible/roles/gitea_runner/molecule/`:

| Scenario | Description | What it verifies |
|----------|-------------|------------------|
| `default` | Rootless Docker runner installation | Basic role convergence — user creation, directory structure, binary download, config file, systemd service template, prune timer templates |
| `multi-instance` | Two isolated runner instances on the same host | Two separate converge plays with different runner names; verifies both instances coexist with independent users, data directories, and service files |
| `lifecycle` | Stop, disable, re-enable, and start sequence | Converge, then side_effect stops and disables the service, then re-enables and starts it; verify confirms the service is active again |
| `template-content` | Verify rendered systemd and prune templates | Checks that the systemd user service file contains expected directives (`Type=simple`, `ExecStart`, `Restart=on-failure`, `DOCKER_HOST`, `XDG_RUNTIME_DIR`), and that the prune service and timer templates are correctly rendered |
| `deregister` | Runner deregistration | Creates a fake `.runner` file, then runs the deregister tasks; verifies the `.runner` file is removed |
| `update` | Runner binary update | Converge, then side_effect runs the update playbook; verifies the binary is updated |
| `remove` | Runner removal | Converge, then side_effect runs the remove playbook; verifies the user, directories, and service files are cleaned up |

All scenarios test idempotence (second run produces zero changes), which is a core requirement of the Ansible role.

### Common scenario configuration

All scenarios use `docker_rootless_setup: false` and `skip_runner_registration: true` in their converge playbooks. This is because:

- **Rootless Docker** requires kernel user namespace support, which is not available in all Docker-in-Docker CI environments. The role handles this gracefully via the `docker_rootless_setup` guard.
- **Runner registration** requires a real Gitea instance. The role handles this via the `skip_runner_registration` flag, which skips the `register.yml` and `integration_test.yml` tasks.

### Platforms

4 platforms are tested:

| Platform | Docker image |
|----------|-------------|
| `ubuntu-2204` | `geerlingguy/docker-ubuntu2204-ansible` |
| `ubuntu-2404` | `geerlingguy/docker-ubuntu2404-ansible` |
| `debian-12` | `geerlingguy/docker-debian12-ansible` |
| `archlinux` | `archlinux:latest` |

The platform list is defined in `devx.molecule.platforms` (single source of truth), shared between `devx.molecule.distribute_molecule` (CI) and `devx.molecule.molecule_all` (local dev tool).

### CI Test Distribution

CI runs all 7 scenarios x 4 platforms (28 test pairs) distributed across available Gitea Actions runners.

The `discover-runners` job runs `devx.molecule.discover_runners` which queries the Gitea API for registered runners at three levels (repo, org, instance) and generates a dynamic matrix. If the API query fails (e.g., no admin access for instance-level runners), it falls back to the `MOLECULE_RUNNERS` repo variable, then to a default of 3.

The `molecule-tests` job uses `fromJSON()` to consume the dynamic matrix, and passes the runner count to `python -m devx.molecule.distribute_molecule --max-runners` so test pairs are evenly distributed.

`devx.molecule.distribute_molecule` discovers all molecule scenarios under `ansible/roles/*/molecule/` and crosses them with the supported OS platform matrix, then splits the resulting test pairs evenly across the requested number of runners. Each pair is encoded as `scenario|platform_name|platform_image|platform_command`.

The CI workflow runs each test pair sequentially via a shell loop that sets the appropriate `MOLECULE_PLATFORM_*` environment variables and invokes `molecule test` directly.

### Path-based CI filtering

The CI workflow includes a `detect-changes` job that checks whether any files under `ansible/` or `.ansible-lint` have changed. If no Ansible files are changed, molecule tests are skipped — this prevents non-Ansible changes (e.g., Python scripts, workflow YAML, docs) from being blocked by molecule test infrastructure flakiness.

## Full Test Suite

```bash
make test-all      # Runs pytest-cov + molecule (Ubuntu 22.04)
```

From the `Makefile`:

- `test-all` — `pytest-cov + molecule` (unit tests with coverage + all 6 molecule scenarios on Ubuntu 22.04)

For a complete test across all platforms, use `make molecule-all` separately.

## Build & Test Commands Summary

```bash
make setup          # Create venv, install deps, set up hooks, install CI tools
make lint-all       # ruff + pyright + bandit + ansible-lint + checkmake + actionlint
make pytest-cov     # Unit tests with 100% coverage enforcement
make test-unit      # Unit tests without coverage
make test-integration # Integration tests
make molecule       # All 6 scenarios on Ubuntu 22.04
make molecule-all   # All 6 scenarios on all 4 supported OSes
make test-all       # pytest-cov + molecule
```

## Known Issues

- `ansible-lint` may warn about `command-instead-of-module` for `systemctl --user` calls — this is expected (systemd module doesn't support user services) and skipped in `.ansible-lint`
- Molecule Docker driver may print "Event loop is closed" warnings on interrupt — harmless