emil
5b05db4e6d
Add /docs/ directory with user and technical documentation extracted from README, AGENTS.md, and source code. Add scripts/sync_wiki.py to sync docs to Gitea wiki via API. Add scripts/doc_coverage.py to check CLI commands, modules, and CI scripts are documented. Add sync-wiki.yml workflow for auto-sync on merge and release. Slim down README.md to lean entry point. 28 new unit tests, 100% coverage maintained. Closes GRM-36
88 lines
2.8 KiB
Markdown
88 lines
2.8 KiB
Markdown
# Testing Strategy
|
||
|
||
## Unit Tests
|
||
|
||
```bash
|
||
make test-unit
|
||
```
|
||
|
||
Runs pytest with 100% coverage requirement.
|
||
|
||
From the `Makefile`:
|
||
|
||
- `test-unit` — `pytest tests/unit/ -v --no-cov` (unit tests without coverage)
|
||
- `pytest-cov` — `pytest tests/unit/ -v --cov=src/gitea_runner_manager --cov=scripts --cov-report=term-missing --cov-fail-under=100` (unit tests with 100% coverage enforcement)
|
||
|
||
The coverage requirement is `--cov-fail-under=100` — 100% test coverage is required.
|
||
|
||
## Molecule Tests
|
||
|
||
```bash
|
||
make molecule # Quick: all 6 scenarios on Ubuntu 22.04
|
||
make molecule-all # Full: all 6 scenarios on all 4 supported OSes
|
||
```
|
||
|
||
Runs six scenarios:
|
||
|
||
- **default** — Rootless Docker runner installation
|
||
- **multi-instance** — Two isolated runner instances on the same host
|
||
- **lifecycle** — Stop, disable, re-enable, and start sequence
|
||
- **template-content** — Verify rendered systemd user service and prune templates
|
||
- **deregister** — Runner deregistration
|
||
- **update** — Runner binary update
|
||
|
||
All scenarios test idempotence (second run produces zero changes).
|
||
|
||
### Platforms
|
||
|
||
4 platforms are tested: `ubuntu-2204`, `ubuntu-2404`, `debian-12`, `archlinux`.
|
||
|
||
The platform list is defined in `scripts/distribute_molecule.py` (single source of truth).
|
||
|
||
### CI Test Distribution
|
||
|
||
CI runs all 6 scenarios × 4 platforms (24 test pairs) distributed across 3 parallel runners.
|
||
|
||
From `.gitea/workflows/ci.yml`, the `molecule-tests` job uses a matrix of `runner-index: [0, 1, 2]` and calls `scripts/distribute_molecule.py --runner-index <index> --max-runners 3` to discover assigned test pairs, then runs `scripts/molecule_ci_guard.py` with those pairs.
|
||
|
||
## Integration Tests
|
||
|
||
```bash
|
||
make test-integration
|
||
```
|
||
|
||
Tests the full CLI lifecycle commands end-to-end (mocked executor boundary).
|
||
|
||
From the `Makefile`:
|
||
|
||
- `test-integration` — `pytest tests/integration/ -v --no-cov`
|
||
|
||
## Full Test Suite
|
||
|
||
```bash
|
||
make test-all # Runs unit tests + linters + molecule
|
||
```
|
||
|
||
From the `Makefile`:
|
||
|
||
- `test-all` — `pytest-cov + molecule` (unit tests with coverage + all 6 molecule scenarios on Ubuntu 22.04)
|
||
|
||
## Build & Test Commands Summary
|
||
|
||
From `AGENTS.md`:
|
||
|
||
```bash
|
||
make setup # Create venv, install deps, set up hooks
|
||
make lint-all # ruff + pyright + bandit + ansible-lint + checkmake
|
||
make pytest-cov # Unit tests with 100% coverage enforcement
|
||
make test-unit # Unit tests without coverage
|
||
make molecule # All 6 scenarios on Ubuntu 22.04
|
||
make molecule-all # All 6 scenarios on all 4 supported OSes
|
||
make test-all # pytest-cov + molecule
|
||
```
|
||
|
||
## Known Issues
|
||
|
||
- `ansible-lint` may warn about `command-instead-of-module` for `systemctl --user` calls — this is expected (systemd module doesn't support user services) and skipped in `.ansible-lint`
|
||
- Molecule Docker driver may print "Event loop is closed" warnings on interrupt — harmless
|