grm/AGENTS.md

# AGENTS.md — Project Conventions for GRM

## Build & Test Commands

```bash
make setup          # Create venv, install deps, set up hooks, install CI tools
make install-tools  # Install actionlint, git-cliff, act_runner to ~/.local/bin
make lint-all       # ruff + pyright + bandit + ansible-lint + checkmake + actionlint
make pytest-cov     # Unit tests with 100% coverage enforcement
make test-unit      # Unit tests without coverage
make molecule       # All 6 scenarios on Ubuntu 22.04
make molecule-all   # All 6 scenarios on all 4 supported OSes
make test-all       # pytest-cov + molecule
make workflow-lint  # Static lint of .gitea/workflows/*.yml (actionlint)
make workflow-dryrun # Dry-run all workflows in Docker (act_runner exec --dryrun)
make workflow-check # workflow-lint + workflow-dryrun
```

`make setup` automatically installs all development tools:
- **Python deps** via `scripts/setup.py` (pip install -e .[dev], ansible-galaxy, pre-commit hooks)
- **checkmake** via `scripts/install_checkmake.py` (Makefile linter)
- **actionlint, git-cliff, act_runner, tea** via `scripts/install_tools.py` (CI/CD tools to ~/.local/bin)
- **tea CLI login** via `scripts/setup.py` (configures `tea login` from `.env` `REPO_TOKEN`)

## Workflow Verification (Before Push)

Workflow YAML files (`.gitea/workflows/*.yml`) are verified with two tools:

1. **actionlint** — Static linter that catches syntax errors, invalid
   expressions, unknown keys, type mismatches, and shellcheck issues.
   Config: `.gitea/actionlint.yaml` (registers custom `docker` runner label).
   Installed automatically by `make setup` via `scripts/install_tools.py`.

2. **act_runner exec --dryrun** — Gitea's own runner in dry-run mode.
   Validates job dependencies, step ordering, and Docker image selection
   without starting containers. Installed automatically by `make setup`.

Both run via `make workflow-check` and are part of `make lint-all`.
The pre-commit hook runs actionlint automatically when workflow files change.
The CI `quality` job runs `make setup` (which installs all tools) then `make lint-all`.
CI also runs a best-effort `make workflow-dryrun` step (skipped if act_runner is not installed in the CI Docker image).

## Architecture

- **Python CLI** (`src/gitea_runner_manager/`) — Click-based CLI that delegates to Ansible
- **Ansible Role** (`ansible/roles/gitea-runner/`) — Idempotent role for rootless Docker runner setup
- **CI Scripts** (`scripts/`) — Automation for auto-merge, post-merge, release, publishing, molecule distribution, PR reviews, failure notifications
- **Versioning** (`cliff.toml`) — git-cliff configuration for automated semver versioning from conventional commits

## PR Workflow (Mandatory)

Every change to master goes through this workflow. No exceptions.

### Branch Protection (Required Gitea Settings)

Branch protection and labels are automatically configured by
`scripts/configure_repo.py`, which runs as a `configure-repo` job in
the post-merge workflow on every push to master.

The following rules are enforced for `master`:
- **Require pull request**: No direct pushes to master
- **Require approval review**: At least 1 `APPROVE` review before merge
- **Require status checks**: CI quality + molecule tests must pass
- **Block force pushes**: No history rewriting on master

The auto-merge workflow enforces the APPROVE review check programmatically
as a defense-in-depth measure, but branch protection is the primary gate.

### 1. Create Vikunja Task
Create a task in Vikunja project 6 to get a `GRM-N` identifier.

### 2. Create Branch
```bash
git checkout master && git pull
git checkout -b GRM-N-short-description
```

### 3. Implement Changes
- Write code following conventions below
- Write/update tests (100% coverage required)
- Update documentation (CHANGELOG, README, AGENTS.md as needed)

### 4. Commit (Conventional Commits)
Branch commits use conventional commit format (no `GRM-N:` prefix):
```
feat: add new feature
fix: resolve bug
docs: update README
```

### 5. Push and Create PR
- **PR title format**: `GRM-N: <vikunja task title>` (must match the Vikunja task title exactly)
- PR body: summary of changes, `Closes GRM-N`
- Add `ready-to-merge` label **only after review is complete**

### 6. Review the PR (Mandatory — Before Adding ready-to-merge Label)

**Review checklist:** Every PR is reviewed against
[REVIEW_CHECKLIST.md](REVIEW_CHECKLIST.md) — 13 categories covering
architecture, code quality, security, i18n, testing, performance,
UX, documentation, workflow compliance, maintainability, resource
management, backwards compatibility, and logging.

**Automated review (CI `pr-review` job):** Every PR triggers an automated
review via `scripts/ci/pr_review.py`. This job posts a review with
`COMMENT` (no issues) or `REQUEST_CHANGES` (issues found) based on
the **[auto]** items in the checklist:

- Architecture compliance (no subprocess in CLI, no hardcoded URLs)
- Best practices (no `print()`, no bare `except`, no `TODO`/`FIXME`,
  no functions > 50 lines)
- Security (no hardcoded secrets, no `shell=True`, no `eval`/`exec`)
- i18n (no raw strings in `click.echo()` without `_()` wrapper)
- Resource management (no `open()` without `with`, no `Popen()` without cleanup)
- Documentation (source changes must include doc updates)
- Test coverage (source changes must include test updates)
- Commit conventions (conventional commit format on PR commits)

The automated review posts inline comments on specific lines and
includes a link to the full checklist. The agent **must** address all
`REQUEST_CHANGES` issues before proceeding.

**Manual review (agent):** After the automated review passes, the agent
must go through **every category** in `REVIEW_CHECKLIST.md` and verify
the **[manual]** items by reviewing the full diff
(`git diff master...HEAD`).

Post review comments using `scripts/ci/review_pr.py`:
```bash
REPO_TOKEN=<token> python3 scripts/ci/review_pr.py <pr_number> <owner/repo> \
  --event REQUEST_CHANGES \
  --body "Review summary" \
  --comments-json comments.json
```

### 7. Address Review Comments
Fix each comment one by one, commit, and push. Re-review until satisfied.

### 8. Approve and Merge
Once all checklist items are verified and comments are addressed, post
an approval review with `--checklist-confirmed` and `--checklist-categories`:
```bash
REPO_TOKEN=<token> python3 scripts/ci/review_pr.py <pr_number> <owner/repo> \
  --event APPROVE --checklist-confirmed \
  --checklist-categories 1,2,3,4,5,6,7,8,9,10,11,12,13 \
  --body "All 13 REVIEW_CHECKLIST.md categories verified. Architecture: <summary>. Security: <summary>. Tests: <summary>. Docs: <summary>."
```

The `--checklist-confirmed` flag is **required** for APPROVE events —
it attests that the reviewer has gone through every checklist category.
The `--checklist-categories` flag is also **required** — it must list at
least 8 of the 13 category numbers, ensuring the reviewer actually
checked each category rather than rubber-stamping. The review body must
be substantive (> 50 characters) — trivial approvals like "LGTM" are
rejected.

Then add the `ready-to-merge` label. The auto-merge workflow will:
1. **Validate** PR title format (`GRM-N: <vikunja task title>`) and match against Vikunja task title
2. **Check** that at least one substantive APPROVE review exists (body > 20 chars or has inline comments)
3. Wait for all CI checks to pass (including the `pr-review` job)
4. Squash-merge with title: `GRM-N <conventional commit message>` (space-separated, no colon after GRM-N)
5. The post-merge workflow marks the Vikunja task as done
6. The release workflow automatically versions, tags, and publishes (see below)

> **IMPORTANT**: Never manually merge PRs via the API. Always use the auto-merge
> workflow by adding the `ready-to-merge` label. Manual merges bypass the
> `GRM-N <conventional>` format enforcement, producing incorrectly named commits.
> The auto-merge script validates the PR title matches the Vikunja task ID
> and conventional commit format before merging.

### CI Path Filtering

The CI workflow includes a `detect-changes` job that checks whether any files
under `ansible/` or `.ansible-lint` have changed. If no Ansible files are
changed, molecule tests are skipped — this prevents non-Ansible changes
(e.g., Python scripts, workflow YAML, docs) from being blocked by molecule
test infrastructure flakiness.

### Dynamic Runner Discovery

Molecule tests are distributed across available Gitea Actions runners
dynamically via `scripts/ci/discover_runners.py`. The `discover-runners`
job queries the Gitea API for runners at all levels (repo, org, instance)
and generates a dynamic matrix. If the API can't see instance-level runners
(no admin scope), it falls back to the `MOLECULE_RUNNERS` repo variable,
then to a default of 3.

**When adding/removing Gitea runners:**
1. Repo/org-level runners are auto-detected via the API
2. For instance-level runners, update the `MOLECULE_RUNNERS` repo variable
3. The workflow automatically scales the matrix to match available runners

### Automated Release Pipeline

After a PR is merged to master, the **post-merge workflow**
(`.gitea/workflows/post-merge.yml`) runs automatically. This single
workflow consolidates release, wiki sync, badge generation, and
Vikunja task updates:

1. **detect-type** — Checks if the commit is a regular merge or a
   release commit (`release: vX.Y.Z`). All subsequent jobs skip for
   release commits (the `[skip ci]` tag also prevents re-triggering).

2. **release** — Runs `scripts/ci/release.py` which:
   - **Checks for user-facing changes** via `scripts/ci/classify_changes.py` — if only
     workflow/infrastructure files changed (`.gitea/`, `scripts/`, `docs/`, `tests/`,
     `AGENTS.md`, `Makefile`, etc.), the release is **skipped entirely** — no version
     bump, no tag, no publish. This prevents unnecessary releases for CI/docs-only changes.
   - Uses **git-cliff** to calculate the next semver version from conventional commits
   - Updates `__version__` in `src/gitea_runner_manager/__init__.py` (single source of truth)
   - Updates `CHANGELOG.md` with the new version section
   - **Runs `make lint-ruff` and `make pytest-cov`** to verify the release is healthy
   - If lint or tests fail, **aborts immediately** — no commit, no tag
   - Commits with `release: vX.Y.Z [skip ci]` prefix (the `[skip ci]` prevents
     re-triggering post-merge on the release commit)
   - Creates an annotated tag `vX.Y.Z` on the release commit
   - Pushes both the commit and tag to master
   - `--skip-tests` flag bypasses test verification (emergency use only, not recommended)
   - Loops are prevented by `has_unreleased_changes` — after a release commit is tagged, the next run finds no unreleased changes and exits

3. **sync-wiki** — Syncs documentation to the Gitea wiki.

4. **badges** — Generates and pushes quality badge SVGs to the `badges` branch.
   Runs **after** the release job (even if release fails or is skipped) so the
   version badge always reflects the latest state. The script fetches the
   latest master before generating badges to pick up any release commits.

5. **vikunja** — Marks the corresponding Vikunja task as done.

The tag push triggers the **publish workflow** (`.gitea/workflows/publish.yml`)
which builds and publishes the package to PyPI.

### Smart CI: User-Facing vs Workflow-Only Changes

Not all changes require the full CI pipeline or a new release. The project
classifies changes into two categories using `scripts/ci/classify_changes.py`:

**Classification strategy (safe-by-default):** Any file NOT in the explicit
workflow-only allowlist is treated as user-facing. This prevents new file
types from accidentally skipping releases.

**Workflow-only paths** (infrastructure → no release needed):
- `.gitea/**` — Gitea Actions workflows
- `scripts/ci/**` — CI/CD automation scripts
- `scripts/setup.py`, `scripts/molecule_all.py`, `scripts/install_tools.py`, `scripts/__init__.py` — Dev tooling and package init
- `docs/**` — Documentation
- `tests/**` — Test files
- `AGENTS.md`, `README.md`, `CHANGELOG.md`, `TROUBLESHOOTING.md` — Project docs
- `Makefile`, `cliff.toml`, `.pre-commit-config.yaml`, `.ansible-lint` — Config
- `.env.example`, `.gitignore`, `.ruff.toml` — Config
- `hooks/**` — Git hooks

**User-facing paths** (tool changes → release needed) — everything else:
- `src/gitea_runner_manager/**` — Python CLI source (except `__init__.py` and `api_clients.py`)
- `ansible/**` — Ansible role
- `pyproject.toml` — Package metadata
- Any new file type not in the allowlist

**Script directory structure:**
- `scripts/` — Dev tools (run locally by developers): `check_test_speed.py`, `configure_repo.py`, `install_checkmake.py`, `install_tools.py`, `setup.py`, `molecule_all.py`, `generate_badges.py`, `gitea_cli.py`
- `scripts/ci/` — CI/CD automation (run by workflows): `release.py`, `publish.py`, `auto_merge.py`, `classify_changes.py`, `detect_release_commit.py`, `push_badges.py`, `doc_coverage.py`, `sync_wiki.py`, `distribute_molecule.py`, `molecule_ci_guard.py`, `discover_runners.py`, `notify_failure.py`, `post_merge.py`, `pr_review.py`, `review_pr.py`, `validate_commit_msg.py`, `platforms.py`

**CI behavior based on classification:**
- **Molecule tests**: Only run when `ansible/` or `.ansible-lint` files change
- **Release dry-run**: Only runs when user-facing files change
- **Quality job** (lint, unit tests, coverage, doc-coverage): Always runs
- **Release workflow**: Skips entirely when no user-facing files changed since last tag

**AI agents must follow these rules:**
- When working on workflow/CI/docs-only changes, use `ci:` or `docs:` commit prefixes
- Do NOT bump the version or create tags for workflow-only changes
- The `classify_changes.py` script enforces this automatically — no manual intervention needed
- When adding a new CI script, place it in `scripts/ci/`. Dev tools go in `scripts/`.

## Script Separation and Import Rules

The codebase enforces strict separation between the GRM tool and CI/dev scripts:

### Directory Layout

| Directory | Purpose | Release impact |
|-----------|---------|----------------|
| `src/gitea_runner_manager/` | User-facing GRM CLI tool | Changes trigger release |
| `scripts/` | Dev tools (run locally) | Workflow-only (no release) |
| `scripts/ci/` | CI/CD automation (run by workflows) | Workflow-only (no release) |
| `ansible/` | Ansible role for runner setup | Changes trigger release |

### Import Rules

1. **`src/gitea_runner_manager/` NEVER imports from `scripts/`** — the tool is self-contained
2. **Scripts MAY import from `gitea_runner_manager`** — one-way dependency (scripts use the tool's API clients, config, i18n)
3. **Cross-script imports** (scripts importing from other scripts) are allowed within `scripts/ci/` but must be documented
4. **`scripts/gitea_cli.py`** is a shared wrapper around the `tea` CLI — CI scripts import from it for Gitea API operations (issues, labels, PRs, releases, reviews)

### tea CLI Integration

The `tea` Gitea CLI tool is used for Gitea API interactions in CI scripts. It is installed by `scripts/install_tools.py` and configured by `scripts/setup.py` (login profile from `.env` `REPO_TOKEN`).

**`scripts/gitea_cli.py`** — Python wrapper around `tea` CLI with JSON output parsing:
- `TeaCLI.create_issue()` — Create issues with labels
- `TeaCLI.list_labels()` / `TeaCLI.create_label()` / `TeaCLI.add_label()` — Label management
- `TeaCLI.create_pr()` / `TeaCLI.merge_pr()` / `TeaCLI.review_pr()` — Pull request operations
- `TeaCLI.create_release()` / `TeaCLI.list_releases()` — Release management
- `TeaCLI.list_branches()` — Branch listing

**Scripts using tea (via `gitea_cli.py`):**
- `scripts/ci/publish.py` — Creates Gitea releases via `tea releases create`
- `scripts/ci/notify_failure.py` — Creates issues via `tea issues create` (falls back to `GiteaClient` if tea not installed)
- `scripts/configure_repo.py` — Creates labels via `tea labels create` (falls back to `GiteaClient` if tea fails; branch protection still uses `GiteaClient` since tea only supports basic protect/unprotect)

**Operations still using `GiteaClient` (not supported by tea):**
- PR reviews (`review_pr.py`) — tea v0.14.1 only supports interactive reviews
- Wiki page management (`sync_wiki.py`)
- Commit status checks (`auto_merge.py`)
- Runner discovery (`discover_runners.py`)
- Branch protection with detailed config (`configure_repo.py`)
- PR file/commit listing (`pr_review.py`)

### PYTHONPATH Configuration

Scripts have different import requirements. Workflows must set `PYTHONPATH` accordingly:

| PYTHONPATH | When to use | Example scripts |
|------------|-------------|-----------------|
| `src` | Script imports from `gitea_runner_manager` | `auto_merge.py`, `pr_review.py`, `review_pr.py`, `sync_wiki.py`, `post_merge.py`, `classify_changes.py`, `discover_runners.py`, `doc_coverage.py` |
| `.:src` | Script imports from both `gitea_runner_manager` and `scripts.gitea_cli` | `publish.py`, `notify_failure.py`, `configure_repo.py` |
| `.` | Script imports from other `scripts.ci.*` modules | `release.py` (imports `classify_changes.has_user_facing_changes`) |
| (none) | Script has no GRM or cross-script imports | `detect_release_commit.py`, `distribute_molecule.py`, `molecule_ci_guard.py`, `push_badges.py`, `validate_commit_msg.py` |

**In workflows**, always use `env:` blocks (not inline `PYTHONPATH=value`):
```yaml
- name: Run script
  env:
    PYTHONPATH: src
  run: python3 scripts/ci/example.py
```

**Locally**, the current directory is in `sys.path` by default, so `PYTHONPATH` is usually not needed.

### Shared Constants

`scripts/ci/platforms.py` is the single source of truth for the molecule
platform matrix. Both `scripts/ci/distribute_molecule.py` (CI) and
`scripts/molecule_all.py` (dev tool) import `PLATFORMS` from it — this
avoids dev tools importing directly from CI scripts.

2. **Publish workflow** (`.gitea/workflows/publish.yml`):
   - Triggers on tag push (`v*`)
   - Validates `PYPI_TOKEN` is set (warns if missing)
   - Builds the Python package
   - Optionally publishes to PyPI (if `PYPI_TOKEN` is set)
   - Creates a Gitea release with git-cliff-generated release notes
   - On failure, creates a Gitea issue via `scripts/ci/notify_failure.py`

### git-cliff Commit Preprocessing

Merge commits on master have the format `GRM-N <conventional commit>`. The
`GRM-N ` prefix is not a valid conventional commit prefix, so `cliff.toml`
includes a `commit_preprocessors` entry that strips it before parsing. This
ensures all merged work appears in the changelog.

### Version Bumping Rules (git-cliff)

| Commit type | Version bump |
|-------------|-------------|
| `feat:` | minor (0.X.0) |
| `fix:` | patch (0.0.X) |
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0: major would be 1.0.0) |
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |

The version source is `__version__` in `src/gitea_runner_manager/__init__.py`, read by setuptools via `dynamic = ["version"]` in `pyproject.toml`. The release script only updates `__init__.py` — no need to touch `pyproject.toml`. `grm --version` reports this version.

### Title Format Summary

| What | Format | Example |
|------|--------|---------|
| Branch name | `GRM-N-short-description` | `GRM-33-add-pr-review-step` |
| Branch commits | `<conventional commit>` | `feat: add review script` |
| PR title | `GRM-N: <vikunja task title>` | `GRM-33: Add mandatory PR review step` |
| Merge commit | `GRM-N <conventional commit>` | `GRM-33 feat: add review script` |

## Key Conventions

- Python 3.12+ required (ruff/pyright target `py312`)
- 100% test coverage required (`--cov-fail-under=100`)
- Conventional commits on feature branches (no `GRM-N:` prefix)
- Branch names must include `GRM-N` task ID
- Line length: 120 chars
- Secrets are passed via temp JSON files, never on the command line (CWE-214)
- CI triggers only on `opened` and `synchronize` PR events (not `labeled`)

## Ansible Role Structure

```
main.yml → systemd_check → user_setup → rootless_docker → install_runner → prune → integration_test
```

- `install_runner.yml` handles: download, config, validate, register, service
- `main.yml` handles: prune, integration_test (NOT install_runner — avoids duplicates)
- `systemctl --user` tasks must be guarded by `docker_rootless_setup`
- Template creation tasks are NOT guarded by `docker_rootless_setup` (they just create files)

## Molecule Scenarios

6 scenarios: `default`, `multi-instance`, `lifecycle`, `template-content`, `deregister`, `update`
4 platforms: `ubuntu-2204`, `ubuntu-2404`, `debian-12`, `archlinux`
Platform list is defined in `scripts/ci/platforms.py` (single source of truth)

## Known Issues

- `ansible-lint` may warn about `command-instead-of-module` for `systemctl --user` calls — this is expected (systemd module doesn't support user services) and skipped in `.ansible-lint`
- Molecule Docker driver may print "Event loop is closed" warnings on interrupt — harmless

## Documentation-as-Code

All documentation lives in `/docs/` and is synced to the Gitea wiki automatically.

### Structure

```
docs/
├── index.md              # Wiki homepage
├── mapping.json          # File-to-wiki-page title mapping
├── user/                 # User documentation
│   ├── getting-started.md
│   ├── installation.md
│   ├── cli-commands.md
│   ├── troubleshooting.md
│   └── faq.md
└── tech/                 # Technical documentation
    ├── architecture.md
    ├── development-setup.md
    ├── ci-cd-workflow.md
    ├── testing-strategy.md
    ├── decision-log.md
    └── contributing.md
```

### Wiki Sync

- **On merge to master**: `sync-wiki.yml` workflow runs `scripts/ci/sync_wiki.py` which pushes all `/docs/` content to the Gitea wiki via API
- **On release tag**: Same sync runs, plus the wiki is tagged with the release version
- `mapping.json` maps each file path to a wiki page title (e.g., `user/getting-started.md` → `Getting-Started`)
- README.md is a lean entry point with links to the wiki — no detailed content

### Documentation Coverage

- `scripts/ci/doc_coverage.py` checks that all CLI commands, Python modules, and CI scripts are documented
- Runs as a CI step in the quality job with `--fail-on-missing` (blocks CI if docs are missing)
- Enforced: 100% coverage for public CLI commands and major architectural components

### Updating Documentation

1. Edit files in `/docs/`
2. If adding a new page, add it to `docs/mapping.json`
3. Commit and create a PR (standard PR workflow)
4. On merge, wiki is automatically synced