grm/docs/tech/ci-cd-workflow.md

# CI/CD Workflow

GRM uses a fully automated CI/CD pipeline built on Gitea Actions. Every change to master goes through a mandatory PR workflow with branch protection, automated review, and auto-merge. Releases are automated via git-cliff and conventional commits.

## Workflow Overview

| Workflow | Trigger | Purpose |
|----------|---------|---------|
| `ci.yml` | PR opened/synchronized | Quality checks (lint, test, coverage) + molecule tests |
| `auto-merge.yml` | PR labeled `ready-to-merge` | Validates and squash-merges the PR |
| `post-merge.yml` | Push to `master` | Release, wiki sync, badges, Vikunja task update |
| `publish.yml` | Tag push (`v*`) | Build and publish package to PyPI, create Gitea release |

Every change to master goes through a mandatory PR workflow. No exceptions.

## PR Workflow

### 1. Create Vikunja Task

Create a task in Vikunja project 6 to get a `GRM-N` identifier.

### 2. Create Branch

```bash
git checkout master && git pull
git checkout -b GRM-N-short-description
```

### 3. Implement Changes

- Write code following conventions
- Write/update tests (100% coverage required)
- Update documentation (CHANGELOG, README, AGENTS.md as needed)

### 4. Commit (Conventional Commits)

Branch commits use conventional commit format (no `GRM-N:` prefix):

```
feat: add new feature
fix: resolve bug
docs: update README
```

### 5. Push and Create PR

- **PR title format**: `GRM-N: <vikunja task title>` (must match the Vikunja task title exactly)
- PR body: summary of changes, `Closes GRM-N`
- Add `ready-to-merge` label **only after review is complete**

### 6. Review the PR (Mandatory — Before Adding ready-to-merge Label)

Review the full diff (`git diff master...HEAD`) focusing on:

- **Functional completeness**: Does the code do what it claims? Are all requirements met?
- **Edge cases**: Are boundary conditions, empty inputs, error paths handled?
- **Technical excellence**:
  - Architecture compliance and evolution
  - Single Responsibility Principle (SRP)
  - Deduplication (no copy-paste, single source of truth)
  - Code smells detection and removal
  - Best industry practices
  - Industry-grade code quality
  - Reusability
  - Clean code
  - Readability
  - Maintainability
  - Extensibility
- **Performance**: No unnecessary allocations, O(n) vs O(n²), efficient data structures
- **Security**: No secrets in logs/process list, input validation, no injection vectors
- **User experience**: Clear error messages, intuitive CLI flags, helpful output
- **Documentation**: Completeness and relevance of docs, CHANGELOG entries, AGENTS.md updates

Post review comments using `devx.ci.pr_review`:

```bash
CI_GITEA_TOKEN=<token> python -m devx.ci.pr_review <pr_number> <owner/repo> \
  --event REQUEST_CHANGES \
  --body "Review summary" \
  --comments-json comments.json
```

### 7. Address Review Comments

Fix each comment one by one, commit, and push. Re-review until satisfied.

### 8. Approve and Merge

Once all comments are addressed:

```bash
CI_GITEA_TOKEN=<token> python -m devx.ci.pr_review <pr_number> <owner/repo> \
  --event APPROVE \
  --body "All comments addressed. LGTM."
```

Then add the `ready-to-merge` label. The auto-merge workflow will:

1. **Validate** PR title format and match against Vikunja task title
2. **Check** that at least one APPROVE review exists
3. Wait for all CI checks to pass
4. Squash-merge with title: `GRM-N <conventional commit message>` (space-separated)
5. The post-merge workflow marks the Vikunja task as done
6. The release workflow automatically versions, tags, and publishes

### 9. Post-Merge Automation

After the squash-merge:

- The **post-merge workflow** (`.gitea/workflows/post-merge.yml`) triggers on push to `master` and runs `devx.ci.post_merge` to mark the Vikunja task as done, extracting the task ID from the merge commit message.
- The **release workflow** (`.gitea/workflows/release.yml`) triggers on push to `master` and automatically versions, tags, and publishes (see below).

## Branch Protection (Required Gitea Settings)

Configure the following branch protection rules for `master` in Gitea repo settings:

- **Require pull request**: No direct pushes to master
- **Require approval review**: At least 1 `APPROVE` review before merge
- **Require status checks**: CI quality + molecule tests must pass
- **Block force pushes**: No history rewriting on master

The auto-merge workflow enforces the APPROVE review check programmatically as a defense-in-depth measure, but branch protection is the primary gate.

## CI Path Filtering

The CI workflow (`.gitea/workflows/ci.yml`) includes a `detect-changes` job that checks whether any files under `ansible/` or `.ansible-lint` have changed. If no Ansible files are changed, molecule tests are skipped — this prevents non-Ansible changes (e.g., Python scripts, workflow YAML, docs) from being blocked by molecule test infrastructure flakiness.

The `detect-changes` job:

- For pull requests: compares `origin/master` against the PR head SHA
- For pushes to master: compares `HEAD~1` against `HEAD`
- Outputs `ansible-changed` as `true` or `false`

The `molecule-tests` job depends on both `quality` and `detect-changes`, and only runs if `ansible-changed == 'true'`.

CI triggers only on `opened` and `synchronize` PR events (not `labeled`).

## CI Quality Job

The `quality` job in `.gitea/workflows/ci.yml` runs:

1. `make setup` — full environment setup
2. `make lint-all` — ruff + pyright + bandit + ansible-lint + checkmake
3. `make pytest-cov` — unit tests with 100% coverage enforcement
4. `python -m devx.tools.check_test_speed --max-seconds 10` — verify unit tests run fast
5. `PYTHONPATH=src python -m devx.ci.release --dry-run` — release dry-run validation

## Automated Release Pipeline

After a PR is merged to master, the release pipeline runs automatically.

### Release Workflow (`.gitea/workflows/release.yml`)

- Triggers on push to `master`
- Sets up full dev environment (`make setup`) so lint and tests can run
- Installs git-cliff (version 2.13.0)
- Configures git as `grm-ci-bot`
- Runs `devx.ci.release` which uses **git-cliff** to:
  - **Checks for user-facing changes** via `devx.ci.classify_changes` — if only workflow/infrastructure files changed, the release is **skipped entirely** — no version bump, no tag, no publish
  - Calculate the next semver version from conventional commits since the last tag
  - Update `__version__` in `src/gitea_runner_manager/__init__.py` (single source of truth)
  - Update `CHANGELOG.md` with the new version section
  - **Run `make lint-ruff` and `make pytest-cov`** to verify the release is healthy
  - If lint or tests fail, **abort immediately** — no commit, no tag
  - Commit with `release: vX.Y.Z [skip ci]` prefix (the `[skip ci]` prevents re-triggering post-merge on the release commit)
  - Create an annotated tag `vX.Y.Z` on the release commit
  - Push both the commit and tag to master
- `--skip-tests` flag bypasses test verification (emergency use only, not recommended)
- Loops are prevented by `has_unreleased_changes` — after a release commit is tagged, the next run finds no unreleased changes and exits
- On failure, creates a Gitea issue via `devx.ci.notify_failure`

### Publish Workflow (`.gitea/workflows/publish.yml`)

- Triggers on tag push (`v*`)
- Installs git-cliff (version 2.13.0)
- Installs build tools (`build`, `twine`, `requests`, `python-dotenv`, `click`)
- Validates `PYPI_TOKEN` is set (warns if missing)
- Builds the Python package
- Optionally publishes to PyPI (if `PYPI_TOKEN` is set)
- Creates a Gitea release with git-cliff-generated release notes
- Uses `devx.ci.publish` for build and publish orchestration
- On failure, creates a Gitea issue via `devx.ci.notify_failure`

### Auto-Merge Workflow (`.gitea/workflows/auto-merge.yml`)

- Triggers on `pull_request` labeled events
- Runs `devx.ci.auto_merge` with the branch name, PR title, repository, PR number, and label name
- Validates PR title format, checks for APPROVE review, waits for CI, and squash-merges

### Post-Merge Workflow (`.gitea/workflows/post-merge.yml`)

- Triggers on push to `master`
- Consolidates release, wiki sync, badge generation, and Vikunja task updates into a single workflow
- **detect-type** — Runs `devx.ci.detect_release_commit` to check if the commit is a release commit (`release: vX.Y.Z`). All subsequent jobs skip for release commits (the `[skip ci]` tag also prevents re-triggering).
- **release** — Runs `devx.ci.release` (see Automated Release Pipeline below)
- **sync-wiki** — Syncs documentation to the Gitea wiki via `devx.ci.sync_wiki`
- **badges** — Generates and pushes quality badge SVGs to the `badges` branch via `devx.ci.push_badges`. Runs after the release job (even if release fails or is skipped) so the version badge always reflects the latest state.
- **vikunja** — Marks the corresponding Vikunja task as done via `devx.ci.post_merge`

### Smart CI: User-Facing vs Workflow-Only Changes

Not all changes require the full CI pipeline or a new release. The project uses
`devx.ci.classify_changes` to classify changed files into two categories.

**Classification strategy (safe-by-default):** Any file NOT in the explicit
workflow-only allowlist is treated as user-facing. This prevents new file types
from accidentally skipping releases. Classification is config-driven via
`[tool.devx.classify]` in `pyproject.toml`.

**User-facing paths** (tool changes → release needed):
- `src/gitea_runner_manager/**` — Python CLI source
- `ansible/**` — Ansible role
- `pyproject.toml` — Package metadata

**Workflow-only paths** (infrastructure → no release needed):
- `.gitea/workflows/**`, `docs/**`, `tests/**`
- `AGENTS.md`, `README.md`, `CHANGELOG.md`, `Makefile`, `cliff.toml`, etc.

**CI behavior based on classification:**
- **Molecule tests**: Only run when `ansible/` or `.ansible-lint` files change
- **Release dry-run**: Only runs when user-facing files change (separate `release-dry-run` job)
- **Quality job** (lint, unit tests, coverage, doc-coverage): Always runs
- **Release workflow**: `release.py` calls `classify_changes` to check if any
  user-facing files changed since the last tag. If not, the release is skipped
  entirely — no version bump, no tag, no publish.

### Dynamic Runner Discovery

Molecule tests are distributed across available Gitea Actions runners
dynamically. The `discover-runners` job runs `devx.molecule.discover_runners` which queries the Gitea API for
registered runners at three levels (repo, org, instance) and generates
a matrix of runner indices. If the API query fails (e.g., no admin
access for instance-level runners), it falls back to the
`MOLECULE_RUNNERS` repo variable, then to a default of 3.

The `molecule-tests` job uses `fromJSON()` to consume the dynamic
matrix, and passes the runner count to `python -m devx.molecule.distribute_molecule
--max-runners` so test pairs are evenly distributed.

When adding or removing Gitea runners:
1. If runners are registered at the repo/org level, they're auto-detected
2. If runners are at the instance level, update the `MOLECULE_RUNNERS` repo variable
3. The workflow automatically scales the matrix to match available runners

### Molecule Test Distribution

`devx.molecule.distribute_molecule` discovers all molecule scenarios
under `ansible/roles/*/molecule/` and crosses them with the supported
OS platform matrix (defined in `devx.molecule.platforms`), then splits
the resulting test pairs evenly across the requested number of runners.
Each pair is encoded as `scenario|platform_name|platform_image|platform_command`.

`devx.molecule.molecule_ci_guard` runs the actual molecule test for a
given test pair, with CI context (Gitea URL, token, run ID) for
reporting results back to the commit status API.

### Commit Message Validation

`devx.ci.validate_commit_msg` validates that commit messages
follow the conventional commit format (`feat:`, `fix:`, `docs:`, etc.).
It is used by the pre-commit hook to enforce conventional commits on
feature branches.

### Release Commit Detection

The `detect-type` job in the post-merge workflow runs
`devx.ci.detect_release_commit` to check whether the latest commit
is a release commit (format: `release: vX.Y.Z`). When a release commit
is detected, all post-merge jobs (release, sync-wiki, badges, vikunja)
are skipped — the tag push triggers the publish workflow instead.

### Badge Generation and Push

The `badges` job in the post-merge workflow runs
`devx.ci.push_badges` which:
1. Fetches the latest master and hard-resets to it (picks up release commits)
2. Generates quality badge SVG files via `devx.tools.generate_badges`
3. Creates an orphan `badges` branch
4. Copies SVG files to the branch root
5. Force-pushes the branch to the remote

The badges job depends on the `release` job and uses `if: always()` so it
runs even if release fails or is skipped. This ensures the version badge
always reflects the actual state of the repository after any release
commits have been pushed.

## git-cliff Commit Preprocessing

Merge commits on master have the format `GRM-N <conventional commit>`. The `GRM-N ` prefix is not a valid conventional commit prefix, so `cliff.toml` includes a `commit_preprocessors` entry that strips it before parsing:

```toml
commit_preprocessors = [
    # Strip GRM-N task ID prefix from merge commits so git-cliff sees conventional commits
    { pattern = "^GRM-\\d+\\s+", replace = "" },
]
```

This ensures all merged work appears in the changelog.

### git-cliff Configuration Highlights (`cliff.toml`)

- `conventional_commits = true` — parse conventional commit format
- `filter_unconventional = true` — skip non-conventional commits
- `render_always = true` — always render the changelog
- `trim = true` — trim whitespace
- Commit parsers group commits into: Features, Bug Fixes, Documentation, Performance, Refactor, Styling, Testing, Miscellaneous Tasks, Security, Revert, Other
- `chore(release): prepare for`, `chore(deps.*)`, `chore(pr)`, `chore(pull)` commits are skipped
- `sort_commits = "oldest"` — oldest commits first

## Version Bumping Rules (git-cliff)

| Commit type | Version bump |
|-------------|-------------|
| `feat:` | minor (0.X.0) |
| `fix:` | patch (0.0.X) |
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0: major would be 1.0.0) |
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |

From `cliff.toml` `[bump]` section:

- `features_always_bump_minor = true`
- `breaking_always_bump_major = false`
- `initial_tag = "0.1.0"`

The version source is `__version__` in `src/gitea_runner_manager/__init__.py`, read by setuptools via `dynamic = ["version"]` in `pyproject.toml`. The release script only updates `__init__.py` — no need to touch `pyproject.toml`. `grm --version` reports this version.

## Title Format Summary

| What | Format | Example |
|------|--------|---------|
| Branch name | `GRM-N-short-description` | `GRM-33-add-pr-review-step` |
| Branch commits | `<conventional commit>` | `feat: add review script` |
| PR title | `GRM-N: <vikunja task title>` | `GRM-33: Add mandatory PR review step` |
| Merge commit | `GRM-N <conventional commit>` | `GRM-33 feat: add review script` |