grm/docs/tech/ci-cd-workflow.md

CI/CD Workflow

GRM uses a fully automated CI/CD pipeline built on Gitea Actions. Every change to master goes through a mandatory PR workflow with branch protection, automated review, and auto-merge. Releases are automated via git-cliff and conventional commits.

Workflow Overview

Workflow	Trigger	Purpose
ci.yml	PR opened/synchronized	Validate (lint, test, coverage, detect-changes, release-dry-run, pr-review, discover-runners) + molecule tests
auto-merge.yml	PR labeled ready-to-merge	Validates and squash-merges the PR
post-merge.yml	Push to master	Detect-and-configure + release-and-maintain (release, publish, wiki sync, badges, Vikunja task update)

Every change to master goes through a mandatory PR workflow. No exceptions.

PR Workflow

1. Create Vikunja Task

Create a task in Vikunja project 6 to get a GRM-N identifier.

2. Create Branch

git checkout master && git pull
git checkout -b GRM-N-short-description

3. Implement Changes

• Write code following conventions
• Write/update tests (100% coverage required)
• Update documentation (CHANGELOG, README, AGENTS.md as needed)

4. Commit (Conventional Commits)

Branch commits use conventional commit format (no GRM-N: prefix):

feat: add new feature
fix: resolve bug
docs: update README

5. Push and Create PR

• PR title format: GRM-N: <vikunja task title> (must match the Vikunja task title exactly)
• PR body: summary of changes, Closes GRM-N
• Add ready-to-merge label only after review is complete

6. Review the PR (Mandatory — Before Adding ready-to-merge Label)

Review the full diff (git diff master...HEAD) focusing on:

• Functional completeness: Does the code do what it claims? Are all requirements met?
• Edge cases: Are boundary conditions, empty inputs, error paths handled?
• Technical excellence:
  • Architecture compliance and evolution
  • Single Responsibility Principle (SRP)
  • Deduplication (no copy-paste, single source of truth)
  • Code smells detection and removal
  • Best industry practices
  • Industry-grade code quality
  • Reusability
  • Clean code
  • Readability
  • Maintainability
  • Extensibility
• Performance: No unnecessary allocations, O(n) vs O(n²), efficient data structures
• Security: No secrets in logs/process list, input validation, no injection vectors
• User experience: Clear error messages, intuitive CLI flags, helpful output
• Documentation: Completeness and relevance of docs, CHANGELOG entries, AGENTS.md updates

Post review comments using devx.ci.pr_review:

CI_GITEA_TOKEN=<token> python -m devx.ci.pr_review <pr_number> <owner/repo> \
  --event REQUEST_CHANGES \
  --body "Review summary" \
  --comments-json comments.json

7. Address Review Comments

Fix each comment one by one, commit, and push. Re-review until satisfied.

8. Approve and Merge

Once all comments are addressed:

CI_GITEA_TOKEN=<token> python -m devx.ci.pr_review <pr_number> <owner/repo> \
  --event APPROVE \
  --body "All comments addressed. LGTM."

Then add the ready-to-merge label. The auto-merge workflow will:

1. Validate PR title format and match against Vikunja task title
2. Check that at least one APPROVE review exists
3. Wait for all CI checks to pass
4. Squash-merge with title: GRM-N <conventional commit message> (space-separated)
5. The post-merge workflow marks the Vikunja task as done
6. The release-and-maintain job automatically versions, tags, and publishes

9. Post-Merge Automation

After the squash-merge:

• The post-merge workflow (.gitea/workflows/post-merge.yml) triggers on push to master. The detect-and-configure job configures the repo and detects the commit type. The release-and-maintain job then runs the release, publish, sync-wiki, badges, and Vikunja steps as appropriate.
• The release step (in the release-and-maintain job) automatically versions, tags, and publishes (see below).

Branch Protection (Required Gitea Settings)

Configure the following branch protection rules for master in Gitea repo settings:

• Require pull request: No direct pushes to master
• Require approval review: At least 1 APPROVE review before merge
• Require status checks: CI validate + molecule tests must pass
• Block force pushes: No history rewriting on master

The auto-merge workflow enforces the APPROVE review check programmatically as a defense-in-depth measure, but branch protection is the primary gate.

CI Path Filtering

The CI workflow (.gitea/workflows/ci.yml) includes a detect-changes step in the validate job that checks whether any files under ansible/ or .ansible-lint have changed. If no Ansible files are changed, molecule tests are skipped — this prevents non-Ansible changes (e.g., Python scripts, workflow YAML, docs) from being blocked by molecule test infrastructure flakiness.

The detect-changes step:

• For pull requests: compares origin/master against the PR head SHA
• For pushes to master: compares HEAD~1 against HEAD
• Outputs ansible-changed as true or false

The molecule-tests job depends on the validate job (which includes the detect-changes step), and only runs if ansible-changed == 'true'.

CI triggers only on opened and synchronize PR events (not labeled).

CI Validate Job

The validate job in .gitea/workflows/ci.yml consolidates the former quality, detect-changes, release-dry-run, pre-merge-check, pr-review, and discover-runners jobs into a single job. It runs:

1. make setup — full environment setup
2. make lint-all — ruff + pyright + bandit + ansible-lint + checkmake
3. make pytest-cov — unit tests with 100% coverage enforcement
4. python -m devx.tools.check_test_speed --max-seconds 10 — verify unit tests run fast
5. PYTHONPATH=src python -m devx.ci.release --dry-run — release dry-run validation (release-dry-run step)
6. Pre-merge validation step — validates branch format, PR title, and Vikunja task match
7. detect-changes step — checks whether Ansible files changed (gates molecule tests)
8. pr-review step — automated PR review via devx.ci.pr_review
9. discover-runners step — dynamic runner discovery for molecule tests (conditional on ansible-changed)

Automated Release Pipeline

After a PR is merged to master, the release pipeline runs automatically.

Release Step (in the release-and-maintain job)

• Runs as a conditional step in the release-and-maintain job (skipped for release commits)
• Sets up full dev environment (make setup) so lint and tests can run
• Installs git-cliff (version 2.13.0)
• Configures git as grm-ci-bot
• Runs devx.ci.release which uses git-cliff to:
  • Checks for user-facing changes via devx.ci.classify_changes — if only workflow/infrastructure files changed, the release is skipped entirely — no version bump, no tag, no publish
  • Calculate the next semver version from conventional commits since the last tag
  • Update __version__ in src/grm/__init__.py (single source of truth)
  • Update CHANGELOG.md with the new version section
  • Run make lint-ruff and make pytest-cov to verify the release is healthy
  • If lint or tests fail, abort immediately — no commit, no tag
  • Commit with release: vX.Y.Z [skip ci] prefix (the [skip ci] prevents re-triggering post-merge on the release commit)
  • Create an annotated tag vX.Y.Z on the release commit
  • Push both the commit and tag to master
• --skip-tests flag bypasses test verification (emergency use only, not recommended)
• Loops are prevented by has_unreleased_changes — after a release commit is tagged, the next run finds no unreleased changes and exits
• On failure, creates a Gitea issue via devx.ci.notify_failure

Publish Step (in the release-and-maintain job)

• Runs as a conditional step in the release-and-maintain job (only if the release step created a tag)
• Checks out the release tag within the same job
• Installs git-cliff (version 2.13.0)
• Installs build tools (build, twine, requests, python-dotenv, click)
• Validates PYPI_TOKEN is set (warns if missing)
• Builds the Python package
• Optionally publishes to PyPI (if PYPI_TOKEN is set)
• Creates a Gitea release with git-cliff-generated release notes
• Uses devx.ci.publish for build and publish orchestration
• On failure, creates a Gitea issue via devx.ci.notify_failure

Auto-Merge Workflow (.gitea/workflows/auto-merge.yml)

• Triggers on pull_request labeled events
• Runs devx.ci.auto_merge with the branch name, PR title, repository, PR number, and label name
• Validates PR title format, checks for APPROVE review, waits for CI, and squash-merges

Post-Merge Workflow (.gitea/workflows/post-merge.yml)

• Triggers on push to master
• Consolidated from 7 jobs into 2 jobs to reduce runner overhead
• detect-and-configure — Configures repo (branch protection, labels), detects release commit, validates commit message. Outputs is-release and is-automated for the next job.
  • detect-type step — Runs devx.ci.detect_release_commit to check if the commit is a release commit (release: vX.Y.Z). All subsequent steps skip for release commits (the [skip ci] tag also prevents re-triggering).
  • validate-commit-msg step — Validates the commit message follows conventional commit format.
  • configure-repo step — Runs devx.tools.configure_repo to set up branch protection and labels.
• release-and-maintain — Runs all post-merge maintenance as conditional steps:
  • release step (if not a release commit) — Runs devx.ci.release (see Automated Release Pipeline below)
  • publish step (if release created a tag) — Builds and publishes the package to the Gitea PyPI registry
  • sync-wiki step (if not automated) — Syncs documentation to the Gitea wiki via devx.ci.sync_wiki
  • vikunja step (if not automated) — Marks the corresponding Vikunja task as done via devx.ci.post_merge
  • badges step (always) — Generates and pushes quality badge SVGs to the badges branch via devx.ci.push_badges. Runs even if release fails or is skipped so the version badge always reflects the latest state.

Smart CI: User-Facing vs Workflow-Only Changes

Not all changes require the full CI pipeline or a new release. The project uses devx.ci.classify_changes to classify changed files into two categories.

Classification strategy (safe-by-default): Any file NOT in the explicit workflow-only allowlist is treated as user-facing. This prevents new file types from accidentally skipping releases. Classification is config-driven via [tool.devx.classify] in pyproject.toml.

User-facing paths (tool changes → release needed):

• src/grm/** — Python CLI source
• ansible/** — Ansible role
• pyproject.toml — Package metadata

Workflow-only paths (infrastructure → no release needed):

• .gitea/workflows/**, docs/**, tests/**
• AGENTS.md, README.md, CHANGELOG.md, Makefile, cliff.toml, etc.

CI behavior based on classification:

• Molecule tests: Only run when ansible/ or .ansible-lint files change
• Release dry-run: Only runs when user-facing files change (release-dry-run step in the validate job)
• Validate job (lint, unit tests, coverage, doc-coverage): Always runs
• Release step: release.py calls classify_changes to check if any user-facing files changed since the last tag. If not, the release is skipped entirely — no version bump, no tag, no publish.

Dynamic Runner Discovery

Molecule tests are distributed across available Gitea Actions runners dynamically. The discover-runners step in the validate job runs devx.molecule.discover_runners which queries the Gitea API for registered runners at three levels (repo, org, instance) and generates a matrix of runner indices. If the API query fails (e.g., no admin access for instance-level runners), it falls back to the MOLECULE_RUNNERS repo variable, then to a default of 3.

The molecule-tests job uses fromJSON() to consume the dynamic matrix, and passes the runner count to python -m devx.molecule.distribute_molecule --max-runners so test pairs are evenly distributed.

When adding or removing Gitea runners:

1. If runners are registered at the repo/org level, they're auto-detected
2. If runners are at the instance level, update the MOLECULE_RUNNERS repo variable
3. The workflow automatically scales the matrix to match available runners

Molecule Test Distribution

devx.molecule.distribute_molecule discovers all molecule scenarios under ansible/roles/*/molecule/ and crosses them with the supported OS platform matrix (defined in devx.molecule.platforms), then splits the resulting test pairs evenly across the requested number of runners. Each pair is encoded as scenario|platform_name|platform_image|platform_command.

The CI workflow runs each test pair sequentially via a shell loop that sets the appropriate MOLECULE_PLATFORM_* environment variables and invokes molecule test directly.

Commit Message Validation

devx.ci.validate_commit_msg validates that commit messages follow the conventional commit format (feat:, fix:, docs:, etc.). It is used by the pre-commit hook to enforce conventional commits on feature branches.

Release Commit Detection

The detect-type step in the detect-and-configure job (post-merge workflow) runs devx.ci.detect_release_commit to check whether the latest commit is a release commit (format: release: vX.Y.Z). When a release commit is detected, all subsequent steps in the release-and-maintain job (release, publish, sync-wiki, vikunja) are skipped — the tag push triggers the publish step instead.

Badge Generation and Push

The badges step in the release-and-maintain job (post-merge workflow) runs devx.ci.push_badges which:

1. Fetches the latest master and hard-resets to it (picks up release commits)
2. Generates quality badge SVG files via devx.tools.generate_badges
3. Creates an orphan badges branch
4. Copies SVG files to the branch root
5. Force-pushes the branch to the remote

The badges step runs with if: always() so it runs even if the release step fails or is skipped. This ensures the version badge always reflects the actual state of the repository after any release commits have been pushed.

git-cliff Commit Preprocessing

Merge commits on master have the format GRM-N <conventional commit>. The GRM-N prefix is not a valid conventional commit prefix, so cliff.toml includes a commit_preprocessors entry that strips it before parsing:

commit_preprocessors = [
    # Strip GRM-N task ID prefix from merge commits so git-cliff sees conventional commits
    { pattern = "^GRM-\\d+\\s+", replace = "" },
]

This ensures all merged work appears in the changelog.

git-cliff Configuration Highlights (cliff.toml)

• conventional_commits = true — parse conventional commit format
• filter_unconventional = true — skip non-conventional commits
• render_always = true — always render the changelog
• trim = true — trim whitespace
• Commit parsers group commits into: Features, Bug Fixes, Documentation, Performance, Refactor, Styling, Testing, Miscellaneous Tasks, Security, Revert, Other
• chore(release): prepare for, chore(deps.*), chore(pr), chore(pull) commits are skipped
• sort_commits = "oldest" — oldest commits first

Version Bumping Rules (git-cliff)

Commit type	Version bump
feat:	minor (0.X.0)
fix:	patch (0.0.X)
feat!: or BREAKING CHANGE	minor (pre-1.0: major would be 1.0.0)
chore:, ci:, docs:	no bump (excluded by cliff.toml)

From cliff.toml [bump] section:

• features_always_bump_minor = true
• breaking_always_bump_major = false
• initial_tag = "0.1.0"

The version source is __version__ in src/grm/__init__.py, read by setuptools via dynamic = ["version"] in pyproject.toml. The release script only updates __init__.py — no need to touch pyproject.toml. grm --version reports this version.

Title Format Summary

What	Format	Example
Branch name	GRM-N-short-description	GRM-33-add-pr-review-step
Branch commits	<conventional commit>	feat: add review script
PR title	GRM-N: <vikunja task title>	GRM-33: Add mandatory PR review step
Merge commit	GRM-N <conventional commit>	GRM-33 feat: add review script