devx/docs/tech/ci-cd-workflow.md

CI/CD Workflow

devx uses Gitea Actions for CI/CD automation. Two workflows implement a complete pipeline: pull request validation and post-merge release automation (including publishing).

Workflow overview

PR opened/synchronized ──► CI (ci.yml)
    │                         ├── validate (quality + detect-changes +
    │                         │              release-dry-run + pr-review +
    │                         │              pre-merge validation)
    │                         └── auto-merge ──► squash-merge to master
    │                                              │
    ▼                                              ▼
Push to master ──► Post-merge (post-merge.yml)
                     ├── detect-and-configure (detect-type +
                     │                          validate-commit-msg +
                     │                          configure-repo)
                     └── release-and-maintain
                          ├── release ──► tag vX.Y.Z
                          ├── publish ──► Gitea PyPI registry + Gitea release
                          ├── sync-wiki
                          ├── vikunja
                          └── badges (always runs)

CI workflow (ci.yml)

Runs on pull requests (opened and synchronize) and manual dispatch.

Jobs

validate

The single validation job. Consolidates the former quality, detect-changes, release-dry-run, pr-review, and pre-merge-check jobs into one job to save checkout+setup overhead. Runs on every PR.

Quality steps

The main quality gate:

1. Lint all — ruff check, ruff format check, pyright, bandit, actionlint (via make lint-all)
2. Unit tests with 100% coverage — make pytest-cov
3. Check unit test speed — python -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
4. Documentation coverage check — python -m devx.ci.doc_coverage --fail-on-missing
5. Translation completeness check — python -m devx.ci.check_translations
6. Dependency security scan — pip-audit --desc --skip-editable (best-effort, non-blocking)
7. Workflow dry-run validation — make workflow-dryrun via act_runner (best-effort, skipped if act_runner is not installed)

detect-changes step

Classifies changes between origin/master and the PR head as user-facing or workflow-only using python -m devx.ci.classify_changes --github-output. Writes user-facing-changed=true|false to the job output for use by downstream steps.

release-dry-run step

Only runs if the detect-changes step detected user-facing changes. Runs python -m devx.ci.release --dry-run to validate that the release script can calculate the next version and generate the changelog without making changes. Non-blocking (uses || true).

pr-review step

Runs on every pull request. Executes python -m devx.ci.pr_review with the PR number and repository. Fetches the PR diff via the Gitea API and runs automated checks, posting a structured review:

• COMMENT — no issues found
• REQUEST_CHANGES — issues found that must be addressed

Checks performed:

1. Architecture compliance — no subprocess in CLI, no hardcoded URLs
2. Best practices — no print(), no bare except, no TODO/FIXME, no functions > 50 lines
3. Security — no hardcoded secrets, no shell=True, no eval/exec
4. i18n — no raw strings in click.echo() without _() wrapper
5. Resource management — no open() without with, no Popen() without cleanup
6. Documentation — source changes must include doc updates
7. Test coverage — source changes must include test updates
8. Commit conventions — conventional commit format on PR commits

Pre-merge validation step

Runs on every pull request. Executes python -m devx.ci.check_auto_merge_ready with the branch name, PR title, repository, and PR number. Validates auto-merge preconditions before the auto-merge job runs:

1. Branch name — must contain a valid task ID (for example, DEVX-12-fix-foo → DEVX-12)
2. PR title format — must be {PREFIX}-N: <vikunja task title>
3. Vikunja task — must exist and the title must match the PR title
4. Branch state — must not be behind master

auto-merge

Depends on validate. The final job in the CI workflow. Runs python -m devx.ci.auto_merge with the branch name, PR title, repository, and PR number:

1. Read task ID from branch name (for example, DEVX-12-fix-foo → DEVX-12)
2. Validate PR title format — must be {PREFIX}-N: <vikunja task title>
3. Validate PR title matches Vikunja task — fetches the Vikunja task and compares the title
4. Extract conventional commit message from PR commits (newest matching conventional format)
5. Squash-merge with title {PREFIX}-N <conventional commit message>
6. If the head branch is behind master (HTTP 405), automatically pulls master, rebases, force-pushes, and retries the merge

The merge commit push to master triggers the post-merge workflow.

Smart CI: user-facing vs workflow-only changes

Not all changes require a new release. The detect-changes step in the validate job classifies changes using python -m devx.ci.classify_changes:

Workflow-only paths (infrastructure — no release needed):

• .gitea/** — Gitea Actions workflows
• tests/** — Test files
• AGENTS.md, README.md, CHANGELOG.md — Project docs
• Makefile, cliff.toml, .pre-commit-config.yaml — Config
• .env.example, .gitignore — Config
• hooks/** — Git hooks
• src/devx/__init__.py — Only contains __version__ (release artifact)

User-facing paths (tool changes — release needed) — everything else:

• src/devx/** — Python package source (except __init__.py)
• pyproject.toml — Package metadata
• Any new file type not in the allowlist

Classification is configured in pyproject.toml under [tool.devx.classify]. The framework provides DEFAULT_INFRASTRUCTURE — a curated list of paths that are infrastructure for any Python project. Projects inherit these automatically and only specify what is different.

Rule priority (first match wins):

1. user_facing_overrides — safety override (highest priority)
2. infrastructure_overrides — explicit per-file
3. infrastructure — DEFAULT_INFRASTRUCTURE + project-specific patterns
4. Default: user-facing (safe — any unknown file triggers release)

Post-merge workflow (post-merge.yml)

Runs on every push to master. Consolidated into 2 jobs (from 7) to reduce runner overhead: detect-and-configure (detect-type + validate-commit-msg + configure-repo) and release-and-maintain (release + publish + sync-wiki + badges + vikunja). Individual steps within release-and-maintain are conditional on the detect-and-configure job's outputs.

Job dependency graph

detect-and-configure
    ├── configure-repo (independent, skip if release commit)
    ├── detect-type → is-release? is-automated?
    └── validate-commit-msg (skip if release commit)
            │
            ▼
release-and-maintain (needs detect-and-configure)
    ├── release (skip if release commit or workflow-only)
    │       └── publish (if release created a tag)
    ├── sync-wiki (skip if automated)
    ├── vikunja (skip if automated)
    └── badges (always runs)

sync-wiki and vikunja run only on non-automated commits (that is, real PR merges) so that the wiki and task tracker are only updated when a human change lands. They skip on release commits and automated commits.

The badges step always runs (even on release commits) so badges (tests, coverage, version, etc.) are always current. It runs last so it picks up any version bump the release step created.

When release creates a release: vX.Y.Z commit, the release commit's post-merge run still updates badges (the version badge picks up the new version). Other steps skip. The publish step builds and publishes the package to the Gitea PyPI registry within the same release-and-maintain job (it checks out the release tag).

Post-merge jobs

detect-and-configure

The first post-merge job. Consolidates the former detect-type, validate-commit-msg, and configure-repo jobs. Outputs is-release, is-automated, and user-facing-changed for the release-and-maintain job.

detect-type step

Checks if the latest commit is a release commit (release: vX.Y.Z [skip ci]) using python -m devx.ci.detect_release_commit. Writes is-release=true or is-release=false (and is-automated) to the job output. The release-and-maintain job uses these to conditionally skip steps for release commits.

validate-commit-msg step

Skips for release/automated commits. Validates the latest commit message using python -m devx.ci.validate_commit_msg --branch master. On master, commits must follow {PREFIX}-N: <conventional commit> format (added by auto-merge).

configure-repo step

Ensures branch protection and labels are configured using python -m devx.tools.configure_repo --repo <name> --owner <owner>:

• Sets up master branch protection (required status checks, block on rejected reviews, block on outdated branch)
• Creates standard labels
• Status check contexts read from DEVX_STATUS_CHECKS or default to CI / validate (pull_request)

On failure, the notify_failure step creates a Gitea issue.

release-and-maintain

Depends on detect-and-configure. The second post-merge job. Consolidates the former release, publish, sync-wiki, badges, and vikunja jobs. Individual steps are conditional on the detect-and-configure job's outputs.

release step

Skips for release commits and workflow-only changes. The core release automation step. Runs python -m devx.ci.release:

1. Classify changes — calls classify_changes.py to check for user-facing changes. If only infrastructure files changed, exits without releasing.
2. Calculate next version — uses git-cliff to determine the next semver version from conventional commits since the last tag
3. Update version file — updates __version__ in src/devx/__init__.py
4. Update changelog — prepends the new version section to CHANGELOG.md using git-cliff output
5. Run tests — executes make lint-ruff and make pytest-cov to verify the release is healthy. If either fails, the release is aborted — no commit, no tag. Use --skip-tests only for emergency releases.
6. Commit — stages the version file and changelog, commits with release: vX.Y.Z [skip ci] (uses --no-verify to bypass the commit-msg hook since release commits are a special case)
7. Create tag — creates an annotated tag vX.Y.Z with the changelog as the tag message
8. Push — pushes both the commit and tag to master

The script is idempotent: if there are no new conventional commits since the last tag, it exits without doing anything. If the tag already exists (for example, from a partial previous run), it skips tag creation and only pushes.

Tag consistency: Before releasing, the script fetches remote tags and verifies all existing tags point to commits whose message matches the tag version. This prevents duplicate release commits and ensures tag/version/commit alignment.

Version bumping rules (git-cliff):

Commit type	Version bump
feat:	minor (0.X.0)
fix:	patch (0.0.X)
feat!: or BREAKING CHANGE	minor (pre-1.0)
chore:, ci:, docs:	no bump (excluded by cliff.toml)

On failure, the notify_failure step creates a Gitea issue via python -m devx.ci.notify_failure.

sync-wiki step

Skips for automated commits. Syncs documentation from docs/ to the Gitea wiki using python -m devx.ci.sync_wiki --repo <owner/repo> --strict:

1. Reads docs/mapping.json to map file paths to wiki page titles
2. Lists existing wiki pages via the Gitea API
3. For each mapped file, reads content and creates or updates the wiki page
4. --strict runs a full integrity check: verifies page count, missing pages, stale pages, and content match. Fails if any page is empty or content doesn't match.

Pages that exist in the wiki but not in the mapping are left untouched (not deleted).

On failure, the notify_failure step creates a Gitea issue.

badges step

Always runs (even on release commits). Generates and pushes quality badges using python -m devx.ci.push_badges:

1. Fetch latest master — git fetch origin master && git reset --hard origin/master (ensures the version badge reflects the current state, even if the release step recently pushed a new version)
2. Generate badges — calls devx.tools.generate_badges which runs pytest-cov, doc-coverage, lint checks, and version extraction, then writes SVG files: coverage.svg, tests.svg, docs.svg, quality.svg, version.svg, python.svg
3. Push to badges branch — creates an orphan badges branch, copies SVG files, commits, and force-pushes
4. Update README/docs — switches back to master, replaces raw/branch/badges/<name>.svg URLs with raw/commit/<sha>/<name>.svg URLs (cache-busting — Gitea caches raw/branch/ URLs for 6 hours), commits, and pushes

Supports --retries for retrying on git push failures (fetches latest master and waits 10s between attempts).

On failure, the notify_failure step creates a Gitea issue.

vikunja step

Skips for automated commits. Updates the Vikunja task after a merge using python -m devx.ci.post_merge --git-sha <sha>:

1. Extracts the task ID from the first line of the commit message
2. Marks the corresponding Vikunja task as done
3. Posts a comment with the merge SHA

On failure, the notify_failure step creates a Gitea issue.

publish step

Only runs if the release step created a tag. Builds and publishes the package within the same release-and-maintain job (checks out the release tag). Runs python -m devx.ci.publish <tag> <owner/repo>:

1. Install dependencies — build, twine, requests, python-dotenv, click, and the project itself
2. Install CI tools — git-cliff and tea via python -m devx.tools.install_tools
3. Configure tea login — tea login add using CI_GITEA_TOKEN
4. Build and publish — python -m devx.ci.publish <tag> <owner/repo>:
   • Build the package with python -m build
   • Publish to the Gitea PyPI registry (default) using twine upload --repository-url <url> -u <token> -p <token>
   • OR publish to standard PyPI if PYPI_TOKEN is set
   • OR skip publishing if --skip-build is passed (non-Python repos)
   • Create a Gitea release with git-cliff-generated release notes via tea create release

Publishing destination resolution (checked in order):

1. Gitea PyPI registry — if --registry-url is given, or DEVX_PYPI_REGISTRY_URL env var is set, or derived from GITEA_API_URL
2. Standard PyPI — if PYPI_TOKEN is set (takes precedence over Gitea registry)
3. Skip — if neither is configured, only the Gitea release is created

On failure, the notify_failure step creates a Gitea issue.

CI scripts

auto_merge.py

Auto-merge PR when all CI checks pass. Reads task ID from the branch name (for example, DEVX-12-fix-foo → DEVX-12). Validates PR title format, checks the Vikunja task exists and the title matches, extracts the conventional commit message from PR commits, and squash-merges with {PREFIX}-N <conventional commit> title.

python -m devx.ci.auto_merge <branch> <pr_title> <owner/repo> <pr_number>

release.py

Automated release using git-cliff. Calculates next semver version from conventional commits, updates __version__ and CHANGELOG.md, runs lint and tests, commits with release: vX.Y.Z [skip ci], creates annotated tag, and pushes. Idempotent — exits if no unreleased changes.

python -m devx.ci.release [--dry-run] [--skip-tests] [--verify]

• --dry-run — preview without making changes
• --skip-tests — skip lint and test verification (emergency only)
• --verify — check tag/version/changelog alignment and exit

publish.py

Builds package, publishes to Gitea PyPI registry or standard PyPI, and creates a Gitea release with git-cliff-generated notes.

python -m devx.ci.publish <tag> <owner/repo> [--registry-url <url>] [--skip-build]

pr_review.py

Automated PR review. Fetches the PR diff via the Gitea API, runs automated checks (architecture, best practices, security, i18n, resource management, documentation, test coverage, commit conventions), and posts a structured review with inline comments.

python -m devx.ci.pr_review <pr_number> <owner/repo>

notify_failure.py

Creates a Gitea issue when a CI workflow fails. Uses the tea CLI for issue creation with failure labels. Supports --auto-login to configure the tea CLI login profile from CI_GITEA_TOKEN.

python -m devx.ci.notify_failure --repo <owner/repo> --run-id <id> \
  --workflow <name> --commit <sha> [--auto-login]

post_merge.py

Updates Vikunja task after a merge to master. Extracts task ID from the commit message, marks the task as done, and posts a comment with the merge SHA.

python -m devx.ci.post_merge <commit_msg> [--commit-sha <sha>] [--git-sha <sha>]

classify_changes.py

Classifies git changes as user-facing or workflow-only. Uses a layered rule system configured in pyproject.toml. Safe-by-default: any unknown file defaults to user-facing.

python -m devx.ci.classify_changes [--base <ref>] [--head <ref>] \
  [--quiet] [--check <category>] [--github-output]

discover_runners.py

Discovers available Gitea Actions runners at repository, organization, and instance levels. Falls back to MOLECULE_RUNNERS repo variable or DEFAULT_MAX_RUNNERS (3).

python -m devx.ci.discover_runners --owner <owner> --repo <repo> [--count] [--indices]

detect_release_commit.py

Detects whether the latest git commit is a release commit. Writes is-release=true|false to $GITHUB_OUTPUT.

python -m devx.ci.detect_release_commit

push_badges.py

Generates SVG badge files, pushes them to the badges branch, and updates README.md and docs/index.md with cache-busting raw/commit/<sha>/ URLs.

python -m devx.ci.push_badges [--output-dir <dir>] [--branch <branch>] \
  [--no-readme-update] [--retries <n>]

distribute_molecule.py

Distributes molecule (scenario, platform) pairs across N parallel runners. Discovers scenarios under ansible/roles/*/molecule/.

python -m devx.molecule.distribute_molecule --runner-index <i> --max-runners <n>
python -m devx.molecule.distribute_molecule --list
python -m devx.molecule.distribute_molecule --list-platforms

validate_commit_msg.py

Validates commit messages. On feature branches: conventional commits only (no {PREFIX}-N prefix). On master: must have {PREFIX}-N prefix from auto-merge, followed by a conventional commit message.

python -m devx.ci.validate_commit_msg <commit_msg_file> [--branch <branch>]

sync_wiki.py

Syncs documentation from docs/ to the Gitea wiki via the API. Reads docs/mapping.json for file-to-page mapping. Supports --dry-run, --verify, and --strict (full integrity check).

python -m devx.ci.sync_wiki [--dry-run] [--repo <owner/repo>] [--verify] [--strict]

check_translations.py

Validates translation files against the Python source code. Checks for missing keys, dead keys, and missing languages.

python -m devx.ci.check_translations [--translations <file>]...

doc_coverage.py

Checks documentation coverage for CLI commands and major modules. Parses Click commands from cli.py and verifies documentation exists.

python -m devx.ci.doc_coverage [--docs-dir <dir>] [--fail-on-missing]

distribute_files.py

Distributes files matching a glob pattern across N parallel runners (round-robin). Writes the assigned file list to $GITHUB_ENV.

python -m devx.ci.distribute_files --pattern <glob> --runner-index <i> \
  --max-runners <n> [--github-env] [--skip-if-excess]

integration_guard.py

Runs pytest with cross-runner failure detection. If any other integration-tests matrix runner reports failure, the current pytest subprocess is killed and this runner exits early.

python -m devx.ci.integration_guard -- <pytest args>

Release process summary

The complete release process from PR to published package:

1. PR merged — auto-merge squash-merges the PR to master with {PREFIX}-N <conventional commit> title
2. Post-merge triggers — the merge push triggers post-merge.yml
3. detect-and-configure — detects release commit, validates commit message, and ensures branch protection/labels
4. release (step in release-and-maintain) — release.py calculates the next version, updates files, runs tests, commits release: vX.Y.Z [skip ci], creates tag vX.Y.Z, and pushes to master
5. publish (step in release-and-maintain) — publish.py builds the package, publishes to the Gitea PyPI registry, and creates a Gitea release with git-cliff notes (checks out the release tag within the same job)
6. sync-wiki (step in release-and-maintain) — documentation is synced to the Gitea wiki
7. vikunja (step in release-and-maintain) — the corresponding Vikunja task is marked as done
8. badges (step in release-and-maintain) — quality badges are regenerated and pushed to the badges branch; README and docs/index.md are updated with cache-busting URLs

The release commit's post-merge run skips all steps except badges (which picks up the new version number). This prevents infinite loops.

Failure handling

Every job in the CI and post-merge workflows has a notify_failure step that runs if: failure(). This creates a Gitea issue with the workflow name, run ID, and commit SHA, ensuring failures that would otherwise go unnoticed in the Actions tab are surfaced as issues. The issue is created via the tea CLI with a bug label if available.