devx/docs/tech/ci-cd-workflow.md

CI/CD Workflow

devx uses Gitea Actions for CI/CD automation. Three workflows implement a complete pipeline: pull request validation, post-merge release automation, and tag-triggered publishing.

Workflow overview

PR opened/synchronized ──► CI (ci.yml)
    │                         ├── quality
    │                         ├── detect-changes
    │                         ├── release-dry-run (if user-facing)
    │                         ├── pr-review
    │                         └── auto-merge ──► squash-merge to master
    │                                              │
    ▼                                              ▼
Push to master ──► Post-merge (post-merge.yml)
                     ├── detect-type
                     ├── validate-commit-msg
                     ├── release ──► tag vX.Y.Z
                     ├── sync-wiki        │
                     ├── badges           │
                     ├── vikunja          │
                     └── configure-repo   │
                                          │
                                          ▼
Tag push (v*) ──► Publish (publish.yml)
                     └── publish ──► Gitea PyPI registry + Gitea release

CI workflow (ci.yml)

Runs on pull requests (opened and synchronize) and manual dispatch.

Jobs

quality

The main quality gate. Runs on every PR:

1. Lint all — ruff check, ruff format check, pyright, bandit, actionlint (via make lint-all)
2. Unit tests with 100% coverage — make pytest-cov
3. Check unit test speed — python -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
4. Documentation coverage check — python -m devx.ci.doc_coverage --fail-on-missing
5. Translation completeness check — python -m devx.ci.check_translations
6. Dependency security scan — pip-audit --desc --skip-editable (best-effort, non-blocking)
7. Workflow dry-run validation — make workflow-dryrun via act_runner (best-effort, skipped if act_runner is not installed)

detect-changes

Classifies changes between origin/master and the PR head as user-facing or workflow-only using python -m devx.ci.classify_changes --github-output. Writes user-facing-changed=true|false to the job output for use by downstream jobs.

release-dry-run

Depends on quality and detect-changes. Only runs if user-facing changes are detected. Runs python -m devx.ci.release --dry-run to validate that the release script can calculate the next version and generate the changelog without making changes. Non-blocking (uses || true).

pr-review

Runs on every pull request. Executes python -m devx.ci.pr_review with the PR number and repository. Fetches the PR diff via the Gitea API and runs automated checks, posting a structured review:

• COMMENT — no issues found
• REQUEST_CHANGES — issues found that must be addressed

Checks performed:

1. Architecture compliance — no subprocess in CLI, no hardcoded URLs
2. Best practices — no print(), no bare except, no TODO/FIXME, no functions > 50 lines
3. Security — no hardcoded secrets, no shell=True, no eval/exec
4. i18n — no raw strings in click.echo() without _() wrapper
5. Resource management — no open() without with, no Popen() without cleanup
6. Documentation — source changes must include doc updates
7. Test coverage — source changes must include test updates
8. Commit conventions — conventional commit format on PR commits

auto-merge

Depends on quality, detect-changes, and pr-review. The final job in the CI workflow. Runs python -m devx.ci.auto_merge with the branch name, PR title, repository, and PR number:

1. Read task ID from branch name (for example, DEVX-12-fix-foo → DEVX-12)
2. Validate PR title format — must be {PREFIX}-N: <vikunja task title>
3. Validate PR title matches Vikunja task — fetches the Vikunja task and compares the title
4. Extract conventional commit message from PR commits (newest matching conventional format)
5. Squash-merge with title {PREFIX}-N <conventional commit message>
6. If the head branch is behind master (HTTP 405), automatically pulls master, rebases, force-pushes, and retries the merge

The merge commit push to master triggers the post-merge workflow.

Smart CI: user-facing vs workflow-only changes

Not all changes require a new release. The detect-changes job classifies changes using python -m devx.ci.classify_changes:

Workflow-only paths (infrastructure — no release needed):

• .gitea/** — Gitea Actions workflows
• tests/** — Test files
• AGENTS.md, README.md, CHANGELOG.md — Project docs
• Makefile, cliff.toml, .pre-commit-config.yaml — Config
• .env.example, .gitignore — Config
• hooks/** — Git hooks
• src/devx/__init__.py — Only contains __version__ (release artifact)

User-facing paths (tool changes — release needed) — everything else:

• src/devx/** — Python package source (except __init__.py)
• pyproject.toml — Package metadata
• Any new file type not in the allowlist

Classification is configured in pyproject.toml under [tool.devx.classify]. The framework provides DEFAULT_INFRASTRUCTURE — a curated list of paths that are infrastructure for any Python project. Projects inherit these automatically and only specify what is different.

Rule priority (first match wins):

1. user_facing_overrides — safety override (highest priority)
2. infrastructure_overrides — explicit per-file
3. infrastructure — DEFAULT_INFRASTRUCTURE + project-specific patterns
4. Default: user-facing (safe — any unknown file triggers release)

Post-merge workflow (post-merge.yml)

Runs on every push to master. A single workflow with conditional jobs replaces separate workflows for release, wiki sync, badges, and Vikunja task updates.

Job dependency graph

detect-type ──┬── validate-commit-msg (skip if release commit)
              ├── release (skip if release commit)
              │       │
              │       ├── sync-wiki (needs release)
              │       ├── badges (needs release, ALWAYS runs)
              │       └── vikunja (needs release)
              └── configure-repo (independent, skip if release commit)

sync-wiki and vikunja depend on release succeeding so that the wiki and task tracker are only updated when the code is actually released. If release fails, they are skipped to avoid leaving the wiki or Vikunja in an inconsistent state.

The badges job uses if: always() with no is-release condition so it runs on every push to master, including release commits. This ensures badges (tests, coverage, version, etc.) are always current.

When release creates a release: vX.Y.Z commit, the release commit's post-merge run still updates badges (the version badge picks up the new version). Other jobs skip. The tag push triggers publish.yml.

Post-merge jobs

detect-type

Checks if the latest commit is a release commit (release: vX.Y.Z [skip ci]) using python -m devx.ci.detect_release_commit. Writes is-release=true or is-release=false to the job output. All subsequent jobs use this to conditionally skip for release commits.

validate-commit-msg

Depends on detect-type. Skips for release commits. Validates the latest commit message using python -m devx.ci.validate_commit_msg --branch master. On master, commits must follow {PREFIX}-N: <conventional commit> format (added by auto-merge).

release

Depends on detect-type. Skips for release commits. The core release automation job. Runs python -m devx.ci.release:

1. Classify changes — calls classify_changes.py to check for user-facing changes. If only infrastructure files changed, exits without releasing.
2. Calculate next version — uses git-cliff to determine the next semver version from conventional commits since the last tag
3. Update version file — updates __version__ in src/devx/__init__.py
4. Update changelog — prepends the new version section to CHANGELOG.md using git-cliff output
5. Run tests — executes make lint-ruff and make pytest-cov to verify the release is healthy. If either fails, the release is aborted — no commit, no tag. Use --skip-tests only for emergency releases.
6. Commit — stages the version file and changelog, commits with release: vX.Y.Z [skip ci] (uses --no-verify to bypass the commit-msg hook since release commits are a special case)
7. Create tag — creates an annotated tag vX.Y.Z with the changelog as the tag message
8. Push — pushes both the commit and tag to master

The script is idempotent: if there are no new conventional commits since the last tag, it exits without doing anything. If the tag already exists (for example, from a partial previous run), it skips tag creation and only pushes.

Tag consistency: Before releasing, the script fetches remote tags and verifies all existing tags point to commits whose message matches the tag version. This prevents duplicate release commits and ensures tag/version/commit alignment.

Version bumping rules (git-cliff):

Commit type	Version bump
feat:	minor (0.X.0)
fix:	patch (0.0.X)
feat!: or BREAKING CHANGE	minor (pre-1.0)
chore:, ci:, docs:	no bump (excluded by cliff.toml)

On failure, the notify_failure step creates a Gitea issue via python -m devx.ci.notify_failure.

sync-wiki

Depends on detect-type and release. Skips for release commits. Syncs documentation from docs/ to the Gitea wiki using python -m devx.ci.sync_wiki --repo <owner/repo> --strict:

1. Reads docs/mapping.json to map file paths to wiki page titles
2. Lists existing wiki pages via the Gitea API
3. For each mapped file, reads content and creates or updates the wiki page
4. --strict runs a full integrity check: verifies page count, missing pages, stale pages, and content match. Fails if any page is empty or content doesn't match.

Pages that exist in the wiki but not in the mapping are left untouched (not deleted).

On failure, the notify_failure step creates a Gitea issue.

badges

Depends on detect-type and release. Uses if: always() so it runs on every push to master, including release commits. Generates and pushes quality badges using python -m devx.ci.push_badges:

1. Fetch latest master — git fetch origin master && git reset --hard origin/master (ensures the version badge reflects the current state, even if the release job just pushed a new version)
2. Generate badges — calls devx.tools.generate_badges which runs pytest-cov, doc-coverage, lint checks, and version extraction, then writes SVG files: coverage.svg, tests.svg, docs.svg, quality.svg, version.svg, python.svg
3. Push to badges branch — creates an orphan badges branch, copies SVG files, commits, and force-pushes
4. Update README/docs — switches back to master, replaces raw/branch/badges/<name>.svg URLs with raw/commit/<sha>/<name>.svg URLs (cache-busting — Gitea caches raw/branch/ URLs for 6 hours), commits, and pushes

Supports --retries for retrying on git push failures (fetches latest master and waits 10s between attempts).

On failure, the notify_failure step creates a Gitea issue.

vikunja

Depends on detect-type and release. Skips for release commits. Updates the Vikunja task after a merge using python -m devx.ci.post_merge --git-sha <sha>:

1. Extracts the task ID from the first line of the commit message
2. Marks the corresponding Vikunja task as done
3. Posts a comment with the merge SHA

On failure, the notify_failure step creates a Gitea issue.

configure-repo

Depends on detect-type. Skips for release commits. Ensures branch protection and labels are configured using python -m devx.tools.configure_repo --repo <name> --owner <owner>:

• Sets up master branch protection (required status checks, block on rejected reviews, block on outdated branch)
• Creates standard labels
• Status check contexts read from DEVX_STATUS_CHECKS or default to CI / quality (pull_request)

On failure, the notify_failure step creates a Gitea issue.

Publish workflow (publish.yml)

Runs on tag pushes matching v*. Triggered by the release job in the post-merge workflow when it creates and pushes a new version tag.

Job: publish

1. Install dependencies — build, twine, requests, python-dotenv, click, and the project itself
2. Install CI tools — git-cliff and tea via python -m devx.tools.install_tools
3. Configure tea login — tea login add using CI_GITEA_TOKEN
4. Build and publish — python -m devx.ci.publish <tag> <owner/repo>:
   • Build the package with python -m build
   • Publish to the Gitea PyPI registry (default) using twine upload --repository-url <url> -u <token> -p <token>
   • OR publish to standard PyPI if PYPI_TOKEN is set
   • OR skip publishing if --skip-build is passed (non-Python repos)
   • Create a Gitea release with git-cliff-generated release notes via tea create release

Publishing destination resolution (checked in order):

1. Gitea PyPI registry — if --registry-url is given, or DEVX_PYPI_REGISTRY_URL env var is set, or derived from GITEA_API_URL
2. Standard PyPI — if PYPI_TOKEN is set (takes precedence over Gitea registry)
3. Skip — if neither is configured, only the Gitea release is created

On failure, the notify_failure step creates a Gitea issue.

CI scripts

auto_merge.py

Auto-merge PR when all CI checks pass. Reads task ID from the branch name (for example, DEVX-12-fix-foo → DEVX-12). Validates PR title format, checks the Vikunja task exists and the title matches, extracts the conventional commit message from PR commits, and squash-merges with {PREFIX}-N <conventional commit> title.

python -m devx.ci.auto_merge <branch> <pr_title> <owner/repo> <pr_number>

release.py

Automated release using git-cliff. Calculates next semver version from conventional commits, updates __version__ and CHANGELOG.md, runs lint and tests, commits with release: vX.Y.Z [skip ci], creates annotated tag, and pushes. Idempotent — exits if no unreleased changes.

python -m devx.ci.release [--dry-run] [--skip-tests] [--verify]

• --dry-run — preview without making changes
• --skip-tests — skip lint and test verification (emergency only)
• --verify — check tag/version/changelog alignment and exit

publish.py

Builds package, publishes to Gitea PyPI registry or standard PyPI, and creates a Gitea release with git-cliff-generated notes.

python -m devx.ci.publish <tag> <owner/repo> [--registry-url <url>] [--skip-build]

pr_review.py

Automated PR review. Fetches the PR diff via the Gitea API, runs automated checks (architecture, best practices, security, i18n, resource management, documentation, test coverage, commit conventions), and posts a structured review with inline comments.

python -m devx.ci.pr_review <pr_number> <owner/repo>

notify_failure.py

Creates a Gitea issue when a CI workflow fails. Uses the tea CLI for issue creation with failure labels. Supports --auto-login to configure the tea CLI login profile from CI_GITEA_TOKEN.

python -m devx.ci.notify_failure --repo <owner/repo> --run-id <id> \
  --workflow <name> --commit <sha> [--auto-login]

post_merge.py

Updates Vikunja task after a merge to master. Extracts task ID from the commit message, marks the task as done, and posts a comment with the merge SHA.

python -m devx.ci.post_merge <commit_msg> [--commit-sha <sha>] [--git-sha <sha>]

classify_changes.py

Classifies git changes as user-facing or workflow-only. Uses a layered rule system configured in pyproject.toml. Safe-by-default: any unknown file defaults to user-facing.

python -m devx.ci.classify_changes [--base <ref>] [--head <ref>] \
  [--quiet] [--check <category>] [--github-output]

discover_runners.py

Discovers available Gitea Actions runners at repository, organization, and instance levels. Falls back to MOLECULE_RUNNERS repo variable or DEFAULT_MAX_RUNNERS (3).

python -m devx.ci.discover_runners --owner <owner> --repo <repo> [--count] [--indices]

detect_release_commit.py

Detects whether the latest git commit is a release commit. Writes is-release=true|false to $GITHUB_OUTPUT.

python -m devx.ci.detect_release_commit

push_badges.py

Generates SVG badge files, pushes them to the badges branch, and updates README.md and docs/index.md with cache-busting raw/commit/<sha>/ URLs.

python -m devx.ci.push_badges [--output-dir <dir>] [--branch <branch>] \
  [--no-readme-update] [--retries <n>]

distribute_molecule.py

Distributes molecule (scenario, platform) pairs across N parallel runners. Discovers scenarios under ansible/roles/*/molecule/.

python -m devx.molecule.distribute_molecule --runner-index <i> --max-runners <n>
python -m devx.molecule.distribute_molecule --list
python -m devx.molecule.distribute_molecule --list-platforms

molecule_ci_guard.py

Runs molecule tests sequentially while polling the Gitea API for other runner failures. Aborts early if another runner fails the same job.

python -m devx.molecule.molecule_ci_guard [--roles-root <dir>] pair1 pair2 ...

validate_commit_msg.py

Validates commit messages. On feature branches: conventional commits only (no {PREFIX}-N prefix). On master: must have {PREFIX}-N prefix from auto-merge, followed by a conventional commit message.

python -m devx.ci.validate_commit_msg <commit_msg_file> [--branch <branch>]

sync_wiki.py

Syncs documentation from docs/ to the Gitea wiki via the API. Reads docs/mapping.json for file-to-page mapping. Supports --dry-run, --verify, and --strict (full integrity check).

python -m devx.ci.sync_wiki [--dry-run] [--repo <owner/repo>] [--verify] [--strict]

check_translations.py

Validates translation files against the Python source code. Checks for missing keys, dead keys, and missing languages.

python -m devx.ci.check_translations [--translations <file>]...

doc_coverage.py

Checks documentation coverage for CLI commands and major modules. Parses Click commands from cli.py and verifies documentation exists.

python -m devx.ci.doc_coverage [--docs-dir <dir>] [--fail-on-missing]

distribute_files.py

Distributes files matching a glob pattern across N parallel runners (round-robin). Writes the assigned file list to $GITHUB_ENV.

python -m devx.ci.distribute_files --pattern <glob> --runner-index <i> \
  --max-runners <n> [--github-env] [--skip-if-excess]

integration_guard.py

Runs pytest with cross-runner failure detection. If any other integration-tests matrix runner reports failure, the current pytest subprocess is killed and this runner exits early.

python -m devx.ci.integration_guard -- <pytest args>

Release process summary

The complete release process from PR to published package:

1. PR merged — auto-merge squash-merges the PR to master with {PREFIX}-N <conventional commit> title
2. Post-merge triggers — the merge push triggers post-merge.yml
3. detect-type — confirms the commit is not a release commit
4. release — release.py calculates the next version, updates files, runs tests, commits release: vX.Y.Z [skip ci], creates tag vX.Y.Z, and pushes to master
5. Tag push triggers publish — the tag push triggers publish.yml
6. publish — publish.py builds the package, publishes to the Gitea PyPI registry, and creates a Gitea release with git-cliff notes
7. sync-wiki — documentation is synced to the Gitea wiki
8. badges — quality badges are regenerated and pushed to the badges branch; README and docs/index.md are updated with cache-busting URLs
9. vikunja — the corresponding Vikunja task is marked as done
10. configure-repo — branch protection and labels are ensured

The release commit's post-merge run skips all jobs except badges (which picks up the new version number). This prevents infinite loops.

Failure handling

Every job in the post-merge and publish workflows has a notify_failure step that runs if: failure(). This creates a Gitea issue with the workflow name, run ID, and commit SHA, ensuring failures that would otherwise go unnoticed in the Actions tab are surfaced as issues. The issue is created via the tea CLI with a bug label if available.