Public Access
Sync wiki from docs/ [skip ci]
Devin CI
+599
@@ -1 +1,600 @@
|
|||||||
# Architecture
|
# Architecture
|
||||||
|
|
||||||
|
devx is a reusable Python package providing development and CI/CD tools for
|
||||||
|
oblachno-oss projects. It is self-contained under `src/devx/` and never imports
|
||||||
|
from scripts outside the package.
|
||||||
|
|
||||||
|
## Package structure
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/devx/
|
||||||
|
├── __init__.py # Version (single source of truth, read by setuptools)
|
||||||
|
├── cli.py # Click-based CLI entry point (devx command)
|
||||||
|
├── config.py # Configuration system (DEVX_ env vars, .env loading)
|
||||||
|
├── api_clients.py # GiteaClient, VikunjaClient — HTTP API wrappers
|
||||||
|
├── gitea_cli.py # TeaCLI — wrapper around tea CLI with JSON parsing
|
||||||
|
├── i18n.py # Translation system (JSON-based, translations.json)
|
||||||
|
├── exceptions.py # Custom exception types (DevxError, APIError)
|
||||||
|
├── opentofu.py # OpenTofu output helpers
|
||||||
|
├── translations.json # Translation strings (en, bg, de, ru, zh, pl)
|
||||||
|
├── ci/ # CI/CD automation modules (run by workflows)
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── _shared.py # Shared utilities (get_latest_tag)
|
||||||
|
│ ├── release.py # Automated versioning, tagging, changelog
|
||||||
|
│ ├── publish.py # Build and publish to Gitea PyPI registry
|
||||||
|
│ ├── auto_merge.py # Squash-merge PRs with task ID validation
|
||||||
|
│ ├── classify_changes.py # User-facing vs workflow-only change detection
|
||||||
|
│ ├── detect_release_commit.py # Detect release commits on master
|
||||||
|
│ ├── validate_commit_msg.py # Conventional commit validation
|
||||||
|
│ ├── pr_review.py # Automated PR review
|
||||||
|
│ ├── post_merge.py # Vikunja task updates after merge
|
||||||
|
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
|
||||||
|
│ ├── push_badges.py # Generate and push quality badges
|
||||||
|
│ ├── notify_failure.py # Create Gitea issues on CI failures
|
||||||
|
│ ├── distribute_files.py # Distribute files across parallel runners
|
||||||
|
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
|
||||||
|
│ ├── discover_runners.py # Dynamic Gitea runner discovery
|
||||||
|
│ ├── check_translations.py # Translation completeness check
|
||||||
|
│ └── doc_coverage.py # Documentation coverage check
|
||||||
|
├── tools/ # Developer tooling modules (run locally or by CI)
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── setup.py # Environment setup (venv, deps, hooks, tea login)
|
||||||
|
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea
|
||||||
|
│ ├── check_test_speed.py # Measure unit test execution time
|
||||||
|
│ ├── check_test_isolation.py # Pytest plugin: detect un-hermetic test patterns
|
||||||
|
│ ├── configure_repo.py # Branch protection and label setup
|
||||||
|
│ ├── generate_badges.py # Badge SVG generation
|
||||||
|
│ ├── generate_cliff_config.py # Generate cliff.toml with correct prefix
|
||||||
|
│ └── install_checkmake.py # Install checkmake (Makefile linter)
|
||||||
|
└── molecule/ # Optional molecule testing helpers (Ansible projects)
|
||||||
|
├── __init__.py
|
||||||
|
├── discover_runners.py # Dynamic Gitea runner discovery
|
||||||
|
├── distribute_molecule.py # Distribute scenarios across runners
|
||||||
|
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
|
||||||
|
├── molecule_all.py # Run all molecule scenarios locally
|
||||||
|
├── start_docker.py # Ensure Docker is available for molecule
|
||||||
|
└── platforms.py # Supported molecule platforms
|
||||||
|
```
|
||||||
|
|
||||||
|
## Core modules
|
||||||
|
|
||||||
|
### `__init__.py`
|
||||||
|
|
||||||
|
Contains only `__version__`, the single source of truth for the package
|
||||||
|
version. Read by setuptools via `dynamic = ["version"]` in `pyproject.toml`.
|
||||||
|
Updated automatically by `devx.ci.release` during the release process. Treated
|
||||||
|
as infrastructure (not user-facing) by the change classifier since it is a
|
||||||
|
release artifact, not user code.
|
||||||
|
|
||||||
|
### `cli.py`
|
||||||
|
|
||||||
|
Click-based CLI entry point. Provides three command groups: `devx ci`,
|
||||||
|
`devx tools`, and `devx molecule`. Each subcommand delegates to the
|
||||||
|
corresponding module via `_run_module()`, which imports the module, sets
|
||||||
|
`sys.argv`, and calls its `main()` function. This design keeps all logic in
|
||||||
|
the modules themselves — `cli.py` is purely a router.
|
||||||
|
|
||||||
|
The CLI is registered as a console script via `pyproject.toml`:
|
||||||
|
```toml
|
||||||
|
[project.scripts]
|
||||||
|
devx = "devx.cli:cli"
|
||||||
|
```
|
||||||
|
|
||||||
|
### `config.py`
|
||||||
|
|
||||||
|
Shared configuration constants for all devx modules. All defaults can be
|
||||||
|
overridden via environment variables with the `DEVX_` prefix. Provides:
|
||||||
|
|
||||||
|
- `GITEA_API_URL` / `VIKUNJA_API_URL` — API endpoints
|
||||||
|
- `REPO_OWNER` — repository owner (must be set per-project)
|
||||||
|
- `TASK_PREFIX` / `TASK_ID_RE` — task ID prefix and regex (for example, `DEVX-N`)
|
||||||
|
- `VIKUNJA_PROJECT_ID` — Vikunja project for task tracking
|
||||||
|
- `DEFAULT_TIMEOUT`, `DEFAULT_PER_PAGE` — HTTP client defaults
|
||||||
|
- `MAX_RETRIES`, `RETRY_BACKOFF_BASE`, `RETRY_STATUS_CODES` — retry config
|
||||||
|
- `CONVENTIONAL_RE` — conventional commit format regex
|
||||||
|
|
||||||
|
### `exceptions.py`
|
||||||
|
|
||||||
|
Custom exception hierarchy:
|
||||||
|
|
||||||
|
- `DevxError` — base exception for all devx errors
|
||||||
|
- `APIError(DevxError)` — raised when a REST API call returns an HTTP error.
|
||||||
|
Carries `status` (HTTP status code) and `message` (error message).
|
||||||
|
|
||||||
|
### `i18n.py`
|
||||||
|
|
||||||
|
Simple i18n system using a JSON translations file (`translations.json`).
|
||||||
|
Supports six languages: `en`, `bg`, `de`, `pl`, `ru`, `zh`. The `_()` function
|
||||||
|
wraps user-facing strings for translation.
|
||||||
|
|
||||||
|
Projects can extend translations by setting `DEVX_TRANSLATIONS_PATH` to a
|
||||||
|
custom JSON file. Keys from the project's file are merged on top of devx's
|
||||||
|
built-in translations, allowing projects to override or add keys without
|
||||||
|
modifying the package.
|
||||||
|
|
||||||
|
### `api_clients.py`
|
||||||
|
|
||||||
|
Reusable HTTP API clients with connection pooling and retry logic. Both
|
||||||
|
clients retry on transient errors (429, 5xx, connection errors) with
|
||||||
|
exponential backoff (2s, 4s, 8s).
|
||||||
|
|
||||||
|
**`GiteaClient`** — Gitea REST API wrapper:
|
||||||
|
- Branch protection (get, create, update)
|
||||||
|
- Labels (list, create, add to issues)
|
||||||
|
- Issues (create, list)
|
||||||
|
- Pull requests (get commits, merge, create review)
|
||||||
|
- Releases (list, create idempotent)
|
||||||
|
- Actions (list runs, list jobs, get job logs)
|
||||||
|
- Actions variables (get, set idempotent)
|
||||||
|
- Wiki pages (list, fetch, create, update, delete)
|
||||||
|
|
||||||
|
**`VikunjaClient`** — Vikunja REST API wrapper:
|
||||||
|
- Tasks (list project tasks, get, update, mark done)
|
||||||
|
- Comments (create)
|
||||||
|
|
||||||
|
### `gitea_cli.py`
|
||||||
|
|
||||||
|
Thin Python wrapper around the `tea` Gitea CLI tool. Parses JSON output for
|
||||||
|
structured data. Used by CI scripts for Gitea API operations that tea handles
|
||||||
|
well, avoiding hand-rolled HTTP requests.
|
||||||
|
|
||||||
|
**`TeaCLI`** operations:
|
||||||
|
- `create_issue()` — Create issues with labels
|
||||||
|
- `list_labels()` / `create_label()` / `add_label()` — Label management
|
||||||
|
- `create_pr()` / `merge_pr()` / `review_pr()` — Pull request operations
|
||||||
|
- `create_release()` / `list_releases()` — Release management
|
||||||
|
- `list_branches()` — Branch listing
|
||||||
|
|
||||||
|
Operations NOT supported via tea (still use `GiteaClient`):
|
||||||
|
- Wiki page management
|
||||||
|
- Commit status checks
|
||||||
|
- Runner discovery
|
||||||
|
- PR file/commit listing (tea has limited support)
|
||||||
|
- Branch protection with detailed config
|
||||||
|
|
||||||
|
### `opentofu.py`
|
||||||
|
|
||||||
|
OpenTofu output helpers for CI/CD deployment scripts. Provides reusable
|
||||||
|
functions for extracting values from `tofu output` in a structured way,
|
||||||
|
eliminating duplicated `subprocess.run` boilerplate:
|
||||||
|
|
||||||
|
- `get_tofu_output(output_name, cwd, env)` — Run `tofu output -json` and return parsed JSON
|
||||||
|
- `get_tofu_vm_ip(output_name, vm_name, cwd, env)` — Extract a VM's IP address
|
||||||
|
- `get_tofu_vm_field(output_name, vm_name, field, cwd, env)` — Extract a VM field
|
||||||
|
|
||||||
|
## CI/CD modules (`devx.ci`)
|
||||||
|
|
||||||
|
Modules in this package are run by Gitea Actions workflows. They may import
|
||||||
|
from `devx.api_clients`, `devx.config`, `devx.gitea_cli`, and `devx.i18n`.
|
||||||
|
|
||||||
|
### `release.py`
|
||||||
|
|
||||||
|
Automated release using git-cliff. Calculates the next semver version from
|
||||||
|
conventional commits since the last tag, updates `__version__` in
|
||||||
|
`__init__.py` and `CHANGELOG.md`, runs lint and tests to verify the release
|
||||||
|
is healthy, commits with `release: vX.Y.Z [skip ci]`, creates an annotated
|
||||||
|
tag, and pushes both to master.
|
||||||
|
|
||||||
|
Idempotent: if there are no new conventional commits since the last tag, it
|
||||||
|
exits without doing anything. If the tag already exists, it skips tag creation
|
||||||
|
and only pushes. Includes a `--verify` mode that checks tag/version/changelog
|
||||||
|
alignment without making changes.
|
||||||
|
|
||||||
|
### `publish.py`
|
||||||
|
|
||||||
|
Builds the Python package with `python -m build`, publishes to a Gitea PyPI
|
||||||
|
registry (or standard PyPI if `PYPI_TOKEN` is set), and creates a Gitea
|
||||||
|
release with git-cliff-generated notes. Supports `--skip-build` for non-Python
|
||||||
|
repos that only need a Gitea release.
|
||||||
|
|
||||||
|
### `auto_merge.py`
|
||||||
|
|
||||||
|
Auto-merges a PR when all CI checks pass. Reads the task ID from the branch
|
||||||
|
name, validates the PR title format against
|
||||||
|
the Vikunja task title, extracts the conventional commit message from PR
|
||||||
|
commits, and squash-merges with title `{PREFIX}-N <conventional commit>`.
|
||||||
|
|
||||||
|
If the head branch is behind master (HTTP 405), it automatically pulls master,
|
||||||
|
rebases, force-pushes, and retries the merge.
|
||||||
|
|
||||||
|
### `classify_changes.py`
|
||||||
|
|
||||||
|
Classifies git changes between two refs as user-facing or workflow-only. Uses
|
||||||
|
a layered rule system configured in `pyproject.toml` under
|
||||||
|
`[tool.devx.classify]`:
|
||||||
|
|
||||||
|
1. **User-facing overrides** (highest priority — safety override)
|
||||||
|
2. **Infrastructure overrides** (explicit per-file)
|
||||||
|
3. **Infrastructure patterns** (DEFAULT_INFRASTRUCTURE + project-specific)
|
||||||
|
4. **Default**: user-facing (safe default — any unknown file triggers release)
|
||||||
|
|
||||||
|
Also supports custom tags (orthogonal to release impact) for CI conditional
|
||||||
|
execution (for example, `ansible` tag to trigger molecule tests).
|
||||||
|
|
||||||
|
### `pr_review.py`
|
||||||
|
|
||||||
|
Automated PR review. Fetches the PR diff via the Gitea API and runs a series
|
||||||
|
of checks, posting a structured review with `COMMENT` (no issues) or
|
||||||
|
`REQUEST_CHANGES` (issues found):
|
||||||
|
|
||||||
|
- Architecture compliance (no subprocess in CLI, no hardcoded URLs)
|
||||||
|
- Best practices (no `print()`, no bare `except`, no `TODO`/`FIXME`, no
|
||||||
|
functions > 50 lines)
|
||||||
|
- Security (no hardcoded secrets, no `shell=True`, no `eval`/`exec`)
|
||||||
|
- i18n (no raw strings in `click.echo()` without `_()` wrapper)
|
||||||
|
- Resource management (no `open()` without `with`, no `Popen()` without cleanup)
|
||||||
|
- Documentation (source changes must include doc updates)
|
||||||
|
- Test coverage (source changes must include test updates)
|
||||||
|
- Commit conventions (conventional commit format on PR commits)
|
||||||
|
|
||||||
|
### `sync_wiki.py`
|
||||||
|
|
||||||
|
Syncs documentation from `docs/` to the Gitea wiki via the API. Reads
|
||||||
|
`docs/mapping.json` to map file paths to wiki page titles, then creates or
|
||||||
|
updates pages. Supports `--dry-run`, `--verify` (check content), and
|
||||||
|
`--strict` (full integrity check: page count, missing pages, stale pages,
|
||||||
|
content match).
|
||||||
|
|
||||||
|
### `push_badges.py`
|
||||||
|
|
||||||
|
Generates SVG badge files using `devx.tools.generate_badges`, pushes them to
|
||||||
|
an orphan `badges` branch, and updates `README.md` and `docs/index.md` on
|
||||||
|
master with cache-busting `raw/commit/<sha>/badge.svg` URLs (Gitea caches
|
||||||
|
`raw/branch/` URLs for 6 hours). Fetches latest master before generating
|
||||||
|
badges so the version badge reflects the current state. Supports `--retries`
|
||||||
|
for retrying on git push failures.
|
||||||
|
|
||||||
|
### `notify_failure.py`
|
||||||
|
|
||||||
|
Creates a Gitea issue when a CI workflow fails. Uses the `tea` CLI for issue
|
||||||
|
creation with failure labels. Supports `--auto-login` to configure the tea
|
||||||
|
CLI login profile from `CI_GITEA_TOKEN` and `DEVX_GITEA_API_URL` before creating
|
||||||
|
the issue.
|
||||||
|
|
||||||
|
### `post_merge.py`
|
||||||
|
|
||||||
|
Updates the Vikunja task after a merge to master. Extracts the task ID from
|
||||||
|
the commit message, marks the task as done, and posts a comment with the
|
||||||
|
merge SHA.
|
||||||
|
|
||||||
|
### `validate_commit_msg.py`
|
||||||
|
|
||||||
|
Validates commit messages. On feature branches: conventional commits only
|
||||||
|
(no `{PREFIX}-N` prefix). On master: must have `{PREFIX}-N` prefix from
|
||||||
|
auto-merge, followed by a conventional commit message.
|
||||||
|
|
||||||
|
### `detect_release_commit.py`
|
||||||
|
|
||||||
|
Detects whether the latest git commit is a release commit
|
||||||
|
(`release: vX.Y.Z [skip ci]`). Writes `is-release=true` or `is-release=false`
|
||||||
|
to `$GITHUB_OUTPUT` for use in CI workflow conditionals.
|
||||||
|
|
||||||
|
### `check_translations.py`
|
||||||
|
|
||||||
|
Validates translation files against the Python source code. Checks for
|
||||||
|
missing keys (used in code but not in translations), dead keys (defined but
|
||||||
|
not used), and missing languages (a key exists but is missing one of the five
|
||||||
|
supported languages). Supports checking additional translation sets via
|
||||||
|
`--translations`.
|
||||||
|
|
||||||
|
### `doc_coverage.py`
|
||||||
|
|
||||||
|
Checks documentation coverage for CLI commands and major modules. Parses
|
||||||
|
Click commands from `cli.py` and verifies each has documentation in
|
||||||
|
`docs/user/cli-commands.md`. Checks that core modules are documented in
|
||||||
|
`architecture.md` and CI scripts in `ci-cd-workflow.md`. Supports
|
||||||
|
`--fail-on-missing` to enforce 100% coverage.
|
||||||
|
|
||||||
|
### `discover_runners.py`
|
||||||
|
|
||||||
|
Discovers available Gitea Actions runners at three levels: repository,
|
||||||
|
organization, and instance (admin). Falls back to the `MOLECULE_RUNNERS` repo
|
||||||
|
variable or `DEFAULT_MAX_RUNNERS` (3). Outputs runner count or a JSON index
|
||||||
|
array for use as a dynamic matrix in Gitea Actions.
|
||||||
|
|
||||||
|
### `distribute_files.py`
|
||||||
|
|
||||||
|
Distributes files matching a glob pattern across N parallel runners
|
||||||
|
(round-robin). Writes the assigned file list for the current runner to
|
||||||
|
`$GITHUB_ENV`. Used for splitting test suites across CI runners.
|
||||||
|
|
||||||
|
### `integration_guard.py`
|
||||||
|
|
||||||
|
Runs pytest with the same cross-runner failure detection mechanism used by
|
||||||
|
`molecule_ci_guard`. If any other integration-tests matrix runner reports
|
||||||
|
failure, the current pytest subprocess is killed and this runner exits early.
|
||||||
|
|
||||||
|
## Developer tools (`devx.tools`)
|
||||||
|
|
||||||
|
Modules in this package are run locally or by CI setup jobs. They may import
|
||||||
|
from `devx.api_clients`, `devx.config`, and `devx.gitea_cli`.
|
||||||
|
|
||||||
|
### `setup.py`
|
||||||
|
|
||||||
|
Project setup: installs Python dependencies (editable mode with extras),
|
||||||
|
Ansible Galaxy collections (if `ansible/requirements.yml` exists in the target repo), pre-commit
|
||||||
|
hooks (pre-commit, commit-msg, pre-push), and configures the `tea` CLI login
|
||||||
|
profile from `.env`. Supports `--extras` to specify dependency groups,
|
||||||
|
`--no-pre-commit` to skip hook installation, and `--no-tea-login` to skip tea
|
||||||
|
configuration.
|
||||||
|
|
||||||
|
### `install_tools.py`
|
||||||
|
|
||||||
|
Installs CI/CD development tools that are not Python packages: actionlint,
|
||||||
|
git-cliff, act_runner, and tea. Each tool is installed to `~/.local/bin` if
|
||||||
|
not already on PATH. Idempotent: skips tools that are already available.
|
||||||
|
Supports `--tool` to install specific tools and `--list` to show status.
|
||||||
|
|
||||||
|
### `check_test_speed.py`
|
||||||
|
|
||||||
|
Runs unit tests and enforces execution-time budgets. Two quality gates:
|
||||||
|
total suite time must not exceed `--max-seconds` (default: 10s), and no
|
||||||
|
individual test may exceed `--max-single-seconds` (default: 0.5s, 0 to
|
||||||
|
disable). Runs `make test-unit` with `PYTEST_ADDOPTS=--durations=0`.
|
||||||
|
|
||||||
|
### `check_test_isolation.py`
|
||||||
|
|
||||||
|
Pytest plugin (auto-discovered via `pytest11` entry point) that
|
||||||
|
statically analyzes test files for un-hermetic patterns causing slow
|
||||||
|
or flaky tests: unpatched `subprocess.run`/`time.sleep` calls, known
|
||||||
|
subprocess-spawning helpers called without `@patch`, and excessive
|
||||||
|
loop iterations (>100). Also available as a standalone CLI for CI
|
||||||
|
gates and pre-commit hooks. See ADR-0001 for design rationale.
|
||||||
|
|
||||||
|
### `configure_repo.py`
|
||||||
|
|
||||||
|
Configures repository branch protection and labels via the Gitea REST API.
|
||||||
|
Sets up master branch protection (required status checks, block on rejected
|
||||||
|
reviews, block on outdated branch) and creates standard labels. Status check
|
||||||
|
contexts are read from `DEVX_STATUS_CHECKS` or default to
|
||||||
|
`CI / validate (pull_request)`.
|
||||||
|
|
||||||
|
### `generate_badges.py`
|
||||||
|
|
||||||
|
Generates self-contained SVG badge files from project metrics. Runs
|
||||||
|
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
|
||||||
|
SVG files that can be served as static files from the Gitea raw file API.
|
||||||
|
Badges generated: coverage, tests, docs, quality, version, python.
|
||||||
|
|
||||||
|
### `generate_cliff_config.py`
|
||||||
|
|
||||||
|
Generates a `cliff.toml` configuration file with the correct task ID prefix
|
||||||
|
preprocessor. Eliminates the need to manually duplicate and maintain
|
||||||
|
`cliff.toml` across repos that use devx. Supports `--prefix` to set the task
|
||||||
|
ID prefix and `--force` to overwrite an existing file.
|
||||||
|
|
||||||
|
### `install_checkmake.py`
|
||||||
|
|
||||||
|
Installs checkmake (Makefile linter) if not already present. Tries
|
||||||
|
`go install` first if Go is available, otherwise downloads the latest
|
||||||
|
pre-built Linux binary from the official GitHub releases.
|
||||||
|
|
||||||
|
## Molecule modules (`devx.molecule`)
|
||||||
|
|
||||||
|
Optional modules for projects with Ansible roles. Requires the `molecule`
|
||||||
|
extra (`pip install devx[molecule]`).
|
||||||
|
|
||||||
|
### `distribute_molecule.py`
|
||||||
|
|
||||||
|
Distributes molecule (scenario, platform) pairs across N parallel runners.
|
||||||
|
Discovers scenarios under `ansible/roles/*/molecule/` and crosses them with
|
||||||
|
the supported OS platform matrix. Supports `--roles-root` for multi-role
|
||||||
|
repositories, `--list` to list scenarios, and `--list-platforms` to list
|
||||||
|
platforms.
|
||||||
|
|
||||||
|
### `molecule_ci_guard.py`
|
||||||
|
|
||||||
|
Runs molecule tests sequentially while polling the Gitea API for other runner
|
||||||
|
failures. If any other molecule matrix runner reports failure, the current
|
||||||
|
molecule subprocess is killed and this runner exits early. Supports both
|
||||||
|
single-role (4-part) and multi-role (5-part) pair encoding.
|
||||||
|
|
||||||
|
### `molecule_all.py`
|
||||||
|
|
||||||
|
Runs all molecule scenarios on all supported OS platforms sequentially.
|
||||||
|
Intended for local development; CI uses the parallel matrix instead.
|
||||||
|
|
||||||
|
### `molecule/discover_runners.py`
|
||||||
|
|
||||||
|
Discovers available Gitea Actions runners for molecule tests. Same logic as
|
||||||
|
`devx.ci.discover_runners` but intended for molecule-specific workflows.
|
||||||
|
|
||||||
|
### `start_docker.py`
|
||||||
|
|
||||||
|
Ensures Docker is available for molecule tests in CI. Verifies Docker is
|
||||||
|
accessible and sets `DOCKER_HOST` explicitly. If the host socket is not
|
||||||
|
available, tries the rootless socket, then starts a local `dockerd` with the
|
||||||
|
vfs storage driver (requires privileged container).
|
||||||
|
|
||||||
|
### `platforms.py`
|
||||||
|
|
||||||
|
Single source of truth for the supported OS platform matrix. Each entry maps
|
||||||
|
a short name to (image, command). Uses the project's pre-built
|
||||||
|
molecule-test-base image with `sleep infinity` (not systemd) to avoid cgroup
|
||||||
|
v2 failures. Supports loading custom platforms from a JSON file.
|
||||||
|
|
||||||
|
## Design principles
|
||||||
|
|
||||||
|
- **Self-contained package** — `src/devx/` never imports from scripts outside
|
||||||
|
the package. This allows devx to be installed and used as a dependency
|
||||||
|
without requiring a specific repo layout in the consumer.
|
||||||
|
- **Module-based invocation** — All tools invoked via `python -m devx.ci.*`,
|
||||||
|
`python -m devx.tools.*`, or `python -m devx.molecule.*`. The `devx` CLI is
|
||||||
|
a thin router that delegates to module `main()` functions.
|
||||||
|
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (not `.:src` since
|
||||||
|
there are no scripts at repo root). The `src` directory is the sole import
|
||||||
|
root.
|
||||||
|
- **Config via env vars** — `DEVX_*` environment variables with `.env` file
|
||||||
|
fallback. Projects override defaults via environment or `.env`, never by
|
||||||
|
editing package code.
|
||||||
|
- **100% test coverage** — enforced by `--cov-fail-under=100` in pytest.
|
||||||
|
- **i18n by default** — all user-facing strings wrapped in `_()` for
|
||||||
|
translation. Five languages supported out of the box.
|
||||||
|
- **Safe-by-default classification** — any file that doesn't match an
|
||||||
|
infrastructure pattern defaults to user-facing, triggering a release. This
|
||||||
|
prevents new file types from accidentally skipping releases.
|
||||||
|
- **Secrets via environment** — secrets are passed via environment variables,
|
||||||
|
never on the command line.
|
||||||
|
|
||||||
|
## Import rules
|
||||||
|
|
||||||
|
1. **`src/devx/` is self-contained** — the package never imports from outside `src/`
|
||||||
|
2. **CI modules** (`devx.ci.*`) may import from `devx.api_clients`,
|
||||||
|
`devx.config`, `devx.gitea_cli`, `devx.i18n`
|
||||||
|
3. **Tool modules** (`devx.tools.*`) may import from `devx.api_clients`,
|
||||||
|
`devx.config`, `devx.gitea_cli`
|
||||||
|
4. **Cross-module imports** within `devx.ci.*` or `devx.tools.*` are allowed
|
||||||
|
but must be documented (for example, `release.py` imports from
|
||||||
|
`classify_changes.py`)
|
||||||
|
|
||||||
|
## Data flow
|
||||||
|
|
||||||
|
### PR lifecycle
|
||||||
|
|
||||||
|
```text
|
||||||
|
Developer creates Vikunja task (DEVX-N)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Developer creates branch (DEVX-N-short-description)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Developer commits (conventional commits, no DEVX-N prefix)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Developer pushes and creates PR (title: "DEVX-N: <vikunja task title>")
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
CI workflow (ci.yml) triggers:
|
||||||
|
│
|
||||||
|
├── validate (single job: quality + detect-changes +
|
||||||
|
│ release-dry-run + pr-review + pre-merge validation)
|
||||||
|
│ ├── quality steps (lint, tests, coverage, test speed, doc coverage,
|
||||||
|
│ │ translation check, dependency scan, workflow dry-run)
|
||||||
|
│ ├── detect-changes (classify_changes.py → user-facing or workflow-only)
|
||||||
|
│ │ └── if user-facing → release-dry-run (release.py --dry-run)
|
||||||
|
│ ├── pre-merge validation (check_auto_merge_ready.py)
|
||||||
|
│ └── pr-review (pr_review.py → posts COMMENT or REQUEST_CHANGES)
|
||||||
|
│
|
||||||
|
└── auto-merge (auto_merge.py)
|
||||||
|
├── validate PR title format
|
||||||
|
├── validate PR title matches Vikunja task title
|
||||||
|
├── extract conventional commit message from PR commits
|
||||||
|
├── squash-merge with "DEVX-N <conventional commit>" title
|
||||||
|
└── push to master
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Post-merge workflow triggers (see below)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Post-merge flow
|
||||||
|
|
||||||
|
```text
|
||||||
|
Push to master (squash-merge commit: "DEVX-N <conventional commit>")
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Post-merge workflow (post-merge.yml) triggers:
|
||||||
|
│
|
||||||
|
├── detect-and-configure (single job)
|
||||||
|
│ ├── configure-repo (configure_repo.py)
|
||||||
|
│ ├── detect-type (detect_release_commit.py)
|
||||||
|
│ │ └── is-release? → skip all steps except badges
|
||||||
|
│ └── validate-commit-msg (validate_commit_msg.py --branch master)
|
||||||
|
│
|
||||||
|
└── release-and-maintain (needs detect-and-configure)
|
||||||
|
├── release (release.py) [skip if release commit or workflow-only]
|
||||||
|
│ ├── classify_changes.py → skip if workflow-only
|
||||||
|
│ ├── git-cliff → calculate next version
|
||||||
|
│ ├── update __version__ in __init__.py
|
||||||
|
│ ├── update CHANGELOG.md
|
||||||
|
│ ├── run make lint-ruff && make pytest-cov
|
||||||
|
│ ├── commit "release: vX.Y.Z [skip ci]"
|
||||||
|
│ ├── create annotated tag vX.Y.Z
|
||||||
|
│ └── push commit + tag to master
|
||||||
|
│ │
|
||||||
|
│ ▼
|
||||||
|
│ publish (publish.py) [if release created a tag]
|
||||||
|
│ ├── build package (python -m build)
|
||||||
|
│ ├── publish to Gitea PyPI registry (twine upload)
|
||||||
|
│ │ OR publish to standard PyPI (if PYPI_TOKEN set)
|
||||||
|
│ │ OR skip publish (if --skip-build)
|
||||||
|
│ └── create Gitea release with git-cliff notes
|
||||||
|
│
|
||||||
|
├── sync-wiki (sync_wiki.py --strict) [skip if automated]
|
||||||
|
│ └── sync docs/ to Gitea wiki with integrity check
|
||||||
|
│
|
||||||
|
├── vikunja (post_merge.py) [skip if automated]
|
||||||
|
│ ├── extract task ID from commit message
|
||||||
|
│ ├── mark Vikunja task as done
|
||||||
|
│ └── post comment with merge SHA
|
||||||
|
│
|
||||||
|
└── badges (push_badges.py) [ALWAYS runs, even on release commits]
|
||||||
|
├── fetch latest master
|
||||||
|
├── generate_badges.py → SVG files
|
||||||
|
├── push to orphan badges branch
|
||||||
|
└── update README.md + docs/index.md with cache-busting URLs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Publish flow
|
||||||
|
|
||||||
|
```text
|
||||||
|
Within release-and-maintain job (after release step creates a tag):
|
||||||
|
│
|
||||||
|
├── install build, twine, git-cliff, tea
|
||||||
|
├── configure tea login
|
||||||
|
├── checkout release tag
|
||||||
|
│
|
||||||
|
└── publish (publish.py)
|
||||||
|
├── build package (python -m build)
|
||||||
|
├── publish to Gitea PyPI registry (twine upload)
|
||||||
|
│ OR publish to standard PyPI (if PYPI_TOKEN set)
|
||||||
|
│ OR skip publish (if --skip-build)
|
||||||
|
└── create Gitea release with git-cliff notes
|
||||||
|
```
|
||||||
|
|
||||||
|
### Badge generation flow
|
||||||
|
|
||||||
|
```text
|
||||||
|
push_badges.py:
|
||||||
|
│
|
||||||
|
├── fetch_latest_master() → git fetch + reset --hard origin/master
|
||||||
|
│
|
||||||
|
├── generate_badges() → devx.tools.generate_badges
|
||||||
|
│ ├── run pytest-cov → parse coverage %
|
||||||
|
│ ├── run pytest → parse test count
|
||||||
|
│ ├── run doc_coverage → parse doc coverage %
|
||||||
|
│ ├── run lint → quality status
|
||||||
|
│ ├── read __version__ from __init__.py
|
||||||
|
│ └── write SVG files to .badges/
|
||||||
|
│
|
||||||
|
├── push_to_badges_branch()
|
||||||
|
│ ├── git checkout --orphan badges
|
||||||
|
│ ├── git rm -rf .
|
||||||
|
│ ├── copy SVG files to root
|
||||||
|
│ ├── git commit "Update badges [skip ci]"
|
||||||
|
│ ├── git push origin badges --force
|
||||||
|
│ └── return commit SHA
|
||||||
|
│
|
||||||
|
└── update_readme_with_badge_sha()
|
||||||
|
├── git checkout master
|
||||||
|
├── replace raw/branch/badges/ URLs with raw/commit/<sha>/ URLs
|
||||||
|
├── git commit "chore: update badge URLs [skip ci]"
|
||||||
|
└── git push origin master
|
||||||
|
```
|
||||||
|
|
||||||
|
## tea CLI integration
|
||||||
|
|
||||||
|
The `tea` Gitea CLI tool is used for Gitea API interactions where tea provides
|
||||||
|
reliable, official support. It is installed by
|
||||||
|
`python -m devx.tools.install_tools` and configured by
|
||||||
|
`python -m devx.tools.setup` (login profile from `.env` `CI_GITEA_TOKEN`).
|
||||||
|
|
||||||
|
`devx.gitea_cli.TeaCLI` wraps tea with JSON output parsing. Operations that
|
||||||
|
tea does not support (wiki management, commit status, runner discovery,
|
||||||
|
detailed branch protection) fall back to `GiteaClient` (direct HTTP).
|
||||||
|
|
||||||
|
## Version source
|
||||||
|
|
||||||
|
The version source is `__version__` in `src/devx/__init__.py`, read by
|
||||||
|
setuptools via `dynamic = ["version"]` in `pyproject.toml`. The release
|
||||||
|
script updates this file, commits it, and tags the commit. This ensures the
|
||||||
|
package version, git tag, and changelog always stay aligned.
|
||||||
|
|||||||
+581
-1
@@ -1 +1,581 @@
|
|||||||
# CI-CD-Workflow
|
# CI/CD Workflow
|
||||||
|
|
||||||
|
devx uses Gitea Actions for CI/CD automation. Two workflows implement a
|
||||||
|
complete pipeline: pull request validation and post-merge release
|
||||||
|
automation (including publishing).
|
||||||
|
|
||||||
|
## Workflow overview
|
||||||
|
|
||||||
|
```text
|
||||||
|
PR opened/synchronized ──► CI (ci.yml)
|
||||||
|
│ ├── validate (quality + detect-changes +
|
||||||
|
│ │ release-dry-run + pr-review +
|
||||||
|
│ │ pre-merge validation)
|
||||||
|
│ └── auto-merge ──► squash-merge to master
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
Push to master ──► Post-merge (post-merge.yml)
|
||||||
|
├── detect-and-configure (detect-type +
|
||||||
|
│ validate-commit-msg +
|
||||||
|
│ configure-repo)
|
||||||
|
└── release-and-maintain
|
||||||
|
├── release ──► tag vX.Y.Z
|
||||||
|
├── publish ──► Gitea PyPI registry + Gitea release
|
||||||
|
├── sync-wiki
|
||||||
|
├── vikunja
|
||||||
|
└── badges (always runs)
|
||||||
|
```
|
||||||
|
|
||||||
|
## CI workflow (`ci.yml`)
|
||||||
|
|
||||||
|
Runs on pull requests (opened and synchronize) and manual dispatch.
|
||||||
|
|
||||||
|
### Jobs
|
||||||
|
|
||||||
|
#### `validate`
|
||||||
|
|
||||||
|
The single validation job. Consolidates the former `quality`,
|
||||||
|
`detect-changes`, `release-dry-run`, `pr-review`, and `pre-merge-check`
|
||||||
|
jobs into one job to save checkout+setup overhead. Runs on every PR.
|
||||||
|
|
||||||
|
**Quality steps**
|
||||||
|
|
||||||
|
The main quality gate:
|
||||||
|
|
||||||
|
1. **Lint all** — ruff check, ruff format check, pyright, bandit, actionlint
|
||||||
|
(via `make lint-all`)
|
||||||
|
2. **Unit tests with 100% coverage** — `make pytest-cov`
|
||||||
|
3. **Check unit test speed** — `python -m devx.tools.check_test_speed
|
||||||
|
--max-seconds 4 --max-single-seconds 0.5`
|
||||||
|
4. **Documentation coverage check** — `python -m devx.ci.doc_coverage
|
||||||
|
--fail-on-missing`
|
||||||
|
5. **Translation completeness check** — `python -m devx.ci.check_translations`
|
||||||
|
6. **Dependency security scan** — `pip-audit --desc --skip-editable`
|
||||||
|
(best-effort, non-blocking)
|
||||||
|
7. **Workflow dry-run validation** — `make workflow-dryrun` via act_runner
|
||||||
|
(best-effort, skipped if act_runner is not installed)
|
||||||
|
|
||||||
|
**`detect-changes` step**
|
||||||
|
|
||||||
|
Classifies changes between `origin/master` and the PR head as user-facing or
|
||||||
|
workflow-only using `python -m devx.ci.classify_changes --github-output`.
|
||||||
|
Writes `user-facing-changed=true|false` to the job output for use by
|
||||||
|
downstream steps.
|
||||||
|
|
||||||
|
**`release-dry-run` step**
|
||||||
|
|
||||||
|
Only runs if the detect-changes step detected user-facing changes. Runs
|
||||||
|
`python -m devx.ci.release --dry-run` to validate that the release script
|
||||||
|
can calculate the next version and generate the changelog without making
|
||||||
|
changes. Non-blocking (uses `|| true`).
|
||||||
|
|
||||||
|
**`pr-review` step**
|
||||||
|
|
||||||
|
Runs on every pull request. Executes `python -m devx.ci.pr_review` with the
|
||||||
|
PR number and repository. Fetches the PR diff via the Gitea API and runs
|
||||||
|
automated checks, posting a structured review:
|
||||||
|
|
||||||
|
- `COMMENT` — no issues found
|
||||||
|
- `REQUEST_CHANGES` — issues found that must be addressed
|
||||||
|
|
||||||
|
Checks performed:
|
||||||
|
1. Architecture compliance — no subprocess in CLI, no hardcoded URLs
|
||||||
|
2. Best practices — no `print()`, no bare `except`, no `TODO`/`FIXME`,
|
||||||
|
no functions > 50 lines
|
||||||
|
3. Security — no hardcoded secrets, no `shell=True`, no `eval`/`exec`
|
||||||
|
4. i18n — no raw strings in `click.echo()` without `_()` wrapper
|
||||||
|
5. Resource management — no `open()` without `with`, no `Popen()` without
|
||||||
|
cleanup
|
||||||
|
6. Documentation — source changes must include doc updates
|
||||||
|
7. Test coverage — source changes must include test updates
|
||||||
|
8. Commit conventions — conventional commit format on PR commits
|
||||||
|
|
||||||
|
**Pre-merge validation step**
|
||||||
|
|
||||||
|
Runs on every pull request. Executes
|
||||||
|
`python -m devx.ci.check_auto_merge_ready` with the branch name, PR title,
|
||||||
|
repository, and PR number. Validates auto-merge preconditions before the
|
||||||
|
`auto-merge` job runs:
|
||||||
|
|
||||||
|
1. **Branch name** — must contain a valid task ID (for example,
|
||||||
|
`DEVX-12-fix-foo` → `DEVX-12`)
|
||||||
|
2. **PR title format** — must be `{PREFIX}-N: <vikunja task title>`
|
||||||
|
3. **Vikunja task** — must exist and the title must match the PR title
|
||||||
|
4. **Branch state** — must not be behind master
|
||||||
|
|
||||||
|
#### `auto-merge`
|
||||||
|
|
||||||
|
Depends on `validate`. The final job in the CI workflow. Runs
|
||||||
|
`python -m devx.ci.auto_merge` with the branch name, PR title, repository,
|
||||||
|
and PR number:
|
||||||
|
|
||||||
|
1. **Read task ID** from branch name (for example, `DEVX-12-fix-foo` → `DEVX-12`)
|
||||||
|
2. **Validate PR title format** — must be `{PREFIX}-N: <vikunja task title>`
|
||||||
|
3. **Validate PR title matches Vikunja task** — fetches the Vikunja task and
|
||||||
|
compares the title
|
||||||
|
4. **Extract conventional commit message** from PR commits (newest matching
|
||||||
|
conventional format)
|
||||||
|
5. **Squash-merge** with title `{PREFIX}-N <conventional commit message>`
|
||||||
|
6. If the head branch is behind master (HTTP 405), automatically pulls master,
|
||||||
|
rebases, force-pushes, and retries the merge
|
||||||
|
|
||||||
|
The merge commit push to master triggers the post-merge workflow.
|
||||||
|
|
||||||
|
### Smart CI: user-facing vs workflow-only changes
|
||||||
|
|
||||||
|
Not all changes require a new release. The `detect-changes` step in the
|
||||||
|
`validate` job classifies changes using
|
||||||
|
`python -m devx.ci.classify_changes`:
|
||||||
|
|
||||||
|
**Workflow-only paths** (infrastructure — no release needed):
|
||||||
|
- `.gitea/**` — Gitea Actions workflows
|
||||||
|
- `tests/**` — Test files
|
||||||
|
- `AGENTS.md`, `README.md`, `CHANGELOG.md` — Project docs
|
||||||
|
- `Makefile`, `cliff.toml`, `.pre-commit-config.yaml` — Config
|
||||||
|
- `.env.example`, `.gitignore` — Config
|
||||||
|
- `hooks/**` — Git hooks
|
||||||
|
- `src/devx/__init__.py` — Only contains `__version__` (release artifact)
|
||||||
|
|
||||||
|
**User-facing paths** (tool changes — release needed) — everything else:
|
||||||
|
- `src/devx/**` — Python package source (except `__init__.py`)
|
||||||
|
- `pyproject.toml` — Package metadata
|
||||||
|
- Any new file type not in the allowlist
|
||||||
|
|
||||||
|
Classification is configured in `pyproject.toml` under
|
||||||
|
`[tool.devx.classify]`. The framework provides `DEFAULT_INFRASTRUCTURE` — a
|
||||||
|
curated list of paths that are infrastructure for any Python project. Projects
|
||||||
|
inherit these automatically and only specify what is different.
|
||||||
|
|
||||||
|
Rule priority (first match wins):
|
||||||
|
1. `user_facing_overrides` — safety override (highest priority)
|
||||||
|
2. `infrastructure_overrides` — explicit per-file
|
||||||
|
3. `infrastructure` — DEFAULT_INFRASTRUCTURE + project-specific patterns
|
||||||
|
4. Default: user-facing (safe — any unknown file triggers release)
|
||||||
|
|
||||||
|
## Post-merge workflow (`post-merge.yml`)
|
||||||
|
|
||||||
|
Runs on every push to master. Consolidated into 2 jobs (from 7) to reduce
|
||||||
|
runner overhead: `detect-and-configure` (detect-type + validate-commit-msg +
|
||||||
|
configure-repo) and `release-and-maintain` (release + publish + sync-wiki +
|
||||||
|
badges + vikunja). Individual steps within `release-and-maintain` are
|
||||||
|
conditional on the `detect-and-configure` job's outputs.
|
||||||
|
|
||||||
|
### Job dependency graph
|
||||||
|
|
||||||
|
```text
|
||||||
|
detect-and-configure
|
||||||
|
├── configure-repo (independent, skip if release commit)
|
||||||
|
├── detect-type → is-release? is-automated?
|
||||||
|
└── validate-commit-msg (skip if release commit)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
release-and-maintain (needs detect-and-configure)
|
||||||
|
├── release (skip if release commit or workflow-only)
|
||||||
|
│ └── publish (if release created a tag)
|
||||||
|
├── sync-wiki (skip if automated)
|
||||||
|
├── vikunja (skip if automated)
|
||||||
|
└── badges (always runs)
|
||||||
|
```
|
||||||
|
|
||||||
|
`sync-wiki` and `vikunja` run only on non-automated commits (that is, real PR
|
||||||
|
merges) so that the wiki and task tracker are only updated when a human
|
||||||
|
change lands. They skip on release commits and automated commits.
|
||||||
|
|
||||||
|
The `badges` step always runs (even on release commits) so badges (tests,
|
||||||
|
coverage, version, etc.) are always current. It runs last so it picks up
|
||||||
|
any version bump the release step created.
|
||||||
|
|
||||||
|
When `release` creates a `release: vX.Y.Z` commit, the release commit's
|
||||||
|
post-merge run still updates badges (the version badge picks up the new
|
||||||
|
version). Other steps skip. The `publish` step builds and publishes the
|
||||||
|
package to the Gitea PyPI registry within the same `release-and-maintain`
|
||||||
|
job (it checks out the release tag).
|
||||||
|
|
||||||
|
### Post-merge jobs
|
||||||
|
|
||||||
|
#### `detect-and-configure`
|
||||||
|
|
||||||
|
The first post-merge job. Consolidates the former `detect-type`,
|
||||||
|
`validate-commit-msg`, and `configure-repo` jobs. Outputs `is-release`,
|
||||||
|
`is-automated`, and `user-facing-changed` for the `release-and-maintain`
|
||||||
|
job.
|
||||||
|
|
||||||
|
**`detect-type` step**
|
||||||
|
|
||||||
|
Checks if the latest commit is a release commit (`release: vX.Y.Z [skip ci]`)
|
||||||
|
using `python -m devx.ci.detect_release_commit`. Writes `is-release=true` or
|
||||||
|
`is-release=false` (and `is-automated`) to the job output. The
|
||||||
|
`release-and-maintain` job uses these to conditionally skip steps for
|
||||||
|
release commits.
|
||||||
|
|
||||||
|
**`validate-commit-msg` step**
|
||||||
|
|
||||||
|
Skips for release/automated commits. Validates the latest commit message
|
||||||
|
using `python -m devx.ci.validate_commit_msg --branch master`. On master,
|
||||||
|
commits must follow `{PREFIX}-N: <conventional commit>` format (added by
|
||||||
|
auto-merge).
|
||||||
|
|
||||||
|
**`configure-repo` step**
|
||||||
|
|
||||||
|
Ensures branch protection and labels are configured using
|
||||||
|
`python -m devx.tools.configure_repo --repo <name> --owner <owner>`:
|
||||||
|
|
||||||
|
- Sets up master branch protection (required status checks, block on rejected
|
||||||
|
reviews, block on outdated branch)
|
||||||
|
- Creates standard labels
|
||||||
|
- Status check contexts read from `DEVX_STATUS_CHECKS` or default to
|
||||||
|
`CI / validate (pull_request)`
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue.
|
||||||
|
|
||||||
|
#### `release-and-maintain`
|
||||||
|
|
||||||
|
Depends on `detect-and-configure`. The second post-merge job. Consolidates
|
||||||
|
the former `release`, `publish`, `sync-wiki`, `badges`, and `vikunja` jobs.
|
||||||
|
Individual steps are conditional on the `detect-and-configure` job's outputs.
|
||||||
|
|
||||||
|
**`release` step**
|
||||||
|
|
||||||
|
Skips for release commits and workflow-only changes. The core release
|
||||||
|
automation step. Runs `python -m devx.ci.release`:
|
||||||
|
|
||||||
|
1. **Classify changes** — calls `classify_changes.py` to check for user-facing
|
||||||
|
changes. If only infrastructure files changed, exits without releasing.
|
||||||
|
2. **Calculate next version** — uses git-cliff to determine the next semver
|
||||||
|
version from conventional commits since the last tag
|
||||||
|
3. **Update version file** — updates `__version__` in `src/devx/__init__.py`
|
||||||
|
4. **Update changelog** — prepends the new version section to `CHANGELOG.md`
|
||||||
|
using git-cliff output
|
||||||
|
5. **Run tests** — executes `make lint-ruff` and `make pytest-cov` to verify
|
||||||
|
the release is healthy. If either fails, the release is aborted — no
|
||||||
|
commit, no tag. Use `--skip-tests` only for emergency releases.
|
||||||
|
6. **Commit** — stages the version file and changelog, commits with
|
||||||
|
`release: vX.Y.Z [skip ci]` (uses `--no-verify` to bypass the commit-msg
|
||||||
|
hook since release commits are a special case)
|
||||||
|
7. **Create tag** — creates an annotated tag `vX.Y.Z` with the changelog as
|
||||||
|
the tag message
|
||||||
|
8. **Push** — pushes both the commit and tag to master
|
||||||
|
|
||||||
|
The script is idempotent: if there are no new conventional commits since the
|
||||||
|
last tag, it exits without doing anything. If the tag already exists (for example,
|
||||||
|
from a partial previous run), it skips tag creation and only pushes.
|
||||||
|
|
||||||
|
**Tag consistency**: Before releasing, the script fetches remote tags and
|
||||||
|
verifies all existing tags point to commits whose message matches the tag
|
||||||
|
version. This prevents duplicate release commits and ensures
|
||||||
|
tag/version/commit alignment.
|
||||||
|
|
||||||
|
**Version bumping rules** (git-cliff):
|
||||||
|
|
||||||
|
| Commit type | Version bump |
|
||||||
|
|-------------|-------------|
|
||||||
|
| `feat:` | minor (0.X.0) |
|
||||||
|
| `fix:` | patch (0.0.X) |
|
||||||
|
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0) |
|
||||||
|
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue via
|
||||||
|
`python -m devx.ci.notify_failure`.
|
||||||
|
|
||||||
|
**`sync-wiki` step**
|
||||||
|
|
||||||
|
Skips for automated commits. Syncs documentation from `docs/` to the Gitea
|
||||||
|
wiki using `python -m devx.ci.sync_wiki --repo <owner/repo> --strict`:
|
||||||
|
|
||||||
|
1. Reads `docs/mapping.json` to map file paths to wiki page titles
|
||||||
|
2. Lists existing wiki pages via the Gitea API
|
||||||
|
3. For each mapped file, reads content and creates or updates the wiki page
|
||||||
|
4. `--strict` runs a full integrity check: verifies page count, missing
|
||||||
|
pages, stale pages, and content match. Fails if any page is empty or
|
||||||
|
content doesn't match.
|
||||||
|
|
||||||
|
Pages that exist in the wiki but not in the mapping are left untouched (not
|
||||||
|
deleted).
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue.
|
||||||
|
|
||||||
|
**`badges` step**
|
||||||
|
|
||||||
|
Always runs (even on release commits). Generates and pushes quality badges
|
||||||
|
using `python -m devx.ci.push_badges`:
|
||||||
|
|
||||||
|
1. **Fetch latest master** — `git fetch origin master && git reset --hard
|
||||||
|
origin/master` (ensures the version badge reflects the current state,
|
||||||
|
even if the release step recently pushed a new version)
|
||||||
|
2. **Generate badges** — calls `devx.tools.generate_badges` which runs
|
||||||
|
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
|
||||||
|
SVG files: `coverage.svg`, `tests.svg`, `docs.svg`, `quality.svg`,
|
||||||
|
`version.svg`, `python.svg`
|
||||||
|
3. **Push to badges branch** — creates an orphan `badges` branch, copies SVG
|
||||||
|
files, commits, and force-pushes
|
||||||
|
4. **Update README/docs** — switches back to master, replaces
|
||||||
|
`raw/branch/badges/<name>.svg` URLs with `raw/commit/<sha>/<name>.svg`
|
||||||
|
URLs (cache-busting — Gitea caches `raw/branch/` URLs for 6 hours),
|
||||||
|
commits, and pushes
|
||||||
|
|
||||||
|
Supports `--retries` for retrying on git push failures (fetches latest master
|
||||||
|
and waits 10s between attempts).
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue.
|
||||||
|
|
||||||
|
**`vikunja` step**
|
||||||
|
|
||||||
|
Skips for automated commits. Updates the Vikunja task after a merge using
|
||||||
|
`python -m devx.ci.post_merge --git-sha <sha>`:
|
||||||
|
|
||||||
|
1. Extracts the task ID from the first line of the commit message
|
||||||
|
2. Marks the corresponding Vikunja task as done
|
||||||
|
3. Posts a comment with the merge SHA
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue.
|
||||||
|
|
||||||
|
**`publish` step**
|
||||||
|
|
||||||
|
Only runs if the `release` step created a tag. Builds and publishes the
|
||||||
|
package within the same `release-and-maintain` job (checks out the release
|
||||||
|
tag). Runs `python -m devx.ci.publish <tag> <owner/repo>`:
|
||||||
|
|
||||||
|
1. **Install dependencies** — build, twine, requests, python-dotenv, click,
|
||||||
|
and the project itself
|
||||||
|
2. **Install CI tools** — git-cliff and tea via
|
||||||
|
`python -m devx.tools.install_tools`
|
||||||
|
3. **Configure tea login** — `tea login add` using `CI_GITEA_TOKEN`
|
||||||
|
4. **Build and publish** — `python -m devx.ci.publish <tag> <owner/repo>`:
|
||||||
|
- Build the package with `python -m build`
|
||||||
|
- Publish to the Gitea PyPI registry (default) using `twine upload
|
||||||
|
--repository-url <url> -u <token> -p <token>`
|
||||||
|
- OR publish to standard PyPI if `PYPI_TOKEN` is set
|
||||||
|
- OR skip publishing if `--skip-build` is passed (non-Python repos)
|
||||||
|
- Create a Gitea release with git-cliff-generated release notes via
|
||||||
|
`tea create release`
|
||||||
|
|
||||||
|
Publishing destination resolution (checked in order):
|
||||||
|
1. **Gitea PyPI registry** — if `--registry-url` is given, or
|
||||||
|
`DEVX_PYPI_REGISTRY_URL` env var is set, or derived from `GITEA_API_URL`
|
||||||
|
2. **Standard PyPI** — if `PYPI_TOKEN` is set (takes precedence over Gitea
|
||||||
|
registry)
|
||||||
|
3. **Skip** — if neither is configured, only the Gitea release is created
|
||||||
|
|
||||||
|
On failure, the `notify_failure` step creates a Gitea issue.
|
||||||
|
|
||||||
|
## CI scripts
|
||||||
|
|
||||||
|
### `auto_merge.py`
|
||||||
|
|
||||||
|
Auto-merge PR when all CI checks pass. Reads task ID from the branch name
|
||||||
|
(for example, `DEVX-12-fix-foo` → `DEVX-12`). Validates PR title format, checks the
|
||||||
|
Vikunja task exists and the title matches, extracts the conventional commit
|
||||||
|
message from PR commits, and squash-merges with
|
||||||
|
`{PREFIX}-N <conventional commit>` title.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.auto_merge <branch> <pr_title> <owner/repo> <pr_number>
|
||||||
|
```
|
||||||
|
|
||||||
|
### `release.py`
|
||||||
|
|
||||||
|
Automated release using git-cliff. Calculates next semver version from
|
||||||
|
conventional commits, updates `__version__` and `CHANGELOG.md`, runs lint and
|
||||||
|
tests, commits with `release: vX.Y.Z [skip ci]`, creates annotated tag, and
|
||||||
|
pushes. Idempotent — exits if no unreleased changes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.release [--dry-run] [--skip-tests] [--verify]
|
||||||
|
```
|
||||||
|
|
||||||
|
- `--dry-run` — preview without making changes
|
||||||
|
- `--skip-tests` — skip lint and test verification (emergency only)
|
||||||
|
- `--verify` — check tag/version/changelog alignment and exit
|
||||||
|
|
||||||
|
### `publish.py`
|
||||||
|
|
||||||
|
Builds package, publishes to Gitea PyPI registry or standard PyPI, and
|
||||||
|
creates a Gitea release with git-cliff-generated notes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.publish <tag> <owner/repo> [--registry-url <url>] [--skip-build]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `pr_review.py`
|
||||||
|
|
||||||
|
Automated PR review. Fetches the PR diff via the Gitea API, runs automated
|
||||||
|
checks (architecture, best practices, security, i18n, resource management,
|
||||||
|
documentation, test coverage, commit conventions), and posts a structured
|
||||||
|
review with inline comments.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.pr_review <pr_number> <owner/repo>
|
||||||
|
```
|
||||||
|
|
||||||
|
### `notify_failure.py`
|
||||||
|
|
||||||
|
Creates a Gitea issue when a CI workflow fails. Uses the tea CLI for issue
|
||||||
|
creation with failure labels. Supports `--auto-login` to configure the tea
|
||||||
|
CLI login profile from `CI_GITEA_TOKEN`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.notify_failure --repo <owner/repo> --run-id <id> \
|
||||||
|
--workflow <name> --commit <sha> [--auto-login]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `post_merge.py`
|
||||||
|
|
||||||
|
Updates Vikunja task after a merge to master. Extracts task ID from the
|
||||||
|
commit message, marks the task as done, and posts a comment with the merge SHA.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.post_merge <commit_msg> [--commit-sha <sha>] [--git-sha <sha>]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `classify_changes.py`
|
||||||
|
|
||||||
|
Classifies git changes as user-facing or workflow-only. Uses a layered rule
|
||||||
|
system configured in `pyproject.toml`. Safe-by-default: any unknown file
|
||||||
|
defaults to user-facing.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.classify_changes [--base <ref>] [--head <ref>] \
|
||||||
|
[--quiet] [--check <category>] [--github-output]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `discover_runners.py`
|
||||||
|
|
||||||
|
Discovers available Gitea Actions runners at repository, organization, and
|
||||||
|
instance levels. Falls back to `MOLECULE_RUNNERS` repo variable or
|
||||||
|
`DEFAULT_MAX_RUNNERS` (3).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.discover_runners --owner <owner> --repo <repo> [--count] [--indices]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `detect_release_commit.py`
|
||||||
|
|
||||||
|
Detects whether the latest git commit is a release commit. Writes
|
||||||
|
`is-release=true|false` to `$GITHUB_OUTPUT`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.detect_release_commit
|
||||||
|
```
|
||||||
|
|
||||||
|
### `push_badges.py`
|
||||||
|
|
||||||
|
Generates SVG badge files, pushes them to the `badges` branch, and updates
|
||||||
|
README.md and docs/index.md with cache-busting `raw/commit/<sha>/` URLs.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.push_badges [--output-dir <dir>] [--branch <branch>] \
|
||||||
|
[--no-readme-update] [--retries <n>]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `distribute_molecule.py`
|
||||||
|
|
||||||
|
Distributes molecule (scenario, platform) pairs across N parallel runners.
|
||||||
|
Discovers scenarios under `ansible/roles/*/molecule/`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.molecule.distribute_molecule --runner-index <i> --max-runners <n>
|
||||||
|
python -m devx.molecule.distribute_molecule --list
|
||||||
|
python -m devx.molecule.distribute_molecule --list-platforms
|
||||||
|
```
|
||||||
|
|
||||||
|
### `molecule_ci_guard.py`
|
||||||
|
|
||||||
|
Runs molecule tests sequentially while polling the Gitea API for other runner
|
||||||
|
failures. Aborts early if another runner fails the same job.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.molecule.molecule_ci_guard [--roles-root <dir>] pair1 pair2 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### `validate_commit_msg.py`
|
||||||
|
|
||||||
|
Validates commit messages. On feature branches: conventional commits only
|
||||||
|
(no `{PREFIX}-N` prefix). On master: must have `{PREFIX}-N` prefix from
|
||||||
|
auto-merge, followed by a conventional commit message.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.validate_commit_msg <commit_msg_file> [--branch <branch>]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `sync_wiki.py`
|
||||||
|
|
||||||
|
Syncs documentation from `docs/` to the Gitea wiki via the API. Reads
|
||||||
|
`docs/mapping.json` for file-to-page mapping. Supports `--dry-run`,
|
||||||
|
`--verify`, and `--strict` (full integrity check).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.sync_wiki [--dry-run] [--repo <owner/repo>] [--verify] [--strict]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `check_translations.py`
|
||||||
|
|
||||||
|
Validates translation files against the Python source code. Checks for
|
||||||
|
missing keys, dead keys, and missing languages.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.check_translations [--translations <file>]...
|
||||||
|
```
|
||||||
|
|
||||||
|
### `doc_coverage.py`
|
||||||
|
|
||||||
|
Checks documentation coverage for CLI commands and major modules. Parses
|
||||||
|
Click commands from `cli.py` and verifies documentation exists.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.doc_coverage [--docs-dir <dir>] [--fail-on-missing]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `distribute_files.py`
|
||||||
|
|
||||||
|
Distributes files matching a glob pattern across N parallel runners
|
||||||
|
(round-robin). Writes the assigned file list to `$GITHUB_ENV`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.distribute_files --pattern <glob> --runner-index <i> \
|
||||||
|
--max-runners <n> [--github-env] [--skip-if-excess]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `integration_guard.py`
|
||||||
|
|
||||||
|
Runs pytest with cross-runner failure detection. If any other
|
||||||
|
integration-tests matrix runner reports failure, the current pytest
|
||||||
|
subprocess is killed and this runner exits early.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m devx.ci.integration_guard -- <pytest args>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Release process summary
|
||||||
|
|
||||||
|
The complete release process from PR to published package:
|
||||||
|
|
||||||
|
1. **PR merged** — `auto-merge` squash-merges the PR to master with
|
||||||
|
`{PREFIX}-N <conventional commit>` title
|
||||||
|
2. **Post-merge triggers** — the merge push triggers `post-merge.yml`
|
||||||
|
3. **detect-and-configure** — detects release commit, validates commit
|
||||||
|
message, and ensures branch protection/labels
|
||||||
|
4. **release** (step in `release-and-maintain`) — `release.py` calculates
|
||||||
|
the next version, updates files, runs tests, commits
|
||||||
|
`release: vX.Y.Z [skip ci]`, creates tag `vX.Y.Z`, and pushes to master
|
||||||
|
5. **publish** (step in `release-and-maintain`) — `publish.py` builds the
|
||||||
|
package, publishes to the Gitea PyPI registry, and creates a Gitea
|
||||||
|
release with git-cliff notes (checks out the release tag within the
|
||||||
|
same job)
|
||||||
|
6. **sync-wiki** (step in `release-and-maintain`) — documentation is synced
|
||||||
|
to the Gitea wiki
|
||||||
|
7. **vikunja** (step in `release-and-maintain`) — the corresponding Vikunja
|
||||||
|
task is marked as done
|
||||||
|
8. **badges** (step in `release-and-maintain`) — quality badges are
|
||||||
|
regenerated and pushed to the `badges` branch; README and docs/index.md
|
||||||
|
are updated with cache-busting URLs
|
||||||
|
|
||||||
|
The release commit's post-merge run skips all steps except `badges` (which
|
||||||
|
picks up the new version number). This prevents infinite loops.
|
||||||
|
|
||||||
|
## Failure handling
|
||||||
|
|
||||||
|
Every job in the CI and post-merge workflows has a `notify_failure` step
|
||||||
|
that runs `if: failure()`. This creates a Gitea issue with the workflow name,
|
||||||
|
run ID, and commit SHA, ensuring failures that would otherwise go unnoticed
|
||||||
|
in the Actions tab are surfaced as issues. The issue is created via the tea
|
||||||
|
CLI with a `bug` label if available.
|
||||||
|
|||||||
+559
-1
@@ -1 +1,559 @@
|
|||||||
# CLI-Commands
|
# CLI Commands
|
||||||
|
|
||||||
|
devx provides a CLI with three command groups: `ci`, `tools`, and `molecule`.
|
||||||
|
Each subcommand delegates to the corresponding Python module via
|
||||||
|
`python -m devx.*`, so `devx ci release` is equivalent to
|
||||||
|
`python -m devx.ci.release`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx --help # show all command groups
|
||||||
|
devx --version # show package version
|
||||||
|
devx ci --help # show CI commands
|
||||||
|
devx tools --help # show tools commands
|
||||||
|
devx molecule --help # show molecule commands
|
||||||
|
```
|
||||||
|
|
||||||
|
## CI Commands
|
||||||
|
|
||||||
|
### `devx ci auto-merge`
|
||||||
|
|
||||||
|
Auto-merge a PR when all CI checks pass. Reads the task ID from the branch
|
||||||
|
name, validates the PR title format against
|
||||||
|
the Vikunja task title, extracts the conventional commit message from PR
|
||||||
|
commits, and squash-merges with `{PREFIX}-N <conventional commit>` title.
|
||||||
|
|
||||||
|
If the head branch is behind master (HTTP 405), automatically pulls master,
|
||||||
|
rebases, force-pushes, and retries the merge.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci auto-merge <branch> <pr_title> <owner/repo> <pr_number>
|
||||||
|
# Example:
|
||||||
|
devx ci auto-merge DEVX-12-add-feature "DEVX-12: Add feature" oblachno-oss/devx 42
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx ci check-translations`
|
||||||
|
|
||||||
|
Check translation files for gaps, dead keys, and missing languages. Validates
|
||||||
|
translation files against the Python source code that uses them. By default,
|
||||||
|
checks `src/devx/translations.json` against `src/devx/**/*.py`.
|
||||||
|
|
||||||
|
Checks performed:
|
||||||
|
- **Missing keys** — a `_()` call in code has no entry in the translations file
|
||||||
|
- **Dead keys** — a key in the translations file is not used in any code
|
||||||
|
- **Missing languages** — a key exists but is missing one of the six
|
||||||
|
supported languages (en, bg, de, ru, zh, pl)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci check-translations
|
||||||
|
devx ci check-translations --translations path/to/translations.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx ci classify-changes`
|
||||||
|
|
||||||
|
Classify git changes as user-facing or workflow-only. Used to skip releases
|
||||||
|
for infrastructure-only changes. Classification rules are configured in
|
||||||
|
`pyproject.toml` under `[tool.devx.classify]`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci classify-changes --base origin/master --head HEAD
|
||||||
|
devx ci classify-changes --base origin/master --head HEAD --github-output
|
||||||
|
devx ci classify-changes --quiet --check user-facing
|
||||||
|
devx ci classify-changes --check ansible # custom tag from pyproject.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--base <ref>` — base ref (default: latest tag)
|
||||||
|
- `--head <ref>` — head ref (default: HEAD)
|
||||||
|
- `--quiet` — only output true/false
|
||||||
|
- `--check <category>` — check specific category: `all` (default),
|
||||||
|
`user-facing`, or any tag name defined in `[tool.devx.classify.tags]`
|
||||||
|
- `--github-output` — write results to `$GITHUB_OUTPUT` for CI workflow steps
|
||||||
|
|
||||||
|
Exit code 2 indicates workflow-only changes (no release needed).
|
||||||
|
|
||||||
|
### `devx ci detect-release-commit`
|
||||||
|
|
||||||
|
Detect whether the latest git commit is a release commit
|
||||||
|
(`release: vX.Y.Z [skip ci]`). Writes `is-release=true` or `is-release=false`
|
||||||
|
to `$GITHUB_OUTPUT` for use in CI workflow conditionals.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci detect-release-commit
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx ci discover-runners`
|
||||||
|
|
||||||
|
Discover available Gitea Actions runners for dynamic job distribution.
|
||||||
|
Queries the Gitea API for registered runners at repository, organization, and
|
||||||
|
instance (admin) levels. Falls back to `MOLECULE_RUNNERS` repo variable or
|
||||||
|
`DEFAULT_MAX_RUNNERS` (3).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci discover-runners --owner oblachno-oss --repo devx
|
||||||
|
devx ci discover-runners --owner oblachno-oss --repo devx --count
|
||||||
|
devx ci discover-runners --owner oblachno-oss --repo devx --indices
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--count` — print the number of available runners
|
||||||
|
- `--indices` — print a JSON array `[0, 1, ..., N-1]` for use as a dynamic
|
||||||
|
matrix in Gitea Actions
|
||||||
|
|
||||||
|
### `devx ci distribute-files`
|
||||||
|
|
||||||
|
Distribute files across parallel runners (round-robin). Discovers files
|
||||||
|
matching a glob pattern, sorts them for deterministic ordering, then assigns
|
||||||
|
them round-robin to `max_runners` groups. The assigned group for
|
||||||
|
`runner_index` is written to `$GITHUB_ENV`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci distribute-files --pattern "tests/integration/test_*.py" \
|
||||||
|
--runner-index 1 --max-runners 3 --github-env
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--pattern <glob>` — glob pattern for files to distribute
|
||||||
|
- `--runner-index <i>` — current runner index (0-based)
|
||||||
|
- `--max-runners <n>` — total number of runners (default: 3)
|
||||||
|
- `--github-env` — write file list to `$GITHUB_ENV`
|
||||||
|
- `--skip-if-excess` — skip if fewer files than runners
|
||||||
|
|
||||||
|
### `devx ci doc-coverage`
|
||||||
|
|
||||||
|
Check documentation coverage for CLI commands and major modules. Parses
|
||||||
|
Click commands from `cli.py` and checks if each has documentation in
|
||||||
|
`docs/user/cli-commands.md`. Verifies core modules are documented in
|
||||||
|
`architecture.md` and CI scripts in `ci-cd-workflow.md`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci doc-coverage
|
||||||
|
devx ci doc-coverage --docs-dir docs/ --source-dir src/ --fail-on-missing
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--docs-dir <dir>` — path to the docs directory (default: `docs/`)
|
||||||
|
- `--source-dir <dir>` — path to the source directory (default: auto-detect)
|
||||||
|
- `--fail-on-missing` — exit with non-zero status if any documentation is
|
||||||
|
missing
|
||||||
|
|
||||||
|
### `devx ci lint-docs`
|
||||||
|
|
||||||
|
Lint documentation files for structure, broken links, heading hierarchy,
|
||||||
|
duplicate headings, TODO/FIXME markers, and trailing whitespace.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci lint-docs
|
||||||
|
devx ci lint-docs --root . --fix
|
||||||
|
devx ci lint-docs --no-check-links --no-check-stale
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--root <dir>` — repository root directory (default: `.`)
|
||||||
|
- `--docs-dir <dir>` — docs directory (default: `<root>/docs`)
|
||||||
|
- `--check-links/--no-check-links` — check internal links (default: yes)
|
||||||
|
- `--check-headings/--no-check-headings` — check heading hierarchy (default: yes)
|
||||||
|
- `--check-todo/--no-check-todo` — check for TODO/FIXME markers (default: yes)
|
||||||
|
- `--check-stale/--no-check-stale` — check for stale docs (default: no)
|
||||||
|
- `--check-trailing/--no-check-trailing` — check trailing whitespace (default: yes)
|
||||||
|
- `--check-duplicates/--no-check-duplicates` — check duplicate headings (default: yes)
|
||||||
|
- `--fix` — auto-fix trailing whitespace
|
||||||
|
|
||||||
|
### `devx ci integration-guard`
|
||||||
|
|
||||||
|
Run pytest with cross-runner failure detection. If any
|
||||||
|
other integration-tests matrix runner reports failure, the current pytest
|
||||||
|
subprocess is killed and this runner exits early with code 1.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci integration-guard -- test_a.py test_b.py
|
||||||
|
devx ci integration-guard -- -x -v --tb=short test_a.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Environment variables:
|
||||||
|
- `GITEA_URL` — base URL of the Gitea instance
|
||||||
|
- `CI_GITEA_TOKEN` — API token with repo access
|
||||||
|
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
|
||||||
|
- `JOB_NAME` — base job name (`GITHUB_JOB`)
|
||||||
|
- `MATRIX_INDEX` — current matrix index (runner-index)
|
||||||
|
- `GITEA_REPOSITORY` — repository in `owner/repo` format
|
||||||
|
|
||||||
|
### `devx ci notify-failure`
|
||||||
|
|
||||||
|
Create a Gitea issue when a CI workflow fails. Uses the tea CLI for issue
|
||||||
|
creation with a `bug` label if available.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci notify-failure --repo oblachno-oss/devx --run-id 123 \
|
||||||
|
--workflow ci --commit abc123def456
|
||||||
|
devx ci notify-failure --repo oblachno-oss/devx --run-id 123 \
|
||||||
|
--workflow post-merge/release --commit abc123def456 --auto-login
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--repo <owner/repo>` — repository (required)
|
||||||
|
- `--run-id <id>` — CI run ID (required)
|
||||||
|
- `--workflow <name>` — workflow name (required)
|
||||||
|
- `--commit <sha>` — commit SHA (required)
|
||||||
|
- `--auto-login` — configure tea CLI login from `CI_GITEA_TOKEN` before creating
|
||||||
|
the issue
|
||||||
|
|
||||||
|
### `devx ci post-merge`
|
||||||
|
|
||||||
|
Update Vikunja task after a merge to master. Extracts the task ID from the
|
||||||
|
commit message, marks the task as done, and posts a comment with the merge SHA.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci post-merge "DEVX-12 feat: add feature" --git-sha abc123def456
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx ci pr-review`
|
||||||
|
|
||||||
|
Run automated PR review. Fetches the PR diff via the Gitea API and runs a
|
||||||
|
series of checks, posting a structured review (`COMMENT` or
|
||||||
|
`REQUEST_CHANGES`).
|
||||||
|
|
||||||
|
Checks: architecture compliance, best practices, security, i18n, resource
|
||||||
|
management, documentation, test coverage, and commit conventions.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci pr-review 42 oblachno-oss/devx
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx ci publish`
|
||||||
|
|
||||||
|
Build package, publish to Gitea PyPI registry (or standard PyPI), and create
|
||||||
|
a Gitea release with git-cliff-generated notes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci publish v1.0.0 oblachno-oss/devx
|
||||||
|
devx ci publish v1.0.0 oblachno-oss/devx --registry-url https://git.example.com/api/packages/owner/pypi
|
||||||
|
devx ci publish v1.0.0 oblachno-oss/devx --skip-build # Gitea release only
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--registry-url <url>` — Gitea PyPI registry URL. Defaults to
|
||||||
|
`DEVX_PYPI_REGISTRY_URL` env var or a URL derived from `GITEA_API_URL`.
|
||||||
|
When set, publishes to Gitea PyPI instead of standard PyPI (unless
|
||||||
|
`PYPI_TOKEN` is also set).
|
||||||
|
- `--skip-build` — skip package build and PyPI publish (for non-Python repos
|
||||||
|
that only need a Gitea release)
|
||||||
|
|
||||||
|
### `devx ci push-badges`
|
||||||
|
|
||||||
|
Generate badge SVG files and push them to the `badges` branch. Also updates
|
||||||
|
`README.md` and `docs/index.md` on master with cache-busting
|
||||||
|
`raw/commit/<sha>/` URLs.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci push-badges
|
||||||
|
devx ci push-badges --output-dir .badges/ --branch master
|
||||||
|
devx ci push-badges --no-readme-update # skip README update (local testing)
|
||||||
|
devx ci push-badges --retries 3 # retry on git push failures
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--output-dir <dir>` — temporary directory for badge files (default:
|
||||||
|
`.badges/`)
|
||||||
|
- `--branch <branch>` — branch to sync before generating badges (default:
|
||||||
|
`master`)
|
||||||
|
- `--no-readme-update` — skip updating README with cache-busting URLs
|
||||||
|
- `--retries <n>` — number of attempts on git push failures (default: 1).
|
||||||
|
Between attempts, fetches latest master and waits 10s.
|
||||||
|
|
||||||
|
### `devx ci release`
|
||||||
|
|
||||||
|
Automated release: calculate next version, update files, tag, and push. Uses
|
||||||
|
git-cliff to determine the next semver version from conventional commits.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci release
|
||||||
|
devx ci release --dry-run # preview without making changes
|
||||||
|
devx ci release --skip-tests # skip lint and tests (emergency only)
|
||||||
|
devx ci release --verify # check tag/version/changelog alignment
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--dry-run` — show what would happen without making changes
|
||||||
|
- `--skip-tests` — skip lint and test verification (NOT recommended — only
|
||||||
|
for emergency releases)
|
||||||
|
- `--verify` — verify tag/version/changelog alignment and exit (no changes
|
||||||
|
made)
|
||||||
|
|
||||||
|
### `devx ci sync-wiki`
|
||||||
|
|
||||||
|
Sync documentation from `docs/` to the Gitea wiki. Reads `docs/mapping.json`
|
||||||
|
for file-to-page mapping. Pages that exist in the wiki but not in the mapping
|
||||||
|
are left untouched.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci sync-wiki --repo oblachno-oss/devx
|
||||||
|
devx ci sync-wiki --repo oblachno-oss/devx --dry-run
|
||||||
|
devx ci sync-wiki --repo oblachno-oss/devx --verify
|
||||||
|
devx ci sync-wiki --repo oblachno-oss/devx --strict
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--dry-run` — show what would happen without making changes
|
||||||
|
- `--repo <owner/repo>` — repository (auto-detected if omitted)
|
||||||
|
- `--verify` — after syncing, verify each page has non-empty content. Exit 1
|
||||||
|
if any page is empty or mismatched.
|
||||||
|
- `--strict` — full integrity check: verify page count, missing pages, stale
|
||||||
|
pages, and content. Implies `--verify`.
|
||||||
|
|
||||||
|
### `devx ci validate-commit-msg`
|
||||||
|
|
||||||
|
Validate commit messages for conventional commit format. On feature branches:
|
||||||
|
conventional commits only (no `{PREFIX}-N` prefix). On master: must have
|
||||||
|
`{PREFIX}-N` prefix from auto-merge, followed by a conventional commit
|
||||||
|
message.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx ci validate-commit-msg commit-msg.txt
|
||||||
|
devx ci validate-commit-msg commit-msg.txt --branch master
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--branch <branch>` — override branch detection (for CI use)
|
||||||
|
|
||||||
|
## Tools Commands
|
||||||
|
|
||||||
|
### `devx tools check-test-speed`
|
||||||
|
|
||||||
|
Run unit tests and enforce execution-time budgets. Two quality gates:
|
||||||
|
|
||||||
|
- **Total suite time** must not exceed `--max-seconds` (default: 10s)
|
||||||
|
- **Per-test time** — no individual test may exceed `--max-single-seconds`
|
||||||
|
(default: 0.5s, 0 to disable)
|
||||||
|
|
||||||
|
Runs `make test-unit` with `PYTEST_ADDOPTS=--durations=0` so pytest emits
|
||||||
|
per-test timing lines.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools check-test-speed
|
||||||
|
devx tools check-test-speed --max-seconds 10
|
||||||
|
devx tools check-test-speed --max-seconds 4 --max-single-seconds 0.5
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx tools check-test-isolation`
|
||||||
|
|
||||||
|
Statically analyze test files for un-hermetic patterns that cause slow
|
||||||
|
or flaky tests. Also available as a **pytest plugin** (auto-discovered
|
||||||
|
via the `pytest11` entry point when devx is installed — runs
|
||||||
|
automatically on every `pytest` invocation and **fails on violations**).
|
||||||
|
|
||||||
|
Detected patterns (hard errors — exit non-zero):
|
||||||
|
|
||||||
|
- **unpatched-subprocess**: `subprocess.run/call/Popen/check_call/check_output`
|
||||||
|
called in a test function without `@patch` or `with patch(...)`
|
||||||
|
- **unpatched-sleep**: `time.sleep` called without `@patch`
|
||||||
|
- **unpatched-helper**: known subprocess-spawning helpers (`update_doc_versions`,
|
||||||
|
`run_cmd`, `run_tests`) called without `@patch` or patching their internal deps
|
||||||
|
- **excessive-iterations**: `for _ in range(N)` where N > 100
|
||||||
|
- **heavy-module-import**: `httpx`, `ansible`, etc. imported at module level
|
||||||
|
- **reload-without-cleanup**: `importlib.reload()` called an odd number of times
|
||||||
|
|
||||||
|
Advisory patterns (exit 0 — runtime audit is authoritative):
|
||||||
|
|
||||||
|
- **transitive-subprocess**: `CliRunner.invoke(target)` where `target`
|
||||||
|
transitively calls `subprocess.run` without being patched. Detected via
|
||||||
|
static call-graph analysis. The runtime subprocess audit catches actual
|
||||||
|
leaks — if a real subprocess runs without `@patch`, the test fails.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools check-test-isolation
|
||||||
|
devx tools check-test-isolation --test-path tests/
|
||||||
|
devx tools check-test-isolation --categories unpatched-subprocess,transitive-subprocess
|
||||||
|
devx tools check-test-isolation --max-loop-iterations 50
|
||||||
|
devx tools check-test-isolation --src-dir src/
|
||||||
|
```
|
||||||
|
|
||||||
|
Pytest plugin options (automatic when devx is installed):
|
||||||
|
|
||||||
|
- `--no-test-isolation` — disable static analysis and runtime subprocess audit
|
||||||
|
- `--test-isolation-max-loop N` — max iterations per loop (default: 100)
|
||||||
|
|
||||||
|
### `devx tools configure-repo`
|
||||||
|
|
||||||
|
Configure repository: branch protection and labels via the Gitea REST API.
|
||||||
|
Sets up master branch protection (required status checks, block on rejected
|
||||||
|
reviews, block on outdated branch) and creates standard labels.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools configure-repo --repo devx --owner oblachno-oss
|
||||||
|
```
|
||||||
|
|
||||||
|
Status check contexts are read from `DEVX_STATUS_CHECKS` (comma-separated) or
|
||||||
|
default to `CI / quality (pull_request)`.
|
||||||
|
|
||||||
|
### `devx tools generate-badges`
|
||||||
|
|
||||||
|
Generate self-contained SVG badge files from project metrics. Runs
|
||||||
|
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
|
||||||
|
SVG files that can be served as static files from the Gitea raw file API.
|
||||||
|
|
||||||
|
Badges generated: `coverage.svg`, `tests.svg`, `docs.svg`, `quality.svg`,
|
||||||
|
`version.svg`, `python.svg`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools generate-badges
|
||||||
|
devx tools generate-badges --output-dir .badges/
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx tools generate-cliff-config`
|
||||||
|
|
||||||
|
Generate a `cliff.toml` configuration file with the correct task ID prefix
|
||||||
|
preprocessor. Eliminates the need to manually duplicate and maintain
|
||||||
|
`cliff.toml` across repos that use devx.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools generate-cliff-config --prefix GRM
|
||||||
|
devx tools generate-cliff-config --prefix GRM --output cliff.toml
|
||||||
|
devx tools generate-cliff-config --prefix GRM --force # overwrite existing
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--prefix <prefix>` — task ID prefix (default: `DEVX_TASK_PREFIX` env var
|
||||||
|
or `DEVX`)
|
||||||
|
- `--output <file>` — output file path (default: `cliff.toml`)
|
||||||
|
- `--force` — overwrite existing file
|
||||||
|
|
||||||
|
### `devx tools install-checkmake`
|
||||||
|
|
||||||
|
Install checkmake (Makefile linter) if not already present. Tries
|
||||||
|
`go install` first if Go is available, otherwise downloads the latest
|
||||||
|
pre-built Linux binary from the official GitHub releases.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools install-checkmake
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx tools install-tools`
|
||||||
|
|
||||||
|
Install CI/CD development tools that are not Python packages: actionlint,
|
||||||
|
git-cliff, act_runner, and tea. Each tool is installed to `~/.local/bin` if
|
||||||
|
not already on PATH. Idempotent: skips tools that are already available.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools install-tools # install all
|
||||||
|
devx tools install-tools --tool actionlint # install one
|
||||||
|
devx tools install-tools --tool git-cliff --tool tea # install specific
|
||||||
|
devx tools install-tools --list # list status
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx tools setup`
|
||||||
|
|
||||||
|
Project setup: install Python dependencies (editable mode with extras),
|
||||||
|
Ansible Galaxy collections (if `ansible/requirements.yml` exists in the target repo), pre-commit
|
||||||
|
hooks (pre-commit, commit-msg, pre-push), and configure the tea CLI login
|
||||||
|
profile from `.env`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools setup --bin .venv/bin
|
||||||
|
devx tools setup --bin .venv/bin --extras "ci,lint"
|
||||||
|
devx tools setup --bin .venv/bin --no-pre-commit --no-tea-login
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--bin <dir>` — virtualenv bin directory (required)
|
||||||
|
- `--extras <groups>` — pip extras to install (default: `dev`)
|
||||||
|
- `--no-pre-commit` — skip pre-commit hook installation
|
||||||
|
- `--no-tea-login` — skip tea CLI login configuration
|
||||||
|
|
||||||
|
### `devx tools rebase`
|
||||||
|
|
||||||
|
Rebase the current branch onto `origin/master` and force-push with
|
||||||
|
`--force-with-lease`. Checks if the branch is behind master first —
|
||||||
|
if up-to-date, exits without doing anything.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools rebase # rebase + force-push
|
||||||
|
devx tools rebase -- --no-push # rebase locally only
|
||||||
|
```
|
||||||
|
|
||||||
|
Options (pass after `--`):
|
||||||
|
- `--no-push` — rebase locally without pushing
|
||||||
|
|
||||||
|
### `devx tools pr-rebase`
|
||||||
|
|
||||||
|
Rebase a pull request's head branch onto master via the Gitea API
|
||||||
|
(server-side). This triggers a new `pull_request synchronize` event,
|
||||||
|
which starts a new CI run. Useful when you don't have the branch
|
||||||
|
checked out locally.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools pr-rebase -- --pr 42 # rebase PR #42
|
||||||
|
devx tools pr-rebase # auto-detect PR from current branch
|
||||||
|
```
|
||||||
|
|
||||||
|
Options (pass after `--`):
|
||||||
|
- `--pr <N>` — PR number (auto-detected from current branch if omitted)
|
||||||
|
|
||||||
|
## Molecule Commands
|
||||||
|
|
||||||
|
Molecule commands require the `molecule` extra (`pip install devx[molecule]`).
|
||||||
|
|
||||||
|
### `devx molecule all`
|
||||||
|
|
||||||
|
Run all molecule scenarios on all supported OS platforms. Sequential
|
||||||
|
execution — CI uses the parallel matrix instead.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx molecule all
|
||||||
|
devx molecule all --bin .venv/bin
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx molecule discover-runners`
|
||||||
|
|
||||||
|
Discover available Gitea Actions runners for molecule tests. Same logic as
|
||||||
|
`devx ci discover-runners` but intended for molecule-specific workflows.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx molecule discover-runners --owner oblachno-oss --repo devx --indices
|
||||||
|
```
|
||||||
|
|
||||||
|
### `devx molecule distribute`
|
||||||
|
|
||||||
|
Distribute molecule (scenario, platform) pairs across N parallel runners.
|
||||||
|
Discovers scenarios under `ansible/roles/*/molecule/` and crosses them with
|
||||||
|
the supported OS platform matrix.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx molecule distribute --runner-index 1 --max-runners 3
|
||||||
|
devx molecule distribute --list # list all scenarios
|
||||||
|
devx molecule distribute --list-platforms # list platforms
|
||||||
|
devx molecule distribute --roles-root ansible/roles # multi-role repos
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--runner-index <i>` — current runner index (0-based)
|
||||||
|
- `--max-runners <n>` — total number of runners (default: 3)
|
||||||
|
- `--list` — list all scenarios, one per line
|
||||||
|
- `--list-platforms` — list all platforms, one per line
|
||||||
|
- `--roles-root <dir>` — roles root directory for multi-role repos (default:
|
||||||
|
`ansible/roles`)
|
||||||
|
|
||||||
|
### `devx molecule guard`
|
||||||
|
|
||||||
|
Run molecule tests sequentially with CI failure polling. A background thread
|
||||||
|
polls the Gitea API. If any other molecule matrix runner reports failure, the
|
||||||
|
current molecule subprocess is killed and this runner exits early with code 1.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx molecule guard pair1 pair2 pair3
|
||||||
|
devx molecule guard --roles-root ansible/roles pair1 pair2
|
||||||
|
```
|
||||||
|
|
||||||
|
Each pair is encoded as:
|
||||||
|
- **Single-role (4-part):** `scenario|platform_name|platform_image|platform_command`
|
||||||
|
- **Multi-role (5-part):** `role|scenario|platform_name|platform_image|platform_command`
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--roles-root <dir>` — roles root directory for multi-role repos
|
||||||
|
|
||||||
|
Environment variables:
|
||||||
|
- `GITEA_URL` — base URL of the Gitea instance
|
||||||
|
- `CI_GITEA_TOKEN` — API token with repo access
|
||||||
|
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
|
||||||
|
- `JOB_NAME` — base job name (`GITHUB_JOB`)
|
||||||
|
- `MATRIX_INDEX` — current matrix index (runner-index)
|
||||||
|
- `GITEA_REPOSITORY` — repository in `owner/repo` format
|
||||||
|
|||||||
+161
-1
@@ -1 +1,161 @@
|
|||||||
# Getting-Started
|
# Getting Started with devx
|
||||||
|
|
||||||
|
This guide walks you through installing devx, configuring it for your project,
|
||||||
|
and setting up a complete CI/CD pipeline.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- **Python 3.12+**
|
||||||
|
- **A Gitea instance** with Actions enabled
|
||||||
|
- **A Gitea API token** with repo, workflow, and organization scopes
|
||||||
|
- **(Optional) Vikunja API token** for task tracking integration
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
devx is published to the Gitea PyPI registry. Configure pip to use it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Configure Gitea PyPI registry
|
||||||
|
pip config set global.extra-index-url https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
|
||||||
|
|
||||||
|
# Install devx
|
||||||
|
pip install devx
|
||||||
|
```
|
||||||
|
|
||||||
|
Or install from source:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://git.oblachno.oblachno.fyi/oblachno-oss/devx.git
|
||||||
|
cd devx
|
||||||
|
make setup
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### 1. Configure environment variables
|
||||||
|
|
||||||
|
Create a `.env` file in your project root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CI_GITEA_TOKEN=your_gitea_api_token
|
||||||
|
VIKUNJA_TOKEN=your_vikunja_api_token # optional
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Add devx to your project
|
||||||
|
|
||||||
|
Add devx to your `pyproject.toml`:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[project]
|
||||||
|
dependencies = [
|
||||||
|
"devx>=0.47.5",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = [
|
||||||
|
"devx>=0.47.5",
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Set up the Makefile
|
||||||
|
|
||||||
|
devx provides a shared Makefile fragment. Add this to your `Makefile`:
|
||||||
|
|
||||||
|
```makefile
|
||||||
|
include devx.mak
|
||||||
|
```
|
||||||
|
|
||||||
|
Run `devx tools setup` to install all development tools (actionlint, git-cliff,
|
||||||
|
tea CLI, etc.) and configure pre-commit hooks.
|
||||||
|
|
||||||
|
### 4. Create the docs structure
|
||||||
|
|
||||||
|
devx expects a `docs/` directory with at minimum:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/
|
||||||
|
├── index.md # Documentation home page
|
||||||
|
├── mapping.json # Wiki page title mappings
|
||||||
|
├── user/ # User-facing documentation
|
||||||
|
│ └── cli-commands.md
|
||||||
|
└── tech/ # Technical documentation
|
||||||
|
├── architecture.md
|
||||||
|
└── ci-cd-workflow.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Example `docs/mapping.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"index.md": "Home",
|
||||||
|
"user/cli-commands.md": "CLI-Commands",
|
||||||
|
"tech/architecture.md": "Architecture",
|
||||||
|
"tech/ci-cd-workflow.md": "CI-CD-Workflow"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Set up CI workflows
|
||||||
|
|
||||||
|
Create `.gitea/workflows/ci.yml` and `.gitea/workflows/post-merge.yml` in your
|
||||||
|
project. See the [CI/CD Workflow guide](ci-cd-workflow) for details.
|
||||||
|
|
||||||
|
### 6. Configure release settings
|
||||||
|
|
||||||
|
Add a `cliff.toml` for git-cliff-based versioning:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devx tools generate-cliff-config
|
||||||
|
```
|
||||||
|
|
||||||
|
Add `[tool.devx]` section to `pyproject.toml` for project-specific config:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[tool.devx]
|
||||||
|
# Vikunja project ID for task tracking
|
||||||
|
vikunja_project_id = 6
|
||||||
|
|
||||||
|
[tool.devx.classify]
|
||||||
|
# File patterns that are infrastructure (no release needed)
|
||||||
|
infrastructure = [
|
||||||
|
".gitea/**",
|
||||||
|
"docs/**",
|
||||||
|
"tests/**",
|
||||||
|
"AGENTS.md",
|
||||||
|
"README.md",
|
||||||
|
"CHANGELOG.md",
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Available Tools
|
||||||
|
|
||||||
|
### CI/CD Automation (`devx.ci.*`)
|
||||||
|
|
||||||
|
- `devx.ci.release` — Automated semver versioning and tagging
|
||||||
|
- `devx.ci.publish` — Package publishing to Gitea PyPI registry
|
||||||
|
- `devx.ci.auto_merge` — Squash-merge automation with task ID validation
|
||||||
|
- `devx.ci.pr_review` — Automated PR review with inline comments
|
||||||
|
- `devx.ci.classify_changes` — User-facing vs workflow-only change detection
|
||||||
|
- `devx.ci.sync_wiki` — Push docs/ to Gitea wiki
|
||||||
|
- `devx.ci.doc_coverage` — Documentation coverage checker
|
||||||
|
- `devx.ci.lint_docs` — Documentation linter (structure, links, headings)
|
||||||
|
- `devx.ci.check_translations` — i18n translation completeness checker
|
||||||
|
- `devx.ci.notify_failure` — Create Gitea issues on CI failures
|
||||||
|
- `devx.ci.distribute_files` — Parallel test file distribution
|
||||||
|
- `devx.ci.distribute_items` — Parallel item distribution across runners
|
||||||
|
- `devx.ci.discover_runners` — Dynamic runner discovery via Gitea API
|
||||||
|
|
||||||
|
### Development Tools (`devx.tools.*`)
|
||||||
|
|
||||||
|
- `devx.tools.setup` — Environment setup (venv, deps, hooks, tools)
|
||||||
|
- `devx.tools.install_tools` — Install CI/CD tools (actionlint, git-cliff, tea)
|
||||||
|
- `devx.tools.create_task` — Create Vikunja tasks
|
||||||
|
- `devx.tools.create_pr` — Create Gitea PRs with task ID in title
|
||||||
|
- `devx.tools.configure_repo` — Configure branch protection and labels
|
||||||
|
- `devx.tools.generate_badges` — Generate quality badge SVGs
|
||||||
|
- `devx.tools.check_test_speed` — Enforce test execution speed limits
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- Read the [CLI Commands reference](cli-commands) for all available commands
|
||||||
|
- Read the [Architecture guide](architecture) to understand internals
|
||||||
|
- Read the [CI/CD Workflow guide](ci-cd-workflow) for pipeline details
|
||||||
|
|||||||
+175
@@ -0,0 +1,175 @@
|
|||||||
|
# devx — Reusable Development & CI/CD Tools
|
||||||
|
|
||||||
|
A Python package providing reusable development and CI/CD automation tools for
|
||||||
|
oblachno-oss projects. devx consolidates release management, PR automation,
|
||||||
|
wiki sync, badge generation, translation checks, documentation coverage,
|
||||||
|
parallel test distribution, and more into a single installable package.
|
||||||
|
|
||||||
|
It was extracted from the [GRM](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
|
||||||
|
project to be reusable across all oblachno-oss repositories.
|
||||||
|
|
||||||
|
> An open-source project from **Oblachno** (облачно means *cloudy* in Bulgarian).
|
||||||
|
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||||
|
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||||
|
[](https://www.python.org/downloads/)
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
devx provides a complete, opinionated CI/CD pipeline for any project hosted on
|
||||||
|
a Gitea instance with Gitea Actions. Install the package, declare configuration
|
||||||
|
via environment variables and `pyproject.toml`, and inherit:
|
||||||
|
|
||||||
|
- **Automated releases** — git-cliff-driven semver versioning, changelog
|
||||||
|
generation, tagging, and publishing to a Gitea PyPI registry.
|
||||||
|
- **PR automation** — squash-merge with task ID validation, automated PR
|
||||||
|
review with inline comments, and conventional commit enforcement.
|
||||||
|
- **Smart change classification** — user-facing vs workflow-only change
|
||||||
|
detection so infrastructure-only changes skip releases.
|
||||||
|
- **Documentation sync** — push `docs/` markdown to the Gitea wiki with
|
||||||
|
integrity verification.
|
||||||
|
- **Quality badges** — self-contained SVG badges for coverage, tests, docs,
|
||||||
|
quality, version, and Python version.
|
||||||
|
- **Translation checks** — validate i18n keys against source code, detect
|
||||||
|
dead keys and missing languages.
|
||||||
|
- **Parallel test distribution** — split test files or molecule scenarios
|
||||||
|
across CI runners with cross-runner fail-fast.
|
||||||
|
- **Developer tools** — environment setup, CI tool installation, test speed
|
||||||
|
enforcement, repository configuration.
|
||||||
|
- **i18n** — built-in translations for English, Bulgarian, German, Russian,
|
||||||
|
Chinese, and Polish; projects can extend with their own keys.
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
devx is published to the Gitea PyPI registry at
|
||||||
|
`https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple`.
|
||||||
|
The registry is publicly readable — no authentication required to install.
|
||||||
|
|
||||||
|
### Quick install (one-off)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install devx --index-url https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
|
||||||
|
```
|
||||||
|
|
||||||
|
### Persistent configuration (recommended)
|
||||||
|
|
||||||
|
Add the registry to `~/.pip/pip.conf`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[global]
|
||||||
|
extra-index-url = https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
|
||||||
|
```
|
||||||
|
|
||||||
|
Then `pip install devx` works without specifying `--index-url`.
|
||||||
|
|
||||||
|
### As a dependency in another project
|
||||||
|
|
||||||
|
Add devx to your `pyproject.toml` dependencies and configure the registry:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[project]
|
||||||
|
dependencies = [
|
||||||
|
"devx>=0.47.5",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tool.pip]
|
||||||
|
extra-index-url = "https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple"
|
||||||
|
```
|
||||||
|
|
||||||
|
Pin a specific version if needed: `"devx==0.47.5"` or `"devx>=0.47.5,<0.48"`.
|
||||||
|
|
||||||
|
### Optional extras
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install "devx[ci,lint]" # CI runners and linting (pytest, ruff, pyright, bandit, build, twine)
|
||||||
|
pip install "devx[molecule]" # Molecule testing for Ansible projects
|
||||||
|
pip install "devx[dev]" # Full local development (ci + lint + build + twine)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
devx is a self-contained Python package under `src/devx/`:
|
||||||
|
|
||||||
|
- **Core modules** — `config.py`, `exceptions.py`, `i18n.py`, `api_clients.py`,
|
||||||
|
`gitea_cli.py`, `cli.py`, `opentofu.py`
|
||||||
|
- **CI automation** (`devx.ci`) — release, publish, auto_merge, pr_review,
|
||||||
|
classify_changes, sync_wiki, push_badges, check_translations, doc_coverage,
|
||||||
|
validate_commit_msg, detect_release_commit, notify_failure, post_merge,
|
||||||
|
discover_runners, distribute_files, integration_guard
|
||||||
|
- **Dev tools** (`devx.tools`) — setup, install_tools, check_test_speed,
|
||||||
|
configure_repo, generate_badges, generate_cliff_config, install_checkmake
|
||||||
|
- **Molecule tools** (`devx.molecule`) — Optional, for projects with Ansible
|
||||||
|
roles: distribute_molecule, molecule_ci_guard, molecule_all, discover_runners,
|
||||||
|
start_docker, platforms
|
||||||
|
|
||||||
|
See [Architecture](Architecture) for the full package structure, module
|
||||||
|
descriptions, design principles, and data flow diagrams.
|
||||||
|
|
||||||
|
## CI/CD pipeline
|
||||||
|
|
||||||
|
devx uses Gitea Actions with three workflows:
|
||||||
|
|
||||||
|
- **CI** (`ci.yml`) — runs on pull requests: quality checks, change detection,
|
||||||
|
release dry-run, automated PR review, and auto-merge.
|
||||||
|
- **Post-merge** (`post-merge.yml`) — runs on every push to master: release
|
||||||
|
versioning, wiki sync, badge generation, Vikunja task updates, and repo
|
||||||
|
configuration.
|
||||||
|
- **Publish** (`publish.yml`) — runs on tag pushes: builds the package,
|
||||||
|
publishes to the Gitea PyPI registry, and creates a Gitea release.
|
||||||
|
|
||||||
|
See [CI/CD Workflow](CI-CD-Workflow) for the full pipeline documentation,
|
||||||
|
including the post-merge job graph, release process, badge generation, and
|
||||||
|
wiki sync details.
|
||||||
|
|
||||||
|
## CLI commands
|
||||||
|
|
||||||
|
devx provides a `devx` CLI with three command groups:
|
||||||
|
|
||||||
|
- `devx ci <command>` — CI/CD automation (17 commands)
|
||||||
|
- `devx tools <command>` — Developer tools (9 commands)
|
||||||
|
- `devx molecule <command>` — Molecule testing (4 commands, optional)
|
||||||
|
|
||||||
|
See [CLI Commands](CLI-Commands) for full command documentation with examples.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
devx reads configuration from `DEVX_*` environment variables with `.env` file
|
||||||
|
fallback. Key variables:
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
|
||||||
|
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
|
||||||
|
| `DEVX_REPO_OWNER` | **(must be set)** | Repository owner |
|
||||||
|
| `DEVX_REPO_NAME` | **(must be set)** | Repository name |
|
||||||
|
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
|
||||||
|
| `DEVX_LANG` | `en` | Language for i18n (en, bg, de, ru, zh, pl) |
|
||||||
|
| `CI_GITEA_TOKEN` | — | Gitea API token |
|
||||||
|
| `VIKUNJA_TOKEN` | — | Vikunja API token |
|
||||||
|
|
||||||
|
See [AGENTS.md](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/AGENTS.md)
|
||||||
|
for the full configuration reference, PR workflow, and project conventions.
|
||||||
|
|
||||||
|
## Wiki pages
|
||||||
|
|
||||||
|
- [Home](Home) — This page
|
||||||
|
- [Getting Started](Getting-Started) — Installation, configuration, and quick start guide
|
||||||
|
- [CLI Commands](CLI-Commands) — Full CLI command documentation with examples
|
||||||
|
- [Architecture](Architecture) — Package structure, module descriptions, design principles
|
||||||
|
- [CI/CD Workflow](CI-CD-Workflow) — Pipeline documentation, workflows, and CI scripts
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- **Source**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx](https://git.oblachno.oblachno.fyi/oblachno-oss/devx)
|
||||||
|
- **Releases**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||||
|
- **Actions**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||||
|
- **GRM (origin project)**: [https://git.oblachno.oblachno.fyi/oblachno-oss/grm](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
GPL-3.0
|
||||||
|
|||||||
Reference in New Issue
Block a user