Public Access
DEVX-155: refactor: extract wait_for_checks, consolidate ansible_checks, deprecate ci/discover_runners
This commit is contained in:
@@ -0,0 +1,108 @@
|
||||
# ADR-0002: Ansible Check Tool Consolidation and wait_for_checks Extraction
|
||||
|
||||
Date: 2026-08-12
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The devx package had two categories of code duplication and inline
|
||||
workflow logic that were hard to test and maintain:
|
||||
|
||||
### 1. Ansible Check Tools — Duplicated Boilerplate
|
||||
|
||||
Five Ansible check tools (`check_ansible_no_log`,
|
||||
`check_ansible_patterns`, `check_ansible_set_fact_to_json`,
|
||||
`check_ansible_no_state_absent_on_db`, `check_jinja_expr`) each
|
||||
implemented their own file discovery, YAML parsing, task iteration, and
|
||||
violation reporting logic. While the check logic differed, the
|
||||
supporting infrastructure was copy-pasted across all five modules:
|
||||
|
||||
- `find_task_files()` — glob YAML files, skip molecule
|
||||
- YAML multi-document parsing with error handling
|
||||
- Task iteration (bare lists, play dicts with `tasks`/`pre_tasks`/`post_tasks`/`handlers`, nested `block` tasks)
|
||||
- Violation formatting (`path:line — message`)
|
||||
|
||||
This made it difficult to add new checks (each new tool repeated the
|
||||
boilerplate) and risky to change shared behavior (fixes had to be
|
||||
applied to all five modules independently).
|
||||
|
||||
### 2. Inline Job Polling in Workflow YAML
|
||||
|
||||
The `grm` repository's `ci.yml` workflow contained ~25 lines of inline
|
||||
shell + Python polling logic to wait for the `molecule-tests` job to
|
||||
complete before the auto-merge step. This logic:
|
||||
|
||||
- Was not testable (embedded in workflow YAML)
|
||||
- Duplicated the Gitea API client pattern already used elsewhere
|
||||
- Had no timeout handling, no error reporting, no retry logic
|
||||
- Could not be reused by other repositories
|
||||
|
||||
### 3. Duplicate discover_runners Modules
|
||||
|
||||
`devx.ci.discover_runners` and `devx.molecule.discover_runners` were
|
||||
near-identical modules. The `ci/` version had better error logging
|
||||
(warnings on non-200 responses, 403 suppression for instance-level
|
||||
queries), while the `molecule/` version silently swallowed errors.
|
||||
Both were imported by different workflows, making it unclear which was
|
||||
canonical.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Composable `ansible_checks/` Subpackage
|
||||
|
||||
Consolidate the five Ansible check tools into a
|
||||
`devx.tools.ansible_checks/` subpackage with shared utilities:
|
||||
|
||||
- `_shared.py` — `AnsibleFileFinder`, `AnsibleYAMLParser`,
|
||||
`ViolationReporter` classes providing composable helpers
|
||||
- `no_log.py`, `patterns.py`, `set_fact_to_json.py`,
|
||||
`no_state_absent_on_db.py`, `jinja_expr.py` — canonical check
|
||||
implementations using the shared utilities
|
||||
|
||||
The old modules (`check_ansible_*.py`, `check_jinja_expr.py`) remain as
|
||||
**thin backward-compat wrappers** that re-export the canonical
|
||||
implementation and preserve the CLI entry point. This avoids breaking
|
||||
existing Makefile targets and workflow references.
|
||||
|
||||
**Composition over inheritance**: each check module picks the helpers it
|
||||
needs. Tools that don't parse YAML (for example line-based scanners) can skip
|
||||
`AnsibleYAMLParser` entirely.
|
||||
|
||||
### 2. Extracted `wait_for_checks` Module
|
||||
|
||||
Extract the inline polling logic into `devx.ci.wait_for_checks`:
|
||||
|
||||
- Polls the Gitea API for job completion status
|
||||
- Configurable job name prefix, timeout, poll interval
|
||||
- Exit codes: 0 (success), 1 (failure), 2 (timeout), 3 (API error)
|
||||
- `--require-success/--no-require-success` flag for flexibility
|
||||
- 100% test coverage with mocked API responses
|
||||
|
||||
This replaces the inline shell polling in `grm` `ci.yml` with a
|
||||
reusable, testable Python module.
|
||||
|
||||
### 3. Deprecated `ci/discover_runners` Wrapper
|
||||
|
||||
Merge the `ci/discover_runners` implementation (with its better error
|
||||
logging) into `molecule/discover_runners` as the canonical version.
|
||||
Make `ci/discover_runners` a deprecated wrapper that:
|
||||
|
||||
- Re-exports all public symbols from `molecule.discover_runners`
|
||||
- Emits a `DeprecationWarning` when run as `__main__`
|
||||
- Preserves backward compatibility for existing workflow references
|
||||
|
||||
New code should import from `devx.molecule.discover_runners` directly.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **New checks are easier to write**: import `_shared` helpers, implement
|
||||
only the check-specific logic
|
||||
- **Shared behavior can be fixed in one place**: file discovery, YAML
|
||||
parsing, violation formatting
|
||||
- **Workflow polling is testable**: `wait_for_checks` has 26 unit tests
|
||||
covering success, failure, timeout, and API error scenarios
|
||||
- **Backward compatibility preserved**: all existing Makefile targets,
|
||||
workflow references, and test imports continue to work via wrappers
|
||||
- **Migration path is gradual**: new code uses the subpackage; old code
|
||||
can migrate at its own pace; wrappers can be removed in a future
|
||||
release once all references are updated
|
||||
+3
-1
@@ -3,5 +3,7 @@
|
||||
"user/getting-started.md": "Getting-Started",
|
||||
"user/cli-commands.md": "CLI-Commands",
|
||||
"tech/architecture.md": "Architecture",
|
||||
"tech/ci-cd-workflow.md": "CI-CD-Workflow"
|
||||
"tech/ci-cd-workflow.md": "CI-CD-Workflow",
|
||||
"decisions/0001-test-isolation-pytest-plugin-and-shift-left-quality-gates.md": "ADR-0001-Test-Isolation",
|
||||
"decisions/0002-ansible-check-consolidation-and-wait-for-checks.md": "ADR-0002-Ansible-Check-Consolidation"
|
||||
}
|
||||
|
||||
@@ -33,7 +33,8 @@ src/devx/
|
||||
│ ├── notify_failure.py # Create Gitea issues on CI failures
|
||||
│ ├── distribute_files.py # Distribute files across parallel runners
|
||||
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
|
||||
│ ├── discover_runners.py # Dynamic Gitea runner discovery
|
||||
│ ├── discover_runners.py # Deprecated wrapper → molecule/discover_runners
|
||||
│ ├── wait_for_checks.py # Poll Gitea Actions for job completion
|
||||
│ ├── check_translations.py # Translation completeness check
|
||||
│ └── doc_coverage.py # Documentation coverage check
|
||||
├── tools/ # Developer tooling modules (run locally or by CI)
|
||||
@@ -48,7 +49,7 @@ src/devx/
|
||||
│ └── install_checkmake.py # Install checkmake (Makefile linter)
|
||||
└── molecule/ # Optional molecule testing helpers (Ansible projects)
|
||||
├── __init__.py
|
||||
├── discover_runners.py # Dynamic Gitea runner discovery
|
||||
├── discover_runners.py # Dynamic Gitea runner discovery (canonical)
|
||||
├── distribute_molecule.py # Distribute scenarios across runners
|
||||
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
|
||||
├── molecule_all.py # Run all molecule scenarios locally
|
||||
@@ -285,13 +286,24 @@ Click commands from `cli.py` and verifies each has documentation in
|
||||
`architecture.md` and CI scripts in `ci-cd-workflow.md`. Supports
|
||||
`--fail-on-missing` to enforce 100% coverage.
|
||||
|
||||
### `discover_runners.py`
|
||||
### `discover_runners.py` (deprecated wrapper)
|
||||
|
||||
> **Deprecated:** Use `devx.molecule.discover_runners` instead. This
|
||||
> module is a thin wrapper that re-exports the canonical implementation.
|
||||
|
||||
Discovers available Gitea Actions runners at three levels: repository,
|
||||
organization, and instance (administrator). Falls back to the `MOLECULE_RUNNERS` repo
|
||||
variable or `DEFAULT_MAX_RUNNERS` (3). Outputs runner count or a JSON index
|
||||
array for use as a dynamic matrix in Gitea Actions.
|
||||
|
||||
### `wait_for_checks.py`
|
||||
|
||||
Polls the Gitea Actions API for job completion status. Used by auto-merge
|
||||
jobs that need to wait for parallel jobs (for example molecule-tests) before
|
||||
proceeding. Replaces inline shell polling in workflow YAML with a
|
||||
reusable, testable Python module. Exit codes: 0 (success), 1 (job
|
||||
failure), 2 (timeout), 3 (API error or no matching jobs).
|
||||
|
||||
### `distribute_files.py`
|
||||
|
||||
Distributes files matching a glob pattern across N parallel runners
|
||||
@@ -396,8 +408,13 @@ Intended for local development; CI uses the parallel matrix instead.
|
||||
|
||||
### `molecule/discover_runners.py`
|
||||
|
||||
Discovers available Gitea Actions runners for molecule tests. Same logic as
|
||||
`devx.ci.discover_runners` but intended for molecule-specific workflows.
|
||||
Discovers available Gitea Actions runners for molecule tests. This is the
|
||||
canonical implementation; `devx.ci.discover_runners` is a deprecated wrapper
|
||||
that re-exports from this module. Queries runners at repository,
|
||||
organization, and instance (administrator) levels, with warnings logged
|
||||
to stderr on non-200 responses (except 403 on instance-level, which is
|
||||
expected without admin scope). Falls back to `MOLECULE_RUNNERS` env var
|
||||
or `DEFAULT_MAX_RUNNERS` (3).
|
||||
|
||||
### `start_docker.py`
|
||||
|
||||
|
||||
@@ -83,6 +83,11 @@ devx ci detect-release-commit
|
||||
|
||||
### `devx ci discover-runners`
|
||||
|
||||
> **Deprecated:** Use `devx molecule discover-runners` instead. This
|
||||
> command is a thin wrapper that re-exports the canonical implementation
|
||||
> from `devx.molecule.discover_runners`. It will be removed in a future
|
||||
> release.
|
||||
|
||||
Discover available Gitea Actions runners for dynamic job distribution.
|
||||
Queries the Gitea API for registered runners at repository, organization, and
|
||||
instance (administrator) levels. Falls back to `MOLECULE_RUNNERS` repo variable or
|
||||
@@ -315,6 +320,32 @@ devx ci validate-commit-msg commit-msg.txt --branch master
|
||||
Options:
|
||||
- `--branch <branch>` — override branch detection (for CI use)
|
||||
|
||||
### `devx ci wait-for-checks`
|
||||
|
||||
Wait for Gitea Actions jobs to complete by polling the API. Used by
|
||||
auto-merge jobs that need to wait for parallel jobs (for example molecule-tests)
|
||||
before proceeding. Replaces inline shell polling in workflow YAML with
|
||||
a reusable, testable Python module.
|
||||
|
||||
Exit codes:
|
||||
- `0` — all matching jobs completed successfully
|
||||
- `1` — one or more matching jobs failed (when `--require-success` is set)
|
||||
- `2` — timeout reached before all jobs completed
|
||||
- `3` — API error or no matching jobs found
|
||||
|
||||
```bash
|
||||
devx ci wait-for-checks --job-name molecule-tests --repo oblachno-oss/grm
|
||||
devx ci wait-for-checks --job-name molecule-tests --timeout 1200 --poll-interval 10
|
||||
devx ci wait-for-checks --job-name molecule-tests --no-require-success
|
||||
```
|
||||
|
||||
Options:
|
||||
- `--job-name <prefix>` — job name prefix to match (required)
|
||||
- `--repo <owner/name>` — repository (default: `$GITHUB_REPOSITORY`)
|
||||
- `--timeout <seconds>` — max wait time (default: 1200 = 20 min)
|
||||
- `--poll-interval <seconds>` — seconds between polls (default: 10)
|
||||
- `--require-success / --no-require-success` — exit 1 if a job failed (default: yes)
|
||||
|
||||
### `devx ci cancel-superseded-runs`
|
||||
|
||||
Cancel in-flight CI runs for the same PR branch when a new push triggers
|
||||
|
||||
Reference in New Issue
Block a user