Compare commits

..
Author SHA1 Message Date
devx-ci-bot 5f08e23e09 release: v0.51.0 [skip ci] 2026-08-25 16:38:30 +00:00
emo 9fa41457f2 DEVX-157: feat: add role defaults path to create_dependency_pr search
Post-merge / detect-and-configure (push) Successful in 14s
Post-merge / release-and-maintain (push) Successful in 1m20s
Co-authored-by: emo <emo@oblachno.com>
2026-08-25 16:37:45 +00:00
gitea-actions-bot f62fe16c1b chore: update badge URLs to commit c35a676f [skip ci] 2026-08-24 22:23:22 +00:00
devx-ci-bot f90eeeb550 release: v0.50.2 [skip ci] 2026-08-24 22:22:33 +00:00
emil f9462bc939 DEVX-156: fix: update check_pr_size usage example with --repo and --pr-number args
Post-merge / release-and-maintain (push) Successful in 1m17s
Post-merge / detect-and-configure (push) Successful in 11s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-24 22:21:49 +00:00
gitea-actions-bot 7eb11e3261 chore: update badge URLs to commit 3e50818d [skip ci] 2026-08-24 21:53:46 +00:00
emil 0ac2bf4a8c DEVX-156: docs: add consumer repo reference to validate_spec docstring
Post-merge / detect-and-configure (push) Successful in 31s
Post-merge / release-and-maintain (push) Successful in 53s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-24 21:52:18 +00:00
gitea-actions-bot d0e3f3918b chore: update badge URLs to commit 041c1730 [skip ci] 2026-08-24 21:48:07 +00:00
emil 2704ec45b5 DEVX-156: docs: expand package docstring with CI module overview
Post-merge / release-and-maintain (push) Successful in 1m0s
Post-merge / detect-and-configure (push) Successful in 31s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-24 21:42:10 +00:00
emil 11c4a1fc9e DEVX-155: Replace pr_review with spec-driven CI gates and pr-review skill
Post-merge / detect-and-configure (push) Failing after 18s
Post-merge / release-and-maintain (push) Skipped
2026-08-24 20:39:09 +00:00
devx-ci-bot a06caa0e88 release: v0.50.1 [skip ci] 2026-08-15 10:06:01 +00:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 1dc27d6e4b DEVX-155: fix: trust /var/run/docker.sock with free=0 when no inner dockerd exists
Post-merge / detect-and-configure (push) Successful in 1m38s
Post-merge / release-and-maintain (push) Failing after 2m20s
The host's rootless Docker socket is mounted as /var/run/docker.sock
inside CI containers. Its data root is on the host filesystem (not
accessible from inside the container), so _get_docker_free_bytes
returns 0. Previously the code didn't trust this and started a local
dockerd on /dev/shm (too small). Now checks pgrep for dockerd processes
— if none found inside the container, the socket is the host's Docker
and should be trusted.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 02:29:55 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> cdbee0a317 DEVX-155: fix: kill dockerd by PID when pkill fails, use /dev/shm for alive daemon
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
Add pgrep diagnostics before/after pkill to identify lingering dockerd
processes. If pkill fails and pgrep still finds dockerd, kill by PID
directly via os.kill. When inner dockerd can't be killed (still alive),
use /dev/shm as data root since the overlay is still full.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 02:12:46 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 3ac3e613e9 DEVX-155: fix: kill inner dockerd with SIGKILL, use alt socket if alive
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The inner dockerd started by the CI image doesn't respond to SIGTERM.
Use pkill -9 to force-kill it, then verify it's actually dead by
running docker info. If the old daemon is still alive (can't be killed),
use /dev/shm/docker.sock as an alternate socket path to avoid conflicts.

Also moved the data root cleanup after the kill verification, so we
don't delete files while the old daemon might still be writing to them.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 01:25:37 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> ae33b86ba2 DEVX-155: fix: use container overlay for local dockerd data root
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
/dev/shm is a 16G tmpfs — too small for the 505MB molecule-test-base
image. Use /tmp/docker-data on the container's overlay instead, after
killing the inner dockerd and cleaning up its data root to free ~2.4GB.

Also use the standard DOCKER_SOCK path (/var/run/docker.sock) for the
local dockerd, since the inner dockerd has been killed and the socket
is free.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 01:03:40 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> a2d47b7efc DEVX-155: fix: start local dockerd instead of using low-space inner dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
When no socket has sufficient space, don't fall back to the low-space
inner dockerd (which will fail on image pulls). Instead, kill the inner
dockerd, clean up its data root to free space, and start a local
dockerd on /dev/shm with vfs storage driver.

Also adds pkill of the inner dockerd and cleanup of its data root
(overlay2, image, volumes, containers) before starting the local
dockerd, to free up the 2.4GB used by the inner dockerd's data.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 00:41:27 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 0e25810b84 DEVX-155: fix: prefer /run/host-docker.sock over inner dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The start_docker.py script was checking /var/run/docker.sock first,
which inside CI containers is an inner dockerd (v29.5.3) with data
root on the container's 38G overlay (often 100% full). When
_get_docker_free_bytes() returned 0 (data root path not accessible
from inside container), the script assumed it was the host Docker
with plenty of space and returned True immediately — without trying
the host's rootless Docker socket at /run/host-docker.sock.

Fix: try /run/host-docker.sock FIRST (before /var/run/docker.sock).
The host socket is mounted by the gitea runner config and has access
to the host's full filesystem (455G). Only trust free_bytes == 0
(= data root not accessible from container) for /run/host-docker.sock,
since the host's root dir is genuinely outside the container. For
other sockets (inner dockerd), free_bytes == 0 means the path doesn't
exist inside the container — don't trust it.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 23:15:03 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 1470cdae27 DEVX-155: fix: use host Docker when root dir is inaccessible (free=0)
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The CI container can't access the host Docker's data root
(/home/grm-ci-runner-X/.local/share/docker) to check disk space.
When free_bytes=0, the root dir is on the host filesystem (455G).
Always use host Docker in that case instead of starting a local
dockerd on the 16G /dev/shm tmpfs (which fills up with images).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 21:15:23 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> d7c9b7fa94 DEVX-155: fix: disable bridge and ip6tables for local dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The new dockerd can't create bridge network or ip6tables chains
(permission denied). Disable both — molecule tests use host network.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 20:53:21 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 36ea71aadc DEVX-155: fix: add --iptables=false to local dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The new dockerd fails with 'iptables: Permission denied' because it
can't create NAT chains. Disable iptables since molecule tests don't
need network isolation.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 20:42:11 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 5e1337e524 DEVX-155: fix: use /dev/shm/docker.sock socket for local dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The existing dockerd holds /var/run/docker.sock and can't be killed
from inside the container (different PID namespace). Use a new socket
path /dev/shm/docker.sock and data root /dev/shm/docker (16G tmpfs).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 20:30:58 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> c61e4b3cab DEVX-155: fix: kill existing dockerd before starting /dev/shm/docker daemon
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The CI container's pre-existing dockerd (v29.5.3) uses the default
data root on the 38G overlay (100% full). Kill it before starting
the new dockerd with --data-root /dev/shm/docker (16G tmpfs).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 20:05:36 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> e332b62fee DEVX-155: fix: use /dev/shm/docker as data-root for inner dockerd
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The CI container's overlay (38G) is often 100% full, causing the
inner DinD daemon to fail pulling images (no space left on device).
/dev/shm is a 16G tmpfs with plenty of space for molecule test images.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 19:41:15 +02:00
emilandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 0c2388f063 DEVX-155: fix: check /run/host-docker.sock for host Docker daemon
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
The gitea_runner config now mounts the rootless Docker socket at
/run/host-docker.sock. start_docker.py checks this path first,
giving CI containers access to the host's full filesystem (455 GB)
instead of the container's limited overlay (38 GB).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 18:03:32 +02:00
gitea-actions-botandDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 23bd480e80 DEVX-155: fix: prefer rootless Docker socket over low-space inner DinD daemon
Post-merge / detect-and-configure (push) Canceled after 0s
Post-merge / release-and-maintain (push) Canceled after 0s
When a CI container has an inner dockerd (DinD) writing to the
container's overlay (e.g. 38 GB), image pulls fail with ENOSPC.
start_docker.py now checks the daemon's free disk space and tries
rootless sockets (which have access to the host's full filesystem)
when the default socket has less than 20 GB free.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-14 17:27:58 +02:00
gitea-actions-bot 0341ee74c9 chore: update badge URLs to commit a506e113 [skip ci] 2026-08-09 22:09:27 +00:00
devx-ci-bot 6c8b02909f release: v0.50.0 [skip ci] 2026-08-09 22:08:35 +00:00
emil eeb291564d DEVX-154: feat: add 5 standalone lint scripts from infra
Post-merge / detect-and-configure (push) Successful in 23s
Post-merge / release-and-maintain (push) Successful in 1m21s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-09 22:07:38 +00:00
gitea-actions-bot 16fb319a47 chore: update badge URLs to commit 6a22c4cb [skip ci] 2026-08-09 09:39:51 +00:00
devx-ci-bot f9513add63 release: v0.49.0 [skip ci] 2026-08-09 01:10:21 +00:00
emil 3d4b4940ff DEVX-153: feat: sync missing features from v0.49.x line to master
Post-merge / detect-and-configure (push) Successful in 16s
Post-merge / release-and-maintain (push) Successful in 1m2s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-09 01:09:20 +00:00
gitea-actions-bot d2aa4c6298 chore: update badge URLs to commit e3830f8b [skip ci] 2026-08-09 01:01:36 +00:00
devx-ci-bot 9cdbdde6da release: v0.48.2 [skip ci] 2026-08-09 01:00:50 +00:00
emil bd4530094e DEVX-152: fix: remove dead translation keys and add missing one
Post-merge / detect-and-configure (push) Successful in 18s
Post-merge / release-and-maintain (push) Successful in 1m13s
Co-authored-by: emil User <emil.simeonov@tutanota.com>
2026-08-09 00:59:55 +00:00
90 changed files with 12161 additions and 6498 deletions
+9 -1
View File
@@ -12,7 +12,6 @@ Quick reference for devx tools when working on the devx repo itself.
| Check CI status | `make devx-pr-status` or `make devx-pr-status PR=42 WAIT=1` |
| Fetch CI failure logs | `make devx-pr-logs` or `make devx-pr-logs PR=42 JOB=quality TAIL=50` |
| Add ready-to-merge label | `make devx-pr-label` or `make devx-pr-label PR=42` |
| Post PR review | `make devx-pr-review PR=42 EVENT=APPROVE BODY="..." CHECKLIST=1,2,3,4,5,6,7,8,9,10,11,12,13` |
| Rebase current branch | `make rebase` |
| Rebase PR via API | `make pr-rebase` or `make pr-rebase PR=42` |
@@ -24,6 +23,15 @@ When the `ready-to-merge` label is added and all CI checks pass:
3. The rebase triggers a new CI run; the next auto-merge attempt merges
4. No manual rebase needed unless the API rebase fails
## Spec-Driven CI Gates (Pre-merge)
Every PR must pass these gates before merge:
| Gate | Module | What it checks |
|------|--------|----------------|
| Spec validation | `devx.ci.validate_spec` | Spec file exists at `docs/specs/<TASK-ID>.md`, has REQ-IDs, all ACs checked |
| PR size | `devx.ci.check_pr_size` | Max 500 lines / 10 files (excludes CHANGELOG, badges, locks) |
## Key Rules
- Never manually merge via API — always use auto-merge with `ready-to-merge` label
+272
View File
@@ -0,0 +1,272 @@
# pr-review
Deep, critical PR review with auto-fix. This skill guides the agent
through a thorough review of a pull request, posting inline comments
for each issue found, auto-fixing them, resolving the discussion threads,
and marking the PR as ready-to-merge when no blocking issues remain.
## When to Invoke
Invoke this skill when asked to review a PR, or when a PR is open and
needs review before merge. Do NOT invoke automatically on every PR —
this is an on-demand deep review, not a CI gate.
## Prerequisites
- The PR must be open in a Gitea repo
- The agent needs Gitea MCP access (gitea server)
- The agent needs git push access to the PR's head branch
- The PR should have passed CI (validate job) before deep review
## Review Categories
Review every PR against these 8 categories. For each issue found, post
an inline comment on the specific line, then auto-fix it.
### 1. Functional Correctness
- Does the code actually do what the spec/PR title claims?
- Are edge cases handled? (empty input, null, boundary values, concurrent access)
- Are error paths tested? Not just happy path.
- Does the code handle all return values? (ignored errors, unchecked None)
- Are there off-by-one errors, wrong comparisons, inverted conditions?
- Do loops terminate correctly? (no infinite loops, correct break/continue)
- Are regex patterns correct? (anchored, escaped, non-greedy where needed)
- Are API responses validated before use? (status codes, response shape)
### 2. Completeness
- Are all requirements from the spec implemented? (check each REQ-ID)
- Are all acceptance criteria in the spec checked off?
- Are tests written for all new code paths?
- Are error messages user-facing (wrapped in `_()`)?
- Are new CLI commands documented in `docs/user/cli-commands.md`?
- Are new modules added to architecture docs?
- Are CHANGELOG entries added for user-facing changes?
- Are translations added for new user-facing strings?
### 3. Architecture
- Does the code follow the repo's layer separation? (no business logic in CLI, no direct subprocess in CLI)
- Are new dependencies justified? (no unnecessary new packages)
- Is configuration via env vars / config.py, not hardcoded?
- Are new modules placed in the correct directory? (ci/ vs tools/ vs molecule/)
- Does the code reuse existing utilities? (no reimplemented helpers)
- Are imports circular? (check import chains)
- Is the code testable? (injectable dependencies, no hidden global state)
- Does the code follow existing patterns in the codebase?
### 4. Reliability
- Are external API calls retried with backoff?
- Are timeouts set on all network operations?
- Are file operations atomic? (write to temp, rename)
- Are database operations transactional where needed?
- Are there race conditions? (check shared mutable state)
- Are resources cleaned up in all paths? (finally blocks, context managers)
- Can the code handle partial failures? (one service down, others up)
- Are idempotency guarantees maintained? (safe to retry)
### 5. Robustness
- Does the code fail gracefully? (meaningful error messages, not stack traces)
- Are unexpected inputs handled? (type checking, validation)
- Are there any crash-on-bad-input paths?
- Does the code degrade under load? (backpressure, queue limits)
- Are there resource leaks? (file handles, connections, memory)
- Does the code survive network partitions? (retry, circuit breaker)
- Are there any unhandled exceptions that could crash the process?
- Is logging sufficient to diagnose production issues?
### 6. Security
- Are there hardcoded secrets, tokens, or passwords?
- Is `shell=True` used with user input? (command injection)
- Is `eval()` or `exec()` used? (code injection)
- Are SQL queries parameterized? (no string concatenation)
- Are file paths validated? (no path traversal)
- Are user inputs sanitized before display? (XSS in web contexts)
- Are SSL/TLS verifications disabled without justification?
- Are secrets logged in error messages or debug output?
- Are permissions checked before privileged operations?
- Is sensitive data in memory longer than necessary?
### 7. Technical Excellence
- Are functions under 50 lines? (refactor if longer)
- Is cyclomatic complexity reasonable? (no deeply nested if/else chains)
- Are names meaningful? (no single-letter vars, no misleading names)
- Is dead code removed? (no commented-out blocks, no unused imports)
- Are comments explaining WHY, not WHAT?
- Is the code DRY? (no copy-pasted blocks that should be shared)
- Is the code SOLID? (single responsibility, open/closed)
- Are magic numbers extracted to named constants?
- Is the code formatted per the repo's linter config?
- Are type hints present on all function signatures?
### 8. Test Quality
- Do tests actually test the behavior? (not just that code runs)
- Are tests independent? (no shared mutable state, no order dependency)
- Are tests fast? (no real sleeps, no real network calls, mocked)
- Are edge cases tested? (empty, None, boundary, error paths)
- Are test names descriptive? (test_what_condition_expected_result)
- Are mocks set up correctly? (mocking the right object, not too broad)
- Is coverage 100% for new code? (every branch, every line)
- Are integration tests added for cross-module changes?
- Do tests clean up after themselves? (tmp_path, fixtures)
## Review Procedure
### Step 1: Gather Context
```
1. Read the PR spec (if exists): docs/specs/<TASK-ID>.md
2. Fetch PR details via Gitea MCP: pull_request_read (get_pr, list_pr_files)
3. Read the full diff: git diff origin/master...HEAD
4. Read the PR description and any existing review comments
5. Identify the repo's task prefix (OBL-INFRA, GRM, SSO, DEVX)
```
### Step 2: Review Each File
For each changed file in the PR:
1. Read the full file (not just the diff) to understand context
2. Go through all 8 review categories
3. For each issue found, note: file path, line number, category, severity, description, suggested fix
### Step 3: Post Inline Comments
For each issue found, post an inline review comment using the Gitea MCP:
```
mcp_call_tool: gitea / pull_request_review_write
method: create
owner: <owner>
repo: <repo>
pull_number: <PR number>
state: PENDING (accumulate comments before submitting)
body: "" (empty for now, summary added on submit)
comments: [
{
path: "<file path>",
new_line_num: <line number>,
body: "**[<category>] [<severity>]** <description>\n\n**Suggested fix:**\n```<lang>\n<fixed code>\n```"
}
]
```
Comment format:
```
**[Security] [error]** `shell=True` used with user input — command injection risk.
**Suggested fix:**
```python
subprocess.run(["git", "log", commit], check=True)
```
```
Severity levels:
- `error` — must fix before merge (security, correctness, crash)
- `warning` — should fix before merge (reliability, best practice)
- `info` — consider fixing (style, minor improvement)
### Step 4: Auto-Fix Issues
For each issue that can be safely auto-fixed:
1. Edit the file using the `edit` tool
2. Commit with message: `fix: address review comment — <short description>`
3. Push to the PR's head branch: `git push origin HEAD`
4. Wait for CI to re-run on the push
Auto-fix ALL issues unless:
- The fix requires an architectural decision (ask the user)
- The fix changes public API behavior (ask the user)
- The fix is ambiguous (multiple valid approaches, ask the user)
### Step 5: Resolve Discussion Threads
After auto-fixing an issue and CI passes:
1. Find the review comment thread for that issue
2. Post a reply: `Fixed in <commit-sha>. Closing this thread.`
3. Resolve the discussion (if Gitea supports it via API)
4. If resolving via API is not available, the reply comment serves as resolution
### Step 6: Submit Final Review
After all issues are addressed (fixed or discussed):
```
mcp_call_tool: gitea / pull_request_review_write
method: submit
owner: <owner>
repo: <repo>
pull_number: <PR number>
review_id: <from step 3 create>
state: COMMENT (or APPROVED if no blocking issues remain)
body: <summary — see below>
```
### Step 7: Post Summary
Post a brief summary as a PR comment (via `issue_write / add_comment`):
```
## Deep Review Summary
- **Files reviewed:** N
- **Issues found:** N (N auto-fixed, N require attention)
- **Categories:** security (N), correctness (N), architecture (N), ...
**Outcome:** ✅ Ready to merge — all issues addressed.
**OR**
**Outcome:** ⚠️ N blocking issue(s) remain — see inline comments.
```
Keep the summary to 5-10 bullet points. Do not paste the full review.
### Step 8: Mark PR Ready
If all issues are addressed and no blocking issues remain:
```
mcp_call_tool: gitea / issue_write
method: add_labels
owner: <owner>
repo: <repo>
issue_number: <PR number>
labels: [<label_id for "ready-to-merge">]
```
If blocking issues remain, do NOT add the label. Post a comment
explaining what needs to be resolved before the PR can merge.
## Gitea MCP Tools Reference
| Action | MCP tool | Method |
|--------|----------|--------|
| Get PR details | `pull_request_read` | `get_pr` |
| List PR files | `pull_request_read` | `list_pr_files` |
| Get PR diff | `pull_request_read` | `get_pr_diff` |
| Create review (pending) | `pull_request_review_write` | `create` (state: PENDING) |
| Submit review | `pull_request_review_write` | `submit` (state: APPROVED/COMMENT/REQUEST_CHANGES) |
| Post PR comment | `issue_write` | `add_comment` |
| Add label | `issue_write` | `add_labels` |
| List labels | `label_read` | `list_repo_labels` |
| Merge PR | `pull_request_write` | `merge` (do NOT use — auto-merge handles this) |
## Important Rules
- **Never merge the PR yourself.** Add the `ready-to-merge` label and let
the auto-merge workflow handle it. This ensures CI passes and the
commit message follows the `<PREFIX>-N: <conventional>` format.
- **Never approve your own PR.** If the agent created the PR, post
COMMENT state, not APPROVED.
- **Always push fixes to the PR branch**, not directly to master.
- **Wait for CI after each push** before resolving the discussion thread.
- **Post one review with all comments**, not multiple reviews.
- **The summary must be brief** — 5-10 bullet points max.
- **Severity matters**: only `error` severity blocks the `ready-to-merge` label.
@@ -0,0 +1,130 @@
# Spec-Driven Development
## Overview
Every change starts with a spec. No spec, no code. No code, no PR.
The spec is a markdown file at `docs/specs/<TASK-ID>.md` in the repo.
It contains structured requirements (REQ-IDs) and acceptance criteria
(AC checklist) that CI validates before merge.
## Workflow
1. **Create Vikunja task**`make create-task -- --title "Title" --description "..."`
2. **Write spec** — Create `docs/specs/<TASK-ID>.md` (see template below)
3. **Create branch**`git checkout -b <PREFIX>-N-short-description`
4. **Implement** — Write code with `# Implements: REQ-N` comments
5. **Check ACs** — Tick all acceptance criteria checkboxes in the spec
6. **Push and create PR**`make push-with-pr`
7. **CI validates** — Spec validation, PR size check, fast molecule, lint, tests
8. **Auto-merge** — Add `ready-to-merge` label after review
9. **Auto-deploy** — Post-merge deploys to staging (if nightly gate is green)
## Spec Template
```markdown
# <TASK-ID>: <Title>
## Problem
<What is broken or missing? Why does this change exist?>
## Approach
<How will you solve it? What are the key design decisions?>
REQ-1: <First requirement description>
REQ-2: <Second requirement description>
REQ-3: <Third requirement description>
## Test Plan
- <How will you verify each REQ is implemented correctly?>
- <Include unit tests, molecule scenarios, integration tests>
## Deploy Plan
- <How will this change be deployed?>
- <What order do components need to deploy in?>
- <Are there migrations or one-time operations?>
## Rollback Plan
- <How do you revert if something goes wrong?>
- <What data/state changes are irreversible?>
## Acceptance Criteria
- [ ] REQ-1: <criterion that proves REQ-1 is done>
- [ ] REQ-2: <criterion that proves REQ-2 is done>
- [ ] REQ-3: <criterion that proves REQ-3 is done>
```
## CI Validation
The `devx.ci.validate_spec` module checks:
1. **Spec file exists** at `docs/specs/<TASK-ID>.md` (TASK-ID from branch name)
2. **Required sections present**: Problem, Approach, Test Plan, Deploy Plan, Rollback Plan, Acceptance Criteria
3. **At least one REQ-ID** line (format: `REQ-N: <description>`)
4. **All AC checkboxes checked** (`- [x]`, not `- [ ]`)
If any check fails, CI blocks the PR before expensive jobs run.
## PR Size Limits
CI enforces max 500 lines / 10 files changed (excluding CHANGELOG.md,
README.md, badges, lock files). Oversized PRs are rejected. Split your
work into smaller PRs.
## Code-to-Spec Linking
Each function, task, or template that implements a requirement should
have a comment:
```python
# Implements: REQ-1
def install_sso_bridge():
...
```
```yaml
# Implements: REQ-2
- name: Clone infra repo
git:
...
```
## Fast Molecule (Pre-merge)
CI runs molecule only for **changed roles** (detected via git diff),
with converge + verify only, single platform. This gives quick feedback
(~5-10 min) without the full molecule suite.
## Full Molecule (Nightly)
The complete molecule suite (all scenarios, all platforms) runs nightly
at 02:00 CET on master. If it fails:
- A Gitea issue is created with the `feedback` label
- The `NIGHTLY_STATUS` repo variable is set to `failed:<run_id>`
- All staging deploys are blocked until nightly passes again
## Auto-Deploy on Merge
Every merged PR auto-deploys to staging (if nightly gate is green).
No manual trigger needed. The deploy runs the full pipeline:
provision → deploy-observability → deploy-customer → configure-oidc.
For grm/sso-bridge: post-merge publishes the package, then auto-creates
an infra PR to bump the pinned version. That infra PR auto-deploys when
merged.
## Key Commands
```bash
# Validate spec locally (before pushing)
python -m devx.ci.validate_spec --branch <PREFIX>-N-description
# Check PR size locally
python -m devx.ci.check_pr_size --base origin/master --head HEAD
# See which roles need fast molecule
python -m devx.ci.fast_molecule --base origin/master --head HEAD
# Check nightly gate status
python -m devx.ci.nightly_gate --repo oblachno/infra --action check
```
@@ -45,6 +45,13 @@ This runs `lint-all` + `pytest-cov`. The pre-push git hook only
validates the Vikunja task exists — it does NOT run tests. You must
run `make pre-push` manually.
### Spec-Driven Workflow
Every PR requires a spec file at `docs/specs/<TASK-ID>.md`. See the
`spec-driven-development` skill for the full workflow and template.
CI validates the spec (via `devx.ci.validate_spec`) and checks PR size
(via `devx.ci.check_pr_size`) before running expensive jobs.
## CI Failure Investigation
When investigating a CI failure:
+30 -14
View File
@@ -106,14 +106,30 @@ jobs:
--pr-title "$PR_TITLE" \
--repo "$REPOSITORY" \
--pr-number "$PR_NUMBER"
- name: Run automated PR review
- name: Validate spec file
if: github.event_name == 'pull_request'
env:
DEVX_TASK_PREFIX: DEVX
PYTHONPATH: ${{ env.PYTHONPATH }}
HEAD_REF: ${{ github.head_ref }}
run: |
. .venv/bin/activate 2>/dev/null || true
set -euo pipefail
python3 -m devx.ci.pr_review \
"${{ github.event.number }}" \
"${{ github.repository }}"
python3 -m devx.ci.validate_spec \
--branch "$HEAD_REF" \
--github-output
- name: Check PR size
if: github.event_name == 'pull_request'
env:
PYTHONPATH: ${{ env.PYTHONPATH }}
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
run: |
. .venv/bin/activate 2>/dev/null || true
python3 -m devx.ci.check_pr_size \
--base "origin/master" \
--head "${{ github.event.pull_request.head.sha || github.sha }}" \
--repo "${{ github.repository }}" \
--pr-number "${{ github.event.number }}" \
--github-output
# --- release-dry-run step (conditional) ---
- name: Release dry-run validation
if: steps.detect.outputs.user-facing-changed == 'true'
@@ -165,18 +181,18 @@ jobs:
- name: Post approval review
env:
REVIEWER_GITEA_API_TOKEN: ${{ secrets.REVIEWER_GITEA_API_TOKEN }}
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
PR_NUMBER: ${{ github.event.number }}
REPOSITORY: ${{ github.repository }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: |
. .venv/bin/activate 2>/dev/null || true
python3 -m devx.ci.pr_review \
"$PR_NUMBER" \
"$REPOSITORY" \
--event APPROVE \
--checklist-confirmed \
--checklist-categories 1,2,3,4,5,6,7,8,9,10,11,12,13 \
--body "Auto-approved: all CI checks passed (validate job)."
# Post APPROVE review via Gitea API to satisfy branch protection
curl -s -X POST \
"${GITHUB_SERVER_URL}/api/v1/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
-H "Authorization: token ${REVIEWER_GITEA_API_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event":"APPROVED","body":"Auto-approved: all CI checks passed (validate job)."}' \
|| echo "::warning::Failed to post approval review (best-effort)."
- name: Squash merge with task ID
env:
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
+9
View File
@@ -0,0 +1,9 @@
extends: existence
message: "Use 'AM' or 'PM' (preceded by a space)."
link: "https://developers.google.com/style/word-list"
level: error
nonword: true
tokens:
- '\d{1,2}[AP]M\b'
- '\d{1,2} ?[ap]m\b'
- '\d{1,2} ?[aApP]\.[mM]\.'
+64
View File
@@ -0,0 +1,64 @@
extends: conditional
message: "Spell out '%s', if it's unfamiliar to the audience."
link: 'https://developers.google.com/style/abbreviations'
level: suggestion
ignorecase: false
# Ensures that the existence of 'first' implies the existence of 'second'.
first: '\b([A-Z]{3,5})\b'
second: '(?:\b[A-Z][a-z]+ )+\(([A-Z]{3,5})\)'
# ... with the exception of these:
exceptions:
- API
- ASP
- CLI
- CPU
- CSS
- CSV
- DEBUG
- DOM
- DPI
- FAQ
- GCC
- GDB
- GET
- GPU
- GTK
- GUI
- HTML
- HTTP
- HTTPS
- IDE
- JAR
- JSON
- JSX
- LESS
- LLDB
- NET
- NOTE
- NVDA
- OSS
- PATH
- PDF
- PHP
- POST
- RAM
- REPL
- RSA
- SCM
- SCSS
- SDK
- SQL
- SSH
- SSL
- SVG
- TBD
- TCP
- TODO
- URI
- URL
- USB
- UTF
- XML
- XSS
- YAML
- ZIP
+12
View File
@@ -0,0 +1,12 @@
extends: existence
message: "Don't attribute human qualities to software or hardware ('%s')."
link: https://developers.google.com/style/anthropomorphism
level: suggestion
ignorecase: true
# Limited to the two verbs the guide itself names. Broader lists (wants, knows,
# thinks) can't tell a software subject from a human one: on a 950-file corpus
# they produced 8 false positives ('the customer wants', 'your audience knows')
# for every 2 real ones.
tokens:
- sees
- tells
+13
View File
@@ -0,0 +1,13 @@
extends: existence
message: "'%s' should be in lowercase."
link: 'https://developers.google.com/style/colons'
level: warning
scope: sentence
# The match is the word itself, not ': X', and `nonword` is off. Both are
# required for a project Vocab to work: Vale compares accept.txt entries
# against the matched text, and `nonword: true` opts out of that entirely.
# So a proper noun after a colon can be exempted by adding it to accept.txt.
# The guide's other exemption, notice labels, is handled by the lookbehinds;
# headings are already excluded by `scope: sentence`. See issue #20.
tokens:
- '(?<!Note: )(?<!Caution: )(?<!Warning: )(?<!Success: )(?<=:\s)[A-Z]\w+'
+30
View File
@@ -0,0 +1,30 @@
extends: substitution
message: "Use '%s' instead of '%s'."
link: 'https://developers.google.com/style/contractions'
level: suggestion
ignorecase: true
action:
name: replace
swap:
are not: aren't
cannot: can't
could not: couldn't
did not: didn't
do not: don't
does not: doesn't
has not: hasn't
have not: haven't
how is: how's
is not: isn't
it is: it's
should not: shouldn't
that is: that's
they are: they're
was not: wasn't
we are: we're
we have: we've
were not: weren't
what is: what's
when is: when's
where is: where's
will not: won't
+9
View File
@@ -0,0 +1,9 @@
extends: existence
message: "Use 'July 31, 2016' format, not '%s'."
link: 'https://developers.google.com/style/dates-times'
ignorecase: true
level: error
nonword: true
tokens:
- '\d{1,2}(?:\.|/)\d{1,2}(?:\.|/)\d{4}'
- '\d{1,2} (?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?) \d{4}'
+9
View File
@@ -0,0 +1,9 @@
extends: existence
message: "In general, don't use an ellipsis."
link: 'https://developers.google.com/style/ellipses'
nonword: true
level: warning
action:
name: remove
tokens:
- '\.\.\.'
+13
View File
@@ -0,0 +1,13 @@
extends: existence
message: "Don't put a space before or after a dash."
link: "https://developers.google.com/style/dashes"
nonword: true
level: error
action:
name: edit
params:
- trim
- " "
tokens:
- '\s[—–]\s'
+14
View File
@@ -0,0 +1,14 @@
extends: existence
message: "Avoid the unverifiable claim '%s'."
link: https://developers.google.com/style/excessive-claims
level: suggestion
ignorecase: true
# The guide also names 'never', 'always', and 'ensure', but in technical writing
# those are usually legitimate instructions ('never commit secrets') rather than
# product claims: they accounted for 125 of 142 hits on a 950-file corpus.
# 'best practices' is a fixed term, not a superlative.
tokens:
- 'best(?! practices?)'
- simplest
- fastest
- guarantees?
+12
View File
@@ -0,0 +1,12 @@
extends: existence
message: "Don't use exclamation points in text."
link: "https://developers.google.com/style/exclamation-points"
nonword: true
level: error
action:
name: edit
params:
- trim_right
- "!"
tokens:
- '\w+!(?:\s|$)'
+15
View File
@@ -0,0 +1,15 @@
extends: existence
message: "Avoid first-person pronouns such as '%s'."
link: 'https://developers.google.com/style/pronouns#personal-pronouns'
ignorecase: true
level: warning
# The 'I' tokens use lookaround rather than consuming the surrounding
# whitespace. Matching ' I ' made the alert span cover both spaces, which shows
# up as a too-wide underline in editors, and read as "such as ' I '". Dropping
# `nonword` also lets a project Vocab apply, which it can't when set. See PR #50.
tokens:
- '(?<=^|\s)I(?=[\s,])'
- "\\bI'm\\b"
- \bme\b
- \bmy\b
- \bmine\b
+9
View File
@@ -0,0 +1,9 @@
extends: existence
message: "Don't use '%s' as a gender-neutral pronoun."
link: 'https://developers.google.com/style/pronouns#gender-neutral-pronouns'
level: error
ignorecase: true
tokens:
- he/she
- s/he
- \(s\)he
+43
View File
@@ -0,0 +1,43 @@
extends: substitution
message: "Consider using '%s' instead of '%s'."
ignorecase: true
link: "https://developers.google.com/style/inclusive-documentation"
level: error
action:
name: replace
swap:
(?:alumna|alumnus): graduate
(?:alumnae|alumni): graduates
air(?:m[ae]n|wom[ae]n): pilot(s)
anchor(?:m[ae]n|wom[ae]n): anchor(s)
authoress: author
camera(?:m[ae]n|wom[ae]n): camera operator(s)
door(?:m[ae]|wom[ae]n): concierge(s)
draft(?:m[ae]n|wom[ae]n): drafter(s)
fire(?:m[ae]n|wom[ae]n): firefighter(s)
fisher(?:m[ae]n|wom[ae]n): fisher(s)
fresh(?:m[ae]n|wom[ae]n): first-year student(s)
garbage(?:m[ae]n|wom[ae]n): waste collector(s)
lady lawyer: lawyer
ladylike: courteous
mail(?:m[ae]n|wom[ae]n): mail carriers
man and wife: husband and wife
man enough: strong enough
mankind: human kind|humanity
manmade: manufactured
manpower: personnel
middle(?:m[ae]n|wom[ae]n): intermediary
news(?:m[ae]n|wom[ae]n): journalist(s)
ombuds(?:man|woman): ombuds
oneupmanship: upstaging
poetess: poet
police(?:m[ae]n|wom[ae]n): police officer(s)
repair(?:m[ae]n|wom[ae]n): technician(s)
sales(?:m[ae]n|wom[ae]n): salesperson or sales people
service(?:m[ae]n|wom[ae]n): soldier(s)
steward(?:ess)?: flight attendant
tribes(?:m[ae]n|wom[ae]n): tribe member(s)
waitress: waiter
woman doctor: doctor
woman scientist[s]?: scientist(s)
work(?:m[ae]n|wom[ae]n): worker(s)
@@ -0,0 +1,13 @@
extends: existence
message: "Don't put a period at the end of a heading."
link: "https://developers.google.com/style/capitalization#capitalization-in-titles-and-headings"
nonword: true
level: warning
scope: heading
action:
name: edit
params:
- trim_right
- "."
tokens:
- '[a-z0-9][.]\s*$'
+32
View File
@@ -0,0 +1,32 @@
extends: capitalization
message: "'%s' should use sentence-style capitalization."
link: "https://developers.google.com/style/capitalization#capitalization-in-titles-and-headings"
level: warning
scope: heading
match: $sentence
# No `indicators: [":"]` here. That makes Vale require a capital after a colon,
# which is the Microsoft convention this rule was originally copied from. This
# guide says the opposite: "the first word after a colon is generally
# lowercase" (developers.google.com/style/colons), and Colons.yml enforces
# exactly that. See issue #58.
exceptions:
- Azure
- CLI
- Cosmos
- Docker
- Emmet
- gRPC
- I
- Kubernetes
- Linux
- macOS
- Marketplace
- MongoDB
- REPL
- Studio
- TypeScript
- URLs
- Visual
- VS
- Windows
- JSON
+13
View File
@@ -0,0 +1,13 @@
extends: existence
message: "Avoid the jargon '%s'."
link: https://developers.google.com/style/jargon
level: suggestion
ignorecase: true
# The guide also cites 'solution', 'support', and 'workload' as overloaded
# terms, but those have ordinary technical meanings and accounted for every hit
# on a 950-file corpus, so only the unambiguous figurative terms are listed.
tokens:
- break-glass
- camel ?case
- out-of-the-box
- swim ?lane
+15
View File
@@ -0,0 +1,15 @@
extends: substitution
message: "Use '%s' instead of '%s'."
link: 'https://developers.google.com/style/abbreviations'
ignorecase: true
level: error
nonword: true
action:
name: replace
# The delimiter is a lookahead so the replacement doesn't swallow the comma or
# space that follows (issue #18). `$` is included so the abbreviation is still
# caught at the end of a heading, table cell, or block, which accounted for 8
# of 10 occurrences on a 950-file corpus.
swap:
'\b(?:eg|e\.g\.)(?=[\s,;]|$)': for example
'\b(?:ie|i\.e\.)(?=[\s,;]|$)': that is
+14
View File
@@ -0,0 +1,14 @@
extends: existence
message: "'%s' doesn't need a hyphen."
link: "https://developers.google.com/style/hyphens"
level: error
ignorecase: false
nonword: true
action:
name: edit
params:
- regex
- "-"
- " "
tokens:
- '\b[^\s-]+ly-\w+\b'
+12
View File
@@ -0,0 +1,12 @@
extends: existence
message: "Don't use plurals in parentheses such as in '%s'."
link: "https://developers.google.com/style/plurals-parentheses"
level: error
nonword: true
action:
name: edit
params:
- trim_right
- "(s)"
tokens:
- '\b\w+\(s\)'
+7
View File
@@ -0,0 +1,7 @@
extends: existence
message: "Spell out all ordinal numbers ('%s') in text."
link: 'https://developers.google.com/style/numbers'
level: error
nonword: true
tokens:
- \d+(?:st|nd|rd|th)
+28
View File
@@ -0,0 +1,28 @@
extends: existence
message: "Use the Oxford comma in '%s'."
link: 'https://developers.google.com/style/commas'
scope: sentence
level: warning
nonword: true
# List items may be several words long, not just one. Four guards keep the
# false-positive rate down:
#
# 1. The comma can't be the one closing a fronted subordinate clause
# ('When your alarm rings, you turn it off and tumble out of bed.') --
# that comma separates clauses, not list items. Only the first comma of
# such a sentence is exempt, so 'When it rains, apples, pears or bananas
# get wet.' is still caught.
# 2. The item can't open with a clause-introducer (', which ...',
# ', specifically ...').
# 3. The item can't open with a subject pronoun followed by a verb, which
# marks a compound predicate rather than a list ('..., you walk to the
# fridge and get a snack.'). A pronoun directly followed by 'and'/'or'
# is a real list item, so ', you and me.' still matches.
# 4. Neither item may contain an auxiliary verb, which is another compound
# predicate signal (', it has some downsides and is officially
# discouraged.').
#
# The trailing anchor allows end-of-scope so list fragments ('Apples, pears
# or bananas') are still caught.
tokens:
- '(?<!^(?i:when|whenever|while|if|unless|until|although|though|because|since|after|before|once|whereas|whether|as)\b[^,]{0,80}),\s(?!(?:which|who|whom|whose|that|where|when|while|because|since|although|though|if|unless|so|but|and|or|however|therefore|thus|specifically|especially|namely|then|take|see|note|consider|make|use|either|neither)\b)(?!(?i:i|you|we|they|he|she|it)\s+(?!(?:and|or)\b))(?:(?!\b(?:is|are|was|were|has|have|had|be|been|being|will|would|can|could|should|may|might|must|do|does|did)\b)\w+ ){0,4}\w+ (?:and|or) (?:(?!\b(?:is|are|was|were|has|have|had|be|been|being|will|would|can|could|should|may|might|must|do|does|did)\b)\w+ ){0,4}\w+(?:[.?!]|$)'
+15
View File
@@ -0,0 +1,15 @@
extends: existence
message: "Use parentheses judiciously."
link: 'https://developers.google.com/style/parentheses'
nonword: true
level: suggestion
# `[^)]` rather than `.+`: a greedy match ran from the first '(' on a line to
# the last ')', so 'Text (one) and more (two).' produced a single alert
# covering everything between them. See issue #30.
# A bare 3-5 letter acronym is skipped: Acronyms.yml requires acronyms to be
# defined as 'Spelled Out Term (ACRONYM)', so flagging those parentheses would
# put the two rules in direct conflict. The acronym has to be the whole
# parenthetical — '(NASA rocket program)' is an ordinary aside and still
# flags. Length matches the {3,5} in Acronyms.yml. See PR #59.
tokens:
- '\((?![A-Z]{3,5}\))[^)]+\)'
+184
View File
@@ -0,0 +1,184 @@
extends: existence
link: 'https://developers.google.com/style/voice'
message: "In general, use active voice instead of passive voice ('%s')."
ignorecase: true
level: suggestion
raw:
- \b(am|are|were|being|is|been|was|be)\b\s*
tokens:
- '[\w]+ed'
- awoken
- beat
- become
- been
- begun
- bent
- beset
- bet
- bid
- bidden
- bitten
- bled
- blown
- born
- bought
- bound
- bred
- broadcast
- broken
- brought
- built
- burnt
- burst
- cast
- caught
- chosen
- clung
- come
- cost
- crept
- cut
- dealt
- dived
- done
- drawn
- dreamt
- driven
- drunk
- dug
- eaten
- fallen
- fed
- felt
- fit
- fled
- flown
- flung
- forbidden
- foregone
- forgiven
- forgotten
- forsaken
- fought
- found
- frozen
- given
- gone
- gotten
- ground
- grown
- heard
- held
- hidden
- hit
- hung
- hurt
- kept
- knelt
- knit
- known
- laid
- lain
- leapt
- learnt
- led
- left
- lent
- let
- lighted
- lost
- made
- meant
- met
- misspelt
- mistaken
- mown
- overcome
- overdone
- overtaken
- overthrown
- paid
- pled
- proven
- put
- quit
- read
- rid
- ridden
- risen
- run
- rung
- said
- sat
- sawn
- seen
- sent
- set
- sewn
- shaken
- shaven
- shed
- shod
- shone
- shorn
- shot
- shown
- shrunk
- shut
- slain
- slept
- slid
- slit
- slung
- smitten
- sold
- sought
- sown
- sped
- spent
- spilt
- spit
- split
- spoken
- spread
- sprung
- spun
- stolen
- stood
- stridden
- striven
- struck
- strung
- stuck
- stung
- stunk
- sung
- sunk
- swept
- swollen
- sworn
- swum
- swung
- taken
- taught
- thought
- thrived
- thrown
- thrust
- told
- torn
- trodden
- understood
- upheld
- upset
- wed
- wept
- withheld
- withstood
- woken
- won
- worn
- wound
- woven
- written
- wrung
+7
View File
@@ -0,0 +1,7 @@
extends: existence
message: "Don't use periods with acronyms or initialisms such as '%s'."
link: 'https://developers.google.com/style/abbreviations'
level: error
nonword: true
tokens:
- '\b(?:[A-Z]\.){3,}'
+7
View File
@@ -0,0 +1,7 @@
extends: existence
message: "Commas and periods go inside quotation marks."
link: 'https://developers.google.com/style/quotation-marks'
level: error
nonword: true
tokens:
- '"[^"]+"[.,?]'
+7
View File
@@ -0,0 +1,7 @@
extends: existence
message: "Don't add words such as 'from' or 'between' to describe a range of numbers."
link: 'https://developers.google.com/style/hyphens'
nonword: true
level: warning
tokens:
- '(?:from|between)\s\d+\s?-\s?\d+'
+8
View File
@@ -0,0 +1,8 @@
extends: existence
message: "Use semicolons judiciously."
link: 'https://developers.google.com/style/semicolons'
nonword: true
scope: sentence
level: suggestion
tokens:
- ';'
+11
View File
@@ -0,0 +1,11 @@
extends: existence
message: "Don't use internet slang abbreviations such as '%s'."
link: 'https://developers.google.com/style/abbreviations'
ignorecase: true
level: error
tokens:
- 'tl;dr'
- ymmv
- rtfm
- imo
- fwiw
+10
View File
@@ -0,0 +1,10 @@
extends: existence
message: "'%s' should have one space."
link: 'https://developers.google.com/style/sentence-spacing'
level: error
nonword: true
action:
name: remove
tokens:
- '[a-z][.?!] {2,}[A-Z]'
- '[a-z][.?!][A-Z]'
+10
View File
@@ -0,0 +1,10 @@
extends: existence
message: "In general, use American spelling instead of '%s'."
link: 'https://developers.google.com/style/spelling'
ignorecase: true
level: warning
tokens:
- '(?:\w+)nised?'
- 'colour'
- 'labour'
- 'centre'
+13
View File
@@ -0,0 +1,13 @@
extends: existence
message: "Avoid time-based words like '%s' in product documentation."
link: https://developers.google.com/style/timeless-documentation
level: suggestion
ignorecase: true
# The guide also names 'now' and 'new', but both have common senses that aren't
# time-anchored ('create a new project'): adding them took a 950-file corpus of
# technical documentation from 14 hits to 117. 'recently' is left out too — every
# hit in that corpus was the UI idiom 'recently used'.
tokens:
- currently
- latest
- soon
+10
View File
@@ -0,0 +1,10 @@
extends: existence
message: "Put a nonbreaking space between the number and the unit in '%s'."
link: "https://developers.google.com/style/units-of-measure"
nonword: true
level: error
tokens:
- '\b\d+(?:B|kB|MB|GB|TB)\b'
- '\b\d+(?:ns|ms|min|h|d)\b'
# Seconds are split out so a decade ('1990s') isn't read as a unit.
- '\b\d+s\b(?<!\b(?:19|20)\d\ds\b)'
+11
View File
@@ -0,0 +1,11 @@
extends: existence
message: "Try to avoid using first-person plural like '%s'."
link: 'https://developers.google.com/style/pronouns#personal-pronouns'
level: warning
ignorecase: true
tokens:
- we
- we'(?:ve|re)
- ours?
- us
- let's
+7
View File
@@ -0,0 +1,7 @@
extends: existence
message: "Avoid using '%s'."
link: 'https://developers.google.com/style/tense'
ignorecase: true
level: warning
tokens:
- will
+29
View File
@@ -0,0 +1,29 @@
extends: substitution
message: "Use '%s' instead of '%s'."
link: "https://developers.google.com/style/word-list"
level: warning
# Case matters here: each key's own capitalization is what's being corrected,
# so ignorecase would make these match their own replacements. The rest of the
# word list lives in WordListCase.yml.
ignorecase: false
action:
name: replace
swap:
Ajax: AJAX
Android device: Android-powered device
android: Android
API explorer: APIs Explorer
authN: authentication
authZ: authorization
CLI: command-line tool
Cloud: Google Cloud Platform|GCP
Container Engine: Kubernetes Engine
Developers Console: Google API Console|API Console
Google account: Google Account
Google accounts: Google Accounts
Googling: search with Google
HTTPs: HTTPS
k8s: Kubernetes
SHA1: SHA-1|HAS-SHA1
url: URL
World Wide Web: web
+68
View File
@@ -0,0 +1,68 @@
extends: substitution
message: "Use '%s' instead of '%s'."
link: "https://developers.google.com/style/word-list"
level: warning
# The case-insensitive half of the word list, so sentence-initial use is caught
# ('Touch the screen', not only 'touch the screen'). Entries that must stay
# case-sensitive are in WordList.yml.
ignorecase: true
action:
name: replace
swap:
"(?:API Console|dev|developer) key": API key
"(?:cell ?phone|smart ?phone)": phone|mobile phone
"(?:dev|developer|APIs) console": API console
"(?:e-mail|Email|E-mail)": email
"(?:file ?path|path ?name)": path
"(?:kill|terminate|abort)": stop|exit|cancel|end
# Longest form first: with the shortest alternative leading, 'OAuth 2' matched
# only 'OAuth', so applying the suggestion produced 'OAuth 2.0 2'. The rule is
# already case-insensitive, so the inline (?i) is redundant. See issue #41.
'\bOauth2\.0\b|\bOAuth ?2\b(?!\.0)|\bOauth\b(?! ?2)': OAuth 2.0
"(?:ok|Okay)": OK|okay
"(?:WiFi|wifi)": Wi-Fi
'[\.]+apk': APK
'3\-D': 3D
'Google (?:I\-O|IO)': Google I/O
"tap (?:&|and) hold": touch & hold
"un(?:check|select)": clear
above: preceding
account name: username
action bar: app bar
admin: administrator
a\.k\.a|aka: or|also known as
application: app
approx\.: approximately
autoupdate: automatically update
cellular data: mobile data
cellular network: mobile network
chapter: documents|pages|sections
check box: checkbox
click on: click|click in
content type: media type
curated roles: predefined roles
data are: data is
disabled?: turn off|off
ephemeral IP address: ephemeral external IP address
fewer data: less data
file name: filename
firewalls: firewall rules
functionality: capability|feature
grayed-out: unavailable
in order to: to
ingest: import|load
long press: touch & hold
network IP address: internal IP address
omnibox: address bar
open-source: open source
overview screen: recents screen
regex: regular expression
sign into: sign in to
'(?<!single )sign-?on': single sign-on
static IP address: static external IP address
stylesheet: style sheet
synch: sync
tablename: table name
tablet: device
'touch(?! ?(?:&|and) hold)': tap
vs\.: versus
+4
View File
@@ -0,0 +1,4 @@
{
"feed": "https://github.com/errata-ai/Google/releases.atom",
"vale_version": ">=1.0.0"
}
View File
+35 -3
View File
@@ -83,7 +83,6 @@ src/devx/
│ ├── classify_changes.py # User-facing vs infrastructure change detection
│ ├── detect_release_commit.py # Detect release commits on master
│ ├── validate_commit_msg.py # Conventional commit validation
│ ├── pr_review.py # Automated PR review + manual reviews (--event, --body, --checklist-confirmed)
│ ├── post_merge.py # Vikunja task updates after merge
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
│ ├── push_badges.py # Generate and push quality badges (--retries for retry on git push failures)
@@ -124,6 +123,10 @@ src/devx/
│ ├── check_docker_init.py # Check Docker Compose services with healthchecks have init: true
│ ├── check_ansible_set_fact_to_json.py # Check set_fact tasks don't misuse to_json
│ ├── check_alert_rules.py # Validate Prometheus alert rules with promtool
│ ├── check_ansible_no_log.py # Check Ansible tasks for missing no_log on secrets
│ ├── check_ansible_patterns.py # Detect dangerous failure-masking patterns
│ ├── check_jinja_expr.py # Validate Jinja2 expressions in Ansible files
│ ├── check_ansible_no_state_absent_on_db.py # Prevent state:absent on DB paths
│ └── _shared.py # Shared tool utilities
├── opentofu.py # OpenTofu output helpers (get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field)
├── utils/ # Shared utilities (reusable across projects)
@@ -141,8 +144,8 @@ src/devx/
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
├── discover_runners.py # Dynamic Gitea runner discovery
├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast (--roles-root)
├── molecule_all.py # Run all molecule scenarios locally
├── molecule_changed.py # Detect which Ansible roles changed and output molecule scenarios
├── start_docker.py # Ensure Docker daemon is running for molecule tests
└── platforms.py # Supported molecule platforms
```
@@ -154,8 +157,37 @@ src/devx/
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (NOT `.:src` since there are no scripts at repo root)
- **Config via env vars** — `DEVX_*` environment variables with `.env` file fallback
## Spec-Driven Development
Every change starts with a spec. No spec, no code.
**Workflow:**
1. Create Vikunja task → get `<PREFIX>-N` task ID
2. Write spec at `docs/specs/<TASK-ID>.md` (see template in `.devin/skills/spec-driven-development/SKILL.md`)
3. Create branch, implement with `# Implements: REQ-N` comments
4. Tick all acceptance criteria checkboxes in spec
5. Push and create PR — CI validates spec before expensive jobs
**CI gates (pre-merge):**
- `devx.ci.validate_spec` — checks spec exists, has required sections, REQ-IDs, all ACs checked
- `devx.ci.check_pr_size` — max 500 lines / 10 files (excludes CHANGELOG, badges, locks)
- `devx.ci.fast_molecule` — converge+verify only for changed roles, single platform
**Nightly (infra only):**
- Full molecule suite (all scenarios, all platforms) + staging deploy + integration tests
- On failure: sets `NIGHTLY_STATUS=failed`, blocks staging deploys
- Post-merge auto-deploy to staging checks this gate before deploying
**Post-merge:**
- Infra: auto-deploys to staging (if nightly gate is green)
- GRM/sso-bridge: auto-publishes package, auto-creates infra dependency PR to bump pinned version
**Skill:** `.devin/skills/spec-driven-development/SKILL.md` — full template and workflow details.
## PR Workflow (Mandatory)
Every change to master goes through this workflow. No exceptions.
### Branch Protection (Required Gitea Settings)
@@ -206,7 +238,7 @@ docs: update README
### 6. Review the PR
**Automated review (CI `validate` job):** Every PR triggers an automated
review via `python -m devx.ci.pr_review` as a step in the `validate` job.
review via the `pr-review` skill (agent-invoked, not a CI step).
This posts a review with
`COMMENT` (no issues) or `REQUEST_CHANGES` (issues found):
+88 -53
View File
@@ -2,73 +2,108 @@
All notable changes to this project will be documented in this file.
## [0.49.5] - 2026-08-07
### Performance
- Skip dep resolution in setup-image with --no-deps
## [0.49.4] - 2026-08-07
### Bug Fixes
- Add container.credentials for private registry auth
## [0.49.3] - 2026-08-07
### Bug Fixes
- Retry ansible-galaxy collection install on transient timeouts
## [0.49.2] - 2026-08-07
### Bug Fixes
- Add fallback URL for tea download
## [0.49.1] - 2026-08-07
### Bug Fixes
- Add container images to build-images workflow
## [0.49.0] - 2026-08-07
## [0.51.0] - 2026-08-25
### Features
- Add --include-roles and --exclude-roles to distribute_molecule
## [0.48.0] - 2026-07-22
- Add role defaults path to create_dependency_pr search
## [0.50.2] - 2026-08-24
### Bug Fixes
- Update check_pr_size usage example with --repo and --pr-number args
## [0.50.1] - 2026-08-15
### Bug Fixes
- Prefer rootless Docker socket over low-space inner DinD daemon
- Check /run/host-docker.sock for host Docker daemon
- Use /dev/shm/docker as data-root for inner dockerd
- Kill existing dockerd before starting /dev/shm/docker daemon
- Use /dev/shm/docker.sock socket for local dockerd
- Add --iptables=false to local dockerd
- Disable bridge and ip6tables for local dockerd
- Use host Docker when root dir is inaccessible (free=0)
- Prefer /run/host-docker.sock over inner dockerd
- Start local dockerd instead of using low-space inner dockerd
- Use container overlay for local dockerd data root
- Kill inner dockerd with SIGKILL, use alt socket if alive
- Kill dockerd by PID when pkill fails, use /dev/shm for alive daemon
- Trust /var/run/docker.sock with free=0 when no inner dockerd exists
## [0.50.0] - 2026-08-09
### Features
- Extract reusable components from infra and grm into devx
- Add 5 standalone lint scripts from infra
## [0.48.0] - 2026-07-22
## [0.49.0] - 2026-08-09
### Features
- Extract reusable components from infra and grm into devx
- Sync missing features from v0.49.x line to master
## [Unreleased]
## [0.48.2] - 2026-08-09
### Bug Fixes
- Remove dead translation keys and add missing one
## [0.48.1] - 2026-08-08
### Bug Fixes
- *(setup)* Extract version from filename for mirror installs
## [0.48.0] - 2026-08-08
### Features
- Extract reusable components from infra and grm into devx:
- `devx.utils.ui.say()` — unified click.echo + logging output
- `devx.utils.api.APIClient` — base HTTP API client class with retry logic
- `devx.utils.jinja` — Jinja2 environment helpers with Ansible-compatible filters
- `devx.i18n.configure_i18n()` — configurable `lang_env_var` and `translations_path_env_var`
- `devx.ci.cancel_superseded_runs` — cancel in-flight CI runs for the same PR branch
- `devx.ci.check_workflow_artifact_deps` — verify artifact download jobs depend on upload jobs
- `devx.ci.check_workflow_tofu_init` — verify tofu-state jobs have a tofu-init step
- `devx.tools.check_docker_init` — check Docker Compose services with healthchecks have init: true
- `devx.tools.check_ansible_set_fact_to_json` — check set_fact tasks don't misuse to_json
- `devx.tools.check_alert_rules` — validate Prometheus alert rules with promtool
- Add `jinja2` and `pyyaml` as core dependencies (previously in `deploy` extras only)
- Register new CLI commands: `devx ci cancel-superseded-runs`, `devx ci check-workflow-artifact-deps`,
`devx ci check-workflow-tofu-init`, `devx tools check-docker-init`,
`devx tools check-ansible-set-fact-to-json`, `devx tools check-alert-rules`
- Add Makefile targets for all new check tools
- *(setup)* Mirror Ansible collections from Gitea registry with auth
## [0.47.10] - 2026-08-05
### Bug Fixes
- Unique molecule container names per CI runner
## [0.47.9] - 2026-08-03
### Bug Fixes
- Unique molecule container names per CI runner
## [0.47.8] - 2026-08-03
### Bug Fixes
- Increase CI_SCALE_FACTOR default from 4 to 6
## [0.47.7] - 2026-08-03
### Bug Fixes
- Scale check_test_speed limits on CI runners
## [0.47.6] - 2026-08-03
### Bug Fixes
- Configure git auth in setup_image for git+https deps
## [0.47.5] - 2026-08-03
### Bug Fixes
- Push wiki to main branch instead of master
## [0.47.4] - 2026-08-03
### Bug Fixes
- Add User-Agent header to _download in install_tools
## [0.47.3] - 2026-07-17
+9 -14
View File
@@ -16,12 +16,12 @@ quality badges.
[![CI](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions/workflows/ci.yml/badge.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![License: GPL-3.0](https://img.shields.io/badge/license-GPL--3.0-blue)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/python.svg)](https://www.python.org/downloads/)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/python.svg)](https://www.python.org/downloads/)
## Why devx?
@@ -87,7 +87,7 @@ extra index and list devx in your dependencies:
```toml
[project]
dependencies = [
"devx>=0.49.5",
"devx>=0.51.0",
]
[tool.pip]
@@ -101,8 +101,8 @@ pip install -e .
```
> **Note:** If your project requires a specific devx version, pin it in
> `dependencies` (for example, `"devx==0.49.5"`) or use a version constraint
> (for example, `"devx>=0.49.5,<0.50"`).
> `dependencies` (for example, `"devx==0.51.0"`) or use a version constraint
> (for example, `"devx>=0.51.0,<0.52"`).
### Optional extras
@@ -226,10 +226,6 @@ python -m devx.molecule.distribute_molecule --runner-index 1 --max-runners 3
python -m devx.molecule.distribute_molecule --list # list all scenarios
python -m devx.molecule.distribute_molecule --list-platforms # list platforms
# Run molecule tests with cross-runner fail-fast
python -m devx.molecule.molecule_ci_guard pair1 pair2
python -m devx.molecule.molecule_ci_guard --roles-root ansible/roles pair1 pair2
# Run all molecule scenarios locally (sequential)
python -m devx.molecule.molecule_all
python -m devx.molecule.molecule_all --bin .venv/bin
@@ -303,7 +299,6 @@ devx --version
| `devx molecule all` | Run all molecule scenarios on all supported platforms |
| `devx molecule discover-runners` | Discover available Gitea Actions runners |
| `devx molecule distribute` | Distribute molecule test pairs across parallel runners |
| `devx molecule guard` | Run molecule tests with CI failure polling |
See [CLI Commands](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki/CLI-Commands)
in the wiki for full command documentation with examples.
+11 -11
View File
@@ -12,12 +12,12 @@ project to be reusable across all oblachno-oss repositories.
[![CI](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions/workflows/ci.yml/badge.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![License: GPL-3.0](https://img.shields.io/badge/license-GPL--3.0-blue)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/8c02351c3190af51b1dfe545a7abbec665c51559/python.svg)](https://www.python.org/downloads/)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/c35a676f16439b78b79bd9ba70ca8f19bf39e4bb/python.svg)](https://www.python.org/downloads/)
## Overview
@@ -74,14 +74,14 @@ Add devx to your `pyproject.toml` dependencies and configure the registry:
```toml
[project]
dependencies = [
"devx>=0.49.5",
"devx>=0.51.0",
]
[tool.pip]
extra-index-url = "https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple"
```
Pin a specific version if needed: `"devx==0.49.5"` or `"devx>=0.49.5,<0.50"`.
Pin a specific version if needed: `"devx==0.51.0"` or `"devx>=0.51.0,<0.52"`.
### Optional extras
@@ -104,8 +104,8 @@ devx is a self-contained Python package under `src/devx/`:
- **Dev tools** (`devx.tools`) — setup, install_tools, check_test_speed,
configure_repo, generate_badges, generate_cliff_config, install_checkmake
- **Molecule tools** (`devx.molecule`) — Optional, for projects with Ansible
roles: distribute_molecule, molecule_ci_guard, molecule_all, discover_runners,
start_docker, platforms
roles: distribute_molecule, molecule_all, discover_runners, start_docker,
platforms
See [Architecture](Architecture) for the full package structure, module
descriptions, design principles, and data flow diagrams.
@@ -132,7 +132,7 @@ devx provides a `devx` CLI with three command groups:
- `devx ci <command>` — CI/CD automation (17 commands)
- `devx tools <command>` — Developer tools (9 commands)
- `devx molecule <command>` — Molecule testing (4 commands, optional)
- `devx molecule <command>` — Molecule testing (3 commands, optional)
See [CLI Commands](CLI-Commands) for full command documentation with examples.
+46
View File
@@ -0,0 +1,46 @@
# DEVX-155: Replace pr_review with spec-driven CI gates and pr-review skill
## Problem
The `devx.ci.pr_review` module was a monolithic automated PR review tool that
ran in CI and posted COMMENT/REQUEST_CHANGES reviews. It duplicated logic now
better handled by an agent-invoked skill, and it blocked the introduction of
spec-driven development gates (validate_spec, check_pr_size) that should run
before expensive CI jobs.
## Approach
Remove `pr_review` and replace it with lightweight, focused CI gates plus a
new `pr-review` skill for deep agent-invoked reviews.
REQ-1: Add `devx.ci.validate_spec` — validates spec file exists, has required sections, REQ-IDs, all ACs checked
REQ-2: Add `devx.ci.check_pr_size` — enforces max 500 lines / 10 files (excludes CHANGELOG, badges, locks)
REQ-3: Add `devx.ci.fast_molecule` — detects changed roles, outputs fast molecule commands (converge+verify, single platform)
REQ-4: Add `devx.ci.nightly_gate` — checks/sets NIGHTLY_STATUS repo variable to block staging deploys on nightly failure
REQ-5: Add `devx.ci.create_dependency_pr` — auto-creates infra PR to bump pinned package version after grm/sso-bridge release
REQ-6: Remove `devx.ci.pr_review` module and `tests/unit/test_pr_review.py`
REQ-7: Update CI workflows to replace pr_review steps with validate_spec + check_pr_size + curl-based APPROVE
REQ-8: Add `spec-driven-development` and `pr-review` skills under `.devin/skills/`
REQ-9: Update AGENTS.md and skill docs to document the new spec-driven workflow
## Test Plan
- Unit tests for each new module (test_validate_spec, test_check_pr_size, test_fast_molecule, test_nightly_gate, test_create_dependency_pr, test_spec_driven_workflows)
- Remove test_pr_review.py and pr_review references from test_cli.py (pr_review.py deleted from source)
- Verify CI workflow YAML passes actionlint
## Deploy Plan
- Merge to master via auto-merge workflow
- devx post-merge publishes new version; downstream repos (grm, infra, sso-bridge) bump their devx pin
## Rollback Plan
- Revert the merge commit; downstream repos keep their current devx pin
- pr_review.py can be restored from git history if needed
## Acceptance Criteria
- [x] REQ-1: `devx.ci.validate_spec` module exists with `--branch` and `--github-output` options
- [x] REQ-2: `devx.ci.check_pr_size` module exists with `--base`, `--head`, `--github-output` options
- [x] REQ-3: `devx.ci.fast_molecule` module exists and outputs changed roles + commands
- [x] REQ-4: `devx.ci.nightly_gate` module exists with `--action check/set-passed/set-failed`
- [x] REQ-5: `devx.ci.create_dependency_pr` module exists with `--repo`, `--package`, `--new-version` options
- [x] REQ-6: The pr_review CI module and its test file are deleted from source tree
- [x] REQ-7: CI workflow uses validate_spec + check_pr_size + curl APPROVE instead of pr_review
- [x] REQ-8: `.devin/skills/spec-driven-development/SKILL.md` and `.devin/skills/pr-review/SKILL.md` exist
- [x] REQ-9: AGENTS.md documents spec-driven development workflow and pr-review skill
+34
View File
@@ -0,0 +1,34 @@
# DEVX-156: Fix commit message format and release new CI modules
## Problem
The DEVX-155 merge commit on master has an invalid format
('DEVX-155: Replace...' missing conventional commit type). This blocks
the post-merge release workflow's `validate_commit_msg` step, preventing
`validate_spec`, `check_pr_size`, `nightly_gate`, and `create_dependency_pr`
from being published to the Gitea PyPI registry. All downstream repos
(grm, infra, sso-bridge) are blocked — their CI fails with
`No module named devx.ci.validate_spec`.
## Approach
Add a trivial user-facing change (version doc comment) with a proper
conventional commit format to trigger the post-merge release workflow.
The release will publish the new CI modules that DEVX-155 introduced.
REQ-1: Add a user-facing change to src/devx/ to trigger release
REQ-2: Ensure the commit message follows conventional format (type: description)
## Test Plan
- Verify post-merge workflow runs successfully after merge
- Verify a new release tag is created (v0.51.0 or similar)
- Verify devx.ci.validate_spec is importable from the published package
## Deploy Plan
- Merge to master via auto-merge workflow
- Post-merge workflow auto-releases and publishes
## Rollback Plan
- Revert the merge commit if release fails
## Acceptance Criteria
- [x] REQ-1: A user-facing change is added to src/devx/
- [x] REQ-2: Commit message follows conventional format
+24
View File
@@ -0,0 +1,24 @@
# DEVX-157: Add role defaults path to create_dependency_pr search
## Problem
`create_dependency_pr` only searches `pyproject.toml` and the infra images vars file for pinned versions. The sso-bridge role pins its version in its role defaults file via `sso_bridge_version`, which is not searched.
## Approach
Add the sso-bridge role defaults path to the search paths.
REQ-1: Add ROLE_DEFAULTS_PATH constant pointing to the sso-bridge role defaults file
REQ-2: Include ROLE_DEFAULTS_PATH in the search loop
## Test Plan
- Verify existing tests pass
- Verify find_pinned_version finds sso_bridge_version in the defaults file
## Deploy Plan
- Merge to master, auto-release new devx version
## Rollback Plan
- Revert the merge commit
## Acceptance Criteria
- [x] REQ-1: ROLE_DEFAULTS_PATH constant added
- [x] REQ-2: search loop includes ROLE_DEFAULTS_PATH
+3 -11
View File
@@ -50,7 +50,6 @@ src/devx/
├── __init__.py
├── discover_runners.py # Dynamic Gitea runner discovery
├── distribute_molecule.py # Distribute scenarios across runners
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
├── molecule_all.py # Run all molecule scenarios locally
├── start_docker.py # Ensure Docker is available for molecule
└── platforms.py # Supported molecule platforms
@@ -300,9 +299,9 @@ Distributes files matching a glob pattern across N parallel runners
### `integration_guard.py`
Runs pytest with the same cross-runner failure detection mechanism used by
`molecule_ci_guard`. If any other integration-tests matrix runner reports
failure, the current pytest subprocess is killed and this runner exits early.
Runs pytest with cross-runner failure detection. A background thread polls
the Gitea API. If any other integration-tests matrix runner reports failure,
the current pytest subprocess is killed and this runner exits early.
## Developer tools (`devx.tools`)
@@ -382,13 +381,6 @@ the supported OS platform matrix. Supports `--roles-root` for multi-role
repositories, `--list` to list scenarios, and `--list-platforms` to list
platforms.
### `molecule_ci_guard.py`
Runs molecule tests sequentially while polling the Gitea API for other runner
failures. If any other molecule matrix runner reports failure, the current
molecule subprocess is killed and this runner exits early. Supports both
single-role (4-part) and multi-role (5-part) pair encoding.
### `molecule_all.py`
Runs all molecule scenarios on all supported OS platforms sequentially.
-9
View File
@@ -478,15 +478,6 @@ python -m devx.molecule.distribute_molecule --list
python -m devx.molecule.distribute_molecule --list-platforms
```
### `molecule_ci_guard.py`
Runs molecule tests sequentially while polling the Gitea API for other runner
failures. Aborts early if another runner fails the same job.
```bash
python -m devx.molecule.molecule_ci_guard [--roles-root <dir>] pair1 pair2 ...
```
### `validate_commit_msg.py`
Validates commit messages. On feature branches: conventional commits only
-26
View File
@@ -631,29 +631,3 @@ Options:
- `--list-platforms` — list all platforms, one per line
- `--roles-root <dir>` — roles root directory for multi-role repos (default:
`ansible/roles`)
### `devx molecule guard`
Run molecule tests sequentially with CI failure polling. A background thread
polls the Gitea API. If any other molecule matrix runner reports failure, the
current molecule subprocess is killed and this runner exits early with code 1.
```bash
devx molecule guard pair1 pair2 pair3
devx molecule guard --roles-root ansible/roles pair1 pair2
```
Each pair is encoded as:
- **Single-role (4-part):** `scenario|platform_name|platform_image|platform_command`
- **Multi-role (5-part):** `role|scenario|platform_name|platform_image|platform_command`
Options:
- `--roles-root <dir>` — roles root directory for multi-role repos
Environment variables:
- `GITEA_URL` — base URL of the Gitea instance
- `CI_GITEA_TOKEN` — API token with repo access
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
- `JOB_NAME` — base job name (`GITHUB_JOB`)
- `MATRIX_INDEX` — current matrix index (runner-index)
- `GITEA_REPOSITORY` — repository in `owner/repo` format
+2 -2
View File
@@ -48,12 +48,12 @@ Add devx to your `pyproject.toml`:
```toml
[project]
dependencies = [
"devx>=0.49.5",
"devx>=0.51.0",
]
[project.optional-dependencies]
dev = [
"devx>=0.49.5",
"devx>=0.51.0",
]
```
+8 -2
View File
@@ -1,3 +1,9 @@
"""devx — reusable development and CI/CD tools for oblachno-oss projects."""
"""devx — reusable development and CI/CD tools for oblachno-oss projects.
__version__ = "0.49.5"
Provides CI/CD automation (validate_spec, check_pr_size, nightly_gate,
create_dependency_pr, auto_merge, release, publish), developer tooling
(setup, install_tools, configure_repo, create_task, create_pr), and
molecule testing helpers for Ansible projects.
"""
__version__ = "0.51.0"
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env python3
# Implements: REQ-2
"""Check PR size and reject oversized PRs.
Enforces max lines changed and max files changed to keep PRs small
and deployable. Generated/excluded files are not counted.
PRs with the ``refactoring`` label bypass the size check large but
legitimate refactoring PRs that touch many files in a coordinated way.
Usage:
python -m devx.ci.check_pr_size --base origin/master --head HEAD \\
--repo oblachno-oss/grm --pr-number 123
In CI, pass ``--github-output`` to set ``pr-size-ok`` and ``pr-size-detail``
for downstream steps.
"""
from __future__ import annotations
import subprocess # nosec B404
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient
from devx.ci._shared import write_github_output
from devx.config import GITEA_API_URL
from devx.i18n import _
from devx.tokens import get_ci_token
load_dotenv()
# Files/patterns excluded from size counting (generated, badges, locks, etc.)
DEFAULT_EXCLUDED_PATTERNS = [
"CHANGELOG.md",
"README.md",
"docs/index.md",
"*.svg",
"uv.lock",
"poetry.lock",
"Pipfile.lock",
"package-lock.json",
"yarn.lock",
"go.sum",
]
DEFAULT_MAX_LINES = 500
DEFAULT_MAX_FILES = 10
REFACTORING_LABEL = "refactoring"
def has_refactoring_label(repo: str, pr_number: int) -> bool:
"""Check if a PR has the 'refactoring' label (bypasses size check)."""
try:
token = get_ci_token()
owner, repo_name = repo.split("/", 1)
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
pr = client.get_pr(pr_number)
labels = pr.get("labels", [])
return any(label.get("name") == REFACTORING_LABEL for label in labels)
except Exception:
return False
def get_diff_stats(base: str, head: str) -> list[tuple[str, int, int]]:
"""Get per-file diff stats (additions, deletions) between base and head.
Returns a list of (filename, additions, deletions) tuples.
"""
result = subprocess.run( # nosec B603 B607
["git", "diff", "--numstat", base, head],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise click.ClickException(_("git diff --numstat failed: {stderr}", stderr=result.stderr.strip()))
stats: list[tuple[str, int, int]] = []
for line in result.stdout.strip().split("\n"):
if not line:
continue
parts = line.split("\t")
if len(parts) != 3:
continue
additions_s, deletions_s, filename = parts
# Binary files show "-" for additions/deletions
additions = int(additions_s) if additions_s.isdigit() else 0
deletions = int(deletions_s) if deletions_s.isdigit() else 0
stats.append((filename, additions, deletions))
return stats
def is_excluded(filename: str, excluded_patterns: list[str]) -> bool:
"""Check if a filename matches any excluded pattern."""
from fnmatch import fnmatch
return any(fnmatch(filename, pat) for pat in excluded_patterns)
def check_size(
stats: list[tuple[str, int, int]],
max_lines: int,
max_files: int,
excluded_patterns: list[str],
) -> tuple[bool, str]:
"""Check diff stats against limits.
Returns (is_ok, detail_message).
"""
included = [(f, a, d) for f, a, d in stats if not is_excluded(f, excluded_patterns)]
total_lines = sum(a + d for _, a, d in included)
total_files = len(included)
if total_files == 0:
return True, "No non-excluded files changed"
if total_files > max_files:
return False, _(
"PR has {file_count} files changed (max {max_files}). Excluded: {excluded_count} files.",
file_count=total_files,
max_files=max_files,
excluded_count=len(stats) - total_files,
)
if total_lines > max_lines:
return False, _(
"PR has {line_count} lines changed (max {max_lines}). Excluded: {excluded_count} files.",
line_count=total_lines,
max_lines=max_lines,
excluded_count=len(stats) - total_files,
)
return True, _(
"PR size OK: {file_count} files, {line_count} lines (max {max_files} files, {max_lines} lines).",
file_count=total_files,
line_count=total_lines,
max_files=max_files,
max_lines=max_lines,
)
@click.command()
@click.option("--base", default="origin/master", help=_("Base ref for diff"))
@click.option("--head", default="HEAD", help=_("Head ref for diff"))
@click.option(
"--max-lines",
type=int,
default=DEFAULT_MAX_LINES,
help=_("Max lines changed (excluded files not counted)"),
)
@click.option(
"--max-files",
type=int,
default=DEFAULT_MAX_FILES,
help=_("Max files changed (excluded files not counted)"),
)
@click.option(
"--github-output",
"github_output",
is_flag=True,
default=False,
help=_("Write results to $GITHUB_OUTPUT"),
)
@click.option(
"--excluded",
"excluded",
multiple=True,
help=_("Additional excluded patterns (in addition to defaults)"),
)
@click.option("--repo", default=None, help=_("Repo (owner/name) for label check"))
@click.option("--pr-number", type=int, default=None, help=_("PR number for label check"))
def cli(
base: str,
head: str,
max_lines: int,
max_files: int,
github_output: bool,
excluded: tuple[str, ...],
repo: str | None,
pr_number: int | None,
) -> None:
"""Check PR size and reject oversized PRs."""
# Check for refactoring label bypass
if repo and pr_number and has_refactoring_label(repo, pr_number):
detail = _("PR has 'refactoring' label — size check bypassed.")
if github_output:
write_github_output("pr-size-ok", "true")
write_github_output("pr-size-detail", detail)
click.echo(f"[pr-size] {detail}")
return
excluded_patterns = list(DEFAULT_EXCLUDED_PATTERNS) + list(excluded)
stats = get_diff_stats(base, head)
is_ok, detail = check_size(stats, max_lines, max_files, excluded_patterns)
if github_output:
write_github_output("pr-size-ok", "true" if is_ok else "false")
write_github_output("pr-size-detail", detail)
if is_ok:
click.echo(f"[pr-size] {detail}")
else:
click.echo(f"[pr-size] FAILED: {detail}", err=True)
click.echo("", err=True)
click.echo("Oversized PRs cannot be reliably reviewed or deployed independently.", err=True)
click.echo("Split your work into smaller PRs, each addressing one concern.", err=True)
raise click.ClickException(_("PR size check failed."))
if __name__ == "__main__": # pragma: no cover
cli()
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env python3
# Implements: REQ-5
"""Auto-create an infra PR to bump a pinned dependency version.
After grm or sso-bridge publishes a new package version, this module
creates a PR in the infra repo to bump the pinned version in
``pyproject.toml`` or ``ansible/group_vars/all/images.yml``.
Reuses ``devx.tools.create_pr`` for PR creation and Vikunja task linking.
Usage:
python -m devx.ci.create_dependency_pr \
--repo oblachno/infra \
--package grm \
--new-version 0.5.2 \
--source-repo oblachno/grm \
--source-run-id 12345
"""
from __future__ import annotations
import re
import subprocess # nosec B404
from pathlib import Path
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
from devx.exceptions import APIError
from devx.i18n import _
from devx.tokens import get_ci_token, get_vikunja_token
from devx.tools.create_pr import find_existing_pr
load_dotenv()
# Where infra pins dependency versions
PYPROJECT_PATH = "pyproject.toml"
IMAGES_YML_PATH = "ansible/group_vars/all/images.yml"
ROLE_DEFAULTS_PATH = "ansible/roles/sso_bridge/defaults/main.yml"
def find_pinned_version(package: str, file_path: str) -> str | None:
"""Find the currently pinned version of a package in a file.
Looks for patterns like:
- ``"grm @ git+...@v0.5.1"``
- ``grm = "0.5.1"``
- ``grm_version: "0.5.1"``
- ``grm_image_version: "0.5.1"``
"""
path = Path(file_path)
if not path.exists():
return None
content = path.read_text(encoding="utf-8")
# Match various pinning patterns
patterns = [
rf"{package}\s*@\s*git\+[^@]+@v?([\d.]+)", # pip: package @ git+url@vX.Y.Z
rf'{package}\s*=\s*"([\d.]+)"', # pyproject: package = "X.Y.Z"
rf'{package}_version:\s*"([\d.]+)"', # ansible vars: package_version: "X.Y.Z"
rf'{package}_image_version:\s*"([\d.]+)"', # ansible vars: package_image_version: "X.Y.Z"
]
for pat in patterns:
match = re.search(pat, content)
if match:
return match.group(1)
return None
def update_pinned_version(file_path: str, package: str, old_version: str, new_version: str) -> bool:
"""Update the pinned version in a file. Returns True if changed."""
path = Path(file_path)
if not path.exists():
return False
content = path.read_text(encoding="utf-8")
# Replace old version with new version in package-related lines
patterns = [
(rf"({package}\s*@\s*git\+[^@]+@v?){old_version}", rf"\g<1>{new_version}"),
(rf'({package}\s*=\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
(rf'({package}_version:\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
(rf'({package}_image_version:\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
]
new_content = content
changed = False
for pat, replacement in patterns:
new_content, n = re.subn(pat, replacement, new_content)
if n > 0:
changed = True
if changed:
path.write_text(new_content, encoding="utf-8")
return changed
def create_vikunja_task(title: str, description: str) -> str | None:
"""Create a Vikunja task and return its identifier (e.g., OBL-INFRA-531)."""
try:
token = get_vikunja_token()
except click.ClickException:
return None
from devx.api_clients import VikunjaClient
client = VikunjaClient(VIKUNJA_API_URL, token)
task = client.create_task(VIKUNJA_PROJECT_ID, title=title, description=description)
return str(task.get("identifier", ""))
@click.command()
@click.option("--repo", default="oblachno/infra", help=_("Target repo (owner/name) to create PR in"))
@click.option("--package", required=True, help=_("Package name to bump (e.g., grm, sso-bridge)"))
@click.option("--new-version", required=True, help=_("New version to pin"))
@click.option("--source-repo", required=True, help=_("Source repo that published (owner/name)"))
@click.option("--source-run-id", default="", help=_("CI run ID that triggered the publish"))
@click.option("--dry-run", is_flag=True, default=False, help=_("Show what would be done without creating PR"))
def cli(
repo: str,
package: str,
new_version: str,
source_repo: str,
source_run_id: str,
dry_run: bool,
) -> None:
"""Create an infra PR to bump a pinned dependency version."""
token = get_ci_token()
if "/" not in repo:
raise click.ClickException(_("Invalid repo format: {repo}", repo=repo))
owner, repo_name = repo.split("/", 1)
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
# Find current pinned version
old_version = None
changed_file = None
for f in [PYPROJECT_PATH, IMAGES_YML_PATH, ROLE_DEFAULTS_PATH]:
old_version = find_pinned_version(package, f)
if old_version:
changed_file = f
break
if not old_version:
click.echo(_("[dep-pr] Could not find pinned version for {pkg} in infra repo.", pkg=package))
if dry_run:
return
raise click.ClickException(_("Could not find pinned version for {pkg}", pkg=package))
if old_version == new_version:
click.echo(_("[dep-pr] {pkg} already at {version} — no PR needed.", pkg=package, version=new_version))
return
click.echo(
_(
"[dep-pr] Bumping {pkg} from {old} to {new} in {file}",
pkg=package,
old=old_version,
new=new_version,
file=changed_file,
)
)
if dry_run:
click.echo(f"[dep-pr] DRY RUN: would update {changed_file} and create PR")
return
# Create a branch
branch_name = f"deps/{package}-{new_version}"
base_branch = "master"
# Check for existing PR (reuse from tools.create_pr)
existing = find_existing_pr(client, branch_name)
if existing:
click.echo(_("[dep-pr] PR already exists: #{number}", number=existing.get("number", "?")))
return
# Create branch via API
try:
master_ref = client._request("GET", "/git/refs/heads/master").json()
master_sha = master_ref.get("object", {}).get("sha", "")
if not master_sha:
raise click.ClickException("Could not get master SHA")
client._request("POST", "/git/refs", json={"ref": f"refs/heads/{branch_name}", "sha": master_sha})
except APIError as e:
if "already exists" in str(e).lower():
click.echo(f"[dep-pr] Branch {branch_name} already exists")
else:
raise click.ClickException(_("Failed to create branch: {error}", error=str(e))) from None
# Clone, update file, commit, push
subprocess.run(["git", "fetch", "origin", f"{branch_name}"], check=False, capture_output=True) # nosec B603 B607
subprocess.run(["git", "checkout", branch_name], check=False, capture_output=True) # nosec B603 B607
if not changed_file or not update_pinned_version(changed_file, package, old_version, new_version):
raise click.ClickException(_("Failed to update {file}", file=changed_file))
subprocess.run(["git", "add", changed_file], check=True) # nosec B603 B607
commit_msg = f"deps: bump {package} from {old_version} to {new_version}"
subprocess.run(["git", "commit", "-m", commit_msg], check=True) # nosec B603 B607
subprocess.run(["git", "push", "origin", branch_name], check=True) # nosec B603 B607
# Create Vikunja task for tracking
task_title = f"Bump {package} to {new_version}"
task_desc = (
f"<p>Auto-created dependency bump PR.</p>"
f"<p>Package: {package}</p>"
f"<p>Version: {old_version}{new_version}</p>"
f"<p>Source: {source_repo} (run #{source_run_id})</p>"
)
task_id = create_vikunja_task(task_title, task_desc)
# Create PR directly (dependency PRs have custom titles, not Vikunja-derived)
pr_title = f"{task_id}: {task_title}" if task_id else task_title
pr_body = (
f"## Dependency Bump\n\n"
f"Bumps **{package}** from `{old_version}` to `{new_version}`.\n\n"
f"- **Source**: {source_repo}\n"
f"- **Triggered by**: CI run #{source_run_id}\n"
f"- **Changed file**: `{changed_file}`\n\n"
f"This PR was auto-created by `devx.ci.create_dependency_pr`.\n"
)
if task_id:
pr_body += f"\nCloses {task_id}"
pr = client.create_pr(title=pr_title, head=branch_name, base=base_branch, body=pr_body)
click.echo(_("[dep-pr] Created PR #{number}: {title}", number=pr.get("number", "?"), title=pr_title))
if __name__ == "__main__": # pragma: no cover
cli()
-2
View File
@@ -44,7 +44,6 @@ REQUIRED_SCRIPTS = [
"auto_merge.py",
"release.py",
"publish.py",
"pr_review.py",
"notify_failure.py",
"post_merge.py",
"classify_changes.py",
@@ -52,7 +51,6 @@ REQUIRED_SCRIPTS = [
"detect_release_commit.py",
"push_badges.py",
"distribute_molecule.py",
"molecule_ci_guard.py",
"validate_commit_msg.py",
]
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
# Implements: REQ-3
"""Detect changed Ansible roles and output fast molecule test commands.
Reuses ``devx.molecule.molecule_changed`` for role detection (which handles
playbookrole mapping and shared infrastructure paths).
Fast molecule = converge + verify only, single platform, no idempotence
check. Used in pre-merge CI to get quick feedback on Ansible changes
without running the full molecule suite (which runs nightly).
Usage:
python -m devx.ci.fast_molecule --base origin/master --head HEAD
Outputs the list of changed roles and the molecule commands to run.
In CI, pass ``--github-output`` to set ``fast-molecule-roles`` (space-
separated) and ``fast-molecule-needed`` (true/false) for downstream steps.
"""
from __future__ import annotations
from pathlib import Path
import click
from dotenv import load_dotenv
from devx.ci._shared import write_github_output
from devx.i18n import _
from devx.molecule.molecule_changed import detect_changed_roles, get_changed_files
load_dotenv()
def get_molecule_scenarios(role_name: str, roles_dir: str = "ansible/roles") -> list[str]:
"""Get list of molecule scenario names for a role."""
mol_dir = Path(roles_dir) / role_name / "molecule"
if not mol_dir.is_dir():
return []
scenarios = []
for p in mol_dir.iterdir():
if p.is_dir() and (p / "molecule.yml").exists():
scenarios.append(p.name)
return sorted(scenarios)
def build_molecule_commands(
roles: set[str],
roles_dir: str = "ansible/roles",
platform: str = "ubuntu-2604",
) -> list[str]:
"""Build molecule test commands for changed roles.
For each role, runs each scenario with converge + verify only
(skip create/destroy between scenarios, skip idempotence).
"""
commands: list[str] = []
for role in sorted(roles):
scenarios = get_molecule_scenarios(role, roles_dir)
if not scenarios:
continue
for scenario in scenarios:
cmd = f"molecule test -s {scenario} --destroy=never --platform-name={platform}"
commands.append(cmd)
return commands
@click.command()
@click.option("--base", default="origin/master", help=_("Base ref for diff"))
@click.option("--head", default="HEAD", help=_("Head ref for diff"))
@click.option("--roles-dir", default="ansible/roles", help=_("Directory containing Ansible roles"))
@click.option("--platform", default="ubuntu-2604", help=_("Single platform to test against"))
@click.option(
"--github-output",
"github_output",
is_flag=True,
default=False,
help=_("Write results to $GITHUB_OUTPUT"),
)
def cli(
base: str,
head: str,
roles_dir: str,
platform: str,
github_output: bool,
) -> None:
"""Detect changed roles and output fast molecule test commands."""
# Use molecule_changed for role detection (handles playbooks, shared infra)
files = get_changed_files(base)
if not files:
click.echo("[fast-molecule] No files changed.")
if github_output:
write_github_output("fast-molecule-needed", "false")
write_github_output("fast-molecule-roles", "")
return
roles = detect_changed_roles(files)
if not roles:
click.echo("[fast-molecule] No Ansible roles changed.")
if github_output:
write_github_output("fast-molecule-needed", "false")
write_github_output("fast-molecule-roles", "")
return
commands = build_molecule_commands(roles, roles_dir, platform)
if github_output:
write_github_output("fast-molecule-needed", "true" if commands else "false")
write_github_output("fast-molecule-roles", " ".join(sorted(roles)))
click.echo(_("[fast-molecule] Changed roles: {roles}", roles=", ".join(sorted(roles))))
if not commands:
click.echo("[fast-molecule] No molecule scenarios found for changed roles.")
return
click.echo(f"[fast-molecule] {len(commands)} scenario(s) to run:")
for cmd in commands:
click.echo(f" {cmd}")
if __name__ == "__main__": # pragma: no cover
cli()
+51 -7
View File
@@ -1,10 +1,9 @@
#!/usr/bin/env python3
"""Run integration tests with cross-runner failure detection.
Wraps ``pytest`` with the same Gitea API polling mechanism used by
``molecule_ci_guard``. If any other integration-tests matrix runner
reports failure, the current pytest subprocess is killed and this runner
exits early with code 1.
Wraps ``pytest`` with Gitea API polling. If any other integration-tests
matrix runner reports failure, the current pytest subprocess is killed
and this runner exits early with code 1.
Usage::
@@ -35,17 +34,62 @@ import threading
import time
import click
import requests
from devx.config import REPO_NAME, REPO_OWNER
from devx.i18n import _
from devx.molecule.molecule_ci_guard import (
poll_for_other_failures,
)
from devx.tokens import get_ci_token
POLL_INTERVAL = 10
def get_running_jobs(gitea_url: str, owner: str, repo: str, token: str, run_id: int) -> list[dict]:
"""Return jobs for the given workflow run."""
url = f"{gitea_url}/api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/jobs"
headers = {"Authorization": f"token {token}"}
response = requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
data = response.json()
return data.get("jobs", [])
def any_other_runner_failed(jobs: list[dict], current_job_name: str, current_index: int) -> bool:
"""Return True if any other matrix job has failed."""
for job in jobs:
name = job.get("name", "")
if not name.startswith(current_job_name):
continue
if name == f"{current_job_name} ({current_index})" or name == current_job_name:
continue
if job.get("conclusion") == "failure":
return True
return False
def poll_for_other_failures(
gitea_url: str,
owner: str,
repo: str,
token: str,
run_id: int,
job_name: str,
current_index: int,
stop_event: threading.Event,
failed_event: threading.Event,
) -> None:
"""Background thread: poll API and signal if another runner fails."""
while not stop_event.is_set():
try:
jobs = get_running_jobs(gitea_url, owner, repo, token, run_id)
if any_other_runner_failed(jobs, job_name, current_index):
click.echo(_("Another runner failed. Stopping this runner early."))
failed_event.set()
return
except requests.RequestException as exc:
click.echo(_("API poll warning: {exc}", exc=exc))
stop_event.wait(POLL_INTERVAL)
@click.command(context_settings={"ignore_unknown_options": True})
@click.argument("pytest_args", nargs=-1, type=click.UNPROCESSED, required=True)
def cli(pytest_args: tuple[str, ...]) -> None:
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env python3
# Implements: REQ-4
"""Check if the nightly CI gate has passed; block staging deploys if it failed.
The nightly gate stores its status as a Gitea Actions repository variable
named ``NIGHTLY_STATUS`` on the infra repo. Values:
- ``passed`` nightly molecule + staging deploy + integration tests passed.
- ``failed:<run_id>`` nightly failed. Staging deploys are blocked until
the nightly passes again.
- (not set) nightly hasn't run yet. First deploy is allowed (bootstrap).
Usage:
python -m devx.ci.nightly_gate --repo oblachno/infra --action check
python -m devx.ci.nightly_gate --repo oblachno/infra --action set-passed --run-id 12345
python -m devx.ci.nightly_gate --repo oblachno/infra --action set-failed --run-id 12345
"""
from __future__ import annotations
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient
from devx.ci._shared import write_github_output
from devx.config import GITEA_API_URL
from devx.i18n import _
from devx.tokens import get_ci_token
load_dotenv()
NIGHTLY_STATUS_VAR = "NIGHTLY_STATUS"
def get_nightly_status(client: GiteaClient) -> str:
"""Get the nightly status variable. Returns empty string if not set."""
val = client.get_repo_variable(NIGHTLY_STATUS_VAR)
return val or ""
def set_nightly_status(client: GiteaClient, status: str) -> None:
"""Set the nightly status variable."""
client.set_repo_variable(NIGHTLY_STATUS_VAR, status)
@click.command()
@click.option("--repo", required=True, help=_("Repository in owner/name format"))
@click.option(
"--action",
type=click.Choice(["check", "set-passed", "set-failed"]),
required=True,
help=_("Action to perform"),
)
@click.option("--run-id", default="", help=_("CI run ID (for set-failed/set-passed)"))
@click.option(
"--github-output",
"github_output",
is_flag=True,
default=False,
help=_("Write results to $GITHUB_OUTPUT"),
)
def cli(repo: str, action: str, run_id: str, github_output: bool) -> None:
"""Check or set the nightly CI gate status."""
token = get_ci_token()
if "/" not in repo:
raise click.ClickException(_("Invalid repo format: {repo}. Expected owner/name.", repo=repo))
owner, repo_name = repo.split("/", 1)
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
if action == "check":
status = get_nightly_status(client)
if not status:
# Bootstrap: no nightly has run yet, allow deploy
click.echo("[nightly-gate] No nightly status set — allowing deploy (bootstrap).")
if github_output:
write_github_output("nightly-gate-passed", "true")
write_github_output("nightly-status", "")
return
if status.startswith("passed"):
click.echo("[nightly-gate] Nightly passed. Deploy allowed.")
if github_output:
write_github_output("nightly-gate-passed", "true")
write_github_output("nightly-status", status)
elif status.startswith("failed"):
run_part = status.split(":", 1)[1] if ":" in status else ""
run_link = f" (run #{run_part})" if run_part else ""
click.echo(
_(
"[nightly-gate] Nightly FAILED{run}. Staging deploys are blocked until nightly passes.",
run=run_link,
),
err=True,
)
if github_output:
write_github_output("nightly-gate-passed", "false")
write_github_output("nightly-status", status)
raise click.ClickException(_("Nightly gate failed — staging deploy blocked."))
else:
click.echo(f"[nightly-gate] Unknown nightly status: {status} — allowing deploy.")
if github_output:
write_github_output("nightly-gate-passed", "true")
write_github_output("nightly-status", status)
elif action == "set-passed":
set_nightly_status(client, f"passed:{run_id}" if run_id else "passed")
click.echo(_("[nightly-gate] Set NIGHTLY_STATUS=passed{run}", run=f":{run_id}" if run_id else ""))
if github_output:
write_github_output("nightly-status", f"passed:{run_id}" if run_id else "passed")
elif action == "set-failed":
set_nightly_status(client, f"failed:{run_id}" if run_id else "failed")
click.echo(_("[nightly-gate] Set NIGHTLY_STATUS=failed{run}", run=f":{run_id}" if run_id else ""))
if github_output:
write_github_output("nightly-status", f"failed:{run_id}" if run_id else "failed")
if __name__ == "__main__": # pragma: no cover
cli()
-715
View File
@@ -1,715 +0,0 @@
#!/usr/bin/env python3
"""Automated PR review: check architecture compliance, best practices, and quality.
Fetches the PR diff via the Gitea API, runs a series of automated checks,
and posts a structured review using GiteaClient.create_review.
Checks performed:
1. Architecture compliance no business logic in CLI, no direct subprocess
calls outside executor, no hardcoded config that should be in config.py
2. Best practices no bare except, no print() (use click.echo), no TODO/FIXME
left in merged code, no functions > 50 lines
3. Security no secrets in code, no shell=True, no eval/exec
4. i18n no raw English strings in click.echo() without _() wrapper
5. Resource management no open() without with statement, no subprocess without cleanup
6. Documentation new CLI commands documented, new modules in architecture.md
7. Test coverage 100% enforced by pytest-cov (checked in quality job)
8. Commit conventions conventional commit format on branch commits
Usage:
CI_GITEA_API_TOKEN=<token> [REVIEWER_GITEA_API_TOKEN=<token>] python3 -m devx.ci.pr_review <pr_number> <owner/repo>
"""
from __future__ import annotations
import os
import re
from dataclasses import dataclass, field
from typing import Any
import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL
from devx.exceptions import APIError
from devx.i18n import _
from devx.tokens import get_ci_token, get_reviewer_token
load_dotenv()
# Files that are exempt from certain checks
WORKFLOW_ONLY_SUFFIXES = (".yml", ".yaml", ".md", ".json", ".toml", ".cfg", ".ini", ".txt")
PYTHON_SUFFIX = ".py"
# Architecture rules
CLI_FILE = "src/devx/cli.py"
EXECUTOR_FILE = "src/devx/executor.py"
CONFIG_FILE = "src/devx/config.py"
# Patterns that indicate business logic in CLI (should be in runner_manager.py)
BUSINESS_LOGIC_IN_CLI = [
(r"subprocess\.(run|call|Popen|check_output|check_call)", "subprocess call in CLI — delegate to executor.py"),
(r"\bos\.system\b", "os.system call in CLI — delegate to executor.py"),
(r"\bansible-playbook\b", "ansible-playbook reference in CLI — delegate to executor.py"),
]
# Patterns that indicate bad practices
BAD_PRACTICES = [
(r"\bprint\s*\(", "print() found — use click.echo() for user output"),
(r"\beval\s*\(", "eval() found — security risk, avoid dynamic code execution"),
(r"\bexec\s*\(", "exec() found — security risk, avoid dynamic code execution"),
(r"shell\s*=\s*True", "shell=True found — security risk, use shell=False with list args"),
(r"except\s*:", "bare except found — catch specific exceptions"),
(r"except\s+Exception\s*:", "broad Exception catch — catch specific exceptions"),
(r"#\s*(TODO|FIXME|HACK|XXX)", "TODO/FIXME found — resolve before merging"),
]
# Patterns for hardcoded config values that should be in config.py
HARDCODED_CONFIG = [
(r"https?://[a-z]+\.[a-z]+\.[a-z]+", "hardcoded URL — move to config.py with env var override"),
]
@dataclass
class ReviewResult:
"""Result of automated review checks."""
issues: list[dict[str, Any]] = field(default_factory=list)
summary: list[str] = field(default_factory=list)
@property
def has_issues(self) -> bool:
return bool(self.issues)
def add_issue(self, file_path: str, line: int, message: str, severity: str = "warning") -> None:
self.issues.append(
{
"path": file_path,
"body": f"[{severity}] {message}",
"new_position": line,
}
)
def add_summary(self, text: str) -> None:
self.summary.append(text)
def is_python_file(path: str) -> bool:
"""Check if a file is a Python source file."""
return path.endswith(PYTHON_SUFFIX) and not path.startswith("tests/")
def is_workflow_only(path: str) -> bool:
"""Check if a file is workflow/config/docs only (not Python source)."""
return path.endswith(WORKFLOW_ONLY_SUFFIXES) or path.startswith((".gitea/", "docs/", "ansible/"))
def check_architecture_compliance(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that changes follow the documented architecture."""
for f in files:
path = f.get("filename", "")
if not is_python_file(path):
continue
patch = f.get("patch", "")
if not patch:
continue
lines = patch.split("\n")
current_line = 0
for line in lines:
if line.startswith("@@"):
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
# Check for business logic in CLI
if path == CLI_FILE:
for pattern, msg in BUSINESS_LOGIC_IN_CLI:
if re.search(pattern, content):
result.add_issue(path, current_line, msg, "error")
if not result.issues:
result.add_summary("- Architecture compliance: OK")
def check_best_practices(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check for common code quality issues."""
for f in files:
path = f.get("filename", "")
if not is_python_file(path):
continue
patch = f.get("patch", "")
if not patch:
continue
lines = patch.split("\n")
current_line = 0
for line in lines:
if line.startswith("@@"):
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
for pattern, msg in BAD_PRACTICES:
if re.search(pattern, content):
result.add_issue(path, current_line, msg, "warning")
if not any(i["body"].startswith("[warning]") for i in result.issues):
result.add_summary("- Best practices: OK")
def check_security(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check for security issues in changed files."""
for f in files:
path = f.get("filename", "")
if not is_python_file(path):
continue
patch = f.get("patch", "")
if not patch:
continue
lines = patch.split("\n")
current_line = 0
for line in lines:
if line.startswith("@@"):
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
# Check for hardcoded secrets
secret_re = r'(token|password|secret|key)\s*=\s*["\'][^"\']{8,}["\']' # nosec B105
is_secret = re.search(secret_re, content, re.IGNORECASE)
is_comment = content.strip().startswith("#")
is_example = "your-" in content or "example" in content
if is_secret and not is_comment and not is_example:
result.add_issue(
path,
current_line,
"potential hardcoded secret — use environment variable",
"error",
)
if not any(i["body"].startswith("[error]") and "secret" in i["body"] for i in result.issues):
result.add_summary("- Security: OK")
def check_i18n(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that user-facing strings are wrapped in _().
Detects ``click.echo()`` calls with raw string literals that are not
wrapped in ``_()``. Only checks ``src/`` files, not tests or scripts.
"""
# Pattern: click.echo("...") or click.echo(f"...") without _() wrapper
raw_echo_re = re.compile(r'click\.echo\s*\(\s*["\']([^"\']+)["\']')
raw_fstring_re = re.compile(r'click\.echo\s*\(\s*f["\']')
# Also check click.ClickException and raise with string
raw_exception_re = re.compile(r'click\.ClickException\s*\(\s*["\']([^"\']+)["\']')
for f in files:
path = f.get("filename", "")
if not is_python_file(path) or not path.startswith("src/"):
continue
patch = f.get("patch", "")
if not patch:
continue
lines = patch.split("\n")
current_line = 0
for line in lines:
if line.startswith("@@"):
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
# Skip comments and docstrings
stripped = content.strip()
if stripped.startswith("#") or stripped.startswith('"""') or stripped.startswith("'''"):
continue
# Check for raw strings in click.echo without _()
for regex, msg in [
(raw_echo_re, "click.echo() with raw string — wrap in _() for i18n"),
(raw_fstring_re, "click.echo() with f-string — wrap in _() for i18n"),
(raw_exception_re, "ClickException with raw string — wrap in _() for i18n"),
]:
if regex.search(content):
result.add_issue(path, current_line, msg, "warning")
if not any("i18n" in i["body"] for i in result.issues):
result.add_summary("- i18n: OK")
def check_resource_management(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check for resource leaks: open() without with, subprocess without cleanup.
Detects:
- ``open()`` calls not in a ``with`` statement
- ``subprocess.Popen()`` without ``.wait()`` or ``.communicate()``
"""
# Pattern: open("...") not preceded by "with" on the same line
open_re = re.compile(r"(?<!with\s)\bopen\s*\(")
popen_re = re.compile(r"subprocess\.Popen\s*\(")
for f in files:
path = f.get("filename", "")
if not is_python_file(path):
continue
patch = f.get("patch", "")
if not patch:
continue
lines = patch.split("\n")
current_line = 0
for line in lines:
if line.startswith("@@"):
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
# Skip comments
if content.strip().startswith("#"):
continue
# Check for open() without with
if open_re.search(content) and "with " not in content:
result.add_issue(
path, current_line, "open() without with statement — potential resource leak", "warning"
)
# Check for Popen without communicate/wait on same line
if popen_re.search(content) and ".communicate" not in content and ".wait" not in content:
result.add_issue(
path,
current_line,
"subprocess.Popen() without immediate .communicate() or .wait() — ensure cleanup",
"warning",
)
if not any("resource" in i["body"].lower() for i in result.issues):
result.add_summary("- Resource management: OK")
def check_function_length(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that no new function is excessively long (> 50 lines)."""
for f in files:
path = f.get("filename", "")
if not is_python_file(path):
continue
patch = f.get("patch", "")
if not patch:
continue
# Count consecutive added lines within a function
lines = patch.split("\n")
current_line = 0
func_start = 0
func_name = ""
added_in_func = 0
for line in lines:
if line.startswith("@@"):
if func_name and added_in_func > 50:
result.add_issue(
path,
func_start,
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
"warning",
)
match = re.search(r"\+(\d+)", line)
if match:
current_line = int(match.group(1)) - 1
func_name = ""
added_in_func = 0
continue
if line.startswith("+") and not line.startswith("+++"):
current_line += 1
content = line[1:]
func_match = re.match(r"\s*def\s+(\w+)\s*\(", content)
if func_match:
if func_name and added_in_func > 50:
result.add_issue(
path,
func_start,
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
"warning",
)
func_name = func_match.group(1)
func_start = current_line
added_in_func = 0
else:
added_in_func += 1
elif line.startswith(" ") or line.startswith("-"):
pass # context or removed line
# Check last function
if func_name and added_in_func > 50:
result.add_issue(
path,
func_start,
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
"warning",
)
def check_documentation(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that documentation is updated for relevant changes."""
has_src_changes = any(
is_python_file(f.get("filename", "")) and f.get("filename", "").startswith("src/") for f in files
)
has_doc_changes = any(
f.get("filename", "").startswith("docs/") or f.get("filename", "") in ("README.md", "AGENTS.md", "CHANGELOG.md")
for f in files
)
has_ansible_changes = any(f.get("filename", "").startswith("ansible/") for f in files)
has_tofu_changes = any(f.get("filename", "").startswith("tofu/") for f in files)
has_workflow_changes = any(f.get("filename", "").startswith(".gitea/") for f in files)
# Check for TODO/FIXME in changed docs
todo_issues: list[str] = []
for f in files:
filename = f.get("filename", "")
if filename.endswith(".md") and filename.startswith(("docs/", "README", "AGENTS")):
# Can't check file content from PR API easily, but flag if patch adds TODO
patch = f.get("patch", "")
if patch and re.search(r"^\+.*\b(TODO|FIXME|HACK|XXX)\b", patch, re.IGNORECASE):
todo_issues.append(f"{filename}: new TODO/FIXME added in documentation")
if has_src_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — source files changed but no docs updated")
elif has_ansible_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — Ansible role changed but no docs updated")
elif has_tofu_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — OpenTofu changes but no docs updated")
elif has_workflow_changes and not has_doc_changes:
result.add_summary("- Documentation: INFO — workflow changes (consider updating CI docs if behavior changed)")
else:
result.add_summary("- Documentation: OK")
if todo_issues:
for issue in todo_issues:
result.add_summary(f"- Documentation: WARNING — {issue}")
def check_test_coverage(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that tests are updated for source changes."""
has_src_changes = any(
is_python_file(f.get("filename", "")) and f.get("filename", "").startswith("src/") for f in files
)
has_test_changes = any(f.get("filename", "").startswith("tests/") for f in files)
if has_src_changes and not has_test_changes:
result.add_summary("- Tests: WARNING — source files changed but no test files updated")
else:
result.add_summary("- Tests: OK")
def check_commit_conventions(client: GiteaClient, pr_number: str, result: ReviewResult) -> None:
"""Check that PR commits follow conventional commit format.
Verifies that at least one commit on the PR branch matches the
conventional commit pattern (type: description). Merge commits
and revert commits are exempt.
"""
try:
commits = client.get_pr_commits(pr_number)
except APIError as e:
result.add_summary(f"- Commit conventions: ERROR — could not fetch commits: {e.message}")
return
if not commits:
result.add_summary("- Commit conventions: OK (no commits to check)")
return
from devx.config import CONVENTIONAL_RE
has_conventional = False
non_conventional: list[str] = []
for commit in commits:
commit_info = commit.get("commit", {})
message = str(commit_info.get("message", "") if isinstance(commit_info, dict) else "").split("\n")[0]
# Skip merge commits and revert commits
if message.startswith(("Merge", "Revert")):
continue
if CONVENTIONAL_RE.match(message):
has_conventional = True
else:
non_conventional.append(message[:60])
if has_conventional:
result.add_summary("- Commit conventions: OK")
elif non_conventional:
result.add_summary(
f"- Commit conventions: WARNING — no conventional commit found. "
f"Non-conventional commits: {', '.join(non_conventional[:3])}"
)
else:
result.add_summary("- Commit conventions: OK (all commits are merges/reverts)")
def run_review(client: GiteaClient, pr_number: str) -> ReviewResult:
"""Run all review checks and return the result."""
result = ReviewResult()
try:
files = client.get_pr_files(pr_number)
except APIError as e:
result.add_summary(f"- ERROR: Could not fetch PR files: {e.message}")
return result
if not files:
result.add_summary("- No files changed in this PR")
return result
# Run all checks
check_architecture_compliance(files, result)
check_best_practices(files, result)
check_security(files, result)
check_i18n(files, result)
check_resource_management(files, result)
check_function_length(files, result)
check_documentation(files, result)
check_test_coverage(files, result)
check_commit_conventions(client, pr_number, result)
return result
def build_review_body(result: ReviewResult) -> str:
"""Build the review body text from the review result."""
lines = ["## Automated PR Review", ""]
for item in result.summary:
lines.append(item)
if result.issues:
lines.append("")
lines.append(f"**{len(result.issues)} issue(s) found:**")
lines.append("")
for issue in result.issues:
lines.append(f"- `{issue['path']}:{issue['new_position']}` — {issue['body']}")
else:
lines.append("")
lines.append("No issues found by automated checks.")
lines.append("")
lines.append("---")
lines.append("**Auto-merge:** If all CI checks pass, this PR will be merged automatically.")
return "\n".join(lines)
def post_review(client: GiteaClient, pr_number: str, result: ReviewResult) -> dict[str, Any]:
"""Post the review to the PR.
Uses REQUEST_CHANGES when issues are found, COMMENT otherwise.
Never uses APPROVE the bot shares the PR author's token, so
Gitea rejects self-approval. The actual APPROVE must come from
the manual review step.
"""
body = build_review_body(result)
event = "REQUEST_CHANGES" if result.has_issues else "COMMENT"
comments = result.issues if result.has_issues else []
return client.create_review(pr_number, event=event, body=body, comments=comments)
def _post_manual_review(
client: GiteaClient,
pr_number: str,
event: str,
body: str | None,
checklist_confirmed: bool,
checklist_categories: str | None,
dry_run: bool,
owner: str | None = None,
repo_name: str | None = None,
) -> None:
"""Post a manual review with validation for APPROVE events.
When self-approval is rejected (reviewer token belongs to PR author),
falls back to the CI token (different user) if available.
"""
if not body or len(body) < 50:
raise click.ClickException(_("Review body must be at least 50 characters."))
if event == "APPROVE":
if not checklist_confirmed:
raise click.ClickException(
_("--checklist-confirmed is required for APPROVE events."),
)
cats = [c.strip() for c in (checklist_categories or "").split(",") if c.strip()]
cat_nums: list[int] = []
for c in cats:
try:
cat_nums.append(int(c))
except ValueError:
raise click.ClickException(
_("Invalid checklist category: {cat}. Must be numbers.", cat=c),
) from None
if len(cat_nums) < 8:
raise click.ClickException(
_("--checklist-categories must list at least 8 of 13 categories. Got {count}.", count=len(cat_nums)),
)
click.echo(f"Manual review event: {event}")
click.echo(f"Body: {body[:80]}...")
if checklist_confirmed:
click.echo(f"Checklist confirmed: {checklist_categories}")
if dry_run:
click.echo("\n[dry-run] Review not posted.")
return
try:
review = client.create_review(pr_number, event=event, body=body)
except APIError as e:
if "approve" in e.message.lower() or "422" in str(e.status):
# Self-approval not allowed (reviewer token belongs to PR author).
# Fall back to CI token (different user) if available.
ci_token = os.environ.get("CI_GITEA_API_TOKEN", "").strip()
if ci_token and owner and repo_name:
click.echo(_("Note: Self-approval not allowed with reviewer token. Retrying with CI token."))
ci_client = GiteaClient(GITEA_API_URL, ci_token, owner, repo_name)
try:
review = ci_client.create_review(pr_number, event=event, body=body)
except APIError:
click.echo(_("Note: CI token also cannot approve. Posting COMMENT instead."))
review = client.create_review(pr_number, event="COMMENT", body=body)
else:
click.echo(_("Note: Self-approval not allowed. Posting COMMENT instead."))
review = client.create_review(pr_number, event="COMMENT", body=body)
else:
raise
review_id = review.get("id", "?")
click.echo(
_(
"\nReview #{review_id} posted on PR #{pr_number} with event '{event}'.",
review_id=review_id,
pr_number=pr_number,
event=event,
)
)
@click.command()
@click.argument("pr_number")
@click.argument("repo")
@click.option("--dry-run", is_flag=True, default=False, help="Print review without posting.")
@click.option(
"--event",
type=click.Choice(["APPROVE", "REQUEST_CHANGES", "COMMENT"], case_sensitive=False),
default=None,
help="Post a manual review with the given event (skips automated checks).",
)
@click.option("--body", default=None, help="Review body text (required with --event).")
@click.option(
"--checklist-confirmed",
is_flag=True,
default=False,
help="Attest that REVIEW_CHECKLIST.md categories were checked (required for APPROVE).",
)
@click.option(
"--checklist-categories",
default=None,
help="Comma-separated checklist category numbers (required for APPROVE, min 8 of 13).",
)
def main(
pr_number: str,
repo: str,
dry_run: bool,
event: str | None,
body: str | None,
checklist_confirmed: bool,
checklist_categories: str | None,
) -> None:
"""Run automated PR review and post results to Gitea.
Without --event: runs automated checks and posts COMMENT/REQUEST_CHANGES.
With --event: posts a manual review (skips automated checks).
"""
try:
token = get_reviewer_token() if (event and event.upper() == "APPROVE") else get_ci_token()
except click.ClickException:
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set.")) from None
owner, repo_name = repo.split("/")
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
if event is not None:
_post_manual_review(
client,
pr_number,
event.upper(),
body,
checklist_confirmed,
checklist_categories,
dry_run,
owner=owner,
repo_name=repo_name,
)
return
result = run_review(client, pr_number)
body = build_review_body(result)
event = "REQUEST_CHANGES" if result.has_issues else "COMMENT"
click.echo(f"Review event: {event}")
click.echo(f"Issues found: {len(result.issues)}")
click.echo("")
click.echo(body)
if dry_run:
click.echo("\n[dry-run] Review not posted.")
return
try:
review = post_review(client, pr_number, result)
except APIError as e:
if "approve" in e.message.lower() or "422" in str(e.status):
click.echo(_("Note: Self-approval not allowed. Posting COMMENT instead."))
review = client.create_review(pr_number, event="COMMENT", body=body)
else:
raise
review_id = review.get("id", "?")
click.echo(
_(
"\nReview #{review_id} posted on PR #{pr_number} with event '{event}' ({num_comments} inline comments).",
review_id=review_id,
pr_number=pr_number,
event=event,
num_comments=len(result.issues),
)
)
if __name__ == "__main__": # pragma: no cover
main()
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env python3
# Implements: REQ-1
"""Validate that a PR has a spec file with required sections and acceptance criteria.
Spec-driven development gate. Runs in CI before expensive jobs.
Used by grm, infra, sso-bridge, and devx itself.
Validates:
1. A spec file exists at ``docs/specs/<TASK-ID>.md`` (TASK-ID extracted from branch).
2. The spec contains required sections: Problem, Approach, Test Plan, Deploy Plan, Rollback Plan.
3. The spec contains REQ-ID lines (``REQ-N: <description>``).
4. The spec contains an Acceptance Criteria checklist with at least one item.
5. All acceptance criteria checkboxes are checked (``- [x]``).
Usage:
python -m devx.ci.validate_spec --branch OBL-INFRA-531-fix-foo
In CI, also pass ``--github-output`` to set ``spec-valid`` and ``spec-path``
for downstream steps.
"""
from __future__ import annotations
import re
from pathlib import Path
import click
from dotenv import load_dotenv
from devx.ci._shared import extract_task_id, write_github_output
from devx.i18n import _
load_dotenv()
REQUIRED_SECTIONS = [
"## Problem",
"## Approach",
"## Test Plan",
"## Deploy Plan",
"## Rollback Plan",
"## Acceptance Criteria",
]
REQ_ID_RE = re.compile(r"^REQ-\d+:\s+.+", re.MULTILINE)
AC_CHECKED_RE = re.compile(r"^\s*- \[x\]\s+.+", re.MULTILINE)
AC_UNCHECKED_RE = re.compile(r"^\s*- \[ \]\s+.+", re.MULTILINE)
def find_spec_file(task_id: str, specs_dir: str = "docs/specs") -> Path | None:
"""Find the spec file for the given task ID.
Looks for ``docs/specs/<TASK-ID>.md`` (case-insensitive filename).
Returns the Path if found, None otherwise.
"""
base = Path(specs_dir)
if not base.is_dir():
return None
# Exact match (case-insensitive)
for p in base.glob("*.md"):
if p.stem.upper() == task_id.upper():
return p
return None
def validate_spec_content(content: str) -> list[str]:
"""Validate spec content and return a list of error messages.
Returns an empty list if the spec is valid.
"""
errors: list[str] = []
# Check required sections
for section in REQUIRED_SECTIONS:
if section not in content:
errors.append(_("Missing required section: {section}", section=section))
# Check for at least one REQ-ID
req_ids = REQ_ID_RE.findall(content)
if not req_ids:
errors.append(_("No REQ-ID lines found. Each requirement must be labeled (e.g., 'REQ-1: <description>')."))
# Check acceptance criteria has at least one item
checked = AC_CHECKED_RE.findall(content)
unchecked = AC_UNCHECKED_RE.findall(content)
if not checked and not unchecked:
errors.append(_("Acceptance Criteria section has no checklist items. Add at least one '- [ ] item'."))
elif unchecked:
errors.append(
_(
"Acceptance Criteria has {count} unchecked item(s). All AC items must be checked (- [x]) before merge.",
count=len(unchecked),
)
)
return errors
@click.command()
@click.option("--branch", required=True, help=_("Branch name (e.g., OBL-INFRA-531-fix-foo)"))
@click.option("--specs-dir", default="docs/specs", help=_("Directory containing spec files"))
@click.option(
"--github-output",
"github_output",
is_flag=True,
default=False,
help=_("Write results to $GITHUB_OUTPUT"),
)
@click.option("--allow-missing", is_flag=True, default=False, help=_("Allow missing spec (warn only, don't fail)"))
def cli(branch: str, specs_dir: str, github_output: bool, allow_missing: bool) -> None:
"""Validate that a spec file exists and has required content."""
task_id = extract_task_id(branch)
if not task_id:
msg = _("No task ID found in branch name '{branch}'. Expected format: <PREFIX>-N-description.", branch=branch)
if allow_missing:
click.echo(f"WARNING: {msg}")
if github_output:
write_github_output("spec-valid", "false")
write_github_output("spec-path", "")
return
raise click.ClickException(msg)
spec_path = find_spec_file(task_id, specs_dir)
if spec_path is None:
msg = _(
"No spec file found for task {task_id} in {dir}/. Expected: {dir}/{task_id}.md",
task_id=task_id,
dir=specs_dir,
)
if allow_missing:
click.echo(f"WARNING: {msg}")
if github_output:
write_github_output("spec-valid", "false")
write_github_output("spec-path", "")
return
raise click.ClickException(msg)
content = spec_path.read_text(encoding="utf-8")
errors = validate_spec_content(content)
if github_output:
write_github_output("spec-valid", "true" if not errors else "false")
write_github_output("spec-path", str(spec_path))
if errors:
click.echo("", err=True)
click.echo("=" * 60, err=True)
click.echo(f"Spec validation FAILED for {spec_path}:", err=True)
click.echo("=" * 60, err=True)
for e in errors:
click.echo(f" - {e}", err=True)
raise click.ClickException(_("Spec validation failed."))
click.echo(_("[spec-check] Spec validated: {path}", path=spec_path))
if __name__ == "__main__": # pragma: no cover
cli()
-14
View File
@@ -116,13 +116,6 @@ def ci_post_merge(args: tuple[str, ...]) -> None:
_run_module("devx.ci.post_merge", list(args))
@ci.command("pr-review")
@click.argument("args", nargs=-1)
def ci_pr_review(args: tuple[str, ...]) -> None:
"""Run automated PR review."""
_run_module("devx.ci.pr_review", list(args))
@ci.command("publish")
@click.argument("args", nargs=-1)
def ci_publish(args: tuple[str, ...]) -> None:
@@ -301,13 +294,6 @@ def molecule_discover_runners(args: tuple[str, ...]) -> None:
_run_module("devx.molecule.discover_runners", list(args))
@molecule.command("guard")
@click.argument("args", nargs=-1)
def molecule_guard(args: tuple[str, ...]) -> None:
"""Run molecule tests sequentially with CI failure polling."""
_run_module("devx.molecule.molecule_ci_guard", list(args))
@molecule.command("all")
@click.argument("args", nargs=-1)
def molecule_all(args: tuple[str, ...]) -> None:
+1 -11
View File
@@ -109,7 +109,7 @@ devx-ensure-venv:
fi
.PHONY: devx-create-task devx-create-pr devx-push devx-push-with-pr devx-check-config
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-pr-review devx-rebase devx-pr-rebase
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-rebase devx-pr-rebase
.PHONY: devx-configure-gitea-pypi devx-install-tools devx-install-checkmake devx-checkmake
.PHONY: devx-workflow-lint devx-workflow-dryrun devx-workflow-dryrun-safe devx-workflow-check
.PHONY: devx-notify-failure devx-install-hooks devx-activate-scripts devx-venv devx-ensure-venv
@@ -171,16 +171,6 @@ devx-pr-label:
$(if $(PR),--pr $(PR)) \
--label $(or $(LABEL),ready-to-merge)
# Usage: make devx-pr-review PR=42 EVENT=APPROVE BODY="..." CHECKLIST=1,2,3,4,5,6,7,8,9,10,11,12,13
# make devx-pr-review PR=42 EVENT=REQUEST_CHANGES BODY="..."
# make devx-pr-review PR=42 (auto review)
devx-pr-review:
@$(DEVX_PYTHON) -m devx.ci.pr_review \
$(PR) $(DEVX_REPO_OWNER)/$(DEVX_REPO_NAME) \
$(if $(EVENT),--event $(EVENT)) \
$(if $(BODY),--body "$(BODY)") \
$(if $(CHECKLIST),--checklist-confirmed --checklist-categories $(CHECKLIST))
# Rebase current branch onto origin/master and force-push
# Usage: make devx-rebase
# make devx-rebase NO_PUSH=1
+158
View File
@@ -0,0 +1,158 @@
"""Detect which Ansible roles changed and output their molecule scenarios.
Usage::
python -m devx.molecule.molecule_changed --print-targets
python -m devx.molecule.molecule_changed --base origin/master --print-roles
Outputs the list of make targets (e.g. molecule-docker-base) for roles
that have changed files vs the base ref. Used by ``make molecule-changed``
to run only the molecule scenarios affected by the current diff.
Role-to-target mapping is derived from the directory structure:
ansible/roles/<role>/ molecule-<role>
For roles with multiple scenarios (e.g. app_container has customer-apps,
nextcloud, postgres-upgrade, simple-app), the base target runs all
scenarios for that role.
Playbooks that change also trigger molecule for the roles they include.
Shared infrastructure changes (ansible.cfg, requirements.yml, molecule/)
trigger all scenarios.
"""
from __future__ import annotations
import subprocess # nosec B404 — used to run git, a trusted binary
from pathlib import Path
import click
REPO_ROOT = Path.cwd()
# Map role names to make targets.
ROLE_TARGET_MAP: dict[str, str] = {
"app_container": "molecule-app-container",
"app_hardening": "molecule-app-hardening",
"crowdsec": "molecule-crowdsec",
"disk_cleanup": "molecule-disk-cleanup",
"docker_base": "molecule-docker-base",
"observability": "molecule-observability",
"restore": "molecule-restore",
"sso_config": "molecule-sso-config",
"storage": "molecule-storage",
"zitadel": "molecule-zitadel",
}
# Playbooks that map to molecule scenarios (via roles they include).
PLAYBOOK_ROLE_MAP: dict[str, list[str]] = {
"ansible/playbooks/deploy-observability.yml": ["observability", "docker_base", "zitadel", "crowdsec"],
"ansible/playbooks/deploy-customer.yml": ["app_container", "docker_base", "app_hardening", "sso_config"],
"ansible/playbooks/configure-oidc.yml": ["sso_config", "app_container"],
"ansible/playbooks/prepare-vms.yml": ["docker_base", "app_hardening", "storage", "disk_cleanup", "crowdsec"],
}
# Shared infrastructure that affects all molecule tests.
SHARED_PATHS = (
"ansible/ansible.cfg",
"ansible/requirements.yml",
"ansible/molecule/",
)
# Minimum path parts for a role file: ansible/roles/<role> (3 parts).
# Files inside the role have more parts, but we only need the role name.
_MIN_ROLE_PATH_PARTS = 3
def _run_git(args: list[str]) -> str: # pragma: no cover
"""Run a git command and return stdout."""
result = subprocess.run( # nosec
["git", *args],
cwd=REPO_ROOT,
capture_output=True,
text=True,
check=False,
)
return result.stdout
def get_changed_files(base: str) -> list[str]:
"""Get list of changed files vs base ref."""
for ref in [base, "master"]:
output = _run_git(["diff", "--name-only", f"{ref}...HEAD"])
if output.strip():
return sorted(output.strip().splitlines())
return []
def detect_changed_roles(changed_files: list[str]) -> set[str]:
"""Detect which roles have changed files."""
roles: set[str] = set()
for filepath in changed_files:
# Check if file is in a role directory
if filepath.startswith("ansible/roles/"):
parts = filepath.split("/")
if len(parts) >= _MIN_ROLE_PATH_PARTS:
roles.add(parts[2])
# Check if file is a playbook that maps to roles
if filepath in PLAYBOOK_ROLE_MAP:
roles.update(PLAYBOOK_ROLE_MAP[filepath])
# Check shared infrastructure — triggers all roles
for shared in SHARED_PATHS:
if filepath.startswith(shared):
return set(ROLE_TARGET_MAP.keys())
return roles
def roles_to_targets(roles: set[str]) -> list[str]:
"""Convert role names to make targets."""
targets = []
for role in sorted(roles):
target = ROLE_TARGET_MAP.get(role)
if target:
targets.append(target)
return targets
@click.command()
@click.option(
"--base",
default="origin/master",
help="Base ref to compare against (default: origin/master).",
)
@click.option(
"--print-targets",
is_flag=True,
help="Print make targets (e.g. molecule-docker-base).",
)
@click.option(
"--print-roles",
is_flag=True,
help="Print role names (default if no --print-targets).",
)
def main(base: str, print_targets: bool, print_roles: bool) -> None:
"""Detect which Ansible roles changed and output molecule scenarios."""
changed_files = get_changed_files(base)
if not changed_files:
click.echo("No changed files detected.", err=True)
return
roles = detect_changed_roles(changed_files)
if not roles:
click.echo("No molecule scenarios affected by changes.", err=True)
return
if print_targets:
for target in roles_to_targets(roles):
click.echo(target)
else:
for role in sorted(roles):
click.echo(role)
if __name__ == "__main__": # pragma: no cover
main()
-333
View File
@@ -1,333 +0,0 @@
#!/usr/bin/env python3
"""Run molecule tests sequentially while polling Gitea for other runner failures.
Each pair is encoded as one of:
- **Single-role (4-part):** ``scenario|platform_name|platform_image|platform_command``
- **Multi-role (5-part):** ``role|scenario|platform_name|platform_image|platform_command``
Pairs are executed one at a time (molecule scenarios share temp directories and
Docker networks, so parallel execution within a single runner is unsafe).
A background thread polls the Gitea API. If any other molecule matrix runner
reports failure, the current molecule subprocess is killed and this runner
exits early with code 1.
Usage::
# Single-role
python3 -m devx.molecule.molecule_ci_guard pair1 pair2 ...
# Multi-role
python3 -m devx.molecule.molecule_ci_guard --roles-root ansible/roles pair1 pair2 ...
Environment variables:
GITEA_URL Base URL of the Gitea instance.
CI_GITEA_API_TOKEN API token with repo access (CI_GITEA_TOKEN accepted for legacy).
RUN_ID Workflow run ID (GITHUB_RUN_ID).
JOB_NAME Base job name (GITHUB_JOB), e.g. "molecule-tests".
MATRIX_INDEX Current matrix index (runner-index).
GITEA_REPOSITORY Repository in "owner/repo" format.
"""
from __future__ import annotations
import contextlib
import os
import signal
import subprocess # nosec B404
import sys
import threading
import time
from pathlib import Path
import click
import requests
from devx.config import REPO_NAME, REPO_OWNER
from devx.i18n import _
from devx.tokens import get_ci_token
POLL_INTERVAL = 10
def get_running_jobs(gitea_url: str, owner: str, repo: str, token: str, run_id: int) -> list[dict]:
"""Return jobs for the given workflow run."""
url = f"{gitea_url}/api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/jobs"
headers = {"Authorization": f"token {token}"}
response = requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
data = response.json()
return data.get("jobs", [])
def any_other_runner_failed(jobs: list[dict], current_job_name: str, current_index: int) -> bool:
"""Return True if any other molecule matrix job has failed."""
for job in jobs:
name = job.get("name", "")
if not name.startswith(current_job_name):
continue
if name == f"{current_job_name} ({current_index})" or name == current_job_name:
continue
if job.get("conclusion") == "failure":
return True
return False
def poll_for_other_failures(
gitea_url: str,
owner: str,
repo: str,
token: str,
run_id: int,
job_name: str,
current_index: int,
stop_event: threading.Event,
failed_event: threading.Event,
) -> None:
"""Background thread: poll API and signal if another runner fails."""
while not stop_event.is_set():
try:
jobs = get_running_jobs(gitea_url, owner, repo, token, run_id)
if any_other_runner_failed(jobs, job_name, current_index):
click.echo(_("Another molecule runner failed. Stopping this runner early."))
failed_event.set()
return
except requests.RequestException as exc:
click.echo(_("API poll warning: {exc}", exc=exc))
stop_event.wait(POLL_INTERVAL)
def build_molecule_cmd(scenario: str) -> list[str]:
"""Build the molecule command for a scenario."""
cmd = ["molecule", "test"]
if scenario != "default":
cmd.extend(["-s", scenario])
return cmd
def parse_pair(pair: str) -> tuple[str, str, str, str, str]:
"""Parse a pair string into (role, scenario, platform_name, platform_image, platform_command).
Supports both 4-part (single-role) and 5-part (multi-role) formats.
For 4-part pairs, role is empty (caller uses default role dir).
Spaces in the command field are encoded as ``__SPACE__`` to survive
shell word-splitting when ``$TEST_PAIRS`` is expanded unquoted.
"""
parts = pair.split("|")
if len(parts) == 4:
return "", parts[0], parts[1], parts[2], parts[3].replace("__SPACE__", " ")
if len(parts) == 5:
return parts[0], parts[1], parts[2], parts[3], parts[4].replace("__SPACE__", " ")
raise click.ClickException(f"Invalid pair format: {pair!r} (expected 4 or 5 pipe-delimited parts)")
def build_env_for_pair(pair: str, base_env: dict[str, str]) -> dict[str, str]:
"""Build environment for a single molecule pair."""
_role, _scenario, platform_name, platform_image, platform_command = parse_pair(pair)
env = base_env.copy()
# Append runner index to platform name when running in CI matrix to avoid
# Docker container name conflicts when multiple runners share the same Docker host.
matrix_index = env.get("MATRIX_INDEX")
if matrix_index:
platform_name = f"{platform_name}-r{matrix_index}"
env["MOLECULE_PLATFORM_NAME"] = platform_name
env["MOLECULE_PLATFORM_IMAGE"] = platform_image
if platform_command:
env["MOLECULE_PLATFORM_COMMAND"] = platform_command
elif "MOLECULE_PLATFORM_COMMAND" in env:
del env["MOLECULE_PLATFORM_COMMAND"]
env["ANSIBLE_ALLOW_BROKEN_CONDITIONALS"] = "true"
# Use a fresh MOLECULE_HOME per pair to avoid stale config cache
# from previous CI runs (causes "Instances missing" errors).
if "MOLECULE_HOME" not in env:
import tempfile
env["MOLECULE_HOME"] = tempfile.mkdtemp(prefix="molecule-ci-")
return env
def resolve_role_dir(role: str, roles_root: Path | None, repo_root: Path) -> Path:
"""Resolve the working directory for a molecule pair.
For multi-role pairs (role non-empty), uses ``roles_root/role``.
For single-role pairs, auto-discovers the first role with a molecule/
subdirectory under ``repo_root/ansible/roles/``.
"""
if role:
if roles_root is None:
roles_root = repo_root / "ansible" / "roles"
return roles_root / role
roles_dir = repo_root / "ansible" / "roles"
if roles_dir.is_dir():
role_dirs = sorted(d for d in roles_dir.iterdir() if (d / "molecule").is_dir())
if role_dirs:
return role_dirs[0]
return roles_dir / "role" # will produce a clear "not found" error
@click.command()
@click.argument("pairs", nargs=-1, required=True)
@click.option(
"--roles-root",
type=click.Path(exists=True, file_okay=False, path_type=Path),
default=None,
help="Root directory for multi-role pairs (e.g. ansible/roles). Required when pairs use 5-part format.",
)
def cli(pairs: tuple[str, ...], roles_root: Path | None) -> None:
"""Run molecule pairs sequentially, stop if another CI runner fails."""
gitea_url = os.environ.get("GITEA_URL", "")
try:
token = get_ci_token()
except click.ClickException:
token = None
run_id = int(os.environ.get("RUN_ID", "0"))
job_name = os.environ.get("JOB_NAME", "molecule-tests")
current_index = int(os.environ.get("MATRIX_INDEX", "0"))
repository = os.environ.get("GITEA_REPOSITORY", "")
owner, _sep, repo = repository.partition("/")
if not owner or not repo:
owner, repo = REPO_OWNER, REPO_NAME
if not all([gitea_url, token, run_id]):
click.echo(_("GITEA_URL/CI_GITEA_TOKEN/RUN_ID not set; running without cross-runner cancellation."))
# When devx is installed as a pip package, __file__ resolves to the
# site-packages directory, not the repo root. Use GITHUB_WORKSPACE
# (set by Gitea Actions) or cwd as the repo root.
repo_root = Path(os.environ.get("GITHUB_WORKSPACE", os.getcwd())).resolve()
base_env = os.environ.copy()
base_env.setdefault("DOCKER_HOST", f"unix:///run/user/{os.getuid()}/docker.sock")
base_env.setdefault("ANSIBLE_INJECT_INVOCATION", "1")
stop_event = threading.Event()
failed_event = threading.Event()
if gitea_url and token and run_id:
poller = threading.Thread(
target=poll_for_other_failures,
args=(
gitea_url,
owner,
repo,
token,
run_id,
job_name,
current_index,
stop_event,
failed_event,
),
daemon=True,
)
poller.start()
try:
for pair in pairs:
if failed_event.is_set():
sys.exit(1)
role, scenario, platform_name, _img, _cmd = parse_pair(pair)
click.echo(_("Running: {scenario} on {platform}", scenario=scenario, platform=platform_name))
cmd = build_molecule_cmd(scenario)
env = build_env_for_pair(pair, base_env)
cwd = resolve_role_dir(role, roles_root, repo_root)
process = subprocess.Popen( # nosec B603
cmd,
cwd=str(cwd),
env=env,
preexec_fn=os.setsid,
)
try:
while process.poll() is None:
if failed_event.is_set():
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGKILL)
process.wait()
# Clean up containers left behind by the killed test.
click.echo(_("Cleaning up: running molecule destroy for {scenario}", scenario=scenario))
destroy_cmd = ["molecule", "destroy"]
if scenario != "default":
destroy_cmd.extend(["-s", scenario])
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
destroy_cmd,
cwd=str(cwd),
env=env,
check=False,
capture_output=True,
timeout=120,
)
sys.exit(1)
time.sleep(1)
except KeyboardInterrupt:
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
process.wait()
# Clean up containers left behind by the interrupted test.
click.echo(_("Cleaning up: running molecule destroy for {scenario}", scenario=scenario))
destroy_cmd = ["molecule", "destroy"]
if scenario != "default":
destroy_cmd.extend(["-s", scenario])
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
destroy_cmd,
cwd=str(cwd),
env=env,
check=False,
capture_output=True,
timeout=120,
)
sys.exit(1)
rc = process.returncode
if rc != 0:
click.echo(_("FAILED: {pair} exited with code {code}", pair=pair, code=rc))
# Run molecule destroy to clean up containers left behind by the
# failed test. Without this, containers stay running and accumulate
# on the runner, consuming disk/memory and degrading CI performance.
click.echo(_("Cleaning up: running molecule destroy for {scenario}", scenario=scenario))
destroy_cmd = ["molecule", "destroy"]
if scenario != "default":
destroy_cmd.extend(["-s", scenario])
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
destroy_cmd,
cwd=str(cwd),
env=env,
check=False,
capture_output=True,
timeout=120,
)
sys.exit(rc)
click.echo(_("PASSED: {pair}", pair=pair))
# Prune Docker data between scenarios to prevent disk exhaustion
# in Docker-in-Docker molecule containers (each scenario pulls
# hundreds of MB of images that accumulate across pairs).
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
["docker", "system", "prune", "-af", "--volumes"],
check=False,
capture_output=True,
timeout=60,
)
click.echo(_("All molecule tests passed."))
finally:
stop_event.set()
sys.exit(0)
if __name__ == "__main__": # pragma: no cover
cli()
+217 -27
View File
@@ -10,6 +10,12 @@ If the host socket is not available, it tries the rootless socket, then
starts a local ``dockerd`` with the vfs storage driver (requires
privileged container).
When the host socket IS available but has limited disk space (e.g. an
inner DinD daemon writing to a 38 GB container overlay), the script
prefers a rootless socket that has more available space. This prevents
"no space left on device" errors during molecule tests that pull images
and create containers via the Docker daemon.
Usage::
python3 -m devx.molecule.start_docker [--timeout 30]
@@ -17,8 +23,10 @@ Usage::
from __future__ import annotations
import contextlib
import glob
import os
import shutil
import subprocess # nosec B404
import sys
import tempfile
@@ -32,6 +40,16 @@ DEFAULT_TIMEOUT = 30
DOCKER_SOCK = "/var/run/docker.sock"
# Rootless socket fallback (e.g. /run/user/994/docker.sock)
ROOTLESS_SOCK = f"/run/user/{os.getuid()}/docker.sock"
# Host Docker socket mounted by gitea_runner config (see runner config
# ``options: "-v /run/user/<uid>/docker.sock:/run/host-docker.sock"``).
# This gives CI containers access to the host's rootless Docker daemon,
# which has the full host filesystem (e.g. 455 GB) instead of the
# container's limited overlay (e.g. 38 GB).
HOST_DOCKER_SOCK = "/run/host-docker.sock"
# Minimum free bytes for a Docker daemon to be considered usable.
# Below this, image pulls and container creation will fail with ENOSPC.
# 20 GB leaves room for molecule-test-base (~500 MB) + a few containers.
MIN_FREE_BYTES = 20 * 1024**3 # 20 GB
def is_docker_ready() -> bool:
@@ -46,6 +64,46 @@ def is_docker_ready() -> bool:
return result.returncode == 0
def _get_docker_free_bytes() -> int:
"""Get free disk space (bytes) at the Docker daemon's data root.
Returns 0 if the daemon is not reachable or the data root cannot be
determined.
"""
docker_host = os.environ.get("DOCKER_HOST", f"unix://{DOCKER_SOCK}")
try:
result = subprocess.run( # nosec B603 B607
[
"docker",
"info",
"--format",
"{{.DockerRootDir}}",
],
capture_output=True,
text=True,
timeout=10,
check=False,
env={**os.environ, "DOCKER_HOST": docker_host},
)
if result.returncode != 0 or not result.stdout.strip():
return 0
data_root = result.stdout.strip()
if not os.path.exists(data_root):
return 0
return shutil.disk_usage(data_root).free
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
return 0
def _try_socket(sock_path: str) -> bool:
"""Set DOCKER_HOST to *sock_path* and check if the daemon is ready.
Returns ``True`` if the daemon responds, ``False`` otherwise.
"""
os.environ["DOCKER_HOST"] = f"unix://{sock_path}"
return is_docker_ready()
def _diagnose_socket() -> None:
"""Print diagnostic info about the Docker socket."""
click.echo(f"DOCKER_HOST = {os.environ.get('DOCKER_HOST', '(not set)')}")
@@ -97,50 +155,177 @@ def _diagnose_socket() -> None:
def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
"""Ensure Docker is ready for molecule tests.
First tries the host socket. If that works, sets ``DOCKER_HOST`` and
returns immediately. If not, tries the rootless socket. If neither
works, starts a local ``dockerd`` with vfs storage driver (requires
privileged container).
Tries sockets in this order, preferring ones with enough disk space:
1. Host rootless socket (``/run/host-docker.sock``) mounted by the
gitea runner config, has access to the host's full filesystem
(e.g. 455 GB). Preferred over the inner dockerd.
2. Default socket (``/var/run/docker.sock``) may be an inner dockerd
started by the CI image (v29.5.3) with data root on the container's
limited overlay (e.g. 38 GB, often 100 % full).
3. Other rootless sockets (``/run/user/*/docker.sock``).
4. Local ``dockerd`` with vfs storage driver last resort.
Returns ``True`` if Docker is ready, ``False`` if it failed to
start within the timeout.
"""
# Point Docker CLI and Python library to the socket explicitly
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Diagnose socket state
click.echo("--- Docker socket diagnostics ---")
_diagnose_socket()
click.echo("--- End diagnostics ---")
# Check if host Docker is already available
if is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Try rootless socket (e.g. /run/user/994/docker.sock)
click.echo(f"Trying rootless socket: {ROOTLESS_SOCK}")
os.environ["DOCKER_HOST"] = f"unix://{ROOTLESS_SOCK}"
if os.path.exists(ROOTLESS_SOCK) and is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Scan for any rootless sockets at other UIDs
# Collect candidate sockets in priority order.
# The host's rootless Docker socket (mounted at /run/host-docker.sock
# by the gitea runner config) is preferred — it has access to the
# host's full filesystem instead of the container's limited overlay.
candidates: list[str] = []
if os.path.exists(HOST_DOCKER_SOCK):
candidates.append(HOST_DOCKER_SOCK)
if os.path.exists(DOCKER_SOCK):
candidates.append(DOCKER_SOCK)
if os.path.exists(ROOTLESS_SOCK):
candidates.append(ROOTLESS_SOCK)
for sock in sorted(glob.glob("/run/user/*/docker.sock")):
if sock == ROOTLESS_SOCK:
if sock not in candidates:
candidates.append(sock)
# Try each candidate socket — prefer one with enough free space
for sock in candidates:
click.echo(f"Trying socket: {sock}")
if not _try_socket(sock):
continue
click.echo(f"Trying alternative rootless socket: {sock}")
os.environ["DOCKER_HOST"] = f"unix://{sock}"
if is_docker_ready():
free_bytes = _get_docker_free_bytes()
free_gb = free_bytes / 1024**3
click.echo(f" Docker daemon ready (free space: {free_gb:.1f} GB)")
if free_bytes >= MIN_FREE_BYTES:
click.echo(_("Docker daemon already running"))
return True
# If free_bytes is 0, the Docker root dir is on the host filesystem
# (not accessible from inside the container). This is expected for
# the host's rootless Docker — it has the full host disk.
# Only trust this for /run/host-docker.sock (known host socket).
# For other sockets (e.g. inner dockerd), free_bytes == 0 means
# the data root path doesn't exist inside the container — the
# inner dockerd may be using the container's full overlay.
if free_bytes == 0 and sock == HOST_DOCKER_SOCK:
click.echo("Host rootless Docker root dir not accessible from container, using it")
return True
# If free_bytes is 0 and there are no dockerd processes inside the
# container, the socket is the host's Docker (mounted from outside).
# The data root is on the host filesystem and has plenty of space.
if free_bytes == 0 and sock == DOCKER_SOCK:
has_inner_dockerd = False
with contextlib.suppress(Exception):
pgrep_result = subprocess.run( # nosec B603 B607
["pgrep", "-f", "dockerd"],
capture_output=True,
text=True,
timeout=5,
)
has_inner_dockerd = pgrep_result.returncode == 0
if not has_inner_dockerd:
click.echo("No inner dockerd found, socket is host Docker (data root on host), using it")
return True
click.echo(f" Insufficient space ({free_gb:.1f} GB), trying next...")
# No socket with sufficient space found.
# Don't fall back to the low-space inner dockerd — it will fail
# on image pulls. Instead, kill the inner dockerd, clean up its
# data root to free space, and start a new dockerd using the
# freed space on the container's overlay.
click.echo(_("Host Docker not available, starting local dockerd..."))
# Reset DOCKER_HOST to host socket for local dockerd
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Kill the inner dockerd (started by the CI image) to free its
# data root and socket. The inner dockerd uses the container's
# overlay (38G, often 100% full). Killing it frees up the
# socket and any space used by its containers/volumes.
# Use SIGKILL (-9) since the inner dockerd may not respond to SIGTERM.
# Try multiple approaches to ensure the inner dockerd is killed.
with contextlib.suppress(Exception):
result = subprocess.run( # nosec B603 B607
["pgrep", "-af", "dockerd"],
capture_output=True,
text=True,
timeout=5,
)
if result.stdout.strip():
click.echo(f" dockerd processes before kill: {result.stdout.strip()}")
for pattern in ["dockerd", "dockerd-entrypoint.sh", "containerd"]:
with contextlib.suppress(Exception):
subprocess.run( # nosec B603 B607
["pkill", "-9", "-f", pattern],
check=False,
timeout=5,
)
time.sleep(3)
# Check if dockerd processes are still alive
with contextlib.suppress(Exception):
result = subprocess.run( # nosec B603 B607
["pgrep", "-af", "dockerd"],
capture_output=True,
text=True,
timeout=5,
)
if result.stdout.strip():
click.echo(f" dockerd processes after kill: {result.stdout.strip()}")
# Try killing by PID directly
for pid_str in result.stdout.split("\n"):
pid = pid_str.split()[0] if pid_str.strip() else ""
if pid:
with contextlib.suppress(Exception):
os.kill(int(pid), 9)
time.sleep(2)
# Verify the inner dockerd is actually dead. If we can still
# connect to /var/run/docker.sock, the old daemon is still running
# and we need to use a different socket path.
old_daemon_alive = False
with contextlib.suppress(Exception):
result = subprocess.run( # nosec B603 B607
["docker", "info"],
env={**os.environ, "DOCKER_HOST": f"unix://{DOCKER_SOCK}"},
capture_output=True,
timeout=5,
)
old_daemon_alive = result.returncode == 0
if old_daemon_alive:
click.echo(" Inner dockerd still alive, using alternate socket")
local_sock = "/dev/shm/docker.sock" # nosec B108
else:
local_sock = DOCKER_SOCK
# Clean up the inner dockerd's data root to free space.
# The inner dockerd stores images, containers, and volumes here.
# Removing them frees up ~2.4GB on the container's overlay.
inner_data_root = "/home/grm-ci-runner-*/.local/share/docker"
rm_paths = " ".join(f"{inner_data_root}/{d}" for d in ("overlay2", "image", "volumes", "containers"))
with contextlib.suppress(Exception):
subprocess.run( # nosec B603 B607
["sh", "-c", f"rm -rf {rm_paths}"],
check=False,
timeout=30,
)
# Use a fresh data root. If the inner dockerd is dead, use the
# container's overlay (38G, with freed space). If the inner
# dockerd is still alive, use /dev/shm (16G tmpfs) — the overlay
# is still full because the inner dockerd's data can't be cleaned.
docker_data_root = "/dev/shm/docker" if old_daemon_alive else "/tmp/docker-data" # nosec B108
# Remove stale socket if present
with contextlib.suppress(OSError):
os.unlink(local_sock)
os.environ["DOCKER_HOST"] = f"unix://{local_sock}"
# Start local dockerd (requires privileged container)
# Use /tmp/docker-data as data root on the container's overlay.
# The inner dockerd's data root has been cleaned up, freeing ~2.4GB.
# vfs storage driver is used since overlay2 may not work inside
# a Docker-in-Docker container without --privileged.
log_file = tempfile.NamedTemporaryFile( # noqa: SIM115
mode="w", suffix="dockerd.log", delete=False
)
@@ -150,8 +335,13 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
"dockerd",
"--storage-driver",
"vfs",
"--data-root",
docker_data_root,
"--iptables=false",
"--ip6tables=false",
"--bridge=none",
"-H",
f"unix://{DOCKER_SOCK}",
f"unix://{local_sock}",
],
stdout=log_file,
stderr=subprocess.STDOUT,
+2 -16
View File
@@ -2,19 +2,16 @@
Centralizes Gitea/Vikunja token discovery with role-based environment
variable names and backwards compatibility with the legacy
``CI_GITEA_TOKEN`` / ``REVIEW_GITEA_TOKEN`` naming convention.
``CI_GITEA_TOKEN`` naming convention.
Roles:
- ``CI_GITEA_API_TOKEN``: CI workflows (read actions, post status, merge, etc.)
- ``REVIEWER_GITEA_API_TOKEN``: PR approval reviews (must be a different user
from the PR author for Gitea to accept the review as an approval)
- ``DEVELOPER_GITEA_API_TOKEN``: local development tools (create-task,
create-pr, setup, etc.)
Fallbacks:
- New role names are checked first.
- Legacy names (``CI_GITEA_TOKEN``, ``REVIEW_GITEA_TOKEN``) are accepted for
backwards compatibility.
- Legacy names (``CI_GITEA_TOKEN``) are accepted for backwards compatibility.
- If no role-specific token is set, the generic CI tokens are tried last.
"""
@@ -28,12 +25,6 @@ from devx.i18n import _
# Token environment variable names, in lookup priority order.
CI_TOKEN_NAMES = ["CI_GITEA_API_TOKEN", "CI_GITEA_TOKEN"]
REVIEWER_TOKEN_NAMES = [
"REVIEWER_GITEA_API_TOKEN",
# Legacy name used before role-based tokens.
"REVIEW_GITEA_TOKEN",
*CI_TOKEN_NAMES,
]
DEVELOPER_TOKEN_NAMES = ["DEVELOPER_GITEA_API_TOKEN", *CI_TOKEN_NAMES]
VIKUNJA_TOKEN_NAMES = ["VIKUNJA_TOKEN"]
@@ -61,11 +52,6 @@ def get_ci_token() -> str:
return get_token(*CI_TOKEN_NAMES)
def get_reviewer_token() -> str:
"""Resolve the reviewer Gitea API token used for PR approvals."""
return get_token(*REVIEWER_TOKEN_NAMES)
def get_developer_token() -> str:
"""Resolve the developer Gitea API token used for local tooling."""
return get_token(*DEVELOPER_TOKEN_NAMES)
+232
View File
@@ -0,0 +1,232 @@
"""Check Ansible tasks for missing no_log on secret-handling tasks.
ansible-lint's built-in ``no-log-password`` rule only fires when a module
parameter is literally named ``*password*`` and there's a loop. It does
NOT catch:
- Shell/command tasks that interpolate ``{{ _secrets.* }}`` or
``{{ *password* }}`` variables
- Template/copy tasks that render secret values without ``no_log``
This script fills that gap by scanning all Ansible task files for
variables that look like secrets (``_secrets.*``, ``*password*``,
``*secret*``, ``*token*``, ``*api_key*``) and verifying that the task
has ``no_log`` set to a non-False value.
Usage::
python -m devx.tools.check_ansible_no_log
python -m devx.tools.check_ansible_no_log --path ansible/roles/my_role
python -m devx.tools.check_ansible_no_log --ansible-dir ansible/roles
Exit code 0 if all secret-handling tasks have no_log, 1 otherwise.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
import click
import yaml
REPO_ROOT = Path.cwd()
DEFAULT_ANSIBLE_DIR = REPO_ROOT / "ansible"
# Patterns that indicate a task is handling secrets.
# We only match Jinja-interpolated variables ({{ ... }}) to avoid false
# positives from field names like "password" in module params or task names.
SECRET_PATTERNS = [
# {{ _secrets.anything }} or {{ _secrets['anything'] }}
re.compile(r"\{\{[^}]*_secrets\.", re.IGNORECASE),
# {{ anything_password }} but NOT the word "password" in a string literal
re.compile(r"\{\{[^}]*password", re.IGNORECASE),
# {{ anything_secret }}
re.compile(r"\{\{[^}]*_secret\b", re.IGNORECASE),
# {{ anything_api_key }}
re.compile(r"\{\{[^}]*api_key", re.IGNORECASE),
# {{ anything_token }} (but not loop tokens like {{ loop_token }})
re.compile(r"\{\{[^}]*(?:vault_token|auth_token|access_token|bot_token)", re.IGNORECASE),
]
# Task keys whose values might contain secret references
TASK_VALUE_KEYS = {
"shell",
"command",
"ansible.builtin.shell",
"ansible.builtin.command",
"ansible.builtin.template",
"ansible.builtin.copy",
"ansible.builtin.debug",
"template",
"copy",
"debug",
"cmd",
"msg",
"content",
}
# Keys that are NOT secret-bearing (task metadata, not values)
NON_VALUE_KEYS = {
"name",
"when",
"loop",
"loop_control",
"changed_when",
"failed_when",
"no_log",
"register",
"tags",
"vars",
"become",
"become_user",
"delegate_to",
"run_once",
"environment",
"with_items",
"with_dict",
"with_list",
}
def _contains_secret(value: object) -> bool:
"""Recursively check if a value contains secret-like variable references."""
if isinstance(value, str):
return any(p.search(value) for p in SECRET_PATTERNS)
if isinstance(value, dict):
return any(_contains_secret(v) for v in value.values())
if isinstance(value, list):
return any(_contains_secret(item) for item in value)
return False
def _has_no_log(task: dict) -> bool:
"""Check if a task has no_log set to a non-False value."""
no_log = task.get("no_log", False)
# Jinja expressions (e.g. "{{ not debug_mode }}") count as set
return no_log is not False and no_log is not None
def _check_task(task: dict, file_path: Path, task_num: int) -> list[str]:
"""Check a single task for missing no_log on secret values.
Returns a list of violation messages (empty if OK).
"""
violations: list[str] = []
# Skip tasks that already have no_log
if _has_no_log(task):
return violations
# Check all string values in the task for secret references
has_secrets = False
for key, value in task.items():
if key in NON_VALUE_KEYS:
continue
# Check action module params (shell, command, copy, template, etc.)
if _contains_secret(value):
has_secrets = True
break
if has_secrets:
task_name = task.get("name", "<unnamed>")
violations.append(
f"{file_path}:{task_num}: Task '{task_name}' references secrets "
f"but has no no_log. Add `no_log: true` or "
f'`no_log: "{{{{ not (debug_mode | default(false) | bool) }}}}"` '
f"to prevent credential leakage in Ansible output."
)
return violations
def check_directory(ansible_dir: Path) -> list[str]:
"""Check all Ansible task files in a directory tree."""
all_violations: list[str] = []
# Find all task files
task_files = list(ansible_dir.rglob("tasks/*.yml"))
task_files += list(ansible_dir.rglob("tasks/*.yaml"))
# Also check playbook files
task_files += list(ansible_dir.glob("playbooks/*.yml"))
for task_file in sorted(task_files):
# Skip molecule test files
if "molecule" in task_file.parts:
continue
try:
with task_file.open() as f:
docs = list(yaml.safe_load_all(f))
except (yaml.YAMLError, OSError):
continue
for doc in docs:
if not doc:
continue
# Task files are bare lists of tasks; playbook files are
# lists of plays (each play is a dict with 'hosts' key)
if isinstance(doc, list):
is_plays = isinstance(doc[0], dict) and "hosts" in doc[0]
if not is_plays:
for i, task in enumerate(doc):
if not isinstance(task, dict):
continue
all_violations.extend(_check_task(task, task_file, i + 1))
continue
plays = doc
elif isinstance(doc, dict):
plays = [doc]
else:
continue
for play in plays:
if not isinstance(play, dict):
continue
for task_section in ("tasks", "pre_tasks", "post_tasks", "handlers"):
tasks = play.get(task_section, [])
if not isinstance(tasks, list):
continue
for i, task in enumerate(tasks):
if not isinstance(task, dict):
continue
all_violations.extend(_check_task(task, task_file, i + 1))
return all_violations
@click.command()
@click.option(
"--path",
type=click.Path(exists=True, path_type=Path),
help="Check a specific file or directory (default: ansible/).",
)
@click.option(
"--ansible-dir",
type=click.Path(exists=True, path_type=Path),
default=None,
help="Override the default ansible directory (default: ansible/).",
)
def main(path: Path | None, ansible_dir: Path | None) -> None:
"""Check that Ansible tasks handling secrets have no_log set."""
target = path or ansible_dir or DEFAULT_ANSIBLE_DIR
if not target.is_dir():
click.echo(f"Error: {target} is not a directory", err=True)
sys.exit(2)
violations = check_directory(target)
if violations:
click.echo(f"Found {len(violations)} task(s) handling secrets without no_log:\n")
for v in violations:
click.echo(f" {v}")
click.echo(f"\nTotal: {len(violations)} violation(s).")
sys.exit(1)
click.echo(f"[check-ansible-no-log] All secret-handling tasks have no_log. ({target})")
if __name__ == "__main__": # pragma: no cover
main()
@@ -0,0 +1,176 @@
"""Check Ansible tasks for ``state: absent`` on database data directories.
This is a static analysis lint check that runs in CI (``make lint-ci``)
to prevent the class of bug that caused the 2026-07-22 production outage
(ADR-0028): a ``state: absent`` on a PostgreSQL data directory path that
fired on every deploy and wiped the ZITADEL database.
The existing unit test ``scripts/tests/test_no_zitadel_db_wipe.py`` covers
the same concern as a regression test. This lint check runs earlier in
the pipeline (before tests) and covers ALL roles and playbooks, not just
the ZITADEL role.
Allowed contexts (where DB recreation is legitimate):
- PostgreSQL major version upgrades (``upgrade-postgres``, ``PG_VERSION``)
- Explicit ``# lint:allow-state-absent`` comment on the task
Usage::
python -m devx.tools.check_ansible_no_state_absent_on_db
python -m devx.tools.check_ansible_no_state_absent_on_db --path ansible/roles/zitadel/tasks/main.yml
Exit code 0 if no violations found, 1 otherwise.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
import click
REPO_ROOT = Path.cwd()
DEFAULT_ANSIBLE_DIRS: list[Path] = [
REPO_ROOT / "ansible" / "playbooks",
REPO_ROOT / "ansible" / "roles",
]
# Database data directory path patterns.
# These match the DIRECTORY path, not individual files within it.
# Removing a stale config file (e.g. postgresql.conf) is safe; removing
# the entire data directory is not.
DB_PATH_PATTERNS = (
re.compile(r"postgres/zitadel-db", re.IGNORECASE),
re.compile(r"postgres/\w+-db", re.IGNORECASE),
re.compile(r"/var/lib/postgresql/data", re.IGNORECASE),
re.compile(r"/var/lib/postgresql/data/\w+-db", re.IGNORECASE),
)
# Destructive operations
DESTRUCTIVE_PATTERNS = (
re.compile(r"state:\s*absent", re.IGNORECASE),
re.compile(r"rm\s+-rf.*\bdb\b", re.IGNORECASE),
)
# Allowed contexts where DB recreation is legitimate
ALLOWED_CONTEXT_KEYWORDS = (
"upgrade-postgres",
"PG_VERSION",
"pg_version",
)
# Comment marker to explicitly allow state: absent on a specific task
ALLOW_MARKER = "lint:allow-state-absent"
def _find_task_files(base: Path) -> list[Path]:
"""Find all YAML task files under a base directory, skipping molecule."""
if base.is_file() and base.suffix in (".yml", ".yaml"):
return [base]
if not base.is_dir():
return []
files: list[Path] = []
for f in sorted(base.rglob("*.yml")) + sorted(base.rglob("*.yaml")):
if "molecule" in f.parts:
continue
files.append(f)
return files
def _check_file(filepath: Path, repo_root: Path) -> list[str]:
"""Check a YAML file for state: absent on DB data directory paths.
Returns a list of violation messages (empty if clean).
"""
try:
content = filepath.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return []
# Quick check: if no DB path pattern appears anywhere, skip
if not any(p.search(content) for p in DB_PATH_PATTERNS):
return []
try:
display_path = filepath.relative_to(repo_root)
except ValueError:
display_path = filepath
violations: list[str] = []
lines = content.splitlines()
for i, line in enumerate(lines):
for db_pattern in DB_PATH_PATTERNS:
if not db_pattern.search(line):
continue
# Check surrounding context (±5 lines) for destructive operations
context_start = max(0, i - 5)
context_end = min(len(lines), i + 6)
context = "\n".join(lines[context_start:context_end])
# Skip if in an allowed context (PG upgrade)
if any(kw in context for kw in ALLOWED_CONTEXT_KEYWORDS):
continue
# Skip if the allow marker comment is in the context
if ALLOW_MARKER in context:
continue
for dp in DESTRUCTIVE_PATTERNS:
if dp.search(context):
violations.append(
f"{display_path}:{i + 1} — destructive operation "
f"({dp.pattern!r}) near DB data directory path "
f"({db_pattern.pattern!r}). "
f"Database directories must never be wiped automatically (ADR-0028). "
f"If this is legitimate (e.g. PG upgrade), add "
f"#{ALLOW_MARKER} to the task."
)
break
return violations
@click.command()
@click.option(
"--path",
type=click.Path(exists=True, path_type=Path),
help="Check a specific file or directory (default: ansible/playbooks + ansible/roles).",
)
@click.option(
"--ansible-dir",
"ansible_dirs",
type=click.Path(exists=True, path_type=Path),
multiple=True,
default=None,
help="Override the default ansible directories (can be repeated). Defaults to ansible/playbooks and ansible/roles.",
)
def main(path: Path | None, ansible_dirs: tuple[Path, ...]) -> None:
"""Check that no Ansible task uses state: absent on a DB data directory."""
dirs = list(ansible_dirs) if ansible_dirs else DEFAULT_ANSIBLE_DIRS
if path:
files = _find_task_files(path)
else:
files: list[Path] = []
for d in dirs:
files.extend(_find_task_files(d))
all_violations: list[str] = []
for f in files:
all_violations.extend(_check_file(f, REPO_ROOT))
if all_violations:
click.echo("[check-ansible-no-state-absent-on-db] FAIL: destructive operations on DB paths:")
for v in all_violations:
click.echo(f" - {v}")
click.echo(f"\nTotal: {len(all_violations)} violation(s).")
click.echo("Database data directories must never be wiped automatically (ADR-0028).")
sys.exit(1)
else:
click.echo("[check-ansible-no-state-absent-on-db] OK: no destructive operations on DB paths.")
if __name__ == "__main__": # pragma: no cover
main()
+345
View File
@@ -0,0 +1,345 @@
"""Check Ansible tasks for dangerous patterns that mask failures.
This check addresses the gap identified in the testing-strategy audit:
the automated PR review only checks Python files, and ``ansible-lint``
runs at ``profile: basic`` which does not catch dangerous patterns like:
- ``|| true`` on tasks that are NOT cleanup/idempotency operations
- ``failed_when: false`` on critical tasks (e.g. DB operations)
- ``2>/dev/null`` on tasks where stderr contains important diagnostics
Most ``|| true`` and ``2>/dev/null`` instances in the codebase are
legitimate (container removal, journalctl, apt-get, docker prune, SUID
removal). This check flags only instances that are NOT in a known-safe
context. Tasks can also opt out with a ``# lint:allow-failure-masking``
comment.
Usage::
python -m devx.tools.check_ansible_patterns
python -m devx.tools.check_ansible_patterns --path ansible/roles/app_container/tasks/main.yml
Exit code 0 if no violations found, 1 otherwise.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
import click
import yaml
REPO_ROOT = Path.cwd()
DEFAULT_ANSIBLE_DIRS: list[Path] = [
REPO_ROOT / "ansible" / "playbooks",
REPO_ROOT / "ansible" / "roles",
]
# Comment marker to explicitly allow a pattern on a specific task
ALLOW_MARKER = "lint:allow-failure-masking"
# Patterns that mask failures when used in shell/command tasks
OR_TRUE_PATTERN = re.compile(r"\|\|\s*true\b", re.IGNORECASE)
REDIRECT_DEVNULL_PATTERN = re.compile(r"2>/dev/null")
# Module keys that accept shell/command strings
SHELL_MODULE_KEYS = frozenset(
{
"shell",
"command",
"ansible.builtin.shell",
"ansible.builtin.command",
"cmd",
"ansible.builtin.raw",
"raw",
}
)
# Task keys whose values might contain shell commands
COMMAND_VALUE_KEYS = frozenset(
{
"shell",
"command",
"ansible.builtin.shell",
"ansible.builtin.command",
"cmd",
"raw",
"ansible.builtin.raw",
}
)
# Legitimate contexts where || true or 2>/dev/null are safe.
# These are command prefixes or task names that indicate cleanup/idempotency.
LEGITIMATE_COMMAND_PREFIXES = (
# Container/process removal (may not exist)
"docker rm",
"docker stop",
"docker rmi",
"docker network rm",
"docker volume rm",
"pkill",
"kill",
# Cleanup commands that are expected to sometimes fail
"journalctl --vacuum",
"apt-get clean",
"apt-get autoremove",
"docker image prune",
"docker container prune",
"docker volume prune",
"docker builder prune",
"find / -name",
# SUID removal (binaries may not exist)
"chmod",
"rm -f",
# Network connection checks (may fail if not connected)
"docker network connect",
# Prometheus snapshot API (may fail if no snapshot)
"curl.*api/v2/admin/tsdb/snapshot",
)
LEGITIMATE_TASK_NAME_KEYWORDS = (
"remove",
"cleanup",
"clean up",
"prune",
"purge",
"disconnect",
"stop",
"kill",
"strip suid",
"suid",
"vacuum",
"ensure.*absent",
"may not exist",
"if exists",
"optional",
"best effort",
"no-op",
"noop",
"idempotent",
"sync",
)
# Tasks with failed_when: false that are critical and should not mask failures.
# Only flag operations that SHOULD fail loudly — writing secrets, provisioning
# users, creating OIDC apps. Do NOT flag stop/start/check/wait/migrate/restore
# operations where failed_when: false is legitimate (container may not exist,
# may already be stopped, etc.).
CRITICAL_TASK_KEYWORDS = (
"password",
"secret",
"provision",
"oidc",
)
# Task name keywords that indicate failed_when: false is legitimate
LEGITIMATE_FAILED_WHEN_KEYWORDS = (
"stop",
"start",
"check",
"wait",
"migrate",
"restart",
"rebuild",
"restore",
"remove",
"cleanup",
"sync",
"download",
"extract",
"verify",
)
def _is_legitimate_or_true(command_str: str, task_name: str) -> bool:
"""Check if a || true in a command is in a legitimate context."""
# Check task name for legitimate keywords
name_lower = task_name.lower()
if any(re.search(kw, name_lower) for kw in LEGITIMATE_TASK_NAME_KEYWORDS):
return True
# Check command prefix for legitimate patterns
cmd_lower = command_str.lower()
return any(re.search(prefix, cmd_lower) for prefix in LEGITIMATE_COMMAND_PREFIXES)
def _is_legitimate_devnull(command_str: str, task_name: str) -> bool:
"""Check if a 2>/dev/null in a command is in a legitimate context."""
# 2>/dev/null is almost always safe — it suppresses stderr noise.
# Only flag it if the task is critical (DB, backup, OIDC) AND
# there's no || true (which is the more dangerous pattern).
return _is_legitimate_or_true(command_str, task_name)
def _check_task(task: dict, filepath: Path, task_num: int, repo_root: Path) -> list[str]:
"""Check a single task for dangerous failure-masking patterns."""
violations: list[str] = []
try:
display_path = filepath.relative_to(repo_root)
except ValueError:
display_path = filepath
task_name = task.get("name", "<unnamed>")
# Check for the allow marker in the task name
# (YAML comments are not preserved by safe_load, so we check the
# task name for the marker as a workaround)
if ALLOW_MARKER in task_name:
return violations
# Check for || true in command/shell values
for key in COMMAND_VALUE_KEYS:
value = task.get(key)
if value is None:
continue
value_str = str(value)
if OR_TRUE_PATTERN.search(value_str) and not _is_legitimate_or_true(value_str, task_name):
violations.append(
f"{display_path}:{task_num} — task '{task_name}' uses "
f"'|| true' in {key} which may mask real failures. "
f"If this is a cleanup/idempotency operation, rename the "
f"task to include 'remove'/'cleanup'/'prune' or add "
f"#{ALLOW_MARKER} to the task."
)
# Check for failed_when: false on critical tasks
failed_when = task.get("failed_when")
if failed_when is False:
name_lower = task_name.lower()
# Skip if the task name indicates a legitimate failed_when: false context
is_legitimate = any(kw in name_lower for kw in LEGITIMATE_FAILED_WHEN_KEYWORDS)
if not is_legitimate:
for kw in CRITICAL_TASK_KEYWORDS:
if kw in name_lower:
violations.append(
f"{display_path}:{task_num} — critical task '{task_name}' "
f"has failed_when: false, which masks failures on "
f"a {kw}-related operation. Remove failed_when: false "
f"or add #{ALLOW_MARKER} if masking is intentional."
)
break
return violations
def _check_file(filepath: Path, repo_root: Path) -> list[str]:
"""Check a YAML file for dangerous failure-masking patterns."""
try:
content = filepath.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return []
# Quick check: if no patterns appear, skip
if not (
OR_TRUE_PATTERN.search(content) or "failed_when: false" in content or REDIRECT_DEVNULL_PATTERN.search(content)
):
return []
# Check for allow markers in comments
has_allow_marker = ALLOW_MARKER in content
try:
docs = list(yaml.safe_load_all(content))
except yaml.YAMLError:
return []
violations: list[str] = []
for doc in docs:
if not doc:
continue
if isinstance(doc, list):
for i, item in enumerate(doc):
if isinstance(item, dict):
if any(k in item for k in ("tasks", "pre_tasks", "post_tasks", "handlers")):
_check_tasks(item, filepath, violations, repo_root)
else:
violations.extend(_check_task(item, filepath, i + 1, repo_root))
block = item.get("block")
if isinstance(block, list):
for j, bt in enumerate(block):
if isinstance(bt, dict):
violations.extend(_check_task(bt, filepath, i + j + 1, repo_root))
elif isinstance(doc, dict):
_check_tasks(doc, filepath, violations, repo_root)
# Filter out violations if the allow marker is present in the file
# (coarse-grained opt-out for files with many legitimate uses)
if has_allow_marker:
violations = []
return violations
def _check_tasks(doc: dict, filepath: Path, errors: list[str], repo_root: Path) -> None:
"""Check top-level tasks and nested task sections in a playbook doc."""
for section_key in ("tasks", "pre_tasks", "post_tasks", "handlers"):
section = doc.get(section_key)
if isinstance(section, list):
for i, task in enumerate(section):
if isinstance(task, dict):
errors.extend(_check_task(task, filepath, i + 1, repo_root))
block = task.get("block")
if isinstance(block, list):
for j, bt in enumerate(block):
if isinstance(bt, dict):
errors.extend(_check_task(bt, filepath, i + j + 1, repo_root))
def _find_task_files(base: Path) -> list[Path]:
"""Find all YAML task files under a base directory, skipping molecule."""
if base.is_file() and base.suffix in (".yml", ".yaml"):
return [base]
if not base.is_dir():
return []
files: list[Path] = []
for f in sorted(base.rglob("*.yml")) + sorted(base.rglob("*.yaml")):
if "molecule" in f.parts:
continue
files.append(f)
return files
@click.command()
@click.option(
"--path",
type=click.Path(exists=True, path_type=Path),
help="Check a specific file or directory (default: ansible/playbooks + ansible/roles).",
)
@click.option(
"--ansible-dir",
"ansible_dirs",
type=click.Path(exists=True, path_type=Path),
multiple=True,
default=None,
help="Override the default ansible directories (can be repeated). Defaults to ansible/playbooks and ansible/roles.",
)
def main(path: Path | None, ansible_dirs: tuple[Path, ...]) -> None:
"""Check Ansible tasks for dangerous failure-masking patterns."""
dirs = list(ansible_dirs) if ansible_dirs else DEFAULT_ANSIBLE_DIRS
if path:
files = _find_task_files(path)
else:
files: list[Path] = []
for d in dirs:
files.extend(_find_task_files(d))
all_violations: list[str] = []
for f in files:
all_violations.extend(_check_file(f, REPO_ROOT))
if all_violations:
click.echo("[check-ansible-patterns] FAIL: dangerous failure-masking patterns found:")
for v in all_violations:
click.echo(f" - {v}")
click.echo(f"\nTotal: {len(all_violations)} violation(s).")
sys.exit(1)
else:
click.echo("[check-ansible-patterns] OK: no dangerous failure-masking patterns.")
if __name__ == "__main__": # pragma: no cover
main()
+292
View File
@@ -0,0 +1,292 @@
"""Validate Jinja2 expressions in Ansible files by rendering them.
Extracts ``{{ ... }}`` expressions from Ansible YAML files and renders
each one with Ansible's Jinja2 environment using mock variables. Catches
errors like reversed filter arguments, undefined filters, and syntax
errors before pushing to CI.
The check is intentionally lightweight it doesn't need real Ansible
facts or variables. It provides common mock values (now(), ansible_*,
etc.) and renders each expression in isolation. Expressions that fail
with undefined variables that aren't in the mock set are skipped (not
all variables can be predicted).
Usage::
python -m devx.tools.check_jinja_expr
python -m devx.tools.check_jinja_expr --path ansible/playbooks/deploy-observability.yml
Exit code 0 if all renderable expressions pass, 1 if any fail.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
import click
from jinja2 import Environment
from jinja2.exceptions import TemplateSyntaxError, UndefinedError
REPO_ROOT = Path.cwd()
def _default_ansible_dirs() -> list[Path]:
"""Return the default directories to scan for Ansible files."""
return [
REPO_ROOT / "ansible" / "playbooks",
REPO_ROOT / "ansible" / "roles",
]
# Mock context for rendering Jinja expressions.
MOCK_CONTEXT: dict[str, object] = {
"now": lambda fmt=None: (
"2026-01-01T00:00:00+00:00"
if fmt
else type(
"Now",
(),
{
"timestamp": lambda self: 1735689600.0,
"strftime": lambda self, fmt: "2026-01-01T00:00:00+00:00",
},
)()
),
"ansible_date_time": {
"iso8601": "2026-01-01T00:00:00+00:00",
"epoch": "1735689600",
},
"ansible_facts": {
"service_mgr": "systemd",
"architecture": "x86_64",
"distribution_release": "noble",
"virtualization_type": "none",
"interfaces": ["eth0", "lo"],
"hostname": "test-host",
},
"ansible_host": "10.0.0.1",
"env": "staging",
"environment": "staging",
"customer_id": "test",
"zitadel_domain": "zitadel.test",
"_env_name": "staging",
"_observability_data_root": "/opt",
"skip_zitadel_stack": False,
"skip_htpasswd": False,
"skip_observability_stack": False,
"backup_enabled": True,
"app_filter": "",
"app_domain": "test.example.com",
"oidc_client_id": "test-client-id",
"oidc_client_secret": "test-secret", # nosec B105 — mock value for Jinja rendering, not a real secret
"s3_backup_bucket": "test-bucket",
"s3_endpoint": "https://s3.test",
"s3_access_key": "test-key",
"s3_secret_key": "test-secret", # nosec B105 — mock value for Jinja rendering, not a real secret
}
# Pattern to find {{ ... }} expressions (non-greedy, single-line).
EXPR_PATTERN = re.compile(r"\{\{(.*?)\}\}", re.DOTALL)
def _find_yaml_files(path: Path) -> list[Path]:
"""Find Ansible YAML files (tasks, playbooks, handlers) in a path."""
if path.is_file():
return [path]
files: list[Path] = []
for pattern in ["**/*.yml", "**/*.yaml"]:
files.extend(path.glob(pattern))
# Exclude molecule scenarios — they have their own variables.
return [f for f in files if "molecule" not in f.parts]
def _extract_expressions(content: str) -> list[str]:
"""Extract Jinja expressions from file content.
Filters out Go template syntax (``{{.Field}}``) used in docker
inspect --format strings, and single-character fragments from
quoted strings that aren't real Jinja expressions.
"""
expressions = []
for match in EXPR_PATTERN.finditer(content):
raw = match.group(1)
# Skip multi-line expressions (often have YAML formatting artifacts).
if "\n" in raw:
continue
expr = raw.strip()
# Skip empty, control flow, and single-char fragments.
if not expr or expr.startswith("%") or len(expr) <= 1:
continue
# Skip Go template syntax (docker inspect --format).
if expr.startswith(".") or "println" in expr:
continue
# Skip expressions containing Go template dot-access patterns.
if ".State." in expr or ".NetworkSettings." in expr:
continue
# Skip expressions with unbalanced parens/brackets/braces —
# the regex captured only part of a larger expression where
# }} appears inside a dict literal (e.g. default({'k': {}})).
if expr.count("(") != expr.count(")"):
continue
if expr.count("{") != expr.count("}"):
continue
if expr.count("[") != expr.count("]"):
continue
expressions.append(expr)
return expressions
def _render_expression(expr: str) -> tuple[bool, str]:
"""Try to render a Jinja expression. Returns (success, error_msg)."""
try:
env = Environment(autoescape=False, keep_trailing_newline=True) # nosec B701 — Ansible Jinja, not web-facing # noqa: S701
# Add common Ansible filters so expressions can render.
# strftime: Ansible's signature is strftime(string_format, second, utc)
# where string_format is the piped value. If the piped value looks like
# a number (epoch) and second looks like a format string, the args are
# reversed — this is the exact bug from OBL-INFRA-508.
def _strftime(string_format: str, second: float | None = None, utc: bool = False) -> str:
if isinstance(string_format, (int, float)) and isinstance(second, str) and "%" in second:
raise ValueError( # noqa: TRY301
"Invalid value for epoch value — strftime filter arguments "
"are reversed. The format string must be the piped value: "
"'%format%' | strftime(epoch), not epoch | strftime('%format%')"
)
return str(string_format)
env.filters["strftime"] = _strftime
env.filters["b64decode"] = lambda x: x
env.filters["b64encode"] = lambda x: x
env.filters["regex_replace"] = lambda x, pattern, replacement="": x
env.filters["int"] = lambda x, default=0: (
int(x) if isinstance(x, (int, float, str)) and str(x).lstrip("-").isdigit() else default
)
env.filters["bool"] = bool
env.filters["basename"] = lambda x: str(x).rsplit("/", 1)[-1]
env.filters["dirname"] = lambda x: str(x).rsplit("/", 1)[0] if "/" in str(x) else "."
env.filters["combine"] = lambda *args, **kwargs: args[0]
env.filters["from_json"] = lambda x: x
env.filters["to_json"] = lambda x: x
env.filters["ternary"] = lambda x, true_val, false_val=None: true_val if x else false_val
env.filters["dict2items"] = lambda x: [
{"key": k, "value": v} for k, v in (x.items() if isinstance(x, dict) else [])
]
env.filters["map"] = lambda x, attribute=None: x
env.filters["default"] = lambda x, default_value="", boolean=False: x if x else default_value
env.filters["from_yaml"] = lambda x: x
env.filters["difference"] = lambda x, y: x
env.filters["join"] = lambda x, sep="": sep.join(str(i) for i in (x if isinstance(x, list) else [x]))
env.filters["list"] = lambda x: list(x) if isinstance(x, (list, tuple)) else [x]
env.filters["length"] = lambda x: len(x) if hasattr(x, "__len__") else 0
env.filters["items"] = lambda x: list(x.items()) if isinstance(x, dict) else []
env.filters["first"] = lambda x: x[0] if isinstance(x, (list, str)) and x else x
env.filters["last"] = lambda x: x[-1] if isinstance(x, (list, str)) and x else x
env.filters["upper"] = lambda x: str(x).upper()
env.filters["lower"] = lambda x: str(x).lower()
env.filters["replace"] = lambda x, old, new: str(x).replace(old, new)
env.filters["split"] = lambda x, sep=None: str(x).split(sep) if sep else str(x).split()
env.filters["trim"] = lambda x: str(x).strip()
env.filters["sort"] = lambda x: sorted(x) if isinstance(x, list) else x
env.filters["unique"] = lambda x: list(set(x)) if isinstance(x, list) else x
env.filters["count"] = lambda x: len(x) if hasattr(x, "__len__") else 0
env.filters["float"] = lambda x, default=0.0: (
float(x) if isinstance(x, (int, float, str)) and str(x).replace(".", "").lstrip("-").isdigit() else default
)
env.filters["string"] = str
env.filters["indent"] = lambda x, width=4: str(x)
env.filters["to_nice_json"] = str
env.filters["to_nice_yaml"] = str
env.filters["from_yaml_all"] = lambda x: x
env.filters["groupby"] = lambda x: x
env.filters["dictsort"] = lambda x: list(x.items()) if isinstance(x, dict) else []
env.filters["max"] = lambda x: max(x) if isinstance(x, list) and x else x
env.filters["min"] = lambda x: min(x) if isinstance(x, list) and x else x
env.filters["reverse"] = lambda x: list(reversed(x)) if isinstance(x, list) else x
env.filters["flatten"] = lambda x: x
env.filters["product"] = lambda x: x
env.filters["zip"] = lambda x: x
env.filters["subelements"] = lambda x: x
env.filters["json_query"] = lambda x: x
env.filters["type_debug"] = lambda x: type(x).__name__
env.globals["lookup"] = lambda *args, **kwargs: ""
env.globals["query"] = lambda *args, **kwargs: []
template = env.from_string("{{ " + expr + " }}")
result = template.render(**MOCK_CONTEXT)
except TemplateSyntaxError as e:
return False, f"Syntax error: {e.message}"
except UndefinedError as e:
# Undefined variable — skip, we can't mock everything.
return True, f"Skipped (undefined: {e})"
except Exception as e:
# Check if it's a filter argument error.
error_msg = str(e)
if "Invalid value for epoch" in error_msg:
return False, f"strftime filter argument error: {error_msg}"
# Other errors might be due to missing mock variables — skip.
return True, f"Skipped ({type(e).__name__}: {error_msg})"
else:
return True, result
def _check_file(filepath: Path, repo_root: Path) -> list[str]:
"""Check all Jinja expressions in a file. Returns list of violations."""
violations = []
content = filepath.read_text()
expressions = _extract_expressions(content)
for expr in expressions:
success, msg = _render_expression(expr)
if not success:
try:
rel_path = filepath.relative_to(repo_root)
except ValueError:
rel_path = filepath
violations.append(f"{rel_path}: `{{{{ {expr} }}}}` — {msg}")
return violations
@click.command()
@click.option(
"--path",
type=click.Path(exists=True, path_type=Path),
help="Check a specific file or directory (default: ansible/playbooks + ansible/roles).",
)
@click.option(
"--ansible-dir",
"ansible_dirs",
type=click.Path(exists=True, path_type=Path),
multiple=True,
default=None,
help="Override the default ansible directories (can be repeated). Defaults to ansible/playbooks and ansible/roles.",
)
def main(path: Path | None, ansible_dirs: tuple[Path, ...]) -> None:
"""Validate Jinja2 expressions in Ansible files."""
dirs = list(ansible_dirs) if ansible_dirs else _default_ansible_dirs()
if path:
files = _find_yaml_files(path)
else:
files: list[Path] = []
for d in dirs:
files.extend(_find_yaml_files(d))
all_violations: list[str] = []
for f in files:
all_violations.extend(_check_file(f, REPO_ROOT))
if all_violations:
click.echo("[check-jinja-expr] FAIL: invalid Jinja expressions found:")
for v in all_violations:
click.echo(f" - {v}")
click.echo("\nFix: test expressions with `ansible localhost -m debug -a 'msg={{ <expr> }}'`")
sys.exit(1)
else:
click.echo("[check-jinja-expr] OK: all Jinja expressions render correctly.")
if __name__ == "__main__": # pragma: no cover
main()
+4865 -3551
View File
File diff suppressed because it is too large Load Diff
+169
View File
@@ -0,0 +1,169 @@
"""Unit tests for devx.ci.check_pr_size."""
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.ci.check_pr_size import (
check_size,
cli,
get_diff_stats,
has_refactoring_label,
is_excluded,
)
class TestIsExcluded:
def test_excludes_changelog(self) -> None:
assert is_excluded("CHANGELOG.md", ["CHANGELOG.md"])
def test_excludes_svg_glob(self) -> None:
assert is_excluded("docs/badges/coverage.svg", ["*.svg"])
def test_does_not_exclude_source(self) -> None:
assert not is_excluded("src/devx/ci/check_pr_size.py", ["CHANGELOG.md", "*.svg"])
def test_excludes_readme(self) -> None:
assert is_excluded("README.md", ["README.md"])
class TestCheckSize:
def test_under_limits_passes(self) -> None:
stats = [("src/main.py", 100, 50), ("tests/test_main.py", 80, 20)]
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
assert ok is True
assert "250" in detail # 100+50+80+20
def test_over_lines_fails(self) -> None:
stats = [("src/main.py", 300, 300)]
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
assert ok is False
assert "600" in detail
def test_over_files_fails(self) -> None:
stats = [(f"src/file{i}.py", 10, 5) for i in range(15)]
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
assert ok is False
assert "15" in detail
def test_excluded_files_not_counted(self) -> None:
stats = [("CHANGELOG.md", 500, 500), ("src/main.py", 10, 5)]
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=["CHANGELOG.md"])
assert ok is True
assert "15" in detail # only 10+5
def test_empty_stats_passes(self) -> None:
ok, detail = check_size([], max_lines=500, max_files=10, excluded_patterns=[])
assert ok is True
class TestGetDiffStats:
@patch("devx.ci.check_pr_size.subprocess.run")
def test_parses_numstat_output(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="10\t5\tsrc/main.py\n20\t10\ttests/test_main.py\n",
stderr="",
)
stats = get_diff_stats("origin/master", "HEAD")
assert len(stats) == 2
assert stats[0] == ("src/main.py", 10, 5)
assert stats[1] == ("tests/test_main.py", 20, 10)
@patch("devx.ci.check_pr_size.subprocess.run")
def test_handles_binary_files(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="-\t-\timage.png\n",
stderr="",
)
stats = get_diff_stats("origin/master", "HEAD")
assert len(stats) == 1
assert stats[0] == ("image.png", 0, 0)
@patch("devx.ci.check_pr_size.subprocess.run")
def test_empty_output(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0, stdout="", stderr="")
stats = get_diff_stats("origin/master", "HEAD")
assert stats == []
@patch("devx.ci.check_pr_size.subprocess.run")
def test_git_diff_failure_raises(self, mock_run: MagicMock) -> None:
import pytest
mock_run.return_value = MagicMock(returncode=1, stdout="", stderr="fatal: bad ref")
with pytest.raises(Exception, match="git diff|bad ref"):
get_diff_stats("origin/master", "HEAD")
@patch("devx.ci.check_pr_size.subprocess.run")
def test_malformed_line_skipped(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="not_a_valid_line\n10\t5\tsrc/main.py\n",
stderr="",
)
stats = get_diff_stats("origin/master", "HEAD")
assert len(stats) == 1
assert stats[0] == ("src/main.py", 10, 5)
class TestCli:
@patch("devx.ci.check_pr_size.subprocess.run")
def test_passes_when_small(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="10\t5\tsrc/main.py\n",
stderr="",
)
runner = CliRunner()
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
assert result.exit_code == 0
assert "OK" in result.output
@patch("devx.ci.check_pr_size.subprocess.run")
def test_fails_when_too_large(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="300\t300\tsrc/main.py\n",
stderr="",
)
runner = CliRunner()
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD", "--max-lines", "500"])
assert result.exit_code != 0
assert "600" in result.output
@patch("devx.ci.check_pr_size.subprocess.run")
@patch("devx.ci.check_pr_size.has_refactoring_label", return_value=True)
def test_bypasses_with_refactoring_label(self, mock_label: MagicMock, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(
returncode=0,
stdout="300\t300\tsrc/main.py\n",
stderr="",
)
runner = CliRunner()
result = runner.invoke(
cli,
["--base", "origin/master", "--head", "HEAD", "--repo", "owner/repo", "--pr-number", "42"],
)
assert result.exit_code == 0
assert "bypassed" in result.output.lower()
class TestHasRefactoringLabel:
@patch("devx.ci.check_pr_size.GiteaClient")
@patch("devx.ci.check_pr_size.get_ci_token", return_value="fake-token")
def test_returns_true_when_label_present(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.get_pr.return_value = {"labels": [{"name": "refactoring"}, {"name": "bug"}]}
assert has_refactoring_label("owner/repo", 42) is True
@patch("devx.ci.check_pr_size.GiteaClient")
@patch("devx.ci.check_pr_size.get_ci_token", return_value="fake-token")
def test_returns_false_when_label_absent(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.get_pr.return_value = {"labels": [{"name": "bug"}]}
assert has_refactoring_label("owner/repo", 42) is False
@patch("devx.ci.check_pr_size.get_ci_token", side_effect=Exception("no token"))
def test_returns_false_on_error(self, mock_token: MagicMock) -> None:
assert has_refactoring_label("owner/repo", 42) is False
-15
View File
@@ -40,7 +40,6 @@ class TestCliGroups:
result = runner.invoke(cli, ["molecule", "--help"])
assert result.exit_code == 0
assert "distribute" in result.output
assert "guard" in result.output
assert "all" in result.output
@@ -108,13 +107,6 @@ class TestCiCommands:
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.post_merge", ["DEVX-1"])
@patch("devx.cli._run_module")
def test_ci_pr_review(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["ci", "pr-review", "42"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.pr_review", ["42"])
@patch("devx.cli._run_module")
def test_ci_publish(self, mock_run: MagicMock) -> None:
runner = CliRunner()
@@ -231,13 +223,6 @@ class TestMoleculeCommands:
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.molecule.discover_runners", [])
@patch("devx.cli._run_module")
def test_molecule_guard(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["molecule", "guard"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.molecule.molecule_ci_guard", [])
@patch("devx.cli._run_module")
def test_molecule_all(self, mock_run: MagicMock) -> None:
runner = CliRunner()
+182
View File
@@ -0,0 +1,182 @@
"""Unit tests for devx.ci.create_dependency_pr."""
from pathlib import Path
from unittest.mock import MagicMock, patch
import click
from click.testing import CliRunner
from devx.ci.create_dependency_pr import (
cli,
create_vikunja_task,
find_existing_pr,
find_pinned_version,
update_pinned_version,
)
class TestFindPinnedVersion:
def test_finds_pip_git_pin(self, tmp_path: Path) -> None:
content = "grm @ git+https://git.example.com/repo.git@v0.5.1"
path = tmp_path / "pyproject.toml"
path.write_text(content)
version = find_pinned_version("grm", str(path))
assert version == "0.5.1"
def test_finds_pyproject_pin(self, tmp_path: Path) -> None:
content = 'grm = "0.5.1"'
path = tmp_path / "pyproject.toml"
path.write_text(content)
version = find_pinned_version("grm", str(path))
assert version == "0.5.1"
def test_finds_ansible_var_pin(self, tmp_path: Path) -> None:
content = 'grm_version: "0.5.1"'
path = tmp_path / "images.yml"
path.write_text(content)
version = find_pinned_version("grm", str(path))
assert version == "0.5.1"
def test_finds_image_version_pin(self, tmp_path: Path) -> None:
content = 'sso_bridge_image_version: "1.2.3"'
path = tmp_path / "images.yml"
path.write_text(content)
version = find_pinned_version("sso_bridge", str(path))
assert version == "1.2.3"
def test_returns_none_when_not_found(self, tmp_path: Path) -> None:
path = tmp_path / "pyproject.toml"
path.write_text('other = "1.0.0"')
assert find_pinned_version("grm", str(path)) is None
def test_returns_none_when_file_missing(self, tmp_path: Path) -> None:
assert find_pinned_version("grm", str(tmp_path / "nonexistent.toml")) is None
class TestUpdatePinnedVersion:
def test_updates_pip_git_pin(self, tmp_path: Path) -> None:
content = "grm @ git+https://git.example.com/repo.git@v0.5.1"
path = tmp_path / "pyproject.toml"
path.write_text(content)
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
assert changed is True
assert "0.5.2" in path.read_text()
assert "0.5.1" not in path.read_text()
def test_updates_pyproject_pin(self, tmp_path: Path) -> None:
content = 'grm = "0.5.1"'
path = tmp_path / "pyproject.toml"
path.write_text(content)
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
assert changed is True
assert 'grm = "0.5.2"' in path.read_text()
def test_no_change_when_version_not_found(self, tmp_path: Path) -> None:
content = 'other = "1.0.0"'
path = tmp_path / "pyproject.toml"
path.write_text(content)
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
assert changed is False
def test_no_change_when_file_missing(self, tmp_path: Path) -> None:
changed = update_pinned_version(str(tmp_path / "nonexistent"), "grm", "0.5.1", "0.5.2")
assert changed is False
class TestFindExistingPr:
@patch("devx.tools.create_pr.GiteaClient")
def test_returns_pr_when_found(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.list_prs.return_value = [
{"head": {"ref": "deps/grm-0.5.2"}, "number": 42},
{"head": {"ref": "other-branch"}, "number": 43},
]
result = find_existing_pr(mock_client, "deps/grm-0.5.2")
assert result is not None
assert result["number"] == 42
@patch("devx.tools.create_pr.GiteaClient")
def test_returns_none_when_not_found(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.list_prs.return_value = []
result = find_existing_pr(mock_client, "deps/grm-0.5.2")
assert result is None
class TestCli:
@patch("devx.ci.create_dependency_pr.find_pinned_version")
@patch("devx.ci.create_dependency_pr.get_ci_token")
def test_same_version_no_pr(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_find.return_value = "0.5.2"
runner = CliRunner()
result = runner.invoke(
cli,
[
"--package",
"grm",
"--new-version",
"0.5.2",
"--source-repo",
"oblachno/grm",
],
)
assert result.exit_code == 0
assert "no pr needed" in result.output.lower()
@patch("devx.ci.create_dependency_pr.find_pinned_version")
@patch("devx.ci.create_dependency_pr.get_ci_token")
def test_dry_run(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_find.return_value = "0.5.1"
runner = CliRunner()
result = runner.invoke(
cli,
[
"--package",
"grm",
"--new-version",
"0.5.2",
"--source-repo",
"oblachno/grm",
"--dry-run",
],
)
assert result.exit_code == 0
assert "DRY RUN" in result.output
@patch("devx.ci.create_dependency_pr.find_pinned_version")
@patch("devx.ci.create_dependency_pr.get_ci_token")
def test_version_not_found_fails(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_find.return_value = None
runner = CliRunner()
result = runner.invoke(
cli,
[
"--package",
"nonexistent",
"--new-version",
"1.0.0",
"--source-repo",
"oblachno/test",
],
)
assert result.exit_code != 0
class TestCreateVikunjaTask:
def test_returns_none_when_no_token(self) -> None:
with patch("devx.ci.create_dependency_pr.get_vikunja_token", side_effect=click.ClickException("no token")):
result = create_vikunja_task("Test", "desc")
assert result is None
def test_returns_identifier_on_success(self) -> None:
with (
patch("devx.ci.create_dependency_pr.get_vikunja_token", return_value="fake-token"),
patch("devx.api_clients.VikunjaClient") as mock_client_cls,
):
mock_client = mock_client_cls.return_value
mock_client.create_task.return_value = {"identifier": "OBL-INFRA-999"}
result = create_vikunja_task("Test", "desc")
assert result == "OBL-INFRA-999"
+82
View File
@@ -0,0 +1,82 @@
"""Unit tests for devx.ci.fast_molecule."""
from pathlib import Path
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.ci.fast_molecule import (
build_molecule_commands,
cli,
get_molecule_scenarios,
)
class TestGetMoleculeScenarios:
def test_finds_scenarios(self, tmp_path: Path) -> None:
roles_dir = tmp_path / "ansible" / "roles" / "myrole" / "molecule"
roles_dir.mkdir(parents=True)
(roles_dir / "default").mkdir()
(roles_dir / "default" / "molecule.yml").write_text("name: default")
(roles_dir / "full").mkdir()
(roles_dir / "full" / "molecule.yml").write_text("name: full")
(roles_dir / "no_scenario").mkdir() # No molecule.yml
scenarios = get_molecule_scenarios("myrole", str(tmp_path / "ansible" / "roles"))
assert sorted(scenarios) == ["default", "full"]
def test_returns_empty_when_no_molecule_dir(self, tmp_path: Path) -> None:
scenarios = get_molecule_scenarios("nonexistent", str(tmp_path / "ansible" / "roles"))
assert scenarios == []
class TestBuildMoleculeCommands:
def test_builds_commands_for_roles(self, tmp_path: Path) -> None:
roles_dir = tmp_path / "ansible" / "roles"
for role in ["role_a", "role_b"]:
mol_dir = roles_dir / role / "molecule" / "default"
mol_dir.mkdir(parents=True)
(mol_dir / "molecule.yml").write_text("name: default")
commands = build_molecule_commands({"role_a", "role_b"}, str(roles_dir))
assert len(commands) == 2
assert all("molecule test -s default" in c for c in commands)
assert all("--destroy=never" in c for c in commands)
assert all("ubuntu-2604" in c for c in commands)
def test_empty_when_no_scenarios(self, tmp_path: Path) -> None:
commands = build_molecule_commands({"nonexistent"}, str(tmp_path / "ansible" / "roles"))
assert commands == []
def test_empty_when_no_roles(self) -> None:
assert build_molecule_commands(set()) == []
class TestCli:
@patch("devx.ci.fast_molecule.get_changed_files")
def test_no_changes(self, mock_get: MagicMock) -> None:
mock_get.return_value = []
runner = CliRunner()
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
assert result.exit_code == 0
assert "No files changed" in result.output
@patch("devx.ci.fast_molecule.detect_changed_roles")
@patch("devx.ci.fast_molecule.get_changed_files")
def test_no_ansible_changes(self, mock_get: MagicMock, mock_detect: MagicMock) -> None:
mock_get.return_value = ["src/main.py", "README.md"]
mock_detect.return_value = set()
runner = CliRunner()
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
assert result.exit_code == 0
assert "No Ansible roles changed" in result.output
@patch("devx.ci.fast_molecule.detect_changed_roles")
@patch("devx.ci.fast_molecule.get_changed_files")
def test_detects_changed_roles(self, mock_get: MagicMock, mock_detect: MagicMock) -> None:
mock_get.return_value = ["ansible/roles/sso_bridge/tasks/main.yml"]
mock_detect.return_value = {"sso_bridge"}
runner = CliRunner()
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
assert result.exit_code == 0
assert "sso_bridge" in result.output
+126 -6
View File
@@ -7,6 +7,7 @@ import subprocess # nosec B404
import time
from unittest.mock import MagicMock, patch
import pytest
from click.testing import CliRunner
from devx.ci.integration_guard import cli
@@ -111,9 +112,8 @@ class TestCli:
clear=True,
),
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
@@ -158,9 +158,8 @@ class TestCli:
clear=True,
),
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg", side_effect=ProcessLookupError("no such process")),
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
@@ -203,9 +202,8 @@ class TestCli:
clear=True,
),
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
@@ -289,3 +287,125 @@ def test_main_module_block() -> None:
namespace = dict(ig.__dict__)
exec(compile(source, ig.__file__, "exec"), namespace)
assert callable(namespace["cli"])
class TestGetRunningJobs:
def test_returns_jobs(self) -> None:
with patch("devx.ci.integration_guard.requests.get") as mock_get:
mock_response = MagicMock()
mock_response.json.return_value = {
"jobs": [
{"name": "integration-tests (0)", "conclusion": "success"},
{"name": "integration-tests (1)", "conclusion": "failure"},
]
}
mock_response.raise_for_status.return_value = None
mock_get.return_value = mock_response
from devx.ci.integration_guard import get_running_jobs
jobs = get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
assert len(jobs) == 2
mock_get.assert_called_once()
def test_raises_on_request_error(self) -> None:
import requests
with patch("devx.ci.integration_guard.requests.get") as mock_get:
mock_get.side_effect = requests.RequestException("boom")
with pytest.raises(requests.RequestException):
from devx.ci.integration_guard import get_running_jobs
get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
class TestAnyOtherRunnerFailed:
def test_detects_other_failure(self) -> None:
from devx.ci.integration_guard import any_other_runner_failed
jobs = [
{"name": "integration-tests (0)", "conclusion": "success"},
{"name": "integration-tests (1)", "conclusion": "failure"},
{"name": "integration-tests (2)", "conclusion": "running"},
]
assert any_other_runner_failed(jobs, "integration-tests", 0) is True
def test_ignores_current_runner(self) -> None:
from devx.ci.integration_guard import any_other_runner_failed
jobs = [
{"name": "integration-tests (0)", "conclusion": "failure"},
{"name": "integration-tests (1)", "conclusion": "success"},
]
assert any_other_runner_failed(jobs, "integration-tests", 0) is False
def test_ignores_non_matching_jobs(self) -> None:
from devx.ci.integration_guard import any_other_runner_failed
jobs = [
{"name": "quality", "conclusion": "failure"},
{"name": "integration-tests (1)", "conclusion": "success"},
]
assert any_other_runner_failed(jobs, "integration-tests", 0) is False
class TestPollForOtherFailures:
def test_sets_failed_event_when_other_runner_fails(self) -> None:
from devx.ci.integration_guard import poll_for_other_failures
stop_event = MagicMock()
failed_event = MagicMock()
def side_effect(*args, **kwargs):
if stop_event.wait.call_count < 1:
return [
{"name": "integration-tests (0)", "conclusion": "success"},
{"name": "integration-tests (1)", "conclusion": "failure"},
]
return []
with patch("devx.ci.integration_guard.get_running_jobs") as mock_get_jobs:
mock_get_jobs.side_effect = side_effect
stop_event.is_set.side_effect = [False, False]
stop_event.wait.return_value = True
poll_for_other_failures(
"https://gitea.example",
"owner",
"repo",
"token",
123,
"integration-tests",
0,
stop_event,
failed_event,
)
failed_event.set.assert_called_once()
def test_poll_warns_on_api_error(self) -> None:
import requests
from devx.ci.integration_guard import poll_for_other_failures
stop_event = MagicMock()
failed_event = MagicMock()
with patch("devx.ci.integration_guard.get_running_jobs") as mock_get_jobs:
mock_get_jobs.side_effect = requests.RequestException("boom")
stop_event.is_set.side_effect = [False, True]
stop_event.wait.return_value = True
poll_for_other_failures(
"https://gitea.example",
"owner",
"repo",
"token",
123,
"integration-tests",
0,
stop_event,
failed_event,
)
failed_event.set.assert_not_called()
+192
View File
@@ -0,0 +1,192 @@
"""Unit tests for devx.molecule.molecule_changed.
Verifies that the script correctly detects changed roles and maps
them to make targets.
"""
from __future__ import annotations
from unittest.mock import patch
from click.testing import CliRunner
from devx.molecule.molecule_changed import (
detect_changed_roles,
get_changed_files,
main,
roles_to_targets,
)
def test_detect_role_change():
"""A file in ansible/roles/<role>/ maps to that role."""
files = ["ansible/roles/docker_base/tasks/main.yml"]
roles = detect_changed_roles(files)
assert "docker_base" in roles
def test_detect_playbook_change():
"""A playbook change maps to its included roles."""
files = ["ansible/playbooks/deploy-observability.yml"]
roles = detect_changed_roles(files)
assert "observability" in roles
assert "docker_base" in roles
assert "zitadel" in roles
def test_detect_shared_infra_triggers_all():
"""ansible.cfg change triggers all roles."""
files = ["ansible/ansible.cfg"]
roles = detect_changed_roles(files)
assert len(roles) == 10 # all roles
def test_detect_no_ansible_changes():
"""Non-Ansible files don't trigger any roles."""
files = ["scripts/molecule_changed.py", "Makefile"]
roles = detect_changed_roles(files)
assert len(roles) == 0
def test_roles_to_targets():
"""Role names map to make targets."""
targets = roles_to_targets({"docker_base", "zitadel"})
assert "molecule-docker-base" in targets
assert "molecule-zitadel" in targets
def test_roles_to_targets_unknown_role():
"""Unknown roles are silently skipped."""
targets = roles_to_targets({"docker_base", "unknown_role"})
assert targets == ["molecule-docker-base"]
def test_main_no_changes():
"""When no files changed, outputs message to stderr."""
with patch("devx.molecule.molecule_changed.get_changed_files", return_value=[]):
runner = CliRunner()
result = runner.invoke(main, ["--print-targets"])
assert result.exit_code == 0
assert "No changed files" in result.output
def test_main_print_targets():
"""--print-targets outputs make targets."""
with patch(
"devx.molecule.molecule_changed.get_changed_files",
return_value=["ansible/roles/docker_base/tasks/main.yml"],
):
runner = CliRunner()
result = runner.invoke(main, ["--print-targets"])
assert result.exit_code == 0
assert "molecule-docker-base" in result.output
def test_main_print_roles():
"""--print-roles outputs role names."""
with patch(
"devx.molecule.molecule_changed.get_changed_files",
return_value=["ansible/roles/zitadel/tasks/main.yml"],
):
runner = CliRunner()
result = runner.invoke(main, ["--print-roles"])
assert result.exit_code == 0
assert "zitadel" in result.output
def test_main_no_ansible_changes():
"""When only non-Ansible files changed, outputs no scenarios message."""
with patch(
"devx.molecule.molecule_changed.get_changed_files",
return_value=["scripts/molecule_changed.py"],
):
runner = CliRunner()
result = runner.invoke(main, ["--print-targets"])
assert result.exit_code == 0
assert "No molecule scenarios" in result.output
def test_get_changed_files_with_mock():
"""get_changed_files returns files from git diff."""
with patch("devx.molecule.molecule_changed._run_git", return_value="file1\nfile2\n"):
files = get_changed_files("origin/master")
assert files == ["file1", "file2"]
def test_get_changed_files_falls_back_to_master():
"""When base ref has no diff, falls back to master."""
calls: list[list[str]] = []
def mock_git(args):
calls.append(args)
# First call (origin/master) returns empty, second (master) returns files
if "origin/master...HEAD" in args[2]:
return ""
return "ansible/roles/docker_base/tasks/main.yml\n"
with patch("devx.molecule.molecule_changed._run_git", side_effect=mock_git):
files = get_changed_files("origin/master")
assert files == ["ansible/roles/docker_base/tasks/main.yml"]
assert len(calls) == 2
def test_get_changed_files_empty():
"""When no changes in either ref, returns empty list."""
with patch("devx.molecule.molecule_changed._run_git", return_value=""):
files = get_changed_files("origin/master")
assert files == []
def test_detect_molecule_shared_path():
"""ansible/molecule/ change triggers all roles."""
files = ["ansible/molecule/Dockerfile"]
roles = detect_changed_roles(files)
assert len(roles) == 10
def test_detect_requirements_yml_triggers_all():
"""ansible/requirements.yml change triggers all roles."""
files = ["ansible/requirements.yml"]
roles = detect_changed_roles(files)
assert len(roles) == 10
def test_detect_configure_oidc_playbook():
"""configure-oidc.yml maps to sso_config and app_container."""
files = ["ansible/playbooks/configure-oidc.yml"]
roles = detect_changed_roles(files)
assert "sso_config" in roles
assert "app_container" in roles
def test_detect_prepare_vms_playbook():
"""prepare-vms.yml maps to all base roles."""
files = ["ansible/playbooks/prepare-vms.yml"]
roles = detect_changed_roles(files)
assert "docker_base" in roles
assert "app_hardening" in roles
assert "storage" in roles
assert "disk_cleanup" in roles
assert "crowdsec" in roles
def test_detect_deploy_customer_playbook():
"""deploy-customer.yml maps to its roles."""
files = ["ansible/playbooks/deploy-customer.yml"]
roles = detect_changed_roles(files)
assert "app_container" in roles
assert "docker_base" in roles
assert "app_hardening" in roles
assert "sso_config" in roles
def test_main_default_base():
"""main() with no --base uses origin/master."""
with patch(
"devx.molecule.molecule_changed.get_changed_files",
return_value=["ansible/roles/zitadel/tasks/main.yml"],
) as mock:
runner = CliRunner()
result = runner.invoke(main, ["--print-roles"])
assert result.exit_code == 0
mock.assert_called_once_with("origin/master")
-591
View File
@@ -1,591 +0,0 @@
"""Unit tests for scripts/ci/molecule_ci_guard.py."""
from __future__ import annotations
import os
import subprocess # nosec B404
import time
from pathlib import Path
from unittest.mock import MagicMock, patch
import click
import pytest
import requests
from devx.molecule.molecule_ci_guard import (
any_other_runner_failed,
build_env_for_pair,
build_molecule_cmd,
cli,
get_running_jobs,
parse_pair,
poll_for_other_failures,
resolve_role_dir,
)
class TestGetRunningJobs:
def test_returns_jobs(self) -> None:
with patch("devx.molecule.molecule_ci_guard.requests.get") as mock_get:
mock_response = MagicMock()
mock_response.json.return_value = {
"jobs": [
{"name": "molecule-tests (0)", "conclusion": "success"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
}
mock_response.raise_for_status.return_value = None
mock_get.return_value = mock_response
jobs = get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
assert len(jobs) == 2
mock_get.assert_called_once()
def test_raises_on_request_error(self) -> None:
with patch("devx.molecule.molecule_ci_guard.requests.get") as mock_get:
mock_get.side_effect = requests.RequestException("boom")
with pytest.raises(requests.RequestException):
get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
class TestAnyOtherRunnerFailed:
def test_detects_other_failure(self) -> None:
jobs = [
{"name": "molecule-tests (0)", "conclusion": "success"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
{"name": "molecule-tests (2)", "conclusion": "running"},
]
assert any_other_runner_failed(jobs, "molecule-tests", 0) is True
def test_ignores_current_runner(self) -> None:
jobs = [
{"name": "molecule-tests (0)", "conclusion": "failure"},
{"name": "molecule-tests (1)", "conclusion": "success"},
]
assert any_other_runner_failed(jobs, "molecule-tests", 0) is False
def test_ignores_non_molecule_jobs(self) -> None:
jobs = [
{"name": "quality", "conclusion": "failure"},
{"name": "molecule-tests (1)", "conclusion": "success"},
]
assert any_other_runner_failed(jobs, "molecule-tests", 0) is False
class TestBuildMoleculeCmd:
def test_default_scenario(self) -> None:
assert build_molecule_cmd("default") == ["molecule", "test"]
def test_named_scenario(self) -> None:
assert build_molecule_cmd("lifecycle") == ["molecule", "test", "-s", "lifecycle"]
class TestBuildEnvForPair:
def test_with_command(self) -> None:
env = build_env_for_pair("default|ubuntu-2204|img:latest|/lib/systemd/systemd", {})
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2204"
assert env["MOLECULE_PLATFORM_IMAGE"] == "img:latest"
assert env["MOLECULE_PLATFORM_COMMAND"] == "/lib/systemd/systemd"
assert env["ANSIBLE_ALLOW_BROKEN_CONDITIONALS"] == "true"
def test_without_command(self) -> None:
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {})
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2204"
assert "MOLECULE_PLATFORM_COMMAND" not in env
def test_without_command_removes_existing(self) -> None:
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {"MOLECULE_PLATFORM_COMMAND": "old"})
assert "MOLECULE_PLATFORM_COMMAND" not in env
def test_preserves_existing_molecule_home(self) -> None:
"""When MOLECULE_HOME is already set, it is not overridden."""
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {"MOLECULE_HOME": "/custom/home"})
assert env["MOLECULE_HOME"] == "/custom/home"
def test_appends_matrix_index_to_platform_name(self) -> None:
"""When MATRIX_INDEX is set, platform name gets a unique suffix."""
env = build_env_for_pair("default|ubuntu-2604|img:latest|sleep infinity", {"MATRIX_INDEX": "3"})
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2604-r3"
def test_no_matrix_index_keeps_platform_name(self) -> None:
"""Without MATRIX_INDEX, platform name is unchanged."""
env = build_env_for_pair("default|ubuntu-2604|img:latest|sleep infinity", {})
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2604"
class TestPollForOtherFailures:
def test_sets_failed_event_when_other_runner_fails(self) -> None:
stop_event = MagicMock()
failed_event = MagicMock()
def side_effect(*args, **kwargs):
if stop_event.wait.call_count < 1:
return [
{"name": "molecule-tests (0)", "conclusion": "success"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
return []
with patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs:
mock_get_jobs.side_effect = side_effect
stop_event.is_set.side_effect = [False, False]
stop_event.wait.return_value = True
poll_for_other_failures(
"https://gitea.example",
"owner",
"repo",
"token",
123,
"molecule-tests",
0,
stop_event,
failed_event,
)
failed_event.set.assert_called_once()
def test_poll_warns_on_api_error(self) -> None:
stop_event = MagicMock()
failed_event = MagicMock()
with patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs:
mock_get_jobs.side_effect = requests.RequestException("boom")
stop_event.is_set.side_effect = [False, True]
stop_event.wait.return_value = True
poll_for_other_failures(
"https://gitea.example",
"owner",
"repo",
"token",
123,
"molecule-tests",
0,
stop_event,
failed_event,
)
failed_event.set.assert_not_called()
class TestCli:
def test_all_pass(self) -> None:
from click.testing import CliRunner
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("time.sleep"),
):
proc = MagicMock()
proc.poll.return_value = 0
proc.returncode = 0
mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 0
assert "All molecule tests passed" in result.output
# Verify Docker prune was called between scenarios
mock_run.assert_called_once_with(
["docker", "system", "prune", "-af", "--volumes"],
check=False,
capture_output=True,
timeout=60,
)
@patch("devx.molecule.molecule_ci_guard.get_ci_token", side_effect=click.ClickException("no token"))
def test_missing_token_runs_without_polling(self, mock_token: MagicMock) -> None:
"""When no token is available, cross-runner polling is skipped."""
from click.testing import CliRunner
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("devx.molecule.molecule_ci_guard.poll_for_other_failures") as mock_poll,
patch("time.sleep"),
):
proc = MagicMock()
proc.poll.return_value = 0
proc.returncode = 0
mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 0
mock_poll.assert_not_called()
def test_invalid_pair_format_raises(self) -> None:
"""Pair with fewer than 2 parts should raise."""
from click.testing import CliRunner
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen"),
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
):
runner = CliRunner()
result = runner.invoke(cli, ["invalid_no_pipe"])
assert result.exit_code != 0
assert "Invalid pair format" in result.output
def test_failure_exits_nonzero(self) -> None:
from click.testing import CliRunner
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("time.sleep"),
):
proc = MagicMock()
proc.poll.return_value = 1
proc.returncode = 1
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
assert "FAILED" in result.output
def test_exits_before_starting_when_already_failed(self) -> None:
from click.testing import CliRunner
with (
patch.dict(
os.environ,
{
"GITEA_URL": "https://gitea.example",
"CI_GITEA_TOKEN": "token",
"RUN_ID": "123",
"JOB_NAME": "molecule-tests",
"MATRIX_INDEX": "0",
"GITEA_REPOSITORY": "my-org/my-repo",
"PATH": os.environ.get("PATH", ""),
},
clear=True,
),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs,
patch("time.sleep"),
):
mock_get_jobs.return_value = [
{"name": "molecule-tests (0)", "conclusion": "running"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
proc = MagicMock()
proc.poll.return_value = 0
proc.returncode = 0
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
assert "Another molecule runner failed" in result.output
def test_keyboard_interrupt_kills_process(self) -> None:
from click.testing import CliRunner
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("time.sleep", side_effect=KeyboardInterrupt),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
):
mock_getpgid.return_value = 123
proc = MagicMock()
proc.poll.return_value = None
proc.wait.return_value = 0
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
mock_killpg.assert_called()
def test_exits_when_other_runner_fails(self) -> None:
from click.testing import CliRunner
real_sleep = time.sleep
call_count = [0]
def get_jobs_side_effect(*args, **kwargs):
call_count[0] += 1
if call_count[0] < 2:
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
return [
{"name": "molecule-tests (0)", "conclusion": "running"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
with (
patch.dict(
os.environ,
{
"GITEA_URL": "https://gitea.example",
"CI_GITEA_TOKEN": "token",
"RUN_ID": "123",
"JOB_NAME": "molecule-tests",
"MATRIX_INDEX": "0",
"GITEA_REPOSITORY": "my-org/my-repo",
"PATH": os.environ.get("PATH", ""),
},
clear=True,
),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
):
mock_getpgid.return_value = 123
proc = MagicMock()
proc.poll.return_value = None
proc.wait.return_value = 0
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
mock_killpg.assert_called()
def test_default_owner_repo_fallback(self) -> None:
from click.testing import CliRunner
real_sleep = time.sleep
with (
patch.dict(
os.environ,
{
"GITEA_URL": "https://gitea.example",
"CI_GITEA_TOKEN": "token",
"RUN_ID": "123",
"JOB_NAME": "molecule-tests",
"MATRIX_INDEX": "0",
"GITEA_REPOSITORY": "invalid",
"PATH": os.environ.get("PATH", ""),
},
clear=True,
),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
):
mock_get_jobs.return_value = [{"name": "molecule-tests (1)", "conclusion": "success"}]
proc = MagicMock()
proc.poll.return_value = 0
proc.returncode = 0
mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 0
def test_exits_when_other_runner_fails_with_process_lookup_error(self) -> None:
from click.testing import CliRunner
real_sleep = time.sleep
call_count = [0]
def get_jobs_side_effect(*args, **kwargs):
call_count[0] += 1
if call_count[0] < 2:
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
return [
{"name": "molecule-tests (0)", "conclusion": "running"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
with (
patch.dict(
os.environ,
{
"GITEA_URL": "https://gitea.example",
"CI_GITEA_TOKEN": "token",
"RUN_ID": "123",
"JOB_NAME": "molecule-tests",
"MATRIX_INDEX": "0",
"GITEA_REPOSITORY": "my-org/my-repo",
"PATH": os.environ.get("PATH", ""),
},
clear=True,
),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
):
mock_getpgid.return_value = 123
mock_killpg.side_effect = ProcessLookupError("no such process")
proc = MagicMock()
proc.poll.return_value = None
proc.wait.return_value = 0
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
def test_exits_when_other_runner_fails_with_timeout(self) -> None:
from click.testing import CliRunner
real_sleep = time.sleep
call_count = [0]
def get_jobs_side_effect(*args, **kwargs):
call_count[0] += 1
if call_count[0] < 2:
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
return [
{"name": "molecule-tests (0)", "conclusion": "running"},
{"name": "molecule-tests (1)", "conclusion": "failure"},
]
with (
patch.dict(
os.environ,
{
"GITEA_URL": "https://gitea.example",
"CI_GITEA_TOKEN": "token",
"RUN_ID": "123",
"JOB_NAME": "molecule-tests",
"MATRIX_INDEX": "0",
"GITEA_REPOSITORY": "my-org/my-repo",
"PATH": os.environ.get("PATH", ""),
},
clear=True,
),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
patch("os.killpg") as mock_killpg,
patch("os.getpgid") as mock_getpgid,
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
):
mock_getpgid.return_value = 123
mock_killpg.side_effect = [None, ProcessLookupError("no such process")]
proc = MagicMock()
proc.poll.return_value = None
proc.wait.side_effect = [subprocess.TimeoutExpired("cmd", 10)]
mock_popen.return_value = proc
runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 1
def test_main_module_block() -> None:
import devx.molecule.molecule_ci_guard as mg
with open(mg.__file__) as f:
source = f.read()
source = source.replace('if __name__ == "__main__":\n cli()\n', "")
namespace = dict(mg.__dict__)
exec(compile(source, mg.__file__, "exec"), namespace)
assert callable(namespace["cli"])
class TestParsePair:
def test_single_role_4_part(self) -> None:
role, scenario, name, image, cmd = parse_pair("default|ubuntu-2204|ubuntu:22.04|")
assert role == ""
assert scenario == "default"
assert name == "ubuntu-2204"
assert image == "ubuntu:22.04"
assert cmd == ""
def test_multi_role_5_part(self) -> None:
role, scenario, name, image, cmd = parse_pair("gitea-runner|default|ubuntu-2204|ubuntu:22.04|")
assert role == "gitea-runner"
assert scenario == "default"
assert name == "ubuntu-2204"
assert image == "ubuntu:22.04"
assert cmd == ""
def test_multi_role_with_command(self) -> None:
role, scenario, name, image, cmd = parse_pair(
"docker-base|lifecycle|archlinux|archlinux:latest|/usr/lib/systemd/systemd"
)
assert role == "docker-base"
assert scenario == "lifecycle"
assert cmd == "/usr/lib/systemd/systemd"
def test_invalid_pair_raises(self) -> None:
with pytest.raises(click.ClickException, match="Invalid pair format"):
parse_pair("only|two|parts")
def test_too_many_parts_raises(self) -> None:
with pytest.raises(click.ClickException, match="Invalid pair format"):
parse_pair("a|b|c|d|e|f")
class TestResolveRoleDir:
def test_multi_role_with_roles_root(self, tmp_path: Path) -> None:
roles_root = tmp_path / "ansible" / "roles"
roles_root.mkdir(parents=True)
result = resolve_role_dir("gitea-runner", roles_root, tmp_path)
assert result == roles_root / "gitea-runner"
def test_multi_role_default_roles_root(self, tmp_path: Path) -> None:
result = resolve_role_dir("docker-base", None, tmp_path)
assert result == tmp_path / "ansible" / "roles" / "docker-base"
def test_single_role_auto_discovers(self, tmp_path: Path) -> None:
"""Single-role mode auto-discovers first role with molecule/ dir."""
roles_dir = tmp_path / "ansible" / "roles"
(roles_dir / "my_role" / "molecule").mkdir(parents=True)
result = resolve_role_dir("", None, tmp_path)
assert result == roles_dir / "my_role"
def test_single_role_no_roles_returns_fallback(self, tmp_path: Path) -> None:
"""When no roles exist, returns a fallback path (will error at runtime)."""
result = resolve_role_dir("", None, tmp_path)
assert "roles" in str(result)
class TestCliMultiRole:
def test_multi_role_pair_passes(self, tmp_path: Path) -> None:
from click.testing import CliRunner
roles_root = tmp_path / "ansible" / "roles"
(roles_root / "gitea-runner").mkdir(parents=True)
with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("time.sleep"),
):
proc = MagicMock()
proc.poll.return_value = 0
proc.returncode = 0
mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner()
result = runner.invoke(
cli,
["--roles-root", str(roles_root), "gitea-runner|default|ubuntu-2204|ubuntu:22.04|"],
)
assert result.exit_code == 0
assert "All molecule tests passed" in result.output
+106
View File
@@ -0,0 +1,106 @@
"""Unit tests for devx.ci.nightly_gate."""
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.ci.nightly_gate import cli, get_nightly_status, set_nightly_status
class TestGetNightlyStatus:
@patch("devx.ci.nightly_gate.GiteaClient")
def test_returns_value_when_set(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.get_repo_variable.return_value = "passed:12345"
result = get_nightly_status(mock_client)
assert result == "passed:12345"
@patch("devx.ci.nightly_gate.GiteaClient")
def test_returns_empty_when_not_set(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
mock_client.get_repo_variable.return_value = None
result = get_nightly_status(mock_client)
assert result == ""
class TestSetNightlyStatus:
@patch("devx.ci.nightly_gate.GiteaClient")
def test_sets_passed(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
set_nightly_status(mock_client, "passed:12345")
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "passed:12345")
@patch("devx.ci.nightly_gate.GiteaClient")
def test_sets_failed(self, mock_client_cls: MagicMock) -> None:
mock_client = mock_client_cls.return_value
set_nightly_status(mock_client, "failed:99999")
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "failed:99999")
class TestCli:
@patch("devx.ci.nightly_gate.GiteaClient")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_check_bootstrap_allows_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_client = mock_client_cls.return_value
mock_client.get_repo_variable.return_value = None
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
assert result.exit_code == 0
assert "bootstrap" in result.output.lower()
@patch("devx.ci.nightly_gate.GiteaClient")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_check_passed_allows_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_client = mock_client_cls.return_value
mock_client.get_repo_variable.return_value = "passed:12345"
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
assert result.exit_code == 0
assert "passed" in result.output.lower()
@patch("devx.ci.nightly_gate.GiteaClient")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_check_failed_blocks_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_client = mock_client_cls.return_value
mock_client.get_repo_variable.return_value = "failed:99999"
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
assert result.exit_code != 0
assert "blocked" in result.output.lower()
@patch("devx.ci.nightly_gate.GiteaClient")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_set_passed(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_client = mock_client_cls.return_value
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "set-passed", "--run-id", "12345"])
assert result.exit_code == 0
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "passed:12345")
@patch("devx.ci.nightly_gate.GiteaClient")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_set_failed(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
mock_token.return_value = "fake-token"
mock_client = mock_client_cls.return_value
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "set-failed", "--run-id", "99999"])
assert result.exit_code == 0
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "failed:99999")
@patch("devx.ci.nightly_gate.get_ci_token")
def test_fails_without_token(self, mock_token: MagicMock) -> None:
import click as click_mod
mock_token.side_effect = click_mod.ClickException("No token")
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
assert result.exit_code != 0
def test_fails_with_invalid_repo(self) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["--repo", "invalid", "--action", "check"])
assert result.exit_code != 0
-1022
View File
@@ -1,1022 +0,0 @@
"""Unit tests for scripts/ci/pr_review.py."""
from unittest.mock import MagicMock, patch
import pytest
from click.testing import CliRunner
from devx.ci.pr_review import (
ReviewResult,
build_review_body,
check_architecture_compliance,
check_best_practices,
check_commit_conventions,
check_documentation,
check_function_length,
check_i18n,
check_resource_management,
check_security,
check_test_coverage,
is_python_file,
is_workflow_only,
main,
post_review,
run_review,
)
from devx.exceptions import APIError
class TestIsPythonFile:
def test_python_file_in_src(self) -> None:
assert is_python_file("src/devx/cli.py") is True
def test_python_file_in_scripts(self) -> None:
assert is_python_file("scripts/ci/release.py") is True
def test_test_file_excluded(self) -> None:
assert is_python_file("tests/unit/test_cli.py") is False
def test_non_python_file(self) -> None:
assert is_python_file("README.md") is False
def test_yaml_file(self) -> None:
assert is_python_file(".gitea/workflows/ci.yml") is False
class TestIsWorkflowOnly:
def test_yaml_is_workflow(self) -> None:
assert is_workflow_only(".gitea/workflows/ci.yml") is True
def test_md_is_workflow(self) -> None:
assert is_workflow_only("README.md") is True
def test_python_is_not_workflow(self) -> None:
assert is_workflow_only("src/devx/cli.py") is False
def test_ansible_is_workflow(self) -> None:
assert is_workflow_only("ansible/tasks/main.yml") is True
class TestReviewResult:
def test_empty_result_has_no_issues(self) -> None:
result = ReviewResult()
assert result.has_issues is False
def test_add_issue_makes_has_issues_true(self) -> None:
result = ReviewResult()
result.add_issue("src/foo.py", 10, "bad code")
assert result.has_issues is True
assert len(result.issues) == 1
assert result.issues[0]["path"] == "src/foo.py"
assert result.issues[0]["new_position"] == 10
def test_add_summary(self) -> None:
result = ReviewResult()
result.add_summary("all good")
assert "all good" in result.summary
class TestCheckArchitectureCompliance:
def test_subprocess_in_cli_triggers_issue(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ subprocess.run(['ls'])\n",
}
]
check_architecture_compliance(files, result)
assert result.has_issues
assert "subprocess" in result.issues[0]["body"].lower()
def test_subprocess_in_other_file_ok(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/executor.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ subprocess.run(['ls'])\n",
}
]
check_architecture_compliance(files, result)
assert not result.has_issues
def test_no_changes_adds_ok_summary(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_architecture_compliance(files, result)
assert any("Architecture compliance: OK" in s for s in result.summary)
def test_non_python_file_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "README.md", "patch": "@@ -1,1 +1,2 @@\n+subprocess.run(['ls'])\n"}]
check_architecture_compliance(files, result)
assert not result.has_issues
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_architecture_compliance(files, result)
assert not result.has_issues
def test_os_system_in_cli_triggers_issue(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ os.system('ls')\n",
}
]
check_architecture_compliance(files, result)
assert result.has_issues
assert "os.system" in result.issues[0]["body"]
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -1,2 @@\n+ subprocess.run(['ls'])\n",
}
]
check_architecture_compliance(files, result)
assert result.has_issues
class TestCheckBestPractices:
def test_print_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ print('hello')\n",
}
]
check_best_practices(files, result)
assert result.has_issues
assert "print()" in result.issues[0]["body"]
def test_bare_except_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/runner_manager.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ except:\n pass\n",
}
]
check_best_practices(files, result)
assert result.has_issues
assert "bare except" in result.issues[0]["body"]
def test_todo_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ # TODO: fix this\n",
}
]
check_best_practices(files, result)
assert result.has_issues
assert "TODO" in result.issues[0]["body"]
def test_clean_code_no_issues(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ click.echo('hello')\n",
}
]
check_best_practices(files, result)
assert not result.has_issues
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_best_practices(files, result)
assert not result.has_issues
def test_non_python_file_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "README.md", "patch": "@@ -1,1 +1,2 @@\n+print('hello')\n"}]
check_best_practices(files, result)
assert not result.has_issues
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -1,2 @@\n+ print('hello')\n",
}
]
check_best_practices(files, result)
assert result.has_issues
assert "print()" in result.issues[0]["body"]
class TestCheckSecurity:
def test_hardcoded_secret_triggers_error(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/config.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ token = 'abc123secrettoken456'\n",
}
]
check_security(files, result)
assert result.has_issues
assert "secret" in result.issues[0]["body"].lower()
def test_example_token_not_flagged(self) -> None:
result = ReviewResult()
files = [
{
"filename": ".env.example",
"patch": "@@ -1,1 +1,2 @@\n+token = your-example-token\n",
}
]
check_security(files, result)
assert not result.has_issues
def test_shell_true_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/executor.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ subprocess.run('ls', shell=True)\n",
}
]
check_best_practices(files, result)
assert result.has_issues
assert "shell=True" in result.issues[0]["body"]
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/config.py", "patch": ""}]
check_security(files, result)
assert not result.has_issues
def test_non_python_file_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "docs/config.md", "patch": "@@ -1,1 +1,2 @@\n+token = 'abc123secrettoken456'\n"}]
check_security(files, result)
assert not result.has_issues
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [
{
"filename": "src/devx/config.py",
"patch": "@@ -1,2 @@\n+ token = 'abc123secrettoken456'\n",
}
]
check_security(files, result)
assert result.has_issues
assert "secret" in result.issues[0]["body"].lower()
class TestCheckI18n:
def test_raw_string_in_echo_triggers_warning(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+click.echo("Hello world")\n'}]
check_i18n(files, result)
assert result.has_issues
assert any("i18n" in i["body"] for i in result.issues)
def test_translated_string_no_warning(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+click.echo(_("Hello world"))\n'}]
check_i18n(files, result)
assert not result.has_issues
def test_fstring_in_echo_triggers_warning(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+click.echo(f"Hello {name}")\n'}]
check_i18n(files, result)
assert result.has_issues
def test_raw_exception_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": '@@ -1,1 +1,1 @@\n+raise click.ClickException("Something went wrong")\n',
}
]
check_i18n(files, result)
assert result.has_issues
def test_non_src_file_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "scripts/ci/test.py", "patch": '@@ -1,1 +1,1 @@\n+click.echo("Hello world")\n'}]
check_i18n(files, result)
assert not result.has_issues
def test_comment_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+# click.echo("Hello world")\n'}]
check_i18n(files, result)
assert not result.has_issues
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_i18n(files, result)
assert not result.has_issues
def test_clean_code_adds_ok_summary(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+click.echo(_("Hello world"))\n'}]
check_i18n(files, result)
assert any("i18n: OK" in s for s in result.summary)
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,2 @@\n+click.echo("Hello world")\n'}]
check_i18n(files, result)
assert result.has_issues
assert any("i18n" in i["body"] for i in result.issues)
class TestCheckResourceManagement:
def test_open_without_with_triggers_warning(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+f = open("file.txt")\n'}]
check_resource_management(files, result)
assert result.has_issues
assert any("resource" in i["body"].lower() for i in result.issues)
def test_open_with_with_no_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": '@@ -1,1 +1,1 @@\n+with open("file.txt") as f:\n+ pass\n',
}
]
check_resource_management(files, result)
assert not result.has_issues
def test_popen_without_cleanup_triggers_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/executor.py",
"patch": '@@ -1,1 +1,1 @@\n+proc = subprocess.Popen(["cmd"])\n',
}
]
check_resource_management(files, result)
assert result.has_issues
def test_popen_with_communicate_no_warning(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/executor.py",
"patch": '@@ -1,1 +1,1 @@\n+out, err = subprocess.Popen(["cmd"], stdout=PIPE).communicate()\n',
}
]
check_resource_management(files, result)
assert not result.has_issues
def test_comment_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,1 +1,1 @@\n+# f = open("file.txt")\n'}]
check_resource_management(files, result)
assert not result.has_issues
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_resource_management(files, result)
assert not result.has_issues
def test_non_python_file_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "docs/config.md", "patch": '@@ -1,1 +1,1 @@\n+f = open("file.txt")\n'}]
check_resource_management(files, result)
assert not result.has_issues
def test_clean_code_adds_ok_summary(self) -> None:
result = ReviewResult()
files = [
{
"filename": "src/devx/cli.py",
"patch": '@@ -1,1 +1,1 @@\n+with open("file.txt") as f:\n+ data = f.read()\n',
}
]
check_resource_management(files, result)
assert any("Resource management: OK" in s for s in result.summary)
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": '@@ -1,2 @@\n+f = open("file.txt")\n'}]
check_resource_management(files, result)
assert result.has_issues
assert any("resource" in i["body"].lower() for i in result.issues)
class TestCheckFunctionLength:
def test_long_function_triggers_warning(self) -> None:
result = ReviewResult()
# Create a patch with a function that adds > 50 lines
added_lines = "\n".join(f"+ x = {i}" for i in range(55))
patch = f"@@ -10,3 +10,59 @@\n+def foo():\n+ pass\n{added_lines}\n"
files = [{"filename": "src/devx/cli.py", "patch": patch}]
check_function_length(files, result)
assert result.has_issues
assert "foo" in result.issues[0]["body"]
def test_short_function_no_warning(self) -> None:
result = ReviewResult()
patch = "@@ -10,3 +10,8 @@\n def foo():\n pass\n+ x = 1\n+ y = 2\n+ z = 3\n"
files = [{"filename": "src/devx/cli.py", "patch": patch}]
check_function_length(files, result)
assert not result.has_issues
def test_empty_patch_skipped(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": ""}]
check_function_length(files, result)
assert not result.has_issues
def test_non_python_file_skipped(self) -> None:
result = ReviewResult()
added_lines = "\n".join(f"+ x = {i}" for i in range(55))
patch = f"@@ -10,3 +10,59 @@\n+def foo():\n+ pass\n{added_lines}\n"
files = [{"filename": "README.md", "patch": patch}]
check_function_length(files, result)
assert not result.has_issues
def test_multiple_functions_resets_count(self) -> None:
"""Two short functions back-to-back should not trigger the length warning."""
result = ReviewResult()
patch = "@@ -10,3 +10,15 @@\n+def foo():\n+ x = 1\n+def bar():\n+ y = 2\n"
files = [{"filename": "src/devx/cli.py", "patch": patch}]
check_function_length(files, result)
assert not result.has_issues
def test_long_function_followed_by_new_hunk(self) -> None:
"""Long function followed by @@ header triggers the warning at hunk boundary."""
result = ReviewResult()
added_lines = "\n".join(f"+ x = {i}" for i in range(55))
patch = (
f"@@ -10,3 +10,59 @@\n+def foo():\n+ pass\n{added_lines}\n@@ -100,3 +100,5 @@\n+def bar():\n+ pass\n"
)
files = [{"filename": "src/devx/cli.py", "patch": patch}]
check_function_length(files, result)
assert result.has_issues
assert "foo" in result.issues[0]["body"]
def test_long_function_followed_by_new_def(self) -> None:
"""Long function followed by another def triggers the warning at def boundary."""
result = ReviewResult()
added_lines = "\n".join(f"+ x = {i}" for i in range(55))
patch = f"@@ -10,3 +10,60 @@\n+def foo():\n+ pass\n{added_lines}\n+def bar():\n+ pass\n"
files = [{"filename": "src/devx/cli.py", "patch": patch}]
check_function_length(files, result)
assert result.has_issues
assert "foo" in result.issues[0]["body"]
def test_malformed_hunk_header_no_line_number(self) -> None:
"""A @@ header without a +N line number is handled gracefully."""
result = ReviewResult()
files = [{"filename": "src/devx/cli.py", "patch": "@@ -1,2 @@\n+def foo():\n+ pass\n"}]
check_function_length(files, result)
assert not result.has_issues
class TestCheckDocumentation:
def test_src_changes_without_docs_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py"}]
check_documentation(files, result)
assert any("WARNING" in s for s in result.summary)
def test_src_changes_with_docs_ok(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py"}, {"filename": "docs/user/cli-commands.md"}]
check_documentation(files, result)
assert any("Documentation: OK" in s for s in result.summary)
def test_ansible_changes_without_docs_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "ansible/roles/gitea-runner/tasks/main.yml"}]
check_documentation(files, result)
assert any("WARNING" in s for s in result.summary)
def test_only_doc_changes_ok(self) -> None:
result = ReviewResult()
files = [{"filename": "README.md"}]
check_documentation(files, result)
assert any("Documentation: OK" in s for s in result.summary)
def test_tofu_changes_without_docs_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "tofu/modules/hetzner-vm/main.tf"}]
check_documentation(files, result)
assert any("WARNING" in s for s in result.summary)
def test_workflow_changes_info(self) -> None:
result = ReviewResult()
files = [{"filename": ".gitea/workflows/ci.yml"}]
check_documentation(files, result)
assert any("INFO" in s for s in result.summary)
def test_todo_in_doc_patch_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "docs/guide.md", "patch": "+TODO: fix this later\n+Some content\n"}]
check_documentation(files, result)
assert any("TODO" in s for s in result.summary)
def test_todo_in_readme_patch_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "README.md", "patch": "+FIXME: broken\n"}]
check_documentation(files, result)
assert any("FIXME" in s for s in result.summary)
def test_no_todo_in_doc_patch_ok(self) -> None:
result = ReviewResult()
files = [{"filename": "docs/guide.md", "patch": "+Some content\n"}]
check_documentation(files, result)
assert not any("TODO" in s for s in result.summary)
class TestCheckTestCoverage:
def test_src_changes_without_tests_warns(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py"}]
check_test_coverage(files, result)
assert any("WARNING" in s for s in result.summary)
def test_src_changes_with_tests_ok(self) -> None:
result = ReviewResult()
files = [{"filename": "src/devx/cli.py"}, {"filename": "tests/unit/test_cli.py"}]
check_test_coverage(files, result)
assert any("Tests: OK" in s for s in result.summary)
def test_only_test_changes_ok(self) -> None:
result = ReviewResult()
files = [{"filename": "tests/unit/test_cli.py"}]
check_test_coverage(files, result)
assert any("Tests: OK" in s for s in result.summary)
class TestBuildReviewBody:
def test_body_contains_summary(self) -> None:
result = ReviewResult()
result.add_summary("- Architecture compliance: OK")
body = build_review_body(result)
assert "Architecture compliance: OK" in body
assert "Automated PR Review" in body
def test_body_contains_issues(self) -> None:
result = ReviewResult()
result.add_issue("src/foo.py", 10, "bad code")
body = build_review_body(result)
assert "1 issue(s) found" in body
assert "src/foo.py:10" in body
assert "bad code" in body
def test_body_contains_no_issues_message(self) -> None:
result = ReviewResult()
body = build_review_body(result)
assert "No issues found" in body
def test_body_contains_auto_merge_note(self) -> None:
"""Review body must mention auto-merge."""
result = ReviewResult()
body = build_review_body(result)
assert "Auto-merge" in body
class TestRunReview:
@patch("devx.ci.pr_review.GiteaClient")
def test_run_review_with_no_files(self, mock_client_class: MagicMock) -> None:
mock_client = mock_client_class.return_value
mock_client.get_pr_files.return_value = []
result = run_review(mock_client, "42")
assert "No files changed" in result.summary[0]
@patch("devx.ci.pr_review.GiteaClient")
def test_run_review_finds_issues(self, mock_client_class: MagicMock) -> None:
mock_client = mock_client_class.return_value
mock_client.get_pr_files.return_value = [
{
"filename": "src/devx/cli.py",
"patch": "@@ -10,3 +10,4 @@\n def foo():\n pass\n+ print('hello')\n",
}
]
mock_client.get_pr_commits.return_value = [{"commit": {"message": "fix: resolve print issue"}}]
result = run_review(mock_client, "42")
assert result.has_issues
def test_run_review_handles_api_error(self) -> None:
client = MagicMock()
client.get_pr_files.side_effect = APIError(404, "Not found")
result = run_review(client, "42")
assert any("ERROR" in s for s in result.summary)
class TestCheckCommitConventions:
def test_conventional_commit_found(self) -> None:
"""Should report OK when at least one commit is conventional."""
client = MagicMock()
client.get_pr_commits.return_value = [
{"commit": {"message": "fix: resolve bug\n\nDetails"}},
{"commit": {"message": "wip: testing"}},
]
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("OK" in s for s in result.summary)
def test_no_conventional_commit(self) -> None:
"""Should warn when no commits are conventional."""
client = MagicMock()
client.get_pr_commits.return_value = [
{"commit": {"message": "updated stuff"}},
{"commit": {"message": "wip: testing"}},
]
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("WARNING" in s for s in result.summary)
def test_merge_commits_excluded(self) -> None:
"""Merge commits should be excluded from the check."""
client = MagicMock()
client.get_pr_commits.return_value = [
{"commit": {"message": "Merge branch 'feature' into master"}},
{"commit": {"message": "fix: resolve bug"}},
]
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("OK" in s for s in result.summary)
def test_all_merges_and_reverts(self) -> None:
"""Should report OK when all commits are merges/reverts."""
client = MagicMock()
client.get_pr_commits.return_value = [
{"commit": {"message": "Merge branch 'feature' into master"}},
{"commit": {"message": "Revert: bad commit"}},
]
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("merges/reverts" in s for s in result.summary)
def test_no_commits(self) -> None:
"""Should report OK when there are no commits."""
client = MagicMock()
client.get_pr_commits.return_value = []
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("no commits" in s for s in result.summary)
def test_api_error(self) -> None:
"""Should report ERROR when API call fails."""
client = MagicMock()
client.get_pr_commits.side_effect = APIError(500, "server error")
result = ReviewResult()
check_commit_conventions(client, "42", result)
assert any("ERROR" in s for s in result.summary)
class TestPostReview:
def test_post_review_with_issues(self) -> None:
client = MagicMock()
result = ReviewResult()
result.add_issue("src/foo.py", 10, "bad code")
post_review(client, "42", result)
client.create_review.assert_called_once()
call_args = client.create_review.call_args
assert call_args[1]["event"] == "REQUEST_CHANGES"
assert call_args[1]["comments"] == result.issues
def test_post_review_without_issues_uses_comment_not_approve(self) -> None:
"""Automated review posts COMMENT, not APPROVE (self-approval not allowed)."""
client = MagicMock()
result = ReviewResult()
post_review(client, "42", result)
client.create_review.assert_called_once()
call_args = client.create_review.call_args
assert call_args[1]["event"] == "COMMENT"
assert call_args[1]["comments"] == []
class TestMain:
@patch("devx.ci.pr_review.run_review")
@patch("devx.ci.pr_review.GiteaClient")
def test_dry_run_does_not_post(self, mock_client_class: MagicMock, mock_run: MagicMock) -> None:
mock_run.return_value = ReviewResult()
runner = CliRunner()
result = runner.invoke(main, ["42", "my-org/my-repo", "--dry-run"], env={"CI_GITEA_TOKEN": "fake"})
assert result.exit_code == 0
assert "[dry-run]" in result.output
mock_client_class.return_value.create_review.assert_not_called()
@patch("devx.ci.pr_review.run_review")
@patch("devx.ci.pr_review.GiteaClient")
def test_post_review_on_success(self, mock_client_class: MagicMock, mock_run: MagicMock) -> None:
mock_run.return_value = ReviewResult()
mock_client_class.return_value.create_review.return_value = {"id": 123}
runner = CliRunner()
result = runner.invoke(main, ["42", "my-org/my-repo"], env={"CI_GITEA_TOKEN": "fake"})
assert result.exit_code == 0
assert "Review #123" in result.output
mock_client_class.return_value.create_review.assert_called_once()
@patch("devx.ci.pr_review.run_review")
@patch("devx.ci.pr_review.GiteaClient")
def test_self_approval_falls_back_to_comment(self, mock_client_class: MagicMock, mock_run: MagicMock) -> None:
"""If REQUEST_CHANGES fails with 422 (self-approval), fall back to COMMENT."""
mock_run.return_value = ReviewResult()
client = mock_client_class.return_value
client.create_review.side_effect = [
APIError(422, "approve your own pull is not allowed"),
{"id": 124},
]
runner = CliRunner()
result = runner.invoke(main, ["42", "my-org/my-repo"], env={"CI_GITEA_TOKEN": "fake"})
assert result.exit_code == 0
assert "Review #124" in result.output
assert client.create_review.call_count == 2
@patch("devx.ci.pr_review.run_review")
@patch("devx.ci.pr_review.GiteaClient")
def test_other_api_error_re_raises(self, mock_client_class: MagicMock, mock_run: MagicMock) -> None:
"""Non-approval API errors should re-raise, not fall back."""
mock_run.return_value = ReviewResult()
client = mock_client_class.return_value
client.create_review.side_effect = APIError(500, "Internal server error")
runner = CliRunner()
result = runner.invoke(main, ["42", "my-org/my-repo"], env={"CI_GITEA_TOKEN": "fake"})
assert result.exit_code != 0
@patch.dict("os.environ", {"CI_GITEA_API_TOKEN": "", "CI_GITEA_TOKEN": ""})
def test_no_token_raises(self) -> None:
runner = CliRunner()
result = runner.invoke(main, ["42", "my-org/my-repo"], env={"CI_GITEA_API_TOKEN": "", "CI_GITEA_TOKEN": ""})
assert result.exit_code != 0
assert "CI_GITEA_TOKEN" in result.output
class TestManualReview:
@patch("devx.ci.pr_review.GiteaClient")
def test_approve_success(self, mock_client_class: MagicMock) -> None:
mock_client_class.return_value.create_review.return_value = {"id": 200}
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"All 13 REVIEW_CHECKLIST.md categories verified. Architecture: clean. Security: no issues.",
"--checklist-confirmed",
"--checklist-categories",
"1,2,3,4,5,6,7,8,9,10,11,12,13",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code == 0
assert "Review #200" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_approve_without_checklist_confirmed_fails(self, mock_client_class: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-categories",
"1,2,3,4,5,6,7,8",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code != 0
assert "checklist-confirmed" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_approve_with_too_few_categories_fails(self, mock_client_class: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-confirmed",
"--checklist-categories",
"1,2,3",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code != 0
assert "at least 8" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_approve_with_short_body_fails(self, mock_client_class: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"LGTM",
"--checklist-confirmed",
"--checklist-categories",
"1,2,3,4,5,6,7,8",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code != 0
assert "50 characters" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_approve_with_invalid_category_fails(self, mock_client_class: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-confirmed",
"--checklist-categories",
"1,2,abc,4",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code != 0
assert "Invalid" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_request_changes_success(self, mock_client_class: MagicMock) -> None:
mock_client_class.return_value.create_review.return_value = {"id": 201}
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"REQUEST_CHANGES",
"--body",
"Please fix the architecture issues in the CLI module before merging.",
],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code == 0
assert "Review #201" in result.output
@patch("devx.ci.pr_review.GiteaClient")
def test_manual_review_dry_run(self, mock_client_class: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(
main,
["42", "oblachno-oss/devx", "--event", "COMMENT", "--body", "x" * 60, "--dry-run"],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code == 0
assert "[dry-run]" in result.output
mock_client_class.return_value.create_review.assert_not_called()
@patch("devx.ci.pr_review.GiteaClient")
def test_manual_review_self_approval_fallback_to_comment(
self, mock_client_class: MagicMock, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Self-approval with no CI token available → fall back to COMMENT."""
monkeypatch.delenv("CI_GITEA_API_TOKEN", raising=False)
monkeypatch.delenv("CI_GITEA_TOKEN", raising=False)
client = mock_client_class.return_value
client.create_review.side_effect = [
APIError(422, "approve your own pull is not allowed"),
{"id": 202},
]
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-confirmed",
"--checklist-categories",
"1,2,3,4,5,6,7,8",
],
env={"REVIEWER_GITEA_API_TOKEN": "fake-reviewer"},
)
assert result.exit_code == 0
assert "Review #202" in result.output
# Without CI_GITEA_API_TOKEN, the fallback is COMMENT
assert "Self-approval not allowed. Posting COMMENT instead." in result.output
assert client.create_review.call_count == 2
assert client.create_review.call_args_list[1].kwargs.get("event") == "COMMENT"
@patch("devx.ci.pr_review.GiteaClient")
def test_manual_review_self_approval_falls_back_to_ci_token(self, mock_client_class: MagicMock) -> None:
"""Self-approval with CI token available → retry APPROVE with CI token (different user)."""
client = mock_client_class.return_value
client.create_review.side_effect = [
APIError(422, "approve your own pull is not allowed"),
{"id": 303},
]
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-confirmed",
"--checklist-categories",
"1,2,3,4,5,6,7,8",
],
env={"REVIEWER_GITEA_API_TOKEN": "fake-reviewer", "CI_GITEA_API_TOKEN": "fake-ci"},
)
assert result.exit_code == 0
assert "Review #303" in result.output
assert "Retrying with CI token" in result.output
# Second call should still be APPROVE (CI token retry)
assert client.create_review.call_count == 2
assert client.create_review.call_args_list[1].kwargs.get("event") == "APPROVE"
@patch("devx.ci.pr_review.GiteaClient")
def test_manual_review_ci_token_also_fails_falls_back_to_comment(self, mock_client_class: MagicMock) -> None:
"""Self-approval + CI token retry also fails → fall back to COMMENT."""
client = mock_client_class.return_value
client.create_review.side_effect = [
APIError(422, "approve your own pull is not allowed"),
APIError(422, "approve your own pull is not allowed"),
{"id": 404},
]
runner = CliRunner()
result = runner.invoke(
main,
[
"42",
"oblachno-oss/devx",
"--event",
"APPROVE",
"--body",
"x" * 60,
"--checklist-confirmed",
"--checklist-categories",
"1,2,3,4,5,6,7,8",
],
env={"REVIEWER_GITEA_API_TOKEN": "fake-reviewer", "CI_GITEA_API_TOKEN": "fake-ci"},
)
assert result.exit_code == 0
assert "Review #404" in result.output
assert "CI token also cannot approve" in result.output
# Third call should be COMMENT (final fallback)
assert client.create_review.call_count == 3
assert client.create_review.call_args_list[2].kwargs.get("event") == "COMMENT"
@patch("devx.ci.pr_review.GiteaClient")
def test_manual_review_other_error_re_raises(self, mock_client_class: MagicMock) -> None:
client = mock_client_class.return_value
client.create_review.side_effect = APIError(500, "Internal server error")
runner = CliRunner()
result = runner.invoke(
main,
["42", "oblachno-oss/devx", "--event", "COMMENT", "--body", "x" * 60],
env={"CI_GITEA_TOKEN": "fake"},
)
assert result.exit_code != 0
def test_main_module_block() -> None:
import devx.ci.pr_review as pr
with patch.object(pr, "main") as mock_main:
with patch.object(pr, "__name__", "__main__"):
pr.main([])
mock_main.assert_called_once_with([])
+835
View File
@@ -0,0 +1,835 @@
"""Structural tests for spec-driven development workflows and skills.
These tests parse the actual workflow YAML files in each repo and assert
that the new spec-driven development steps, jobs, and env vars are present
and correctly wired. They also validate that the spec-driven-development
skill exists in each repo's .devin/skills/ directory with required sections.
This is a "contract test" it verifies that the workflows we wrote match
the intended structure, catching regressions if someone edits a workflow
and accidentally removes a step or breaks a job dependency.
"""
from __future__ import annotations
from pathlib import Path
import pytest
import yaml
# Repo root paths
# __file__ = .../devx/tests/unit/test_spec_driven_workflows.py
# parents[3] = .../oblachno (the monorepo root containing all repos)
_OBLACHNO_ROOT = Path(__file__).resolve().parents[3]
_INFRA = _OBLACHNO_ROOT / "infra"
_GRM = _OBLACHNO_ROOT / "grm"
_SSO_BRIDGE = _OBLACHNO_ROOT / "sso-bridge"
_DEVX = _OBLACHNO_ROOT / "devx"
def _load_workflow(repo_path: Path, filename: str) -> dict:
"""Load a workflow YAML file and return parsed dict."""
path = repo_path / ".gitea" / "workflows" / filename
if not path.exists():
pytest.skip(f"Workflow {filename} not found in {repo_path.name}")
with open(path, encoding="utf-8") as f:
return yaml.safe_load(f)
def _skip_if_repo_missing(repo_name: str) -> None:
"""Skip test if the sibling repo directory doesn't exist (CI only checks out one repo)."""
repo_path = _OBLACHNO_ROOT / repo_name
if not repo_path.is_dir():
pytest.skip(f"Repo {repo_name} not found at {repo_path} (CI only checks out devx)")
def _read_skill(repo_name: str, skill_name: str) -> str:
"""Read a skill file from a repo, skipping if the repo or file doesn't exist."""
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / skill_name / "SKILL.md"
if not skill_path.exists():
pytest.skip(f"SKILL.md not found in {repo_name}/{skill_name}")
return skill_path.read_text(encoding="utf-8")
def _get_step_names(job: dict) -> list[str]:
"""Extract step names from a job dict."""
names = []
for step in job.get("steps", []):
if "name" in step:
names.append(step["name"])
return names
def _find_step(job: dict, name_part: str) -> dict | None:
"""Find a step by partial name match."""
for step in job.get("steps", []):
if "name" in step and name_part.lower() in step["name"].lower():
return step
return None
def _get_run_commands(step: dict) -> str:
"""Get the run command from a step."""
return step.get("run", "")
# ============================================================================
# Infra ci.yml — spec validation + PR size + fast molecule
# ============================================================================
class TestInfraCiWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_INFRA, "ci.yml")
def test_validate_job_exists(self, workflow: dict) -> None:
assert "validate" in workflow["jobs"]
def test_has_spec_validation_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Validate spec file" in s for s in steps), "validate job must have 'Validate spec file' step"
def test_has_pr_size_check_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Check PR size" in s for s in steps), "validate job must have 'Check PR size' step"
def test_spec_validation_uses_correct_module(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Validate spec file")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.validate_spec" in cmd
assert "--github-output" in cmd
def test_pr_size_uses_correct_module(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Check PR size")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.check_pr_size" in cmd
assert "--github-output" in cmd
def test_spec_validation_sets_task_prefix(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Validate spec file")
assert step is not None
env = step.get("env", {})
assert env.get("DEVX_TASK_PREFIX") == "OBL-INFRA"
def test_has_fast_molecule_job(self, workflow: dict) -> None:
assert "fast-molecule" in workflow["jobs"], "ci.yml must have 'fast-molecule' job (replaced molecule-tests)"
def test_no_full_molecule_tests_job(self, workflow: dict) -> None:
assert "molecule-tests" not in workflow["jobs"], "ci.yml must NOT have 'molecule-tests' job (moved to nightly)"
def test_no_staging_deploy_in_ci(self, workflow: dict) -> None:
# The staging deploy was moved to post-merge (auto-deploy-staging)
job_names = list(workflow["jobs"].keys())
assert "staging-health-gate" not in job_names, "staging-health-gate removed from ci.yml (moved to nightly)"
assert "pre-deploy-checks" not in job_names, "pre-deploy-checks removed from ci.yml (moved to nightly)"
assert "deploy" not in job_names, "deploy job removed from ci.yml (moved to post-merge)"
def test_fast_molecule_uses_devx_module(self, workflow: dict) -> None:
job = workflow["jobs"]["fast-molecule"]
step = _find_step(job, "Detect changed roles")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.fast_molecule" in cmd
assert "--github-output" in cmd
def test_fast_molecule_timeout_is_short(self, workflow: dict) -> None:
job = workflow["jobs"]["fast-molecule"]
assert job.get("timeout-minutes", 999) <= 30, (
"fast-molecule timeout should be <= 30 min (was 120 for full suite)"
)
def test_fast_molecule_no_matrix(self, workflow: dict) -> None:
job = workflow["jobs"]["fast-molecule"]
assert "strategy" not in job or "matrix" not in job.get("strategy", {}), (
"fast-molecule should not use matrix (single runner)"
)
def test_auto_merge_depends_on_fast_molecule(self, workflow: dict) -> None:
job = workflow["jobs"].get("auto-merge", {})
needs = job.get("needs", [])
assert "fast-molecule" in needs, "auto-merge must depend on fast-molecule (not deploy)"
def test_auto_merge_does_not_depend_on_deploy(self, workflow: dict) -> None:
job = workflow["jobs"].get("auto-merge", {})
needs = job.get("needs", [])
assert "deploy" not in needs, "auto-merge must NOT depend on deploy (removed from PR pipeline)"
# ============================================================================
# Infra nightly.yml — full molecule + staging deploy + gate
# ============================================================================
class TestInfraNightlyWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_INFRA, "nightly.yml")
def test_nightly_workflow_exists(self, workflow: dict) -> None:
assert workflow is not None
def test_has_full_molecule_job(self, workflow: dict) -> None:
assert "full-molecule" in workflow["jobs"]
def test_has_set_gate_status_job(self, workflow: dict) -> None:
assert "set-gate-status" in workflow["jobs"]
def test_has_staging_deploy_job(self, workflow: dict) -> None:
assert "staging-deploy" in workflow["jobs"]
def test_full_molecule_uses_matrix(self, workflow: dict) -> None:
job = workflow["jobs"]["full-molecule"]
strategy = job.get("strategy", {})
assert "matrix" in strategy, "full-molecule must use matrix (6 runners)"
assert "runner-index" in strategy["matrix"]
def test_full_molecule_timeout_is_long(self, workflow: dict) -> None:
job = workflow["jobs"]["full-molecule"]
assert job.get("timeout-minutes", 0) >= 90, "full-molecule timeout should be >= 90 min (full suite)"
def test_set_gate_status_depends_on_full_molecule(self, workflow: dict) -> None:
job = workflow["jobs"]["set-gate-status"]
needs = job.get("needs", [])
assert "full-molecule" in needs
def test_set_gate_status_uses_nightly_gate_module(self, workflow: dict) -> None:
job = workflow["jobs"]["set-gate-status"]
step = _find_step(job, "Set nightly gate")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.nightly_gate" in cmd
assert "set-passed" in cmd or "set-failed" in cmd
def test_staging_deploy_depends_on_gate(self, workflow: dict) -> None:
job = workflow["jobs"]["staging-deploy"]
needs = job.get("needs", [])
assert "set-gate-status" in needs
assert "full-molecule" in needs
def test_staging_deploy_only_on_success(self, workflow: dict) -> None:
job = workflow["jobs"]["staging-deploy"]
if_cond = job.get("if", "")
assert "success" in if_cond, "staging-deploy must only run when full-molecule succeeds"
def test_nightly_runs_on_schedule(self, workflow: dict) -> None:
on = workflow.get("on", workflow.get(True, {}))
# YAML may parse 'on' as True (boolean)
if isinstance(on, dict):
assert "schedule" in on, "nightly must have schedule trigger"
else:
pytest.fail("Could not parse 'on' trigger from nightly.yml")
# ============================================================================
# Infra post-merge.yml — auto-deploy staging with nightly gate
# ============================================================================
class TestInfraPostMergeWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_INFRA, "post-merge.yml")
def test_has_auto_deploy_staging_job(self, workflow: dict) -> None:
assert "auto-deploy-staging" in workflow["jobs"], "post-merge must have 'auto-deploy-staging' job"
def test_has_staging_deploy_job(self, workflow: dict) -> None:
assert "staging-deploy" in workflow["jobs"], "post-merge must have 'staging-deploy' reusable workflow job"
def test_auto_deploy_staging_checks_nightly_gate(self, workflow: dict) -> None:
job = workflow["jobs"]["auto-deploy-staging"]
step = _find_step(job, "Check nightly gate")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.nightly_gate" in cmd
assert "--action check" in cmd
def test_staging_deploy_depends_on_auto_deploy_staging(self, workflow: dict) -> None:
job = workflow["jobs"]["staging-deploy"]
needs = job.get("needs", [])
assert "auto-deploy-staging" in needs
def test_staging_deploy_gated_on_gate_passed(self, workflow: dict) -> None:
job = workflow["jobs"]["staging-deploy"]
if_cond = job.get("if", "")
assert "gate-passed" in if_cond, "staging-deploy must check gate-passed output"
def test_auto_deploy_production_waits_for_staging(self, workflow: dict) -> None:
job = workflow["jobs"].get("auto-deploy-production", {})
needs = job.get("needs", [])
assert "staging-deploy" in needs, "auto-deploy-production must wait for staging-deploy"
# ============================================================================
# GRM ci.yml — spec validation + PR size
# ============================================================================
class TestGrmCiWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_GRM, "ci.yml")
def test_has_spec_validation_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Validate spec file" in s for s in steps)
def test_has_pr_size_check_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Check PR size" in s for s in steps)
def test_spec_validation_sets_task_prefix(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Validate spec file")
assert step is not None
env = step.get("env", {})
assert env.get("DEVX_TASK_PREFIX") == "GRM"
# ============================================================================
# GRM post-merge.yml — auto-create infra dependency PR
# ============================================================================
class TestGrmPostMergeWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_GRM, "post-merge.yml")
def test_has_create_dependency_pr_step(self, workflow: dict) -> None:
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None, "grm post-merge must have 'Create infra dependency PR' step"
def test_dependency_pr_uses_correct_module(self, workflow: dict) -> None:
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.create_dependency_pr" in cmd
assert "--package grm" in cmd
assert "--repo oblachno/infra" in cmd
def test_dependency_pr_is_best_effort(self, workflow: dict) -> None:
import re
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None
cmd = _get_run_commands(step)
# Must not fail the workflow if PR creation fails.
# The || echo may be split across lines with backslash continuation in YAML.
# Normalize: remove backslashes and collapse whitespace.
cmd_normalized = " ".join(cmd.replace("\\", " ").split())
assert bool(re.search(r"\|\|\s*echo", cmd_normalized)) or "continue-on-error" in step, (
"dependency PR step must be best-effort (|| echo or continue-on-error)"
)
# ============================================================================
# sso-bridge ci.yml — spec validation + PR size
# ============================================================================
class TestSsoBridgeCiWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_SSO_BRIDGE, "ci.yml")
def test_has_spec_validation_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Validate spec file" in s for s in steps)
def test_has_pr_size_check_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Check PR size" in s for s in steps)
def test_spec_validation_sets_task_prefix(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Validate spec file")
assert step is not None
env = step.get("env", {})
assert env.get("DEVX_TASK_PREFIX") == "SSO"
# ============================================================================
# sso-bridge post-merge.yml — auto-publish + auto-create dependency PR
# ============================================================================
class TestSsoBridgePostMergeWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_SSO_BRIDGE, "post-merge.yml")
def test_post_merge_workflow_exists(self, workflow: dict) -> None:
assert workflow is not None
def test_has_release_and_maintain_job(self, workflow: dict) -> None:
assert "release-and-maintain" in workflow["jobs"]
def test_has_create_dependency_pr_step(self, workflow: dict) -> None:
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None
def test_dependency_pr_uses_correct_module(self, workflow: dict) -> None:
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None
cmd = _get_run_commands(step)
assert "devx.ci.create_dependency_pr" in cmd
assert "--package sso_bridge" in cmd
assert "--repo oblachno/infra" in cmd
def test_dependency_pr_is_best_effort(self, workflow: dict) -> None:
import re
job = workflow["jobs"].get("release-and-maintain", {})
step = _find_step(job, "Create infra dependency PR")
assert step is not None
cmd = _get_run_commands(step)
# The || echo may be split across lines with backslash continuation in YAML.
cmd_normalized = " ".join(cmd.replace("\\", " ").split())
assert bool(re.search(r"\|\|\s*echo", cmd_normalized)) or "continue-on-error" in step
def test_has_publish_step(self, workflow: dict) -> None:
job = workflow["jobs"].get("release-and-maintain", {})
steps = _get_step_names(job)
assert any("publish" in s.lower() for s in steps), "sso-bridge post-merge must have a publish step"
# ============================================================================
# devx ci.yml — spec validation + PR size
# ============================================================================
class TestDevxCiWorkflow:
@pytest.fixture
def workflow(self) -> dict:
return _load_workflow(_DEVX, "ci.yml")
def test_has_spec_validation_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Validate spec file" in s for s in steps)
def test_has_pr_size_check_step(self, workflow: dict) -> None:
steps = _get_step_names(workflow["jobs"]["validate"])
assert any("Check PR size" in s for s in steps)
def test_spec_validation_sets_task_prefix(self, workflow: dict) -> None:
step = _find_step(workflow["jobs"]["validate"], "Validate spec file")
assert step is not None
env = step.get("env", {})
assert env.get("DEVX_TASK_PREFIX") == "DEVX"
# ============================================================================
# Skill files — spec-driven-development SKILL.md in all repos
# ============================================================================
class TestSpecDrivenDevelopmentSkill:
REQUIRED_SECTIONS = [
"## Overview",
"## Workflow",
"## Spec Template",
"## CI Validation",
"## Acceptance Criteria",
]
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_skill_exists_in_repo(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
assert skill_path.exists(), f"SKILL.md not found in {repo_name}"
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_skill_has_required_sections(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
content = skill_path.read_text(encoding="utf-8")
for section in self.REQUIRED_SECTIONS:
assert section in content, f"SKILL.md in {repo_name} missing section: {section}"
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_skill_mentions_req_ids(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
content = skill_path.read_text(encoding="utf-8")
assert "REQ-" in content, f"SKILL.md in {repo_name} must mention REQ-ID format"
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_skill_mentions_pr_size_limit(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
content = skill_path.read_text(encoding="utf-8")
assert "500" in content, f"SKILL.md in {repo_name} must mention 500 line PR size limit"
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_skill_mentions_nightly_gate(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
skill_path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
content = skill_path.read_text(encoding="utf-8")
assert "nightly" in content.lower(), f"SKILL.md in {repo_name} must mention nightly gate"
def test_skill_exists_in_shared_dir(self) -> None:
skill_path = _OBLACHNO_ROOT / ".devin" / "skills" / "spec-driven-development" / "SKILL.md"
if not skill_path.exists():
pytest.skip("Shared .devin/skills/ not found (CI only checks out devx repo)")
assert skill_path.exists(), "SKILL.md not found in shared .devin/skills/"
# ============================================================================
# devx-workflow skill — exists in repos with PR workflow, mentions spec gates
# ============================================================================
class TestDevxWorkflowSkill:
# Repos that have a PR workflow and need the devx-workflow skill
REPOS_WITH_PR_WORKFLOW = ["infra", "grm", "sso-bridge", "devx"]
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_skill_exists(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
assert path.exists(), f"devx-workflow SKILL.md not found in {repo_name}"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_spec_validation(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "validate_spec" in content, f"devx-workflow skill in {repo_name} must mention validate_spec"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_pr_size_check(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "check_pr_size" in content, f"devx-workflow skill in {repo_name} must mention check_pr_size"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_pr_workflow_commands(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "make create-pr" in content or "make push-with-pr" in content, (
f"devx-workflow skill in {repo_name} must mention PR creation commands"
)
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_auto_merge(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "auto-merge" in content.lower() or "ready-to-merge" in content, (
f"devx-workflow skill in {repo_name} must mention auto-merge"
)
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_has_correct_task_prefix(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
"""Each repo's devx-workflow skill must mention its correct task prefix."""
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
expected_prefixes = {
"infra": "OBL-INFRA",
"grm": "GRM",
"sso-bridge": "SSO",
"devx": "DEVX",
}
prefix = expected_prefixes[repo_name]
assert prefix in content, f"devx-workflow skill in {repo_name} must mention task prefix {prefix}"
def test_not_in_mattermost_oidc(self) -> None:
"""mattermost-oidc has no PR workflow — should NOT have devx-workflow skill."""
path = _OBLACHNO_ROOT / "mattermost-oidc" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
assert not path.exists(), "mattermost-oidc should NOT have devx-workflow skill (no PR workflow)"
# Repo-specific content checks
def test_infra_mentions_nightly_gate(self) -> None:
_skip_if_repo_missing("infra")
path = _OBLACHNO_ROOT / "infra" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "nightly" in content.lower(), "infra devx-workflow skill must mention nightly gate"
assert "nightly_gate" in content, "infra devx-workflow skill must mention devx.ci.nightly_gate module"
def test_infra_mentions_fast_molecule(self) -> None:
_skip_if_repo_missing("infra")
path = _OBLACHNO_ROOT / "infra" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "fast_molecule" in content, "infra devx-workflow skill must mention devx.ci.fast_molecule"
def test_infra_mentions_auto_deploy_staging(self) -> None:
_skip_if_repo_missing("infra")
path = _OBLACHNO_ROOT / "infra" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "staging" in content.lower(), "infra devx-workflow skill must mention staging auto-deploy"
def test_grm_mentions_dependency_pr(self) -> None:
_skip_if_repo_missing("grm")
path = _OBLACHNO_ROOT / "grm" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "create_dependency_pr" in content, "grm devx-workflow skill must mention create_dependency_pr"
def test_sso_bridge_mentions_dependency_pr(self) -> None:
_skip_if_repo_missing("sso-bridge")
path = _OBLACHNO_ROOT / "sso-bridge" / ".devin" / "skills" / "devx-workflow" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "create_dependency_pr" in content, "sso-bridge devx-workflow skill must mention create_dependency_pr"
# ============================================================================
# testing-and-debugging skill — exists in all repos, mentions spec workflow
# ============================================================================
class TestTestingAndDebuggingSkill:
# All repos have a testing-and-debugging skill
ALL_REPOS = ["infra", "grm", "sso-bridge", "devx", "mattermost-oidc"]
@pytest.mark.parametrize("repo_name", ALL_REPOS)
def test_skill_exists(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
assert path.exists(), f"testing-and-debugging SKILL.md not found in {repo_name}"
@pytest.mark.parametrize("repo_name", ALL_REPOS)
def test_has_required_sections(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
content = path.read_text(encoding="utf-8")
# All testing-and-debugging skills should have a CI failure investigation section
assert "CI Failure Investigation" in content or "CI failure" in content, (
f"testing-and-debugging skill in {repo_name} must have CI failure section"
)
# Repos with PR workflow should mention spec-driven workflow
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_mentions_spec_driven_workflow(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "spec" in content.lower(), (
f"testing-and-debugging skill in {repo_name} must mention spec-driven workflow"
)
def test_infra_mentions_nightly(self) -> None:
_skip_if_repo_missing("infra")
path = _OBLACHNO_ROOT / "infra" / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "nightly" in content.lower(), "infra testing-and-debugging skill must mention nightly tests"
def test_infra_mentions_fast_molecule(self) -> None:
_skip_if_repo_missing("infra")
path = _OBLACHNO_ROOT / "infra" / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "fast" in content.lower() and "molecule" in content.lower(), (
"infra testing-and-debugging skill must mention fast molecule"
)
def test_mattermost_oidc_no_spec_mention(self) -> None:
"""mattermost-oidc has no spec-driven workflow — skill should NOT mention it."""
_skip_if_repo_missing("mattermost-oidc")
path = _OBLACHNO_ROOT / "mattermost-oidc" / ".devin" / "skills" / "testing-and-debugging" / "SKILL.md"
content = path.read_text(encoding="utf-8")
# mattermost-oidc has no PR workflow, no spec validation
assert "validate_spec" not in content, (
"mattermost-oidc testing-and-debugging skill should NOT mention validate_spec"
)
# ============================================================================
# pr-review skill — deep review with auto-fix, exists in repos with PR workflow
# ============================================================================
class TestPrReviewSkill:
REPOS_WITH_PR_WORKFLOW = ["infra", "grm", "sso-bridge", "devx"]
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_skill_exists(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
assert path.exists(), f"pr-review SKILL.md not found in {repo_name}"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_all_review_categories(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
required_categories = [
"Functional Correctness",
"Completeness",
"Architecture",
"Reliability",
"Robustness",
"Security",
"Technical Excellence",
"Test Quality",
]
for cat in required_categories:
assert cat in content, f"pr-review skill in {repo_name} missing category: {cat}"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_auto_fix(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "auto-fix" in content.lower() or "auto fix" in content.lower(), (
f"pr-review skill in {repo_name} must mention auto-fix"
)
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_gitea_mcp(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "mcp" in content.lower(), f"pr-review skill in {repo_name} must mention Gitea MCP"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_inline_comments(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "inline" in content.lower(), f"pr-review skill in {repo_name} must mention inline comments"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_ready_to_merge(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "ready-to-merge" in content, f"pr-review skill in {repo_name} must mention ready-to-merge label"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_resolve_discussion(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "resolve" in content.lower(), f"pr-review skill in {repo_name} must mention resolving discussions"
@pytest.mark.parametrize("repo_name", REPOS_WITH_PR_WORKFLOW)
def test_mentions_summary(self, repo_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / "pr-review" / "SKILL.md"
content = path.read_text(encoding="utf-8")
assert "summary" in content.lower(), f"pr-review skill in {repo_name} must mention posting a summary"
def test_not_in_mattermost_oidc(self) -> None:
"""mattermost-oidc has no PR workflow — should NOT have pr-review skill."""
path = _OBLACHNO_ROOT / "mattermost-oidc" / ".devin" / "skills" / "pr-review" / "SKILL.md"
assert not path.exists(), "mattermost-oidc should NOT have pr-review skill (no PR workflow)"
def test_no_pr_review_module_remains(self) -> None:
"""The old devx.ci.pr_review module should be deleted."""
path = _DEVX / "src" / "devx" / "ci" / "pr_review.py"
assert not path.exists(), "devx.ci.pr_review module should be deleted (replaced by pr-review skill)"
def test_no_pr_review_test_remains(self) -> None:
"""The old test_pr_review.py should be deleted."""
path = _DEVX / "tests" / "unit" / "test_pr_review.py"
assert not path.exists(), "tests/unit/test_pr_review.py should be deleted"
def test_no_pr_review_in_workflows(self) -> None:
"""No CI workflow should reference devx.ci.pr_review."""
for repo_name in ["infra", "grm", "sso-bridge", "devx"]:
wf_dir = _OBLACHNO_ROOT / repo_name / ".gitea" / "workflows"
if not wf_dir.exists():
continue
for wf_file in wf_dir.glob("*.yml"):
content = wf_file.read_text(encoding="utf-8")
assert "devx.ci.pr_review" not in content, (
f"{repo_name}/{wf_file.name} still references devx.ci.pr_review"
)
# ============================================================================
# Skill consistency — all skills have proper structure
# ============================================================================
class TestSkillConsistency:
ALL_SKILLS = [
("infra", "devx-workflow"),
("infra", "testing-and-debugging"),
("infra", "spec-driven-development"),
("infra", "pr-review"),
("grm", "devx-workflow"),
("grm", "testing-and-debugging"),
("grm", "spec-driven-development"),
("grm", "pr-review"),
("sso-bridge", "devx-workflow"),
("sso-bridge", "testing-and-debugging"),
("sso-bridge", "spec-driven-development"),
("sso-bridge", "pr-review"),
("devx", "devx-workflow"),
("devx", "testing-and-debugging"),
("devx", "spec-driven-development"),
("devx", "pr-review"),
("mattermost-oidc", "testing-and-debugging"),
]
@pytest.mark.parametrize("repo_name, skill_name", ALL_SKILLS)
def test_skill_has_title(self, repo_name: str, skill_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / skill_name / "SKILL.md"
content = path.read_text(encoding="utf-8")
first_line = content.strip().split("\n")[0]
assert first_line.startswith("# "), f"{repo_name}/{skill_name}: SKILL.md must start with a # title"
@pytest.mark.parametrize("repo_name, skill_name", ALL_SKILLS)
def test_skill_not_empty(self, repo_name: str, skill_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / skill_name / "SKILL.md"
content = path.read_text(encoding="utf-8").strip()
assert len(content) > 100, f"{repo_name}/{skill_name}: SKILL.md is too short ({len(content)} chars)"
@pytest.mark.parametrize("repo_name, skill_name", ALL_SKILLS)
def test_skill_has_sections(self, repo_name: str, skill_name: str) -> None:
_skip_if_repo_missing(repo_name)
path = _OBLACHNO_ROOT / repo_name / ".devin" / "skills" / skill_name / "SKILL.md"
content = path.read_text(encoding="utf-8")
# Must have at least 2 ## sections
section_count = content.count("\n## ")
assert section_count >= 2, (
f"{repo_name}/{skill_name}: SKILL.md must have at least 2 sections (found {section_count})"
)
# ============================================================================
# AGENTS.md — spec-driven development section in all repos
# ============================================================================
class TestAgentsMdSpecSection:
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_agents_md_has_spec_driven_section(self, repo_name: str) -> None:
path = _OBLACHNO_ROOT / repo_name / "AGENTS.md"
if not path.exists():
pytest.skip(f"AGENTS.md not found in {repo_name}")
content = path.read_text(encoding="utf-8")
assert "## Spec-Driven Development" in content, (
f"AGENTS.md in {repo_name} must have '## Spec-Driven Development' section"
)
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_agents_md_mentions_validate_spec(self, repo_name: str) -> None:
path = _OBLACHNO_ROOT / repo_name / "AGENTS.md"
if not path.exists():
pytest.skip(f"AGENTS.md not found in {repo_name}")
content = path.read_text(encoding="utf-8")
assert "validate_spec" in content or "devx.ci.validate_spec" in content, (
f"AGENTS.md in {repo_name} must mention devx.ci.validate_spec"
)
@pytest.mark.parametrize("repo_name", ["infra", "grm", "sso-bridge", "devx"])
def test_agents_md_pr_workflow_section_intact(self, repo_name: str) -> None:
"""Ensure the PR Workflow section wasn't accidentally deleted."""
path = _OBLACHNO_ROOT / repo_name / "AGENTS.md"
if not path.exists():
pytest.skip(f"AGENTS.md not found in {repo_name}")
content = path.read_text(encoding="utf-8")
assert "## PR Workflow" in content, f"AGENTS.md in {repo_name} must still have '## PR Workflow' section"
+407 -42
View File
@@ -7,13 +7,32 @@ from click.testing import CliRunner
from devx.molecule.start_docker import (
DOCKER_SOCK,
HOST_DOCKER_SOCK,
ROOTLESS_SOCK,
_diagnose_socket,
_get_docker_free_bytes,
_try_socket,
is_docker_ready,
main,
start_docker_daemon,
)
def _pgrep_empty() -> MagicMock:
"""Mock for pgrep returning no dockerd processes."""
return MagicMock(stdout="", returncode=1)
def _docker_info_alive() -> MagicMock:
"""Mock for docker info showing daemon alive."""
return MagicMock(returncode=0)
def _docker_info_dead() -> MagicMock:
"""Mock for docker info showing daemon dead."""
return MagicMock(returncode=1)
class TestIsDockerReady:
@patch("devx.molecule.start_docker.subprocess.run")
def test_ready(self, mock_run: MagicMock) -> None:
@@ -40,6 +59,42 @@ class TestIsDockerReady:
assert call_kwargs.kwargs["env"]["DOCKER_HOST"] == rootless
class TestGetDockerFreeBytes:
@patch("devx.molecule.start_docker.shutil.disk_usage")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.subprocess.run")
def test_returns_free_bytes(self, mock_run: MagicMock, mock_exists: MagicMock, mock_du: MagicMock) -> None:
mock_run.return_value = MagicMock(stdout="/var/lib/docker\n", returncode=0, text="")
mock_du.return_value = MagicMock(free=50 * 1024**3)
with patch.dict("os.environ", {"DOCKER_HOST": f"unix://{DOCKER_SOCK}"}, clear=False):
assert _get_docker_free_bytes() == 50 * 1024**3
@patch("devx.molecule.start_docker.subprocess.run")
def test_daemon_not_reachable(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=1, stdout="", text="")
assert _get_docker_free_bytes() == 0
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.subprocess.run")
def test_data_root_not_accessible(self, mock_run: MagicMock, mock_exists: MagicMock) -> None:
mock_run.return_value = MagicMock(stdout="/some/path\n", returncode=0, text="")
assert _get_docker_free_bytes() == 0
@patch("devx.molecule.start_docker.subprocess.run", side_effect=FileNotFoundError)
def test_subprocess_not_found(self, mock_run: MagicMock) -> None:
assert _get_docker_free_bytes() == 0
class TestTrySocket:
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
def test_ready(self, mock_ready: MagicMock) -> None:
assert _try_socket("/run/user/999/docker.sock") is True
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
def test_not_ready(self, mock_ready: MagicMock) -> None:
assert _try_socket("/run/user/999/docker.sock") is False
class TestDiagnoseSocket:
@patch("devx.molecule.start_docker.os.stat")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@@ -86,81 +141,373 @@ class TestDiagnoseSocket:
mock_exists.assert_called_with(DOCKER_SOCK)
def _exists_map(paths: set[str]) -> MagicMock:
"""Return a mock os.path.exists that returns True only for *paths*."""
return MagicMock(side_effect=lambda p: p in paths)
class TestStartDockerDaemon:
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=100 * 1024**3)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
def test_host_socket_available(self, mock_ready: MagicMock, mock_diag: MagicMock) -> None:
"""Should return immediately if host Docker is available."""
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_host_socket_available_with_space(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should return immediately if /var/run/docker.sock has enough space."""
assert start_docker_daemon(timeout=5) is True
mock_ready.assert_called_once()
mock_diag.assert_called_once()
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.is_docker_ready")
def test_rootless_socket_available(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({HOST_DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_host_rootless_inaccessible_root_uses_host(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should use rootless socket if host socket fails."""
# First check (host) fails, second check (rootless) succeeds
"""Should use host rootless Docker when root dir is inaccessible (free=0)."""
assert start_docker_daemon(timeout=5) is True
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
@patch("devx.molecule.start_docker.time.sleep")
def test_inner_dockerd_free_zero_starts_local(
self,
mock_sleep: MagicMock,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Inner dockerd with free=0 (not host socket) should NOT be trusted — start local."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
# is_docker_ready: first check (inner dockerd) True, then local daemon checks
mock_ready.side_effect = [True, False, False, False, False, True]
# subprocess.run: pgrep(socket check, finds dockerd), pgrep(before kill),
# pkill x3, pgrep(after kill), docker info(dead), rm
mock_run.side_effect = [
MagicMock(returncode=0, stdout="123 /usr/bin/dockerd\n"), # socket check pgrep
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
_pgrep_empty(),
_docker_info_dead(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
def test_no_inner_dockerd_trusts_host_socket(
self,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should trust /var/run/docker.sock with free=0 when no inner dockerd exists."""
# pgrep finds no dockerd processes (returncode=1)
mock_run.return_value = MagicMock(returncode=1, stdout="")
assert start_docker_daemon(timeout=5) is True
assert os.environ.get("DOCKER_HOST") == f"unix://{DOCKER_SOCK}"
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
@patch("devx.molecule.start_docker.time.sleep")
def test_host_socket_low_space_starts_local(
self,
mock_sleep: MagicMock,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should start local dockerd if host Docker has low space and no rootless sockets."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
mock_ready.side_effect = [True, False, False, False, False, True]
mock_free.return_value = 5 * 1024**3
# subprocess.run: pgrep(before), pkill x3, pgrep(after), docker info(dead), rm
mock_run.side_effect = [
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
_pgrep_empty(),
_docker_info_dead(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
@patch("devx.molecule.start_docker.time.sleep")
def test_inner_dockerd_alive_uses_alt_sock(
self,
mock_sleep: MagicMock,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should use /dev/shm/docker.sock if inner dockerd can't be killed."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
mock_ready.side_effect = [True, False, False, False, False, True]
mock_free.return_value = 5 * 1024**3
# subprocess.run: pgrep(before), pkill x3, pgrep(after), docker info(alive), rm
mock_run.side_effect = [
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
_pgrep_empty(),
_docker_info_alive(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
assert os.environ.get("DOCKER_HOST") == "unix:///dev/shm/docker.sock"
@patch("devx.molecule.start_docker.os.kill")
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
@patch("devx.molecule.start_docker.time.sleep")
def test_inner_dockerd_killed_by_pid(
self,
mock_sleep: MagicMock,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
mock_kill: MagicMock,
) -> None:
"""Should kill dockerd by PID when pkill fails and pgrep finds processes."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
mock_ready.side_effect = [True, False, False, False, False, True]
mock_free.return_value = 5 * 1024**3
# pgrep(before) empty, pkill x3, pgrep(after) finds PID 12345, docker info(dead), rm
mock_run.side_effect = [
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
MagicMock(stdout="12345 /usr/bin/dockererd\n", returncode=0),
_docker_info_dead(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
mock_kill.assert_called_once_with(12345, 9)
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK, HOST_DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_host_rootless_preferred_over_inner_dockerd(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should prefer /run/host-docker.sock over /var/run/docker.sock."""
# host rootless ready with space, inner dockerd never tried
mock_ready.side_effect = [True]
mock_free.side_effect = [200 * 1024**3]
assert start_docker_daemon(timeout=5) is True
# DOCKER_HOST should be set to host socket
assert os.environ.get("DOCKER_HOST") == f"unix://{HOST_DOCKER_SOCK}"
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=100 * 1024**3)
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK, HOST_DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_host_rootless_not_ready_falls_to_inner(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should fall through to inner dockerd if host rootless socket is not ready."""
mock_ready.side_effect = [False, True]
with patch("devx.molecule.start_docker.glob.glob", return_value=[]):
assert start_docker_daemon(timeout=5) is True
assert start_docker_daemon(timeout=5) is True
assert os.environ.get("DOCKER_HOST") == f"unix://{DOCKER_SOCK}"
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.is_docker_ready")
def test_alt_rootless_socket_found(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=100 * 1024**3)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
@patch(
"devx.molecule.start_docker.os.path.exists",
side_effect=_exists_map({DOCKER_SOCK, HOST_DOCKER_SOCK, ROOTLESS_SOCK, "/run/user/999/docker.sock"}),
)
@patch("devx.molecule.start_docker.glob.glob", return_value=["/run/user/999/docker.sock"])
def test_glob_finds_extra_rootless_socket(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should find rootless socket at a different UID via glob scan."""
# Host fails, own rootless fails, alt rootless succeeds
mock_ready.side_effect = [False, False, True]
alt_sock = "/run/user/999/docker.sock"
with patch("devx.molecule.start_docker.glob.glob", return_value=[alt_sock]):
assert start_docker_daemon(timeout=5) is True
"""Should include rootless sockets found via glob scan."""
assert start_docker_daemon(timeout=5) is True
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
def test_alt_rootless_socket_skips_own(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK, HOST_DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_inner_dockerd_fallback_when_host_rootless_low_space(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should skip the own rootless socket in glob scan (already tried)."""
# Host fails, own rootless fails, alt rootless also fails, dockerd fails
mock_ready.side_effect = [False, False, False, False, False, False]
own_sock = f"/run/user/{os.getuid()}/docker.sock"
alt_sock = "/run/user/999/docker.sock"
with (
patch("devx.molecule.start_docker.glob.glob", return_value=[own_sock, alt_sock]),
patch("devx.molecule.start_docker.time.sleep"),
patch("devx.molecule.start_docker.subprocess.Popen"),
patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile") as mock_ntf,
patch("builtins.open", mock_open(read_data="err")),
):
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
assert start_docker_daemon(timeout=2) is False
"""Should fall back to inner dockerd if host rootless has low space."""
# host rootless ready low space, inner dockerd ready with space
mock_ready.side_effect = [True, True]
mock_free.side_effect = [5 * 1024**3, 200 * 1024**3]
assert start_docker_daemon(timeout=5) is True
assert os.environ.get("DOCKER_HOST") == f"unix://{DOCKER_SOCK}"
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes")
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
@patch("devx.molecule.start_docker.time.sleep")
def test_all_sockets_low_space_starts_local(
self,
mock_sleep: MagicMock,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""Should start local dockerd if all sockets have low space."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
mock_ready.side_effect = [True, False, False, False, False, True]
mock_free.return_value = 5 * 1024**3
# subprocess.run: pgrep(before), pkill x3, pgrep(after), docker info(dead), rm
mock_run.side_effect = [
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
_pgrep_empty(),
_docker_info_dead(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_starts_local_daemon(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
# Host fails, rootless doesn't exist, local daemon starts
# No sockets exist, local daemon starts
mock_ready.side_effect = [False, False, False, False, True]
# subprocess.run: pgrep(before), pkill x3, pgrep(after), rm
mock_run.side_effect = [
_pgrep_empty(),
MagicMock(),
MagicMock(),
MagicMock(),
_pgrep_empty(),
MagicMock(),
]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
popen_args = mock_popen.call_args.args[0]
@@ -171,18 +518,22 @@ class TestStartDockerDaemon:
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_fails_after_timeout(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
@@ -190,22 +541,26 @@ class TestStartDockerDaemon:
with patch("builtins.open", mock_open(read_data="dockerd error log")):
assert start_docker_daemon(timeout=3) is False
mock_popen.assert_called_once()
assert mock_sleep.call_count == 3
assert mock_sleep.call_count == 5 # 1 after pkill + 1 after pgrep + 3 timeout retries
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_fails_log_read_error(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
@@ -216,35 +571,45 @@ class TestStartDockerDaemon:
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=0)
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.run")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_local_daemon_ready_on_first_check(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_run: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
# Host fails, rootless doesn't exist, local ready on first loop check
# No sockets exist, local ready on second loop check
mock_ready.side_effect = [False, False, True]
assert start_docker_daemon(timeout=5) is True
assert mock_popen.call_count == 1
assert mock_sleep.call_count == 1
assert mock_sleep.call_count == 4 # 1 after pkill + 1 after pgrep + 2 loop retries
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker._get_docker_free_bytes", return_value=100 * 1024**3)
@patch("devx.molecule.start_docker.os.environ")
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
@patch("devx.molecule.start_docker.os.path.exists", side_effect=_exists_map({DOCKER_SOCK}))
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
def test_sets_docker_host(
self,
mock_glob: MagicMock,
mock_exists: MagicMock,
mock_ready: MagicMock,
mock_environ: MagicMock,
mock_free: MagicMock,
mock_diag: MagicMock,
) -> None:
"""DOCKER_HOST must be set so molecule connects to correct socket."""
@@ -0,0 +1,367 @@
"""Unit tests for devx.tools.check_ansible_no_log."""
from __future__ import annotations
from pathlib import Path
from click.testing import CliRunner
from devx.tools.check_ansible_no_log import _check_task, check_directory, main
def _make_task(name: str, action: str, value: str, **extra: object) -> dict:
"""Build a minimal task dict for testing."""
task: dict = {"name": name, action: value}
task.update(extra)
return task
class TestCheckTask:
def test_task_with_secret_and_no_log_passes(self):
task = _make_task(
"Safe task",
"ansible.builtin.shell",
"echo {{ _secrets.mattermost_admin_password }}",
no_log=True,
)
assert _check_task(task, Path("test.yml"), 1) == []
def test_task_with_secret_and_no_no_log_fails(self):
task = _make_task(
"Unsafe task",
"ansible.builtin.shell",
"echo {{ _secrets.mattermost_admin_password }}",
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
assert "no_log" in violations[0]
def test_task_without_secret_passes(self):
task = _make_task(
"Normal task",
"ansible.builtin.shell",
"echo hello world",
)
assert _check_task(task, Path("test.yml"), 1) == []
def test_task_with_jinja_no_log_passes(self):
task = _make_task(
"Safe task with jinja no_log",
"ansible.builtin.shell",
"echo {{ _secrets.mattermost_admin_password }}",
no_log="{{ not (debug_mode | default(false) | bool) }}",
)
assert _check_task(task, Path("test.yml"), 1) == []
def test_task_with_password_in_name_only_no_false_positive(self):
"""Task name contains 'password' but no secret value — should not flag."""
task = _make_task(
"Configure passwdqc in common-password",
"ansible.builtin.lineinfile",
"password required pam_passwdqc.so min=disabled,disabled,16,12,8",
)
assert _check_task(task, Path("test.yml"), 1) == []
def test_task_with_password_in_module_param_no_false_positive(self):
"""Module param named 'password' but value is a literal — no Jinja."""
task = {
"name": "Set user password",
"ansible.builtin.user": {
"name": "deploy",
"password_lock": True,
},
}
assert _check_task(task, Path("test.yml"), 1) == []
def test_task_with_vault_password_variable_fails(self):
task = _make_task(
"Unsafe vault task",
"ansible.builtin.shell",
"echo {{ vault_zitadel_db_password }}",
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_nested_dict_secret_fails(self):
"""Secrets in nested dict values (e.g. set_fact) should be caught."""
task = {
"name": "Set secrets",
"ansible.builtin.set_fact": {
"db_password": "{{ vault_db_password }}",
"api_key": "{{ vault_api_key }}",
},
}
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_no_log_none_passes(self):
"""no_log: None should count as not set (flagged)."""
task = _make_task(
"Unsafe task",
"ansible.builtin.shell",
"echo {{ _secrets.db_password }}",
no_log=None,
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_secret_in_list_value_fails(self):
"""Secrets inside list values should be caught."""
task = {
"name": "Task with list secret",
"ansible.builtin.set_fact": {
"items": ["{{ _secrets.api_key }}", "normal_value"],
},
}
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_api_key_secret_fails(self):
"""api_key in Jinja expression should be caught."""
task = _make_task(
"Unsafe task",
"ansible.builtin.shell",
"echo {{ my_api_key }}",
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_secret_in_jinja_fails(self):
"""_secret in Jinja expression should be caught."""
task = _make_task(
"Unsafe task",
"ansible.builtin.shell",
"echo {{ my_secret }}",
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_access_token_fails(self):
"""access_token in Jinja expression should be caught."""
task = _make_task(
"Unsafe task",
"ansible.builtin.shell",
"echo {{ my_access_token }}",
)
violations = _check_task(task, Path("test.yml"), 1)
assert len(violations) == 1
def test_task_with_non_secret_non_dict_non_list_value(self):
"""Non-str, non-dict, non-list values (e.g. int) should not crash."""
task = _make_task(
"Task with int",
"ansible.builtin.shell",
"echo hello",
some_int=42,
)
assert _check_task(task, Path("test.yml"), 1) == []
class TestCheckDirectory:
def test_clean_directory_passes(self, tmp_path: Path):
"""A directory with no secret-handling tasks should pass."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Normal task\n ansible.builtin.shell: echo hello\n changed_when: false\n"
)
assert check_directory(role_dir) == []
def test_unsafe_task_is_caught(self, tmp_path: Path):
"""A task with secrets but no no_log should be flagged."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Unsafe task\n ansible.builtin.shell: echo {{ _secrets.db_password }}\n changed_when: false\n"
)
violations = check_directory(role_dir)
assert len(violations) == 1
assert "Unsafe task" in violations[0]
def test_molecule_files_are_skipped(self, tmp_path: Path):
"""Molecule test files should not be scanned."""
role_dir = tmp_path / "roles" / "test_role"
mol_dir = role_dir / "molecule" / "default" / "tasks"
mol_dir.mkdir(parents=True)
(mol_dir / "main.yml").write_text(
"- name: Unsafe task in molecule\n ansible.builtin.shell: echo {{ _secrets.db_password }}\n"
)
assert check_directory(role_dir) == []
def test_playbook_format_is_parsed(self, tmp_path: Path):
"""Playbook files (list of plays with 'hosts') should be parsed."""
pb_dir = tmp_path / "playbooks"
pb_dir.mkdir(parents=True)
(pb_dir / "test.yml").write_text(
"---\n"
"- name: Test play\n"
" hosts: all\n"
" tasks:\n"
" - name: Unsafe task\n"
" ansible.builtin.shell: echo {{ _secrets.db_password }}\n"
)
violations = check_directory(tmp_path)
assert len(violations) == 1
assert "Unsafe task" in violations[0]
def test_invalid_yaml_is_skipped(self, tmp_path: Path):
"""Invalid YAML files should be skipped, not crash."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text("{{ invalid yaml: [")
assert check_directory(role_dir) == []
def test_empty_yaml_doc_is_skipped(self, tmp_path: Path):
"""Empty YAML documents (None) should be skipped."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text("---\n")
assert check_directory(role_dir) == []
def test_non_dict_non_list_doc_is_skipped(self, tmp_path: Path):
"""YAML docs that are neither dict nor list should be skipped."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text("just a string\n")
assert check_directory(role_dir) == []
def test_task_file_with_non_dict_task_skipped(self, tmp_path: Path):
"""Non-dict items in a task list should be skipped."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- just a string\n- name: Safe task\n ansible.builtin.shell: echo hello\n"
)
assert check_directory(role_dir) == []
def test_secret_in_list_value_is_caught(self, tmp_path: Path):
"""Secrets inside list values should be caught."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Task with list secret\n"
" ansible.builtin.set_fact:\n"
" items:\n"
' - "{{ _secrets.api_key }}"\n'
" - normal_value\n"
)
violations = check_directory(role_dir)
assert len(violations) == 1
def test_single_play_dict_format(self, tmp_path: Path):
"""A playbook that's a bare dict (not list of plays) should be parsed."""
pb_dir = tmp_path / "playbooks"
pb_dir.mkdir(parents=True)
(pb_dir / "test.yml").write_text(
"---\n"
"name: Single play\n"
"hosts: all\n"
"tasks:\n"
" - name: Unsafe task\n"
" ansible.builtin.shell: echo {{ _secrets.db_password }}\n"
)
violations = check_directory(tmp_path)
assert len(violations) == 1
def test_play_with_non_dict_play_skipped(self, tmp_path: Path):
"""Non-dict plays in a playbook list should be skipped."""
pb_dir = tmp_path / "playbooks"
pb_dir.mkdir(parents=True)
# First play is valid (makes is_plays=True), second is a non-dict
(pb_dir / "test.yml").write_text(
"---\n"
"- name: Safe play\n"
" hosts: all\n"
" tasks:\n"
" - name: Safe task\n"
" ansible.builtin.shell: echo hello\n"
'- "just a string as second play"\n'
)
assert check_directory(tmp_path) == []
def test_play_with_non_list_tasks_skipped(self, tmp_path: Path):
"""Plays where tasks is not a list should be skipped."""
pb_dir = tmp_path / "playbooks"
pb_dir.mkdir(parents=True)
(pb_dir / "test.yml").write_text('---\n- name: Play with bad tasks\n hosts: all\n tasks: "not a list"\n')
assert check_directory(tmp_path) == []
def test_play_with_non_dict_task_in_playbook(self, tmp_path: Path):
"""Non-dict tasks in a playbook should be skipped."""
pb_dir = tmp_path / "playbooks"
pb_dir.mkdir(parents=True)
(pb_dir / "test.yml").write_text(
"---\n"
"- name: Play\n"
" hosts: all\n"
" tasks:\n"
' - "just a string"\n'
" - name: Safe task\n"
" ansible.builtin.shell: echo hello\n"
)
assert check_directory(tmp_path) == []
def test_yaml_file_with_oserror_skipped(self, tmp_path: Path):
"""YAML files that can't be opened should be skipped."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
# Create a file that will cause OSError when opened
# (use a directory with .yml extension)
bad_file = role_dir / "tasks" / "main.yml"
bad_file.mkdir()
assert check_directory(role_dir) == []
class TestMain:
def test_main_passes_on_clean_dir(self, tmp_path: Path):
"""main() should exit 0 on a clean directory."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Normal task\n ansible.builtin.shell: echo hello\n changed_when: false\n"
)
runner = CliRunner()
result = runner.invoke(main, ["--path", str(role_dir)])
assert result.exit_code == 0
assert "OK" in result.output or "no_log" in result.output
def test_main_fails_on_unsafe_dir(self, tmp_path: Path):
"""main() should exit 1 when violations are found."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Unsafe task\n ansible.builtin.shell: echo {{ _secrets.db_password }}\n"
)
runner = CliRunner()
result = runner.invoke(main, ["--path", str(role_dir)])
assert result.exit_code == 1
assert "Unsafe task" in result.output
def test_main_returns_2_on_missing_dir(self, tmp_path: Path):
"""main() should exit 2 when the directory doesn't exist."""
runner = CliRunner()
result = runner.invoke(main, ["--path", str(tmp_path / "nonexistent")])
assert result.exit_code == 2
def test_main_with_ansible_dir_option(self, tmp_path: Path):
"""main() --ansible-dir should work like --path."""
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text(
"- name: Unsafe task\n ansible.builtin.shell: echo {{ _secrets.db_password }}\n"
)
runner = CliRunner()
result = runner.invoke(main, ["--ansible-dir", str(tmp_path)])
assert result.exit_code == 1
def test_main_no_path_no_ansible_dir_uses_default(self, tmp_path: Path, monkeypatch):
"""main() with no args uses DEFAULT_ANSIBLE_DIR."""
import devx.tools.check_ansible_no_log as mod
role_dir = tmp_path / "roles" / "test_role"
(role_dir / "tasks").mkdir(parents=True)
(role_dir / "tasks" / "main.yml").write_text("- name: Normal task\n ansible.builtin.shell: echo hello\n")
monkeypatch.setattr(mod, "DEFAULT_ANSIBLE_DIR", tmp_path)
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
@@ -0,0 +1,198 @@
"""Unit tests for devx.tools.check_ansible_no_state_absent_on_db."""
from __future__ import annotations
from pathlib import Path
from click.testing import CliRunner
from devx.tools.check_ansible_no_state_absent_on_db import _check_file, _find_task_files, main
class TestCheckFile:
def test_clean_file_no_db_paths(self, tmp_path: Path):
"""A file with no DB paths should produce no violations."""
p = tmp_path / "test.yml"
p.write_text("- name: Safe task\n ansible.builtin.file:\n path: /opt/app/data\n state: directory\n")
assert _check_file(p, tmp_path) == []
def test_state_absent_on_zitadel_db_fails(self, tmp_path: Path):
"""state: absent on zitadel-db path should be flagged."""
p = tmp_path / "test.yml"
p.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /opt/postgres/zitadel-db\n state: absent\n"
)
violations = _check_file(p, tmp_path)
assert len(violations) >= 1
assert "state" in violations[0].lower() or "absent" in violations[0].lower()
def test_state_absent_on_var_lib_postgresql_fails(self, tmp_path: Path):
"""state: absent on /var/lib/postgresql/data should be flagged."""
p = tmp_path / "test.yml"
p.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /var/lib/postgresql/data\n state: absent\n"
)
violations = _check_file(p, tmp_path)
assert len(violations) >= 1
def test_state_absent_on_app_db_fails(self, tmp_path: Path):
"""state: absent on any *-db path should be flagged."""
p = tmp_path / "test.yml"
p.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /opt/postgres/gitea-db\n state: absent\n"
)
violations = _check_file(p, tmp_path)
assert len(violations) >= 1
def test_state_absent_with_pg_upgrade_context_passes(self, tmp_path: Path):
"""state: absent near DB path with upgrade-postgres context should pass."""
p = tmp_path / "test.yml"
p.write_text(
"- name: PG upgrade — remove old data\n"
" ansible.builtin.file:\n"
" path: /opt/postgres/zitadel-db\n"
" state: absent\n"
" when: pg_version_changed | default(false)\n"
)
violations = _check_file(p, tmp_path)
assert violations == []
def test_state_absent_with_pg_version_context_passes(self, tmp_path: Path):
"""state: absent near DB path with PG_VERSION context should pass."""
p = tmp_path / "test.yml"
p.write_text(
"- name: PG upgrade\n"
" ansible.builtin.file:\n"
" path: /opt/postgres/zitadel-db\n"
" state: absent\n"
" when: PG_VERSION is defined\n"
)
violations = _check_file(p, tmp_path)
assert violations == []
def test_state_absent_with_allow_marker_passes(self, tmp_path: Path):
"""state: absent with lint:allow-state-absent comment should pass."""
p = tmp_path / "test.yml"
p.write_text(
"# lint:allow-state-absent\n"
"- name: Intentional wipe\n"
" ansible.builtin.file:\n"
" path: /opt/postgres/zitadel-db\n"
" state: absent\n"
)
violations = _check_file(p, tmp_path)
assert violations == []
def test_state_present_on_db_path_passes(self, tmp_path: Path):
"""state: present (not absent) on DB path should pass."""
p = tmp_path / "test.yml"
p.write_text(
"- name: Safe task\n ansible.builtin.file:\n path: /opt/postgres/zitadel-db\n state: directory\n"
)
assert _check_file(p, tmp_path) == []
def test_nonexistent_file_returns_empty(self):
"""A nonexistent file should return no violations."""
assert _check_file(Path("/nonexistent/path/file.yml"), Path.cwd()) == []
def test_rm_rf_db_fails(self, tmp_path: Path):
"""rm -rf on a DB path should be flagged."""
p = tmp_path / "test.yml"
p.write_text("- name: Dangerous wipe\n ansible.builtin.shell: rm -rf /opt/postgres/zitadel-db\n")
violations = _check_file(p, tmp_path)
assert len(violations) >= 1
def test_relative_path_outside_repo(self, tmp_path: Path):
"""Files outside repo_root use the full path in display."""
p = tmp_path / "test.yml"
p.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /opt/postgres/zitadel-db\n state: absent\n"
)
violations = _check_file(p, Path("/other/repo"))
assert len(violations) >= 1
assert str(tmp_path) in violations[0] or "test.yml" in violations[0]
class TestFindTaskFiles:
def test_find_yml_files_in_directory(self, tmp_path: Path):
"""Should find .yml files in a directory."""
(tmp_path / "tasks").mkdir()
(tmp_path / "tasks" / "main.yml").write_text("[]")
(tmp_path / "tasks" / "other.yaml").write_text("[]")
files = _find_task_files(tmp_path)
assert len(files) == 2
def test_skip_molecule_files(self, tmp_path: Path):
"""Should skip files in molecule directories."""
(tmp_path / "molecule").mkdir()
(tmp_path / "molecule" / "test.yml").write_text("[]")
(tmp_path / "main.yml").write_text("[]")
files = _find_task_files(tmp_path)
assert len(files) == 1
assert "molecule" not in files[0].parts
def test_single_file_input(self, tmp_path: Path):
"""Should return the file itself if it's a .yml file."""
f = tmp_path / "test.yml"
f.write_text("[]")
files = _find_task_files(f)
assert files == [f]
def test_nonexistent_path_returns_empty(self):
"""A path that is neither a file nor a dir should return []."""
files = _find_task_files(Path("/nonexistent/path/that/does/not/exist"))
assert files == []
def test_non_yaml_file_skipped(self, tmp_path: Path):
"""Non-YAML files should not be included."""
f = tmp_path / "readme.txt"
f.write_text("not yaml")
assert _find_task_files(f) == []
class TestMain:
def test_main_no_violations_exit_zero(self, tmp_path: Path):
"""main() with a clean file should exit 0."""
f = tmp_path / "clean.yml"
f.write_text("- name: Safe task\n ansible.builtin.file:\n path: /opt/app\n state: directory\n")
result = CliRunner().invoke(main, ["--path", str(f)])
assert result.exit_code == 0
assert "OK" in result.output
def test_main_with_violations_exit_one(self, tmp_path: Path):
"""main() with a state: absent on a DB path should exit 1."""
f = tmp_path / "dangerous.yml"
f.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /opt/postgres/zitadel-db\n state: absent\n"
)
result = CliRunner().invoke(main, ["--path", str(f)])
assert result.exit_code == 1
assert "FAIL" in result.output
def test_main_path_to_clean_file(self, tmp_path: Path):
"""main() --path pointing to a specific clean file should exit 0."""
f = tmp_path / "tasks.yml"
f.write_text("- name: Safe\n ansible.builtin.file:\n path: /opt/app\n state: directory\n")
result = CliRunner().invoke(main, ["--path", str(f)])
assert result.exit_code == 0
def test_main_default_dirs_no_violations(self, tmp_path: Path, monkeypatch):
"""main() with no --path scans default dirs and exits 0."""
import devx.tools.check_ansible_no_state_absent_on_db as mod
(tmp_path / "clean.yml").write_text(
"- name: Safe\n ansible.builtin.file:\n path: /opt/app\n state: directory\n"
)
monkeypatch.setattr(mod, "DEFAULT_ANSIBLE_DIRS", [tmp_path])
result = CliRunner().invoke(main, [])
assert result.exit_code == 0
assert "OK" in result.output
def test_main_custom_ansible_dirs(self, tmp_path: Path):
"""main() --ansible-dir should work."""
f = tmp_path / "dangerous.yml"
f.write_text(
"- name: Dangerous wipe\n ansible.builtin.file:\n path: /opt/postgres/zitadel-db\n state: absent\n"
)
result = CliRunner().invoke(main, ["--ansible-dir", str(tmp_path)])
assert result.exit_code == 1
@@ -0,0 +1,340 @@
"""Unit tests for devx.tools.check_ansible_patterns."""
from __future__ import annotations
from pathlib import Path
from click.testing import CliRunner
from devx.tools.check_ansible_patterns import (
_check_file,
_check_task,
_check_tasks,
_find_task_files,
_is_legitimate_devnull,
_is_legitimate_or_true,
main,
)
def _make_task(name: str, action: str, value: str, **extra: object) -> dict:
"""Build a minimal task dict for testing."""
task: dict = {"name": name, action: value}
task.update(extra)
return task
class TestIsLegitimateOrTrue:
def test_cleanup_task_name_is_legitimate(self):
assert _is_legitimate_or_true("docker rm old-container", "Remove old container")
def test_prune_task_name_is_legitimate(self):
assert _is_legitimate_or_true("docker image prune -f", "Prune unused images")
def test_docker_rm_command_is_legitimate(self):
assert _is_legitimate_or_true("docker rm -f mycontainer", "Some task")
def test_provision_task_is_not_legitimate(self):
assert not _is_legitimate_or_true("curl -X POST https://api/app || true", "Provision OIDC client")
def test_sync_task_name_is_legitimate(self):
assert _is_legitimate_or_true("psql -c 'ALTER USER' || true", "Sync PostgreSQL password")
class TestCheckTask:
def test_or_true_on_provision_task_fails(self, tmp_path: Path):
task = _make_task(
"Provision OIDC client",
"ansible.builtin.shell",
"curl -X POST https://zitadel/api || true",
)
violations = _check_task(task, tmp_path / "test.yml", 1, tmp_path)
assert len(violations) >= 1
assert "|| true" in violations[0]
def test_or_true_on_cleanup_task_passes(self, tmp_path: Path):
task = _make_task(
"Remove old container",
"ansible.builtin.shell",
"docker rm -f old-container || true",
)
assert _check_task(task, tmp_path / "test.yml", 1, tmp_path) == []
def test_failed_when_false_on_provision_fails(self, tmp_path: Path):
task = _make_task(
"Provision OIDC client",
"ansible.builtin.shell",
"curl -X POST https://zitadel/api",
failed_when=False,
)
violations = _check_task(task, tmp_path / "test.yml", 1, tmp_path)
assert any("failed_when" in v for v in violations)
def test_failed_when_false_on_stop_passes(self, tmp_path: Path):
task = _make_task(
"Stop ZITADEL containers",
"ansible.builtin.shell",
"docker stop zitadel",
failed_when=False,
)
assert _check_task(task, tmp_path / "test.yml", 1, tmp_path) == []
def test_failed_when_false_on_check_passes(self, tmp_path: Path):
task = _make_task(
"Check if ZITADEL is running",
"ansible.builtin.shell",
"docker inspect zitadel",
failed_when=False,
)
assert _check_task(task, tmp_path / "test.yml", 1, tmp_path) == []
def test_allow_marker_in_name_passes(self, tmp_path: Path):
task = _make_task(
"Provision OIDC #lint:allow-failure-masking",
"ansible.builtin.shell",
"curl -X POST https://zitadel/api || true",
failed_when=False,
)
assert _check_task(task, tmp_path / "test.yml", 1, tmp_path) == []
def test_safe_task_no_violations(self, tmp_path: Path):
task = _make_task(
"Create directory",
"ansible.builtin.file",
"path=/opt/app state=directory",
)
assert _check_task(task, tmp_path / "test.yml", 1, tmp_path) == []
def test_relative_path_outside_repo(self, tmp_path: Path):
"""Files outside repo_root use the full path in display."""
task = _make_task(
"Provision OIDC",
"ansible.builtin.shell",
"curl || true",
)
other_dir = Path("/tmp/other")
violations = _check_task(task, other_dir / "test.yml", 1, tmp_path)
assert len(violations) >= 1
class TestCheckFile:
def test_clean_file_passes(self, tmp_path: Path):
p = tmp_path / "test.yml"
p.write_text("- name: Safe task\n ansible.builtin.file:\n path: /opt/app\n state: directory\n")
assert _check_file(p, tmp_path) == []
def test_dangerous_pattern_detected(self, tmp_path: Path):
p = tmp_path / "test.yml"
p.write_text(
"- name: Provision OIDC\n"
" ansible.builtin.shell: |\n"
" curl -X POST https://api/app || true\n"
" failed_when: false\n"
)
violations = _check_file(p, tmp_path)
assert len(violations) >= 1
def test_file_level_allow_marker_passes(self, tmp_path: Path):
p = tmp_path / "test.yml"
p.write_text(
"# lint:allow-failure-masking\n"
"- name: Provision OIDC\n"
" ansible.builtin.shell: |\n"
" curl -X POST https://api/app || true\n"
" failed_when: false\n"
)
assert _check_file(p, tmp_path) == []
def test_nonexistent_file_returns_empty(self):
assert _check_file(Path("/nonexistent/path/file.yml"), Path.cwd()) == []
def test_yaml_parse_error_returns_empty(self, tmp_path: Path):
p = tmp_path / "test.yml"
p.write_text("name: Provision OIDC\n shell: curl || true\n: invalid: [")
assert _check_file(p, tmp_path) == []
def test_dict_doc_playbook_with_tasks(self, tmp_path: Path):
p = tmp_path / "playbook.yml"
p.write_text(
"- hosts: all\n"
" tasks:\n"
" - name: Provision OIDC\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
)
violations = _check_file(p, tmp_path)
assert any("|| true" in v for v in violations)
def test_dict_doc_with_pre_tasks_and_post_tasks(self, tmp_path: Path):
p = tmp_path / "playbook.yml"
p.write_text(
"- hosts: all\n"
" pre_tasks:\n"
" - name: Provision secret\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
" post_tasks:\n"
" - name: Provision OIDC\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
" handlers:\n"
" - name: Provision password\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
)
violations = _check_file(p, tmp_path)
assert len(violations) >= 3
def test_block_tasks_in_list_item(self, tmp_path: Path):
p = tmp_path / "tasks.yml"
p.write_text(
"- name: Outer task\n"
" block:\n"
" - name: Provision OIDC\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
" - name: Provision secret\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
)
violations = _check_file(p, tmp_path)
assert any("|| true" in v for v in violations)
def test_empty_doc_skipped(self, tmp_path: Path):
p = tmp_path / "test.yml"
p.write_text("---\nnull\n---\n- name: Provision OIDC\n ansible.builtin.shell: curl || true\n")
violations = _check_file(p, tmp_path)
assert any("|| true" in v for v in violations)
def test_pure_dict_doc_with_tasks(self, tmp_path: Path):
p = tmp_path / "playbook.yml"
p.write_text(
"hosts: all\n"
"tasks:\n"
" - name: Provision OIDC\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
)
violations = _check_file(p, tmp_path)
assert any("|| true" in v for v in violations)
class TestIsLegitimateDevnull:
def test_cleanup_task_is_legitimate(self):
assert _is_legitimate_devnull("docker rm old-container 2>/dev/null", "Remove old container")
def test_provision_task_is_not_legitimate(self):
assert not _is_legitimate_devnull("curl -X POST https://api/app 2>/dev/null", "Provision OIDC client")
class TestCheckTasks:
def test_tasks_section_checked(self, tmp_path: Path):
doc = {
"tasks": [
{"name": "Provision OIDC", "ansible.builtin.shell": "curl || true"},
],
}
errors: list[str] = []
_check_tasks(doc, tmp_path / "test.yml", errors, tmp_path)
assert any("|| true" in e for e in errors)
def test_block_inside_tasks_section(self, tmp_path: Path):
doc = {
"tasks": [
{
"name": "Outer",
"block": [
{"name": "Provision secret", "ansible.builtin.shell": "curl || true"},
],
},
],
}
errors: list[str] = []
_check_tasks(doc, tmp_path / "test.yml", errors, tmp_path)
assert any("|| true" in e for e in errors)
def test_non_list_section_ignored(self, tmp_path: Path):
doc = {"tasks": "not a list"}
errors: list[str] = []
_check_tasks(doc, tmp_path / "test.yml", errors, tmp_path)
assert errors == []
def test_non_dict_task_ignored(self, tmp_path: Path):
doc = {"tasks": ["just a string"]}
errors: list[str] = []
_check_tasks(doc, tmp_path / "test.yml", errors, tmp_path)
assert errors == []
class TestFindTaskFiles:
def test_single_file(self, tmp_path: Path):
p = tmp_path / "main.yml"
p.write_text("- name: test\n")
assert _find_task_files(p) == [p]
def test_single_yaml_file(self, tmp_path: Path):
p = tmp_path / "main.yaml"
p.write_text("- name: test\n")
assert _find_task_files(p) == [p]
def test_non_yaml_file_returns_empty(self, tmp_path: Path):
p = tmp_path / "main.txt"
p.write_text("hello\n")
assert _find_task_files(p) == []
def test_directory_finds_yaml_files(self, tmp_path: Path):
(tmp_path / "a.yml").write_text("- name: a\n")
(tmp_path / "sub").mkdir()
(tmp_path / "sub" / "b.yaml").write_text("- name: b\n")
(tmp_path / "ignore.txt").write_text("nope\n")
result = _find_task_files(tmp_path)
names = {f.name for f in result}
assert names == {"a.yml", "b.yaml"}
def test_directory_skips_molecule(self, tmp_path: Path):
(tmp_path / "a.yml").write_text("- name: a\n")
(tmp_path / "molecule").mkdir()
(tmp_path / "molecule" / "scenario.yml").write_text("- name: mol\n")
result = _find_task_files(tmp_path)
assert all("molecule" not in f.parts for f in result)
def test_nonexistent_path_returns_empty(self):
assert _find_task_files(Path("/nonexistent/path/xyz")) == []
class TestMain:
def test_main_clean_file_exit_zero(self, tmp_path: Path):
p = tmp_path / "clean.yml"
p.write_text("- name: Safe task\n ansible.builtin.file:\n path: /opt/app\n state: directory\n")
result = CliRunner().invoke(main, ["--path", str(p)])
assert result.exit_code == 0
assert "OK" in result.output
def test_main_violation_exit_one(self, tmp_path: Path):
p = tmp_path / "bad.yml"
p.write_text(
"- name: Provision OIDC\n"
" ansible.builtin.shell: curl -X POST https://api/app || true\n"
" failed_when: false\n"
)
result = CliRunner().invoke(main, ["--path", str(p)])
assert result.exit_code == 1
assert "FAIL" in result.output
def test_main_directory(self, tmp_path: Path):
(tmp_path / "clean.yml").write_text(
"- name: Safe task\n ansible.builtin.file:\n path: /opt\n state: directory\n"
)
result = CliRunner().invoke(main, ["--path", str(tmp_path)])
assert result.exit_code == 0
def test_main_default_dirs(self, tmp_path: Path, monkeypatch):
import devx.tools.check_ansible_patterns as mod
(tmp_path / "clean.yml").write_text(
"- name: Safe task\n ansible.builtin.file:\n path: /opt\n state: directory\n"
)
monkeypatch.setattr(mod, "DEFAULT_ANSIBLE_DIRS", [tmp_path])
result = CliRunner().invoke(main, [])
assert result.exit_code == 0
assert "OK" in result.output
def test_main_custom_ansible_dirs(self, tmp_path: Path):
(tmp_path / "bad.yml").write_text(
"- name: Provision OIDC\n ansible.builtin.shell: curl -X POST https://api/app || true\n"
)
result = CliRunner().invoke(main, ["--ansible-dir", str(tmp_path)])
assert result.exit_code == 1
+264
View File
@@ -0,0 +1,264 @@
"""Unit tests for devx.tools.check_jinja_expr.
Verifies that the check correctly validates Jinja2 expressions,
catches reversed strftime filter arguments (the OBL-INFRA-508 bug),
and passes on valid expressions.
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from devx.tools.check_jinja_expr import (
_check_file,
_default_ansible_dirs,
_extract_expressions,
_render_expression,
main,
)
def test_render_valid_expression():
"""Valid Jinja expression renders without error."""
ok, _ = _render_expression("'%Y-%m-%dT%H:%M:%S+00:00' | strftime(1735689600)")
assert ok
def test_render_reversed_strftime_args():
"""Reversed strftime filter args are detected as an error."""
ok, msg = _render_expression("(now().timestamp() | int + 3600) | strftime('%Y-%m-%dT%H:%M:%S+00:00')")
assert not ok
assert "reversed" in msg.lower()
def test_render_correct_strftime_args():
"""Correct strftime filter args pass."""
ok, _ = _render_expression("'%Y-%m-%dT%H:%M:%S+00:00' | strftime((now().timestamp() | int) + 3600)")
assert ok
def test_render_unknown_filter():
"""Unknown filter is reported as an error."""
ok, msg = _render_expression("'test' | nonexistent_filter")
assert not ok
assert "filter" in msg.lower()
def test_extract_skips_go_templates():
"""Go template syntax ({{.Field}}) is not extracted."""
content = "cmd: docker inspect --format '{{.State.Running}}' container"
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_single_char():
"""Single-character fragments are not extracted."""
content = 'value: "{{ \' }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_multiline():
"""Multi-line expressions are skipped."""
content = 'value: "{{\n something\n}}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_unbalanced():
"""Expressions with unbalanced braces (from partial capture) are skipped."""
content = "value: \"{{ default({'k': {}}, true) }}\""
expressions = _extract_expressions(content)
# The regex captures {{ default({'k': {}} — unbalanced parens
# because the inner }} terminates the match early.
# All extracted expressions should have balanced braces.
for expr in expressions:
assert expr.count("{") == expr.count("}")
def test_extract_valid_expression():
"""Valid Jinja expressions are extracted."""
content = "value: \"{{ my_var | default('x') }}\""
expressions = _extract_expressions(content)
assert "my_var | default('x')" in expressions
def test_main_passes_on_clean_file(tmp_path: Path) -> None:
"""A file with valid expressions passes."""
test_file = tmp_path / "tasks.yml"
test_file.write_text("value: \"{{ my_var | default('x') }}\"\nother: \"{{ '%Y' | strftime(1735689600) }}\"\n")
runner = CliRunner()
result = runner.invoke(main, ["--path", str(test_file)])
assert result.exit_code == 0
def test_main_no_violations_empty_dir(tmp_path: Path) -> None:
"""An empty directory passes."""
runner = CliRunner()
result = runner.invoke(main, ["--path", str(tmp_path)])
assert result.exit_code == 0
def test_main_catches_reversed_strftime(tmp_path: Path) -> None:
"""A file with reversed strftime args is flagged."""
test_file = tmp_path / "test.yml"
test_file.write_text("value: \"{{ (now().timestamp() | int + 3600) | strftime('%Y-%m-%dT%H:%M:%S+00:00') }}\"\n")
runner = CliRunner()
result = runner.invoke(main, ["--path", str(test_file)])
assert result.exit_code == 1
assert "reversed" in result.output.lower()
def test_render_skips_undefined_var():
"""Undefined variables are skipped (MockDict returns mock for missing keys)."""
ok, _ = _render_expression("nonexistent_var_in_mock | upper")
assert ok
def test_render_skips_other_errors():
"""Non-filter errors from missing mocks are skipped."""
ok, _ = _render_expression("some_undefined.attr.method()")
assert ok
def test_extract_skips_backtick():
"""Backtick fragments are skipped (caught by single-char check)."""
content = 'value: "{{ ` }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_network_settings():
"""Expressions with .NetworkSettings. patterns are skipped."""
content = 'value: "{{ foo.NetworkSettings.IPAddress }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_unbalanced_parens():
"""Expressions with unbalanced parens are skipped."""
content = 'value: "{{ foo(bar }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_unbalanced_braces():
"""Expressions with unbalanced braces are skipped."""
content = 'value: "{{ foo{bar }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_unbalanced_brackets():
"""Expressions with unbalanced brackets are skipped."""
content = 'value: "{{ foo[0 }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_control_flow():
"""Control flow fragments starting with % are skipped."""
content = 'value: "{{ % if x }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_check_file_outside_repo(tmp_path: Path) -> None:
"""Files outside REPO_ROOT are handled (no relative_to error)."""
test_file = tmp_path / "test.yml"
test_file.write_text("value: \"{{ (now().timestamp() | int + 3600) | strftime('%Y-%m-%dT%H:%M:%S+00:00') }}\"\n")
violations = _check_file(test_file, Path("/other/repo"))
assert len(violations) == 1
assert "reversed" in violations[0].lower()
def test_render_mock_dict_missing_key():
"""MockDict returns a mock for missing keys (no UndefinedError)."""
ok, _ = _render_expression("undefined_var.some_attr | upper")
assert ok
def test_render_syntax_error():
"""Syntax errors are reported as failures."""
ok, msg = _render_expression("{{ invalid syntax +")
assert not ok
assert "Syntax error" in msg
def test_render_unknown_filter_error():
"""Unknown filters are reported as failures (not skipped)."""
ok, msg = _render_expression("'test' | nonexistent_filter")
assert not ok
assert "filter" in msg.lower()
def test_render_generic_exception_skipped():
"""Non-filter exceptions from missing mocks are skipped."""
# replace() with no args triggers TypeError (missing required args)
# which is not a filter-not-found or strftime error — should be skipped.
ok, msg = _render_expression("my_var | replace")
assert ok
assert "Skipped" in msg
def test_default_ansible_dirs():
"""_default_ansible_dirs returns playbooks and roles paths."""
dirs = _default_ansible_dirs()
assert Path.cwd() / "ansible" / "playbooks" in dirs
assert Path.cwd() / "ansible" / "roles" in dirs
def test_main_default_dirs(tmp_path: Path) -> None:
"""Running with no --path scans default dirs (uses small temp fixture)."""
(tmp_path / "playbooks").mkdir()
(tmp_path / "roles").mkdir()
(tmp_path / "playbooks" / "test.yml").write_text("value: \"{{ my_var | default('x') }}\"\n")
with patch(
"devx.tools.check_jinja_expr._default_ansible_dirs",
return_value=[tmp_path / "playbooks", tmp_path / "roles"],
):
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
def test_main_custom_ansible_dirs(tmp_path: Path) -> None:
"""--ansible-dir option works."""
(tmp_path / "test.yml").write_text("value: \"{{ my_var | default('x') }}\"\n")
runner = CliRunner()
result = runner.invoke(main, ["--ansible-dir", str(tmp_path)])
assert result.exit_code == 0
def test_extract_skips_println():
"""Expressions with 'println' (Go template) are skipped."""
content = 'value: "{{ println something }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_extract_skips_state_dot():
"""Expressions with .State. patterns are skipped."""
content = 'value: "{{ foo.State.Running }}"'
expressions = _extract_expressions(content)
assert len(expressions) == 0
def test_render_now_with_format():
"""now() with a format argument works."""
ok, _ = _render_expression("now('%Y-%m-%d')")
assert ok
def test_find_yaml_files_skips_molecule(tmp_path: Path) -> None:
"""Molecule directories are excluded from file search."""
from devx.tools.check_jinja_expr import _find_yaml_files
(tmp_path / "tasks.yml").write_text("value: test\n")
(tmp_path / "molecule").mkdir()
(tmp_path / "molecule" / "test.yml").write_text("value: test\n")
files = _find_yaml_files(tmp_path)
assert all("molecule" not in f.parts for f in files)
+223
View File
@@ -0,0 +1,223 @@
"""Unit tests for devx.ci.validate_spec."""
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from devx.ci.validate_spec import (
AC_CHECKED_RE,
AC_UNCHECKED_RE,
REQ_ID_RE,
cli,
find_spec_file,
validate_spec_content,
)
VALID_SPEC = """\
# OBL-INFRA-531: Fix sso-bridge role for pip install
## Problem
The sso-bridge role uses scripts.sso_bridge.listener but the pip
package uses sso_bridge.listener.
## Approach
REQ-1: Update molecule verify.yml to use sso_bridge.listener
REQ-2: Add infra repo clone task to sso_bridge role
## Test Plan
- Run molecule test for sso_bridge role
- Verify pip package is installed correctly
## Deploy Plan
- Merge PR
- Auto-deploy to staging
## Rollback Plan
- Revert PR
- Re-deploy previous version
## Acceptance Criteria
- [x] Molecule test passes with sso_bridge.listener
- [x] Infra repo is cloned by sso_bridge role
"""
SPEC_MISSING_SECTION = """\
# OBL-INFRA-531: Fix sso-bridge
## Problem
Something is broken.
## Approach
REQ-1: Fix it
## Test Plan
Run tests
"""
SPEC_UNCHECKED_AC = """\
# OBL-INFRA-531: Fix sso-bridge
## Problem
Broken.
## Approach
REQ-1: Fix it
## Test Plan
Run tests
## Deploy Plan
Deploy
## Rollback Plan
Revert
## Acceptance Criteria
- [x] Fixed
- [ ] Verified in staging
"""
SPEC_NO_REQ_IDS = """\
# OBL-INFRA-531: Fix sso-bridge
## Problem
Broken.
## Approach
Fix it.
## Test Plan
Run tests
## Deploy Plan
Deploy
## Rollback Plan
Revert
## Acceptance Criteria
- [x] Fixed
"""
class TestFindSpecFile:
def test_finds_exact_match(self, tmp_path: Path) -> None:
specs_dir = tmp_path / "specs"
specs_dir.mkdir()
(specs_dir / "OBL-INFRA-531.md").write_text("content")
result = find_spec_file("OBL-INFRA-531", str(specs_dir))
assert result is not None
assert result.name == "OBL-INFRA-531.md"
def test_finds_case_insensitive(self, tmp_path: Path) -> None:
specs_dir = tmp_path / "specs"
specs_dir.mkdir()
(specs_dir / "obl-infra-531.md").write_text("content")
result = find_spec_file("OBL-INFRA-531", str(specs_dir))
assert result is not None
def test_returns_none_when_not_found(self, tmp_path: Path) -> None:
specs_dir = tmp_path / "specs"
specs_dir.mkdir()
result = find_spec_file("OBL-INFRA-999", str(specs_dir))
assert result is None
def test_returns_none_when_dir_missing(self, tmp_path: Path) -> None:
result = find_spec_file("OBL-INFRA-531", str(tmp_path / "nonexistent"))
assert result is None
class TestValidateSpecContent:
def test_valid_spec_passes(self) -> None:
errors = validate_spec_content(VALID_SPEC)
assert errors == []
def test_missing_sections(self) -> None:
errors = validate_spec_content(SPEC_MISSING_SECTION)
assert len(errors) >= 3 # Missing Deploy Plan, Rollback Plan, Acceptance Criteria
assert any("Deploy Plan" in e for e in errors)
assert any("Rollback Plan" in e for e in errors)
assert any("Acceptance Criteria" in e for e in errors)
def test_unchecked_ac_fails(self) -> None:
errors = validate_spec_content(SPEC_UNCHECKED_AC)
assert len(errors) == 1
assert "unchecked" in errors[0].lower()
def test_no_req_ids_fails(self) -> None:
errors = validate_spec_content(SPEC_NO_REQ_IDS)
assert any("REQ-ID" in e for e in errors)
def test_empty_content_fails(self) -> None:
errors = validate_spec_content("")
assert len(errors) >= 2 # Missing sections + no REQ-IDs
class TestRegexPatterns:
def test_req_id_re_matches(self) -> None:
assert REQ_ID_RE.search("REQ-1: Do something")
assert REQ_ID_RE.search("REQ-42: Another thing")
assert not REQ_ID_RE.search("REQ: no number")
def test_ac_checked_re_matches(self) -> None:
assert AC_CHECKED_RE.search("- [x] Done")
assert AC_CHECKED_RE.search(" - [x] Indented")
assert not AC_CHECKED_RE.search("- [ ] Not done")
def test_ac_unchecked_re_matches(self) -> None:
assert AC_UNCHECKED_RE.search("- [ ] Not done")
assert AC_UNCHECKED_RE.search(" - [ ] Indented")
assert not AC_UNCHECKED_RE.search("- [x] Done")
class TestCli:
def test_fails_without_task_id(self) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "no-task-id"])
assert result.exit_code != 0
def test_allow_missing_succeeds_without_task_id(self) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "no-task-id", "--allow-missing"])
assert result.exit_code == 0
assert "WARNING" in result.output
def test_fails_when_spec_not_found(self, tmp_path: Path) -> None:
runner = CliRunner()
with patch("devx.ci.validate_spec.extract_task_id", return_value="OBL-INFRA-999"):
result = runner.invoke(
cli,
["--branch", "OBL-INFRA-999-test", "--specs-dir", str(tmp_path / "specs")],
)
assert result.exit_code != 0
assert "No spec file found" in result.output
def test_passes_with_valid_spec(self, tmp_path: Path) -> None:
specs_dir = tmp_path / "specs"
specs_dir.mkdir()
(specs_dir / "OBL-INFRA-531.md").write_text(VALID_SPEC)
runner = CliRunner()
with patch("devx.ci.validate_spec.extract_task_id", return_value="OBL-INFRA-531"):
result = runner.invoke(
cli,
["--branch", "OBL-INFRA-531-fix-foo", "--specs-dir", str(specs_dir)],
)
assert result.exit_code == 0
assert "Spec validated" in result.output
def test_fails_with_unchecked_ac(self, tmp_path: Path) -> None:
specs_dir = tmp_path / "specs"
specs_dir.mkdir()
(specs_dir / "OBL-INFRA-531.md").write_text(SPEC_UNCHECKED_AC)
runner = CliRunner()
with patch("devx.ci.validate_spec.extract_task_id", return_value="OBL-INFRA-531"):
result = runner.invoke(
cli,
["--branch", "OBL-INFRA-531-fix-foo", "--specs-dir", str(specs_dir)],
)
assert result.exit_code != 0
assert "unchecked" in result.output.lower()