Public Access
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 devx-ci-bot
|
a7f5f47564 | ||
 emil
|
d623a64344 | ||
|
gitea-actions-bot
|
268a4e7988 | ||
|
emil
|
7daaf9e4a9 | ||
|
gitea-actions-bot
|
b7c9334881 |
@@ -0,0 +1,98 @@
|
||||
# testing-and-debugging
|
||||
|
||||
Make targets for testing, debugging, and CI investigation. **Use these
|
||||
instead of raw `pytest`, `ruff`, or `actionlint` commands.**
|
||||
|
||||
## Why Make Targets
|
||||
|
||||
Make targets encapsulate the correct venv activation, PYTHONPATH, env
|
||||
vars, and flags. Running raw commands bypasses venv activation and
|
||||
produces false failures (missing dependencies, wrong Python version).
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| Task | Command | Notes |
|
||||
|------|---------|-------|
|
||||
| Run all unit tests | `make test-unit` | Fast, no coverage |
|
||||
| Run with coverage | `make pytest-cov` | **Required before push** — enforces 100% |
|
||||
| Run single test | `make pytest-cov TEST=tests/test_foo.py::test_bar` | |
|
||||
| Check test speed | `make check-test-speed` | Fails if tests > 10s total or > 0.5s each |
|
||||
| Check test coverage | `make check-test-coverage` | Fails if source changed but tests didn't |
|
||||
|
||||
## Linting
|
||||
|
||||
| Task | Command | Notes |
|
||||
|------|---------|-------|
|
||||
| Full lint | `make lint-all` | ruff + workflow-lint + lint-dockerfiles |
|
||||
| Ruff only | `make lint-ruff` | |
|
||||
| Format check | `make lint-format` | |
|
||||
| Type check | `make typecheck` | pyright |
|
||||
| Bandit | `make lint-bandit` | Security linter |
|
||||
| Workflow lint | `make workflow-check` | actionlint + act_runner dry-run |
|
||||
| Dockerfile lint | `make lint-dockerfiles` | hadolint on all Dockerfiles |
|
||||
| Check mutable globals | `make check-mutable-globals` | Detects module-level mutable state |
|
||||
| Check dep docs | `make check-dep-docs` | Verifies pyproject.toml deps have comments |
|
||||
|
||||
## Pre-Push Verification
|
||||
|
||||
**Before pushing any branch:**
|
||||
|
||||
```bash
|
||||
make pre-push
|
||||
```
|
||||
|
||||
This runs `lint-all` + `pytest-cov`. The pre-push git hook only
|
||||
validates the Vikunja task exists — it does NOT run tests. You must
|
||||
run `make pre-push` manually.
|
||||
|
||||
## CI Failure Investigation
|
||||
|
||||
When investigating a CI failure:
|
||||
|
||||
1. **Fetch logs via MCP** — use `mcp_call_tool` with gitea server,
|
||||
`actions_run_read` method, `download_job_log` tool
|
||||
2. **Reproduce locally** — use `make pytest-cov` or `make lint-all`
|
||||
depending on which CI job failed
|
||||
3. **Never run raw pytest** — always use the make target
|
||||
|
||||
## Virtual Environment
|
||||
|
||||
All commands run inside `.venv`. `make` targets handle activation
|
||||
automatically. For raw commands (rare), activate first:
|
||||
|
||||
```bash
|
||||
source activate.sh # bash/zsh
|
||||
source activate.fish # fish
|
||||
source activate.zsh # zsh
|
||||
```
|
||||
|
||||
If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Coverage Verification Before Push
|
||||
|
||||
**Always run `make pytest-cov` before pushing** — CI enforces 100%
|
||||
coverage and will fail the PR if any lines are uncovered. This is the
|
||||
most common cause of CI quality job failures after code changes. The
|
||||
pre-push git hook only validates Vikunja task existence, not tests.
|
||||
|
||||
### API Response Type Checking
|
||||
|
||||
Never use `is True`/`is False` identity checks on API response values.
|
||||
Many APIs return boolean values as strings (`"true"`/`"false"`). Use
|
||||
the `is_truthy()`/`is_falsy()` helpers from `devx.utils.api` or compare
|
||||
against string values.
|
||||
|
||||
### Time Mocking in Tests
|
||||
|
||||
Always mock `time.sleep` and `time.monotonic` in unit tests using
|
||||
`@patch` decorators. Real sleep calls make tests slow and exceed test
|
||||
speed limits (10s total, 0.5s per test).
|
||||
|
||||
### Mutable Global State
|
||||
|
||||
The `check-mutable-globals` tool detects module-level mutable state
|
||||
(lists, dicts, sets) that can cause test pollution. Avoid module-level
|
||||
mutable defaults — use factory functions or `None` with initialization
|
||||
inside functions.
|
||||
@@ -169,7 +169,10 @@ jobs:
|
||||
if: needs.detect-type.outputs.is-release == 'false'
|
||||
runs-on: docker
|
||||
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
|
||||
timeout-minutes: 10
|
||||
timeout-minutes: 15
|
||||
concurrency:
|
||||
group: sync-wiki-${{ github.repository }}
|
||||
cancel-in-progress: false
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [0.33.3] - 2026-07-06
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Make wiki sync resilient to API timeouts and stale page lists
|
||||
|
||||
## [0.33.2] - 2026-07-05
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
@@ -16,12 +16,12 @@ quality badges.
|
||||
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
|
||||
## Why devx?
|
||||
|
||||
|
||||
+6
-6
@@ -12,12 +12,12 @@ project to be reusable across all oblachno-oss repositories.
|
||||
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
|
||||
## Overview
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""devx — reusable development and CI/CD tools for oblachno-oss projects."""
|
||||
|
||||
__version__ = "0.33.2"
|
||||
__version__ = "0.33.3"
|
||||
|
||||
+44
-17
@@ -122,6 +122,9 @@ def sync_page(
|
||||
"""Create or update a single wiki page.
|
||||
|
||||
Returns "created", "updated", or "skipped" (if dry-run).
|
||||
|
||||
If a create fails with HTTP 400 "already exists" (the page list was
|
||||
stale), re-lists the wiki and falls back to an update.
|
||||
"""
|
||||
if dry_run:
|
||||
click.echo(_("[dry-run] Would sync page: {title} ({chars} chars)", title=page_title, chars=len(content)))
|
||||
@@ -144,16 +147,36 @@ def sync_page(
|
||||
return "updated"
|
||||
|
||||
# Create new page via POST /wiki/new
|
||||
client._request(
|
||||
"POST",
|
||||
"/wiki/new",
|
||||
json={
|
||||
"title": page_title,
|
||||
"content_base64": content_b64,
|
||||
"message": f"Sync from docs/ — create {page_title}",
|
||||
},
|
||||
)
|
||||
return "created"
|
||||
try:
|
||||
client._request(
|
||||
"POST",
|
||||
"/wiki/new",
|
||||
json={
|
||||
"title": page_title,
|
||||
"content_base64": content_b64,
|
||||
"message": f"Sync from docs/ — create {page_title}",
|
||||
},
|
||||
)
|
||||
return "created"
|
||||
except APIError as e:
|
||||
if e.status == 400 and "already exists" in e.message.lower():
|
||||
# The page list was stale (e.g. after a timeout-retry returned
|
||||
# incomplete data). Re-list and fall back to update.
|
||||
click.echo(_(" Page '{title}' already exists (stale list). Re-listing and updating...", title=page_title))
|
||||
fresh_pages = _list_wiki_pages_with_retry(client)
|
||||
if page_title in fresh_pages:
|
||||
sub_url = fresh_pages[page_title]
|
||||
client._request(
|
||||
"PATCH",
|
||||
f"/wiki/page/{sub_url}",
|
||||
json={
|
||||
"title": page_title,
|
||||
"content_base64": content_b64,
|
||||
"message": f"Sync from docs/ — update {page_title} (create→update fallback)",
|
||||
},
|
||||
)
|
||||
return "updated"
|
||||
raise
|
||||
|
||||
|
||||
def verify_wiki_page(
|
||||
@@ -173,15 +196,15 @@ def verify_wiki_page(
|
||||
def _list_wiki_pages_with_retry(client: GiteaClient) -> dict[str, str]:
|
||||
"""List wiki pages with tenacity retry on APIError.
|
||||
|
||||
The Gitea API can be briefly unavailable right after a batch of wiki
|
||||
page updates. Uses the same tenacity pattern as ``api_clients`` for
|
||||
exponential backoff.
|
||||
The Gitea wiki API can be slow (it renders pages on each request)
|
||||
and may time out. Uses 5 attempts with exponential backoff to handle
|
||||
transient slowness.
|
||||
"""
|
||||
_logger = logging.getLogger("sync_wiki")
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(3),
|
||||
wait=wait_exponential(multiplier=2, min=2, max=8),
|
||||
stop=stop_after_attempt(5),
|
||||
wait=wait_exponential(multiplier=2, min=2, max=16),
|
||||
retry=retry_if_exception_type(APIError),
|
||||
before_sleep=before_sleep_log(_logger, logging.WARNING),
|
||||
reraise=True,
|
||||
@@ -291,10 +314,14 @@ def main(dry_run: bool, repo: str | None, verify: bool, strict: bool) -> None:
|
||||
click.echo(_("Syncing {count} documentation pages to wiki...", count=len(mapping)))
|
||||
|
||||
try:
|
||||
existing_pages = list_wiki_pages(client)
|
||||
existing_pages = _list_wiki_pages_with_retry(client)
|
||||
except APIError as e:
|
||||
raise click.ClickException(
|
||||
_("Failed to list existing wiki pages: {error}. Aborting to avoid creating duplicate pages.", error=e)
|
||||
_(
|
||||
"Failed to list existing wiki pages after retries: {error}. "
|
||||
"Aborting to avoid creating duplicate pages.",
|
||||
error=e,
|
||||
)
|
||||
) from e
|
||||
if existing_pages:
|
||||
click.echo(_("Found {count} existing wiki pages.", count=len(existing_pages)))
|
||||
|
||||
@@ -3343,12 +3343,20 @@
|
||||
"ru": "Директория для сканирования (по умолчанию: tests/integration). Можно повторять.",
|
||||
"zh": "要扫描的目录（默认：tests/integration）。可重复。"
|
||||
},
|
||||
"Failed to list existing wiki pages: {error}. Aborting to avoid creating duplicate pages.": {
|
||||
"bg": "Неуспешно извличане на съществуващи wiki страници: {error}. Прекратяване, за да се избегне създаване на дублирани страници.",
|
||||
"de": "Abrufen bestehender Wiki-Seiten fehlgeschlagen: {error}. Abbruch, um doppelte Seiten zu vermeiden.",
|
||||
"en": "Failed to list existing wiki pages: {error}. Aborting to avoid creating duplicate pages.",
|
||||
"pl": "Nie udało się wylistować istniejących stron wiki: {error}. Przerywanie, aby uniknąć tworzenia zduplikowanych stron.",
|
||||
"ru": "Не удалось получить список существующих wiki-страниц: {error}. Прерывание, чтобы избежать создания дубликатов страниц.",
|
||||
"zh": "列出现有 wiki 页面失败：{error}。正在中止以避免创建重复页面。"
|
||||
"Failed to list existing wiki pages after retries: {error}. Aborting to avoid creating duplicate pages.": {
|
||||
"bg": "Неуспешно извличане на съществуващи wiki страници след повторни опити: {error}. Прекратяване, за да се избегне създаване на дублирани страници.",
|
||||
"de": "Abrufen bestehender Wiki-Seiten nach Wiederholungen fehlgeschlagen: {error}. Abbruch, um doppelte Seiten zu vermeiden.",
|
||||
"en": "Failed to list existing wiki pages after retries: {error}. Aborting to avoid creating duplicate pages.",
|
||||
"pl": "Nie udało się wylistować istniejących stron wiki po ponownych próbach: {error}. Przerywanie, aby uniknąć tworzenia zduplikowanych stron.",
|
||||
"ru": "Не удалось получить список существующих wiki-страниц после повторных попыток: {error}. Прерывание, чтобы избежать создания дубликатов страниц.",
|
||||
"zh": "重试后列出现有 wiki 页面失败：{error}。正在中止以避免创建重复页面。"
|
||||
},
|
||||
" Page '{title}' already exists (stale list). Re-listing and updating...": {
|
||||
"bg": " Страницата '{title}' вече съществува (остарял списък). Пресписване и обновяване...",
|
||||
"de": " Seite '{title}' existiert bereits (veraltete Liste). Neu auflisten und aktualisieren...",
|
||||
"en": " Page '{title}' already exists (stale list). Re-listing and updating...",
|
||||
"pl": " Strona '{title}' już istnieje (nieaktualna lista). Ponowne listowanie i aktualizacja...",
|
||||
"ru": " Страница '{title}' уже существует (устаревший список). Повторное получение списка и обновление...",
|
||||
"zh": " 页面 '{title}' 已存在（列表过期）。重新列出并更新..."
|
||||
}
|
||||
}
|
||||
|
||||
@@ -171,6 +171,35 @@ class TestSyncPage:
|
||||
assert "content" not in payload
|
||||
assert base64.b64decode(payload["content_base64"]).decode("utf-8") == "# Updated"
|
||||
|
||||
def test_create_falls_back_to_update_on_already_exists(self) -> None:
|
||||
"""When create fails with 400 'already exists', re-list and update."""
|
||||
client = MagicMock()
|
||||
# First call: POST /wiki/new → 400 already exists
|
||||
# Second call: PATCH /wiki/page/{sub_url} → success
|
||||
create_error = APIError(400, "wiki page already exists [title: Test-Page]")
|
||||
client._request.side_effect = [create_error, MagicMock()]
|
||||
with patch("devx.ci.sync_wiki._list_wiki_pages_with_retry", return_value={"Test-Page": "Test-Page.-"}):
|
||||
result = sync_page(client, "Test-Page", "# Content", {}, dry_run=False)
|
||||
assert result == "updated"
|
||||
# Verify PATCH was called (second call)
|
||||
patch_call = client._request.call_args_list[1]
|
||||
assert patch_call.args[0] == "PATCH"
|
||||
assert "/wiki/page/Test-Page.-" in patch_call.args[1]
|
||||
|
||||
def test_create_raises_non_400_error(self) -> None:
|
||||
"""Non-400 errors from create should propagate, not trigger fallback."""
|
||||
client = MagicMock()
|
||||
client._request.side_effect = APIError(500, "server error")
|
||||
with pytest.raises(APIError):
|
||||
sync_page(client, "Test-Page", "# Content", {}, dry_run=False)
|
||||
|
||||
def test_create_raises_400_not_already_exists(self) -> None:
|
||||
"""400 errors that don't mention 'already exists' should propagate."""
|
||||
client = MagicMock()
|
||||
client._request.side_effect = APIError(400, "invalid title")
|
||||
with pytest.raises(APIError):
|
||||
sync_page(client, "Test-Page", "# Content", {}, dry_run=False)
|
||||
|
||||
|
||||
class TestVerifyWikiPage:
|
||||
def test_verifies_matching_content(self) -> None:
|
||||
@@ -538,14 +567,14 @@ class TestMain:
|
||||
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.sync_wiki.GiteaClient")
|
||||
def test_initial_list_api_error_aborts(self, mock_client_cls: MagicMock) -> None:
|
||||
"""When the initial page list fails, sync aborts to avoid duplicate pages."""
|
||||
"""When the initial page list fails after retries, sync aborts to avoid duplicate pages."""
|
||||
mock_client = MagicMock()
|
||||
mock_client_cls.return_value = mock_client
|
||||
with patch("devx.ci.sync_wiki.MAPPING_FILE") as mock_mapping:
|
||||
mock_mapping.exists.return_value = True
|
||||
with patch("devx.ci.sync_wiki.load_mapping", return_value={"index.md": "Home"}):
|
||||
with patch("devx.ci.sync_wiki.read_doc_content", return_value="# Home"):
|
||||
with patch("devx.ci.sync_wiki.list_wiki_pages", side_effect=APIError(0, "timeout")):
|
||||
with patch("devx.ci.sync_wiki._list_wiki_pages_with_retry", side_effect=APIError(0, "timeout")):
|
||||
with patch("devx.ci.sync_wiki.sync_page", return_value="created"):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--repo", "owner/repo"])
|
||||
@@ -559,17 +588,15 @@ class TestMain:
|
||||
"""When --verify re-fetch fails after retries, verification is skipped gracefully."""
|
||||
mock_client = MagicMock()
|
||||
mock_client_cls.return_value = mock_client
|
||||
# Initial list succeeds, but verify re-fetch fails
|
||||
list_side_effect = [{"Home": "Home"}, APIError(0, "timeout")]
|
||||
with patch("devx.ci.sync_wiki.MAPPING_FILE") as mock_mapping:
|
||||
mock_mapping.exists.return_value = True
|
||||
with patch("devx.ci.sync_wiki.load_mapping", return_value={"index.md": "Home"}):
|
||||
with patch("devx.ci.sync_wiki.read_doc_content", return_value="# Home"):
|
||||
with patch("devx.ci.sync_wiki.list_wiki_pages", return_value={"Home": "Home"}):
|
||||
with patch("devx.ci.sync_wiki._list_wiki_pages_with_retry", side_effect=list_side_effect):
|
||||
with patch("devx.ci.sync_wiki.sync_page", return_value="updated"):
|
||||
with patch(
|
||||
"devx.ci.sync_wiki._list_wiki_pages_with_retry",
|
||||
side_effect=APIError(0, "timeout"),
|
||||
):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--repo", "owner/repo", "--verify"])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--repo", "owner/repo", "--verify"])
|
||||
assert result.exit_code == 0
|
||||
assert "Skipping content verification" in result.output
|
||||
|
||||
Reference in New Issue
Block a user