99 lines
3.1 KiB
Markdown
99 lines
3.1 KiB
Markdown
# testing-and-debugging
|
|
|
|
Make targets for testing, debugging, and CI investigation. **Use these
|
|
instead of raw `pytest`, `ruff`, or `molecule` commands.**
|
|
|
|
## Why Make Targets
|
|
|
|
Make targets encapsulate the correct venv activation, PYTHONPATH, env
|
|
vars, and flags. Running raw commands bypasses venv activation and
|
|
produces false failures (missing dependencies, wrong Python version).
|
|
|
|
## Unit Tests
|
|
|
|
| Task | Command | Notes |
|
|
|------|---------|-------|
|
|
| Run all unit tests | `make test-unit` | Fast, no coverage |
|
|
| Run with coverage | `make pytest-cov` | **Required before push** — enforces 100% |
|
|
| Run single test | `make pytest-cov TEST=tests/test_foo.py::test_bar` | |
|
|
|
|
## Linting
|
|
|
|
| Task | Command | Notes |
|
|
|------|---------|-------|
|
|
| Full lint | `make lint-all` | ruff + pyright + bandit + ansible-lint + checkmake + actionlint |
|
|
| Ruff only | `make lint-ruff` | |
|
|
| Type check | `make typecheck` | pyright |
|
|
| Bandit | `make lint-bandit` | Security linter |
|
|
| Workflow lint | `make workflow-check` | actionlint + act_runner dry-run |
|
|
|
|
## Molecule Tests
|
|
|
|
| Task | Command | Notes |
|
|
|------|---------|-------|
|
|
| All scenarios | `make molecule` | All 6 scenarios on Ubuntu 22.04 |
|
|
| All platforms | `make molecule-all` | All 6 scenarios on all 4 OSes |
|
|
| Parallel | `make molecule-all-parallel` | MOLECULE_JOBS=4 |
|
|
|
|
### Spec-Driven Workflow
|
|
|
|
Every PR requires a spec file at `docs/specs/<TASK-ID>.md`. See the
|
|
`spec-driven-development` skill for the full workflow and template.
|
|
CI validates the spec before running expensive jobs.
|
|
|
|
## Pre-Push Verification
|
|
|
|
**Before pushing any branch:**
|
|
|
|
```bash
|
|
make pre-push
|
|
```
|
|
|
|
This runs `lint-all` + `pytest-cov`. The pre-push git hook only
|
|
validates the Vikunja task exists — it does NOT run tests. You must
|
|
run `make pre-push` manually.
|
|
|
|
## CI Failure Investigation
|
|
|
|
When investigating a CI failure:
|
|
|
|
1. **Fetch logs via MCP** — use `mcp_call_tool` with gitea server,
|
|
`actions_run_read` method, `download_job_log` tool
|
|
2. **Reproduce locally** — use `make pytest-cov` or `make lint-ci`
|
|
depending on which CI job failed
|
|
3. **Never run raw pytest** — always use the make target
|
|
|
|
## Virtual Environment
|
|
|
|
All commands run inside `.venv`. `make` targets handle activation
|
|
automatically. For raw commands (rare), activate first:
|
|
|
|
```bash
|
|
source activate.sh # bash/zsh
|
|
source activate.fish # fish
|
|
source activate.zsh # zsh
|
|
```
|
|
|
|
If `.venv` doesn't exist, run `make setup` first.
|
|
|
|
## Common Pitfalls
|
|
|
|
### Coverage Verification Before Push
|
|
|
|
**Always run `make pytest-cov` before pushing** — CI enforces 100%
|
|
coverage and will fail the PR if any lines are uncovered. This is the
|
|
most common cause of CI validate job failures after code changes. The
|
|
pre-push git hook only validates Vikunja task existence, not tests.
|
|
|
|
### API Response Type Checking
|
|
|
|
Never use `is True`/`is False` identity checks on API response values.
|
|
Many APIs return boolean values as strings (`"true"`/`"false"`). Use
|
|
string comparison or truthy/falsy helpers instead.
|
|
|
|
### Time Mocking in Tests
|
|
|
|
Always mock `time.sleep` and `time.monotonic` in unit tests using
|
|
`@patch` decorators. Real sleep calls make tests slow and exceed test
|
|
speed limits.
|