# AGENTS.md — Project Conventions for devx

## Build & Test Commands

```bash
make setup           # Create venv, install deps, set up hooks, install CI tools
make install-tools   # Install actionlint, git-cliff, act_runner to ~/.local/bin
make lint-all        # ruff + pyright + bandit + actionlint
make pytest-cov      # Unit tests with 100% coverage enforcement
make test-unit       # Unit tests without coverage
make workflow-lint   # Static lint of .gitea/workflows/*.yml (actionlint)
make workflow-dryrun # Dry-run all workflows in Docker (act_runner exec --dryrun)
make workflow-check  # workflow-lint + workflow-dryrun
make clean           # Remove caches, build artifacts, coverage data
```

`make setup` automatically installs all development tools:
- **Python deps** via `python -m devx.tools.setup` (pip install -e .[dev], pre-commit hooks)
- **actionlint, git-cliff, act_runner, tea** via `python -m devx.tools.install_tools` (CI/CD tools to ~/.local/bin)
- **tea CLI login** via `python -m devx.tools.setup` (configures `tea login` from `.env` `REPO_TOKEN`)

## Workflow Verification (Before Push)

Workflow YAML files (`.gitea/workflows/*.yml`) are verified with two tools:

1. **actionlint** — Static linter that catches syntax errors, invalid
   expressions, unknown keys, type mismatches, and shellcheck issues.
   Config: `.gitea/actionlint.yaml` (registers custom `docker` runner label).
   Installed automatically by `make setup` via `python -m devx.tools.install_tools`.

2. **act_runner exec --dryrun** — Gitea's own runner in dry-run mode.
   Validates job dependencies, step ordering, and Docker image selection
   without starting containers. Installed automatically by `make setup`.

Both run via `make workflow-check` and are part of `make lint-all`.
The pre-commit hook runs actionlint automatically when workflow files change.
The CI `quality` job runs `make setup-quality` then `make lint-all`.
CI also runs a best-effort `make workflow-dryrun` step (skipped if act_runner is not installed in the CI Docker image).

## Architecture

devx is a reusable Python package providing development and CI/CD tools for oblachno-oss projects.

### Package Structure

```
src/devx/
├── __init__.py          # Version (single source of truth, read by setuptools)
├── cli.py               # Click-based CLI entry point (devx command)
├── config.py            # Configuration system (DEVX_ env vars, .env loading)
├── api_clients.py       # GiteaClient, VikunjaClient — HTTP API wrappers
├── gitea_cli.py         # TeaCLI — wrapper around tea CLI with JSON parsing
├── i18n.py              # Translation system (gettext-based, translations.json)
├── exceptions.py        # Custom exception types
├── translations.json    # Translation strings (en, bg)
├── ci/                  # CI/CD automation modules (run by workflows)
│   ├── release.py       # Automated versioning, tagging, changelog
│   ├── publish.py       # Build and publish to Gitea PyPI registry (--skip-build for non-Python repos)
│   ├── auto_merge.py    # Squash-merge PRs with task ID validation
│   ├── _shared.py      # Shared utilities (get_latest_tag)
│   ├── classify_changes.py  # User-facing vs workflow-only change detection
│   ├── detect_release_commit.py  # Detect release commits on master
│   ├── validate_commit_msg.py    # Conventional commit validation
│   ├── pr_review.py     # Automated PR review
│   ├── post_merge.py    # Vikunja task updates after merge
│   ├── sync_wiki.py     # Sync documentation to Gitea wiki
│   ├── push_badges.py   # Generate and push quality badges (--retries for retry on git push failures)
│   ├── notify_failure.py    # Create Gitea issues on CI failures (--auto-login)
│   ├── distribute_files.py  # Distribute files across parallel runners (LPT scheduling)
│   ├── integration_guard.py # Run pytest with cross-runner fail-fast
│   ├── check_translations.py  # Translation completeness check
│   └── doc_coverage.py  # Documentation coverage check
├── tools/               # Developer tooling modules (run locally or by CI)
│   ├── setup.py         # Environment setup (venv, deps, hooks)
│   ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea
│   ├── check_test_speed.py  # Measure unit test execution time
│   ├── configure_repo.py    # Branch protection and label setup
│   └── generate_badges.py   # Badge SVG generation
├── opentofu.py          # OpenTofu output helpers (get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field)
└── molecule/            # Optional molecule testing helpers (for Ansible projects)
    ├── discover_runners.py    # Dynamic Gitea runner discovery
    ├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
    ├── molecule_ci_guard.py   # Run molecule with cross-runner fail-fast (--roles-root)
    ├── molecule_all.py        # Run all molecule scenarios locally
    └── platforms.py           # Supported molecule platforms
```

### Key Design Principles

- **Self-contained package** — `src/devx/` never imports from scripts outside the package
- **Module-based invocation** — All tools invoked via `python -m devx.ci.*` or `python -m devx.tools.*`
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (NOT `.:src` since there are no scripts at repo root)
- **Config via env vars** — `DEVX_*` environment variables with `.env` file fallback

## PR Workflow (Mandatory)

Every change to master goes through this workflow. No exceptions.

### Branch Protection (Required Gitea Settings)

Branch protection and labels are automatically configured by
`python -m devx.tools.configure_repo`, which runs as a `configure-repo` job in
the post-merge workflow on every push to master.

The following rules are enforced for `master`:
- **Require pull request**: No direct pushes to master
- **Require approval review**: At least 1 `APPROVE` review before merge
- **Require status checks**: CI quality must pass
- **Block force pushes**: No history rewriting on master

### 1. Create Vikunja Task
Create a task in Vikunja to get a `DEVX-N` identifier.

### 2. Create Branch
```bash
git checkout master && git pull
git checkout -b DEVX-N-short-description
```

### 3. Implement Changes
- Write code following conventions below
- Write/update tests (100% coverage required)
- Update documentation (CHANGELOG, README, AGENTS.md as needed)

### 4. Commit (Conventional Commits)
Branch commits use conventional commit format (no `DEVX-N:` prefix):
```
feat: add new feature
fix: resolve bug
docs: update README
```

### 5. Push and Create PR
- **PR title format**: `DEVX-N: <vikunja task title>` (must match the Vikunja task title exactly)
- PR body: summary of changes, `Closes DEVX-N`
- Add `ready-to-merge` label **only after review is complete**

### 6. Review the PR

**Automated review (CI `pr-review` job):** Every PR triggers an automated
review via `python -m devx.ci.pr_review`. This job posts a review with
`COMMENT` (no issues) or `REQUEST_CHANGES` (issues found):

- Architecture compliance (no subprocess in CLI, no hardcoded URLs)
- Best practices (no `print()`, no bare `except`, no `TODO`/`FIXME`,
  no functions > 50 lines)
- Security (no hardcoded secrets, no `shell=True`, no `eval`/`exec`)
- i18n (no raw strings in `click.echo()` without `_()` wrapper)
- Resource management (no `open()` without `with`, no `Popen()` without cleanup)
- Documentation (source changes must include doc updates)
- Test coverage (source changes must include test updates)
- Commit conventions (conventional commit format on PR commits)

The agent **must** address all `REQUEST_CHANGES` issues before proceeding.

### 7. Address Review Comments
Fix each comment one by one, commit, and push. Re-review until satisfied.

### 8. Approve and Merge
Once all checklist items are verified and comments are addressed, approve
the PR. Then add the `ready-to-merge` label. The auto-merge workflow will:
1. **Validate** PR title format (`DEVX-N: <vikunja task title>`) and match against Vikunja task title
2. **Check** that at least one substantive APPROVE review exists
3. Wait for all CI checks to pass (including the `pr-review` job)
4. Squash-merge with title: `DEVX-N: <conventional commit message>`
5. The post-merge workflow marks the Vikunja task as done
6. The release workflow automatically versions, tags, and publishes

> **IMPORTANT**: Never manually merge PRs via the API. Always use the auto-merge
> workflow by adding the `ready-to-merge` label.

### Automated Release Pipeline

After a PR is merged to master, the **post-merge workflow**
(`.gitea/workflows/post-merge.yml`) runs automatically:

1. **detect-type** — Checks if the commit is a regular merge or a
   release commit (`release: vX.Y.Z`). All subsequent jobs skip for
   release commits (except badges).

2. **release** — Runs `python -m devx.ci.release` which:
   - Checks for user-facing changes via `python -m devx.ci.classify_changes`
   - Uses **git-cliff** to calculate the next semver version from conventional commits
   - Updates `__version__` in `src/devx/__init__.py` (single source of truth)
   - Updates `CHANGELOG.md` with the new version section
   - Runs `make lint-ruff` and `make pytest-cov` to verify the release is healthy
   - Commits with `release: vX.Y.Z [skip ci]` prefix
   - Creates an annotated tag `vX.Y.Z` on the release commit
   - Pushes both the commit and tag to master

3. **sync-wiki** — Syncs documentation to the Gitea wiki. Runs for ALL
   non-release commits (not just when release succeeds), so docs-only
   changes still update the wiki.

4. **badges** — Generates and pushes quality badge SVGs to the `badges` branch.
   Uses `if: always()` so it runs on every push, including release commits.

5. **vikunja** — Marks the corresponding Vikunja task as done. Runs for ALL
   non-release commits (not just when release succeeds), so infrastructure-only
   changes still update the task tracker.

The tag push triggers the **publish workflow** (`.gitea/workflows/publish.yml`)
which builds and publishes the package to the Gitea PyPI registry.

### Smart CI: User-Facing vs Workflow-Only Changes

Not all changes require a new release. The project classifies changes
using `python -m devx.ci.classify_changes`:

**Workflow-only paths** (infrastructure → no release needed):
- `.gitea/**` — Gitea Actions workflows
- `tests/**` — Test files
- `AGENTS.md`, `README.md`, `CHANGELOG.md` — Project docs
- `Makefile`, `cliff.toml`, `.pre-commit-config.yaml` — Config
- `.env.example`, `.gitignore` — Config
- `hooks/**` — Git hooks

**User-facing paths** (tool changes → release needed) — everything else:
- `src/devx/**` — Python package source (except `__init__.py`)
- `pyproject.toml` — Package metadata
- Any new file type not in the allowlist

## Script Separation and Import Rules

### Directory Layout

| Directory | Purpose | Release impact |
|-----------|---------|----------------|
| `src/devx/` | User-facing devx package | Changes trigger release |
| `src/devx/ci/` | CI/CD automation (run by workflows) | Part of package |
| `src/devx/tools/` | Developer tooling (run locally or by CI) | Part of package |
| `tests/` | Test files | Workflow-only (no release) |

### Import Rules

1. **`src/devx/` is self-contained** — the package never imports from outside `src/`
2. **CI modules** (`devx.ci.*`) may import from `devx.api_clients`, `devx.config`, `devx.gitea_cli`, `devx.i18n`
3. **Tool modules** (`devx.tools.*`) may import from `devx.api_clients`, `devx.config`, `devx.gitea_cli`
4. **Cross-module imports** within `devx.ci.*` or `devx.tools.*` are allowed but must be documented

### PYTHONPATH Configuration

All workflows use `PYTHONPATH: src` — devx has no scripts at the repo root,
so `.:src` is not needed. The `src` directory is the sole import root.

```yaml
- name: Run script
  env:
    PYTHONPATH: src
  run: python -m devx.ci.example
```

### tea CLI Integration

The `tea` Gitea CLI tool is used for Gitea API interactions. It is installed
by `python -m devx.tools.install_tools` and configured by
`python -m devx.tools.setup` (login profile from `.env` `REPO_TOKEN`).

**`devx.gitea_cli.TeaCLI`** — Python wrapper around `tea` CLI with JSON output parsing:
- `create_issue()` — Create issues with labels
- `list_labels()` / `create_label()` / `add_label()` — Label management
- `create_pr()` / `merge_pr()` / `review_pr()` — Pull request operations
- `create_release()` / `list_releases()` — Release management

### git-cliff Commit Preprocessing

Merge commits on master have the format `DEVX-N: <conventional commit>`. The
`cliff.toml` includes a `commit_preprocessors` entry that strips the `DEVX-N `
prefix before parsing. This ensures all merged work appears in the changelog.

### Version Bumping Rules (git-cliff)

| Commit type | Version bump |
|-------------|-------------|
| `feat:` | minor (0.X.0) |
| `fix:` | patch (0.0.X) |
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0) |
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |

The version source is `__version__` in `src/devx/__init__.py`, read by
setuptools via `dynamic = ["version"]` in `pyproject.toml`.

### Title Format Summary

| What | Format | Example |
|------|--------|---------|
| Branch name | `DEVX-N-short-description` | `DEVX-12-add-release-script` |
| Branch commits | `<conventional commit>` | `feat: add release script` |
| PR title | `DEVX-N: <vikunja task title>` | `DEVX-12: Add release automation` |
| Merge commit | `DEVX-N: <conventional commit>` | `DEVX-12: feat: add release script` |

### Task ID Resolution

`auto_merge` resolves the task ID solely from the branch name (e.g.
`DEVX-12-fix-foo` → `DEVX-12`). Branch names must include the task ID
prefix — there is no `.taskid` file fallback. If a stale `.taskid` file
exists in the repo, a deprecation warning is printed advising its removal.

### Workflow `auto-merge` Job and `always()`

When `auto-merge` depends on a job that can be skipped (e.g.
`molecule-tests`), the `if:` condition MUST include `always() &&`
at the start. Without it, Gitea Actions skips `auto-merge` when any
dependency is skipped, even if the condition explicitly allows
`result == 'skipped'`.

```yaml
auto-merge:
  needs: [quality, detect-changes, pr-review, molecule-tests]
  if: >-
    always() &&
    github.event_name == 'pull_request' &&
    needs.quality.result == 'success' &&
    needs.pr-review.result == 'success' &&
    (needs.molecule-tests.result == 'success' || needs.molecule-tests.result == 'skipped')
```

### LPT Test Distribution Algorithm

`distribute_molecule` and `distribute_files` use **LPT (Longest Processing
Time first)** scheduling instead of naive round-robin. This produces a more
balanced distribution when test items have varying costs:

1. **Weight estimation**: Each item is assigned a weight:
   - Molecule scenarios: heuristic by name (`nextcloud`=10, `gitea`=8,
     `binary`=2, default=3). See `_SCENARIO_WEIGHTS` in
     `distribute_molecule.py`.
   - Integration test files: weight by file size in bytes (as a proxy
     for test runtime).
2. **LPT assignment**: Items are sorted by weight (descending), then
   each is assigned to the runner with the least total weight.

This ensures heavy scenarios (e.g. `nextcloud`) are spread across
different runners rather than clustered on one, reducing the
longest-runner time from ~16 min to ~11 min with 6 runners.

## Config System

devx uses environment variables with `.env` file fallback for configuration.

### DEVX_ Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_REPO_OWNER` | **(none — must be set)** | Repository owner for API calls |
| `DEVX_REPO_NAME` | **(none — must be set)** | Repository name (or `owner/repo`) |
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
| `DEVX_VIKUNJA_PROJECT_ID` | `6` | Vikunja project ID |
| `DEVX_LANG` | `en` | Language for i18n (en, bg) |
| `REPO_TOKEN` | (from .env) | Gitea API token |
| `VIKUNJA_TOKEN` | (from .env) | Vikunja API token |

### Per-Project Overrides

Projects using devx can override the default API URLs and language by setting
`DEVX_*` environment variables or entries in their `.env` file. The config
system loads `.env` automatically via `python-dotenv`.

## Key Conventions

- Python 3.12+ required (ruff/pyright target `py312`)
- 100% test coverage required (`--cov-fail-under=100`)
- Conventional commits on feature branches (no `DEVX-N:` prefix)
- Branch names must include `DEVX-N` task ID
- Line length: 120 chars
- Secrets are passed via environment variables, never on the command line
- All user-facing strings wrapped in `_()` for i18n