241 lines
9.0 KiB
Markdown
241 lines
9.0 KiB
Markdown
# Development Setup
|
|
|
|
## Project Structure
|
|
|
|
```text
|
|
.
|
|
├── src/grm/ # Python CLI source
|
|
│ ├── cli.py # Click commands
|
|
│ ├── runner_manager.py # Ansible orchestration + registry integration
|
|
│ ├── executor.py # Ansible subprocess execution
|
|
│ ├── registry.py # Local JSON runner registry
|
|
│ ├── i18n.py # Translations (en, bg, de, ru, zh, pl)
|
|
│ ├── exceptions.py # Custom exceptions
|
|
│ ├── logging_config.py # Logging to ~/.local/state/grm/logs/
|
|
│ ├── report.py # Operation report tracking
|
|
│ ├── ui.py # Colorised console output
|
|
│ └── translations.json # Translation strings
|
|
├── ansible/
|
|
│ ├── roles/gitea-runner/ # Main Ansible role
|
|
│ │ ├── defaults/main.yml # Default variables
|
|
│ │ ├── tasks/ # Task files (13 files)
|
|
│ │ ├── templates/ # Jinja2 templates (4 files)
|
|
│ │ └── molecule/ # Test scenarios (7 scenarios)
|
|
│ ├── install-runner.yml # Install playbook
|
|
│ ├── update-runner.yml # Update playbook
|
|
│ ├── start-runner.yml # Start playbook
|
|
│ ├── stop-runner.yml # Stop playbook
|
|
│ ├── enable-runner.yml # Enable playbook
|
|
│ ├── disable-runner.yml # Disable playbook
|
|
│ ├── status-runner.yml # Status playbook
|
|
│ └── remove-runner.yml # Remove playbook
|
|
├── tests/
|
|
│ ├── unit/ # Unit tests
|
|
│ └── integration/ # Integration tests
|
|
├── .gitea/workflows/ # CI/CD workflows
|
|
├── docs/ # Documentation (synced to wiki)
|
|
├── Makefile # Build & test automation
|
|
├── pyproject.toml # Python project metadata
|
|
├── cliff.toml # git-cliff configuration
|
|
└── .env.example # Environment variable template
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- **Python 3.12+** — Required. The Makefile verifies this before creating the venv. Use `pyenv` to manage Python versions if needed.
|
|
- **Git** — For cloning the repository and checking out release tags.
|
|
- **Docker** — Only needed for running Molecule tests locally (`make molecule`).
|
|
- **Go** — Only needed if you want to install `checkmake` manually (alternatively, `make setup` installs it via `devx.tools.install_checkmake`).
|
|
|
|
## Setup Development Environment
|
|
|
|
### Step 1: Clone and checkout latest release
|
|
|
|
```bash
|
|
git clone https://git.oblachno.oblachno.fyi/oblachno-oss/grm.git
|
|
cd grm
|
|
git checkout $(git describe --tags --abbrev=0) # Checkout latest stable release
|
|
```
|
|
|
|
> **Important:** Always checkout the latest release tag before running `make setup`. The `master` branch may contain unreleased changes that are not yet stable. To see all available releases, run `git tag --sort=-version:refname` or check the [releases page](https://git.oblachno.oblachno.fyi/oblachno-oss/grm/releases).
|
|
|
|
### Step 2: Ensure Python 3.12+ is available
|
|
|
|
If you use pyenv:
|
|
|
|
```bash
|
|
pyenv install 3.12
|
|
pyenv local 3.12
|
|
```
|
|
|
|
Verify your Python version:
|
|
|
|
```bash
|
|
python3 --version # Must be 3.12 or higher
|
|
```
|
|
|
|
### Step 3: Run make setup
|
|
|
|
```bash
|
|
make setup
|
|
source .venv/bin/activate
|
|
```
|
|
|
|
The `make setup` target performs the following:
|
|
|
|
1. Verifies Python 3.12+ is installed
|
|
2. Creates a virtualenv in `.venv`
|
|
3. Installs/updates `pip`, `setuptools`, and `wheel`
|
|
4. Creates `.env` from `.env.example` if not present
|
|
5. Generates shell activation scripts (`activate.sh`, `activate.fish`, `activate.zsh`)
|
|
6. Installs the `devx` package from the Oblachno PyPI registry (provides CI/CD tools)
|
|
7. Installs `checkmake` via `devx.tools.install_checkmake` (Makefile linter)
|
|
8. Installs CI/CD tools via `devx.tools.install_tools` (actionlint, git-cliff, act_runner, tea) to `~/.local/bin`
|
|
9. Runs `python -m devx.tools.setup` to install Python dependencies, Ansible Galaxy collections, pre-commit hooks, and configure tea CLI login
|
|
|
|
### Step 4: Configure Gitea credentials
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env:
|
|
# GITEA_URL=https://git.example.com
|
|
# GITEA_REGISTRATION_TOKEN=your-registration-token
|
|
```
|
|
|
|
`GITEA_REGISTRATION_TOKEN` is the runner registration token obtained from your Gitea instance (Admin → Actions → Runners → Create Registration Token).
|
|
|
|
#### Admin API token (optional)
|
|
|
|
Set `CI_GITEA_TOKEN` to enable informational API checks during integration test. This is **optional** — the test primarily verifies the runner by checking:
|
|
|
|
1. **`.runner` registration file** exists and contains valid JSON (proves successful registration)
|
|
2. **Systemd user service** is active (proves daemon is polling for jobs)
|
|
|
|
API checks, if enabled, are purely informational and do not affect pass/fail.
|
|
|
|
### Step 5: Verify the setup
|
|
|
|
```bash
|
|
grm --version # Should print the version
|
|
make lint-all # Should pass with no errors
|
|
make pytest-cov # Should pass with 100% coverage
|
|
```
|
|
|
|
## Shell activation scripts
|
|
|
|
`make setup` generates convenience activation scripts for different shells:
|
|
|
|
```bash
|
|
source activate.sh # bash
|
|
source activate.fish # fish
|
|
source activate.zsh # zsh
|
|
```
|
|
|
|
These scripts activate the `.venv` virtualenv from the project root.
|
|
|
|
## Running Linters
|
|
|
|
```bash
|
|
make lint # Python (ruff + format check + pyright + bandit)
|
|
make lint-bandit # Security scan only
|
|
make ansible-lint # Ansible
|
|
make makefile-lint # Makefile
|
|
make workflow-lint # Gitea Actions workflows (actionlint)
|
|
```
|
|
|
|
The full lint target (`make lint-all`) runs all of the above:
|
|
|
|
```bash
|
|
make lint-all # ruff + pyright + bandit + ansible-lint + checkmake + actionlint
|
|
```
|
|
|
|
Individual lint targets from the `Makefile`:
|
|
|
|
| Target | Description |
|
|
|--------|-------------|
|
|
| `lint-ruff` | `ruff check src/ tests/` |
|
|
| `lint-format` | `ruff format --check src/ tests/` |
|
|
| `typecheck` | `pyright` |
|
|
| `lint-bandit` | `bandit -r src/` |
|
|
| `lint-deps` | `pip-audit` — checks dependencies for known vulnerabilities |
|
|
| `ansible-lint` | `ansible-lint ansible/` |
|
|
| `makefile-lint` | `checkmake Makefile` |
|
|
| `lint` | ruff + format check + pyright + bandit |
|
|
| `lint-all` | lint + ansible-lint + makefile-lint + workflow-lint |
|
|
|
|
## Running Tests
|
|
|
|
### Unit tests
|
|
|
|
```bash
|
|
make test-unit # Without coverage
|
|
make pytest-cov # With 100% coverage enforcement
|
|
```
|
|
|
|
The coverage requirement is `--cov-fail-under=100` — 100% test coverage is required for all code in `src/grm/`.
|
|
|
|
### Integration tests
|
|
|
|
```bash
|
|
make test-integration
|
|
```
|
|
|
|
Tests the full CLI lifecycle commands end-to-end (mocked executor boundary).
|
|
|
|
### Molecule tests
|
|
|
|
```bash
|
|
make molecule # Quick: all 6 scenarios on Ubuntu 22.04
|
|
make molecule-all # Full: all 6 scenarios on all 4 supported OSes
|
|
```
|
|
|
|
Requires Docker to be installed and running on your machine. Molecule creates Docker containers as test hosts, applies the Ansible role, and verifies the results.
|
|
|
|
### Full test suite
|
|
|
|
```bash
|
|
make test-all # pytest-cov + molecule
|
|
```
|
|
|
|
## Workflow verification
|
|
|
|
GRM includes Gitea Actions workflow files in `.gitea/workflows/`. These are verified with two tools:
|
|
|
|
```bash
|
|
make workflow-lint # Static lint via actionlint
|
|
make workflow-dryrun # Dry-run via act_runner exec --dryrun
|
|
make workflow-check # Both of the above
|
|
```
|
|
|
|
The pre-commit hook runs actionlint automatically when workflow files change.
|
|
|
|
## Pre-commit hooks
|
|
|
|
`make setup` installs pre-commit hooks that run:
|
|
|
|
- **pre-commit**: `ruff check`, `ruff format --check`, conventional commit message validation
|
|
- **pre-push**: `make pytest-cov` (ensures tests pass before pushing)
|
|
|
|
## Make targets reference
|
|
|
|
| Target | Description |
|
|
|--------|-------------|
|
|
| `make setup` | Full setup: venv, deps, hooks, CI tools |
|
|
| `make setup-ci` | Lean setup for CI jobs (pytest + lint, no Ansible collections) |
|
|
| `make setup-quality` | Setup for the validate CI job (lint + test deps) |
|
|
| `make setup-molecule` | Full setup for molecule testing |
|
|
| `make setup-release` | Setup for release jobs (git-cliff, tea, lint tools) |
|
|
| `make install-tools` | Install actionlint, git-cliff, act_runner, tea to `~/.local/bin` |
|
|
| `make install-devx` | Install the devx package from the Oblachno PyPI registry |
|
|
| `make lint-all` | ruff + pyright + bandit + ansible-lint + checkmake + actionlint |
|
|
| `make pytest-cov` | Unit tests with 100% coverage enforcement |
|
|
| `make test-unit` | Unit tests without coverage |
|
|
| `make test-integration` | Integration tests |
|
|
| `make molecule` | All 6 Molecule scenarios on Ubuntu 22.04 |
|
|
| `make molecule-all` | All 6 scenarios on all 4 supported OSes |
|
|
| `make test-all` | pytest-cov + molecule |
|
|
| `make workflow-lint` | Static lint of workflow YAML (actionlint) |
|
|
| `make workflow-dryrun` | Dry-run all workflows in Docker |
|
|
| `make workflow-check` | workflow-lint + workflow-dryrun |
|
|
| `make clean` | Remove `__pycache__`, `.pyc`, `.coverage`, `htmlcov/`, `.molecule/` |
|