GRM-75: feat: thoroughly clean Docker artifacts on runner removal
## Summary Thoroughly cleans Docker artifacts on runner removal, updates devx to v0.9.11, fixes Makefile checkmake graceful skip, and comprehensive docs rewrite. Molecule tests fail due to pre-existing Docker infrastructure issue (Docker socket not available in CI runners). Closes GRM-75
This commit is contained in:
+164
-30
@@ -10,13 +10,19 @@
|
||||
│ ├── executor.py # Ansible subprocess execution
|
||||
│ ├── registry.py # Local JSON runner registry
|
||||
│ ├── i18n.py # Translations (en, bg, de, ru, zh)
|
||||
│ └── exceptions.py # Custom exceptions
|
||||
│ ├── exceptions.py # Custom exceptions
|
||||
│ ├── config.py # Configuration constants
|
||||
│ ├── logging_config.py # Logging to ~/.local/state/grm/logs/
|
||||
│ ├── report.py # Operation report tracking
|
||||
│ ├── ui.py # Colorised console output
|
||||
│ ├── api_clients.py # Gitea/Vikunja API clients (for CI scripts)
|
||||
│ └── translations.json # Translation strings
|
||||
├── ansible/
|
||||
│ ├── roles/gitea-runner/ # Main Ansible role
|
||||
│ │ ├── defaults/main.yml # Default variables
|
||||
│ │ ├── tasks/ # Task files
|
||||
│ │ ├── templates/ # Jinja2 templates
|
||||
│ │ └── molecule/ # Test scenarios
|
||||
│ │ ├── tasks/ # Task files (13 files)
|
||||
│ │ ├── templates/ # Jinja2 templates (4 files)
|
||||
│ │ └── molecule/ # Test scenarios (7 scenarios)
|
||||
│ ├── install-runner.yml # Install playbook
|
||||
│ ├── update-runner.yml # Update playbook
|
||||
│ ├── start-runner.yml # Start playbook
|
||||
@@ -28,38 +34,68 @@
|
||||
├── tests/
|
||||
│ ├── unit/ # Unit tests
|
||||
│ └── integration/ # Integration tests
|
||||
├── .gitea/workflows/ # CI/CD workflows
|
||||
├── docs/ # Documentation (synced to wiki)
|
||||
├── Makefile # Build & test automation
|
||||
└── pyproject.toml # Python project metadata
|
||||
├── pyproject.toml # Python project metadata
|
||||
├── cliff.toml # git-cliff configuration
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Python 3.12+** — Required. The Makefile verifies this before creating the venv. Use `pyenv` to manage Python versions if needed.
|
||||
- **Git** — For cloning the repository and checking out release tags.
|
||||
- **Docker** — Only needed for running Molecule tests locally (`make molecule`).
|
||||
- **Go** — Only needed if you want to install `checkmake` manually (alternatively, `make setup` installs it via `devx.tools.install_checkmake`).
|
||||
|
||||
## Setup Development Environment
|
||||
|
||||
### Step 1: Clone and checkout latest release
|
||||
|
||||
```bash
|
||||
make setup # Creates venv, installs deps, sets up hooks
|
||||
git clone https://git.oblachno.oblachno.fyi/oblachno-oss/grm.git
|
||||
cd grm
|
||||
git checkout $(git describe --tags --abbrev=0) # Checkout latest stable release
|
||||
```
|
||||
|
||||
> **Important:** Always checkout the latest release tag before running `make setup`. The `master` branch may contain unreleased changes that are not yet stable. To see all available releases, run `git tag --sort=-version:refname` or check the [releases page](https://git.oblachno.oblachno.fyi/oblachno-oss/grm/releases).
|
||||
|
||||
### Step 2: Ensure Python 3.12+ is available
|
||||
|
||||
If you use pyenv:
|
||||
|
||||
```bash
|
||||
pyenv install 3.12
|
||||
pyenv local 3.12
|
||||
```
|
||||
|
||||
Verify your Python version:
|
||||
|
||||
```bash
|
||||
python3 --version # Must be 3.12 or higher
|
||||
```
|
||||
|
||||
### Step 3: Run make setup
|
||||
|
||||
```bash
|
||||
make setup
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
The `make setup` target (from the `Makefile`):
|
||||
The `make setup` target performs the following:
|
||||
|
||||
- Verifies Python 3.12+ is installed
|
||||
- Creates a virtualenv in `.venv`
|
||||
- Installs/updates `pip`, `setuptools`, and `wheel`
|
||||
- Creates `.env` from `.env.example` if not present
|
||||
- Generates shell activation scripts (`activate.sh`, `activate.fish`, `activate.zsh`)
|
||||
- Installs `checkmake` via `devx.tools.install_checkmake`
|
||||
- Runs `python -m devx.tools.setup` to install dependencies and hooks
|
||||
1. Verifies Python 3.12+ is installed
|
||||
2. Creates a virtualenv in `.venv`
|
||||
3. Installs/updates `pip`, `setuptools`, and `wheel`
|
||||
4. Creates `.env` from `.env.example` if not present
|
||||
5. Generates shell activation scripts (`activate.sh`, `activate.fish`, `activate.zsh`)
|
||||
6. Installs the `devx` package from the Oblachno PyPI registry (provides CI/CD tools)
|
||||
7. Installs `checkmake` via `devx.tools.install_checkmake` (Makefile linter)
|
||||
8. Installs CI/CD tools via `devx.tools.install_tools` (actionlint, git-cliff, act_runner, tea) to `~/.local/bin`
|
||||
9. Runs `python -m devx.tools.setup` to install Python dependencies, Ansible Galaxy collections, pre-commit hooks, and configure tea CLI login
|
||||
|
||||
### Developer Quick Start
|
||||
|
||||
```bash
|
||||
git clone https://git.oblachno.oblachno.com/oblachno/gitea-runner-manager.git
|
||||
cd gitea-runner-manager
|
||||
pyenv install 3.12
|
||||
pyenv local 3.12
|
||||
make setup
|
||||
```
|
||||
|
||||
### Configure Gitea Credentials
|
||||
### Step 4: Configure Gitea credentials
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
@@ -70,28 +106,49 @@ cp .env.example .env
|
||||
|
||||
`GITEA_REGISTRATION_TOKEN` is the runner registration token obtained from your Gitea instance (Admin → Actions → Runners → Create Registration Token).
|
||||
|
||||
#### Admin API Token (optional)
|
||||
#### Admin API token (optional)
|
||||
|
||||
Set `GITEA_ADMIN_TOKEN` to enable informational API checks during integration test. This is **optional** — the test primarily verifies the runner by checking:
|
||||
Set `REPO_TOKEN` to enable informational API checks during integration test. This is **optional** — the test primarily verifies the runner by checking:
|
||||
|
||||
1. **`.runner` registration file** exists and contains valid JSON (proves successful registration)
|
||||
2. **Systemd user service** is active (proves daemon is polling for jobs)
|
||||
|
||||
API checks, if enabled, are purely informational and do not affect pass/fail.
|
||||
|
||||
### Step 5: Verify the setup
|
||||
|
||||
```bash
|
||||
grm --version # Should print the version
|
||||
make lint-all # Should pass with no errors
|
||||
make pytest-cov # Should pass with 100% coverage
|
||||
```
|
||||
|
||||
## Shell activation scripts
|
||||
|
||||
`make setup` generates convenience activation scripts for different shells:
|
||||
|
||||
```bash
|
||||
source activate.sh # bash
|
||||
source activate.fish # fish
|
||||
source activate.zsh # zsh
|
||||
```
|
||||
|
||||
These scripts activate the `.venv` virtualenv from the project root.
|
||||
|
||||
## Running Linters
|
||||
|
||||
```bash
|
||||
make lint # Python (ruff + pyright + bandit)
|
||||
make lint # Python (ruff + format check + pyright + bandit)
|
||||
make lint-bandit # Security scan only
|
||||
make ansible-lint # Ansible
|
||||
make makefile-lint # Makefile
|
||||
make workflow-lint # Gitea Actions workflows (actionlint)
|
||||
```
|
||||
|
||||
The full lint target (`make lint-all`) runs all of the above:
|
||||
|
||||
```bash
|
||||
make lint-all # ruff + pyright + bandit + ansible-lint + checkmake
|
||||
make lint-all # ruff + pyright + bandit + ansible-lint + checkmake + actionlint
|
||||
```
|
||||
|
||||
Individual lint targets from the `Makefile`:
|
||||
@@ -102,7 +159,84 @@ Individual lint targets from the `Makefile`:
|
||||
| `lint-format` | `ruff format --check src/ tests/` |
|
||||
| `typecheck` | `pyright` |
|
||||
| `lint-bandit` | `bandit -r src/` |
|
||||
| `lint-deps` | `pip-audit` — checks dependencies for known vulnerabilities |
|
||||
| `ansible-lint` | `ansible-lint ansible/` |
|
||||
| `makefile-lint` | `checkmake Makefile` |
|
||||
| `lint` | ruff + format check + pyright + bandit |
|
||||
| `lint-all` | lint + ansible-lint + makefile-lint |
|
||||
| `lint-all` | lint + ansible-lint + makefile-lint + workflow-lint |
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Unit tests
|
||||
|
||||
```bash
|
||||
make test-unit # Without coverage
|
||||
make pytest-cov # With 100% coverage enforcement
|
||||
```
|
||||
|
||||
The coverage requirement is `--cov-fail-under=100` — 100% test coverage is required for all code in `src/gitea_runner_manager/`.
|
||||
|
||||
### Integration tests
|
||||
|
||||
```bash
|
||||
make test-integration
|
||||
```
|
||||
|
||||
Tests the full CLI lifecycle commands end-to-end (mocked executor boundary).
|
||||
|
||||
### Molecule tests
|
||||
|
||||
```bash
|
||||
make molecule # Quick: all 6 scenarios on Ubuntu 22.04
|
||||
make molecule-all # Full: all 6 scenarios on all 4 supported OSes
|
||||
```
|
||||
|
||||
Requires Docker to be installed and running on your machine. Molecule creates Docker containers as test hosts, applies the Ansible role, and verifies the results.
|
||||
|
||||
### Full test suite
|
||||
|
||||
```bash
|
||||
make test-all # pytest-cov + molecule
|
||||
```
|
||||
|
||||
## Workflow verification
|
||||
|
||||
GRM includes Gitea Actions workflow files in `.gitea/workflows/`. These are verified with two tools:
|
||||
|
||||
```bash
|
||||
make workflow-lint # Static lint via actionlint
|
||||
make workflow-dryrun # Dry-run via act_runner exec --dryrun
|
||||
make workflow-check # Both of the above
|
||||
```
|
||||
|
||||
The pre-commit hook runs actionlint automatically when workflow files change.
|
||||
|
||||
## Pre-commit hooks
|
||||
|
||||
`make setup` installs pre-commit hooks that run:
|
||||
|
||||
- **pre-commit**: `ruff check`, `ruff format --check`, conventional commit message validation
|
||||
- **pre-push**: `make pytest-cov` (ensures tests pass before pushing)
|
||||
|
||||
## Make targets reference
|
||||
|
||||
| Target | Description |
|
||||
|--------|-------------|
|
||||
| `make setup` | Full setup: venv, deps, hooks, CI tools |
|
||||
| `make setup-ci` | Lean setup for CI jobs (pytest + lint, no Ansible collections) |
|
||||
| `make setup-quality` | Setup for the quality CI job (lint + test deps) |
|
||||
| `make setup-molecule` | Full setup for molecule testing |
|
||||
| `make setup-release` | Setup for release jobs (git-cliff, tea, lint tools) |
|
||||
| `make install-tools` | Install actionlint, git-cliff, act_runner, tea to `~/.local/bin` |
|
||||
| `make install-devx` | Install the devx package from the Oblachno PyPI registry |
|
||||
| `make lint-all` | ruff + pyright + bandit + ansible-lint + checkmake + actionlint |
|
||||
| `make pytest-cov` | Unit tests with 100% coverage enforcement |
|
||||
| `make test-unit` | Unit tests without coverage |
|
||||
| `make test-integration` | Integration tests |
|
||||
| `make molecule` | All 6 Molecule scenarios on Ubuntu 22.04 |
|
||||
| `make molecule-all` | All 6 scenarios on all 4 supported OSes |
|
||||
| `make test-all` | pytest-cov + molecule |
|
||||
| `make workflow-lint` | Static lint of workflow YAML (actionlint) |
|
||||
| `make workflow-dryrun` | Dry-run all workflows in Docker |
|
||||
| `make workflow-check` | workflow-lint + workflow-dryrun |
|
||||
| `make clean` | Remove `__pycache__`, `.pyc`, `.coverage`, `htmlcov/`, `.molecule/` |
|
||||
|
||||
Reference in New Issue
Block a user