# Gitea Runner Manager (GRM) – Complete Project Plan --- ## 1. Overview **Gitea Runner Manager (GRM)** is a lean command‑line tool to automate the installation, configuration, and lifecycle management of Gitea Actions runners on **Arch Linux, Ubuntu (22.04, 24.04, 26.04), and Debian (12, 13)** hosts. It is designed to: - Be **simple and focused** – no unnecessary features. - Be **secure** – no hardcoded secrets, uses scoped tokens. - Be **idempotent** – can be run multiple times safely. - Be **flexible** – accepts a plain IP address or hostname, and allows specifying the SSH user and private key. GRM provides a unified CLI (`grm.py`) and a `make install` target to: - List registered runners in a Gitea instance. - Generate registration tokens. - Install and configure a runner on a remote host (Docker, `act_runner`, systemd service, safe Docker pruning). - Update the `act_runner` binary without losing registration. - (Future) Uninstall a runner. --- ## 2. Key Design Decisions | Area | Decision | Rationale | |------|----------|-----------| | Target OS | Arch Linux, Ubuntu 22.04/24.04/26.04, Debian 12/13 | Covers 99% of use cases; avoids complexity. | | Architecture | amd64 only | Hetzner and most cloud providers use x86_64. | | Backup | Lightweight config backup (optional) | Runner state is stored in Gitea; re‑registration is trivial. | | Monitoring | None | Gitea UI shows runner status; manual checks are enough. | | Logging | Systemd `journald` | Sufficient for debugging; no centralised logging needed. | | Pruning | Only runner‑labelled resources | Prevents accidental deletion of unrelated containers. | | Integration tests | Run after installation; fail if not successful | Ensures runner is functional from the start. | | Token storage | `.env` file or `--token` flag | No secrets in code; supports CI/CD. | | Host specification | Plain IP or hostname; SSH user and key overridable | Simplifies inventory management, works with any host. | --- ## 3. Architecture GRM consists of three layers: 1. **CLI (Python)**: User commands, Gitea API interactions, Ansible invocation. 2. **Ansible Playbook**: Idempotent installation of runner on target host, adapting to OS distribution. 3. **Integration Tests**: Run after installation; verify runner is online in Gitea. ```text +----------------+ +----------------+ +-----------------+ | User / CI | ----> | grm.py CLI | ----> | Gitea API | +----------------+ +----------------+ +-----------------+ | v +------------------+ | Ansible Playbook | +------------------+ | v +------------------+ | Remote Host | | (Arch/Ubuntu | | /Debian) | +------------------+ | v +------------------+ | Integration Tests| | (post-install) | +------------------+ ``` --- ## 4. Project Structure ``` gitea-runner-manager/ ├── .python-version # 3.11.11 ├── .env.example # Environment variables template ├── .gitignore ├── README.md ├── LICENSE (GPL-3.0) ├── Makefile # Targets: setup, install, update, lint, ansible-lint, test, etc. ├── pyproject.toml # Single source for Python dependencies ├── setup.py # Minimal setup for editable install ├── grm.py # CLI entrypoint ├── src/ │ └── gitea_runner_manager/ │ ├── __init__.py │ ├── cli.py # CLI logic (click commands) │ ├── runner_manager.py # Core logic (API calls, Ansible invocation) │ ├── api_client.py # Gitea API interactions │ └── exceptions.py # Custom exceptions ├── tests/ │ ├── __init__.py │ ├── unit/ │ │ ├── test_runner_manager.py │ │ └── test_api_client.py │ └── integration/ │ └── test_provision.py # Integration tests for installation ├── ansible/ │ ├── requirements.yml # Ansible collections │ ├── install-runner.yml # Main playbook │ ├── update-runner.yml # Update playbook (future) │ ├── inventory.example # Optional static inventory (not required) │ ├── group_vars/ │ │ └── all.yml │ └── roles/ │ └── gitea-runner/ │ ├── tasks/ │ │ ├── main.yml │ │ ├── docker.yml # Install Docker (OS-specific) │ │ ├── download_act_runner.yml # Download binary │ │ ├── validate.yml # Validate binary │ │ ├── register.yml # Register with Gitea │ │ ├── config.yml # Create config file │ │ ├── service.yml # Systemd service │ │ ├── prune.yml # Docker prune timer │ │ └── integration_test.yml # Post-install validation │ ├── handlers/ │ │ └── main.yml │ ├── templates/ │ │ ├── act-runner.service.j2 │ │ ├── act-runner-config.toml.j2 │ │ ├── docker-prune.service.j2 │ │ └── docker-prune.timer.j2 │ ├── vars/ │ │ └── main.yml │ └── molecule/ │ └── default/ │ ├── molecule.yml │ ├── converge.yml │ ├── verify.yml │ └── prepare.yml └── .pre-commit-config.yaml # Pre-commit and pre-push hooks ``` --- ## 5. Python Environment and Dependencies - **Python version**: 3.11.11 (managed by pyenv). - **Virtual environment**: Created automatically by `make setup` (or manually with `python -m venv .venv`). **Dependencies** (defined in `pyproject.toml`): | Type | Packages | |------|----------| | Runtime | `requests`, `python-dotenv`, `click`, `ansible` | | Development | `pytest`, `pytest-cov`, `ruff`, `pyright`, `molecule`, `molecule-docker`, `ansible-lint`, `pre-commit` | All dependencies are installed with `make setup` or `pip install -e .[dev]`. --- ## 6. Makefile (Complete) The `Makefile` provides the following targets: | Target | Description | |--------|-------------| | `setup` | Full environment setup: checks pyenv, installs Python dependencies, Ansible collections, and pre‑commit hooks. | | `install` | Installs a runner on a host. **Requires `HOST`**, optional `USER`, `KEY`, `NAME`, `TOKEN`. Example: `make install HOST=192.168.1.10 USER=arch NAME=my-runner` | | `update` | Updates the `act_runner` binary on the specified host (future). | | `lint` | Runs Python linters (`ruff`, `pyright`). | | `ansible-lint` | Runs `ansible-lint` on all playbooks and roles. | | `lint-all` | Runs `lint` and `ansible-lint`. | | `test-unit` | Runs unit tests with coverage. | | `pytest-cov` | Runs unit tests with **100% coverage requirement**. | | `molecule` | Runs Ansible Molecule tests. | | `test-all` | Runs `pytest-cov` and `molecule`. | | `clean` | Removes temporary files and caches. | **Example usage**: ```bash make setup # Initialize development environment # Install runner on a host (plain IP) with default user (ansible_user in inventory) make install HOST=192.168.1.10 # With custom user and SSH private key make install HOST=192.168.1.10 USER=arch KEY=~/.ssh/id_ed25519 # With custom runner name and token (token auto-generated if omitted) make install HOST=runner.example.com USER=ubuntu NAME=prod-runner make ansible-lint # Lint Ansible code make test-all # Run all tests (unit + molecule) ``` --- ## 7. Pre-commit and Pre-push Hooks Defined in `.pre-commit-config.yaml`. Hooks run automatically on `git commit` and `git push`. | Hook | Stage | Purpose | |------|-------|---------| | `ruff-lint` | commit | Lint Python code | | `ruff-format` | commit | Format Python code | | `pyright` | commit | Type‑check Python code | | `ansible-lint` | commit | Lint Ansible playbooks/roles | | `detect-secrets` | commit | Prevent committing secrets | | `pytest-cov` | push | **100% unit test coverage** | | `test-all` | push | Run all tests (unit + molecule) | If any hook fails, the commit or push is blocked. --- ## 8. CLI – `grm.py` The CLI is built with `click` and provides the following commands: ```bash # List all registered runners ./grm.py list # Generate a new registration token ./grm.py token # Install and configure a runner on a remote host ./grm.py install --user [--key ] [--name ] [--token ] # Update runner binary ./grm.py update --user [--key ] [--version ] ``` **Options**: - `--user`: SSH user (default: from environment or `ansible_user` in inventory, fallback to `root`). - `--key`: Path to private SSH key (optional, uses default key if not provided). - `--name`: Runner name (default: hostname). - `--token`: Registration token (auto-generated if not provided). **Environment**: - Reads `.env` file if present. - Uses `GITEA_URL`, `GITEA_TOKEN`, and optionally `GITEA_RUNNER_USER`, `GITEA_RUNNER_KEY` from environment. **Implementation** (`src/gitea_runner_manager/cli.py`): - `list` → calls `api_client.get_runners()`. - `token` → calls `api_client.create_registration_token()`. - `install` → generates token (if not provided), builds an Ansible command with `-i ,` and `--user ` and `--private-key `. - `update` → similar to install but with the update playbook. **Ansible invocation**: ```bash ansible-playbook install-runner.yml \ -i "," \ -u \ --private-key \ --extra-vars "registration_token= runner_name=" ``` --- ## 9. Ansible Role – `gitea-runner` The role performs the following tasks in order, adapting to the OS distribution using `ansible_facts['os_family']` and `ansible_distribution`. ### 9.1. `docker.yml` – OS‑specific Docker installation - **For Debian/Ubuntu**: - Install `apt-transport-https`, `ca-certificates`, `curl`. - Add Docker GPG key and repository. - Install `docker-ce`, `docker-ce-cli`, `containerd.io`, `docker-compose-plugin`. - **For Arch Linux**: - Install `docker`, `docker-compose` using `pacman`. - Ensure the `docker` systemd service is enabled and started. - Add the current user to the `docker` group. The playbook detects the OS family and executes the appropriate block. ### 9.2. `download_act_runner.yml` - Fetches the latest (or specified) `act_runner` binary from Gitea releases. - Extracts it to `/usr/local/bin/act_runner` and sets executable permissions. - Uses `ansible_architecture` to choose the correct binary (`linux_amd64`). ### 9.3. `validate.yml` - Checks that `/usr/local/bin/act_runner` exists and is executable. - Runs `act_runner --version` to ensure it works. - Checks Docker connectivity (`docker version`). - Sets `runner_validated: true` if all checks pass. ### 9.4. `config.yml` - Creates `/etc/act-runner/config.toml` with the following content: ```toml log.level = "info" runner.file = ".runner" container.label = "gitea-runner=true" ``` - This ensures all spawned containers are labelled, enabling safe pruning. ### 9.5. `register.yml` - Ensures work directory (`/var/lib/gitea-runner`) exists. - Runs `act_runner register` with the provided `registration_token`, `runner_name`, `labels`, and `gitea_url`. - Skips registration if `.act_runner` already exists (idempotent). ### 9.6. `service.yml` - Creates systemd service file `/etc/systemd/system/act-runner-{{ runner_name }}.service`. - Points to the config file with `--config /etc/act-runner/config.toml`. - Enables and starts the service. ### 9.7. `prune.yml` - Creates systemd service and timer for daily Docker prune: - `docker-prune.service`: runs `docker system prune` and `docker volume prune` with filters for `label=gitea-runner=true` and `until=24h`. - `docker-prune.timer`: triggers daily. - Enables and starts the timer. ### 9.8. `integration_test.yml` - Waits up to 2 minutes for the runner to appear in the Gitea API. - Checks that the runner status is `"online"`. - Fails the playbook if the runner is not found or not online. - This ensures that the runner is fully functional after installation. --- ## 10. Integration Tests (Detailed) After registration and service start, the playbook runs `integration_test.yml`. It uses the Gitea API to verify the runner is online. The test is written in Ansible and uses the `uri` module. **Conditions**: - Retry every 10 seconds for up to 12 attempts (2 minutes total). - If the runner is not found or not online, the playbook fails with a clear error message. **Why this matters**: - Catches registration failures early. - Ensures the runner can communicate with Gitea. - Prevents deploying a broken runner. --- ## 11. Safe Docker Pruning The runner labels all its containers with `gitea-runner=true` (via `container.label` in the config file). The prune service uses `--filter "label=gitea-runner=true"` to ensure it only removes resources created by the runner. This guarantees that other services on the same host are not affected. --- ## 12. Quality Gates - **100% unit test coverage** (`make pytest-cov`). - **All linters pass** (ruff, pyright, ansible-lint). - **Molecule tests pass** (role validation in Docker container). - **Integration tests pass** (post‑installation validation). These gates are enforced by pre‑push hooks. --- ## 13. Installation & Usage ### 13.1. Developer Setup ```bash git clone https://git.oblachno.oblachno.com/oblachno/gitea-runner-manager.git cd gitea-runner-manager pyenv install 3.11.11 pyenv local 3.11.11 make setup ``` ### 13.2. Configure Gitea Credentials ```bash cp .env.example .env # Edit .env: # GITEA_URL=https://git.oblachno.oblachno.com # GITEA_TOKEN=your-personal-access-token # Optional: GITEA_RUNNER_USER=ubuntu # default SSH user # Optional: GITEA_RUNNER_KEY=~/.ssh/id_rsa ``` The token needs `admin:runner` scope (or `admin` for full management). ### 13.3. Install a Runner Using the CLI (recommended for flexibility): ```bash ./grm.py install 192.168.1.10 --user ubuntu --key ~/.ssh/id_ed25519 --name prod-runner ``` Using Make: ```bash make install HOST=192.168.1.10 USER=ubuntu KEY=~/.ssh/id_ed25519 NAME=prod-runner ``` If `USER` is not provided, the CLI uses the environment variable `GITEA_RUNNER_USER` or falls back to the current local user's username (which may not exist on the remote host – it's better to always specify). ### 13.4. Verify Runner Check Gitea admin UI under **Actions → Runners**. The runner should appear as **Online**. ### 13.5. Update Runner Binary (Future) ```bash ./grm.py update 192.168.1.10 --user ubuntu ``` --- ## 14. Molecule Tests The Ansible role is tested with Molecule using a systemd‑enabled Docker container. For Arch Linux, we may use a different Docker image (e.g., `archlinux/archlinux`). The test suite will include scenarios for Ubuntu, Debian, and Arch Linux. The `default` scenario: - Verifies Docker installation. - Checks that `act_runner` binary is present and executable. - Asserts that the systemd service is enabled and running. - Ensures the prune timer is active. - Runs the `integration_test` task with a mock Gitea API (or skips it if `runner_register=false`). Molecule tests run as part of `make test-all`. --- ## 15. Logging - All Ansible output goes to stdout (visible in CLI). - `act_runner` logs go to `journald` via the systemd service. - To view runner logs: `sudo journalctl -u act-runner- -f`. --- ## 16. Future Extensions (Optional) - **Uninstall**: A playbook to stop the service, remove the binary, and delete the work directory. - **Version pinning**: Allow specifying a particular `act_runner` version via CLI. - **Additional distributions**: Extend the role to support more distros if needed. --- ## 17. Success Criteria - [ ] `make setup` configures the development environment. - [ ] `make install HOST=... USER=...` provisions a runner on Ubuntu, Debian, and Arch Linux. - [ ] Integration tests pass after installation; installation fails if they do not. - [ ] Pre‑commit and pre‑push hooks enforce quality gates (100% coverage, linting). - [ ] Molecule tests pass for all supported OS. - [ ] Docker prune only affects runner‑labelled resources. - [ ] `make ansible-lint` runs successfully. - [ ] Documentation is complete and accurate. --- ## 18. License - **GPL‑3.0** – open source, free to use and modify. --- **This GRM plan is production‑ready, lean, cross‑distribution, and flexible.** It supports Arch, Ubuntu, and Debian, and accepts plain IP addresses with configurable SSH user and key. All components are specified, and the `make setup` command gets a developer from zero to a fully configured environment in minutes.