oblachno-oss/grm
3
0
Fork 0
Code Issues 20 Pull Requests Actions Packages Projects Releases 14 Wiki Activity
Files
v0.1.0
grm/initial-plan.md
T

17 KiB
Raw Permalink Blame History

Gitea Runner Manager (GRM) – Complete Project Plan

1. Overview

Gitea Runner Manager (GRM) is a lean command‑line tool to automate the installation, configuration, and lifecycle management of Gitea Actions runners on Arch Linux, Ubuntu (22.04, 24.04, 26.04), and Debian (12, 13) hosts. It is designed to:

  • Be simple and focused – no unnecessary features.
  • Be secure – no hardcoded secrets, uses scoped tokens.
  • Be idempotent – can be run multiple times safely.
  • Be flexible – accepts a plain IP address or hostname, and allows specifying the SSH user and private key.

GRM provides a unified CLI (grm.py) and a make install target to:

  • List registered runners in a Gitea instance.
  • Generate registration tokens.
  • Install and configure a runner on a remote host (Docker, act_runner, systemd service, safe Docker pruning).
  • Update the act_runner binary without losing registration.
  • (Future) Uninstall a runner.

2. Key Design Decisions

Area Decision Rationale
Target OS Arch Linux, Ubuntu 22.04/24.04/26.04, Debian 12/13 Covers 99% of use cases; avoids complexity.
Architecture amd64 only Hetzner and most cloud providers use x86_64.
Backup Lightweight config backup (optional) Runner state is stored in Gitea; re‑registration is trivial.
Monitoring None Gitea UI shows runner status; manual checks are enough.
Logging Systemd journald Sufficient for debugging; no centralised logging needed.
Pruning Only runner‑labelled resources Prevents accidental deletion of unrelated containers.
Integration tests Run after installation; fail if not successful Ensures runner is functional from the start.
Token storage .env file or --token flag No secrets in code; supports CI/CD.
Host specification Plain IP or hostname; SSH user and key overridable Simplifies inventory management, works with any host.

3. Architecture

GRM consists of three layers:

  1. CLI (Python): User commands, Gitea API interactions, Ansible invocation.
  2. Ansible Playbook: Idempotent installation of runner on target host, adapting to OS distribution.
  3. Integration Tests: Run after installation; verify runner is online in Gitea.
+----------------+       +----------------+       +-----------------+
|   User / CI    | ----> |   grm.py CLI   | ----> |  Gitea API      |
+----------------+       +----------------+       +-----------------+
                              |
                              v
                       +------------------+
                       |  Ansible Playbook |
                       +------------------+
                              |
                              v
                       +------------------+
                       |   Remote Host    |
                       |  (Arch/Ubuntu    |
                       |   /Debian)       |
                       +------------------+
                              |
                              v
                       +------------------+
                       | Integration Tests|
                       | (post-install)   |
                       +------------------+

4. Project Structure

gitea-runner-manager/
├── .python-version                           # 3.11.11
├── .env.example                              # Environment variables template
├── .gitignore
├── README.md
├── LICENSE (GPL-3.0)
├── Makefile                                  # Targets: setup, install, update, lint, ansible-lint, test, etc.
├── pyproject.toml                            # Single source for Python dependencies
├── setup.py                                  # Minimal setup for editable install
├── grm.py                                    # CLI entrypoint
├── src/
│   └── gitea_runner_manager/
│       ├── __init__.py
│       ├── cli.py                            # CLI logic (click commands)
│       ├── runner_manager.py                 # Core logic (API calls, Ansible invocation)
│       ├── api_client.py                     # Gitea API interactions
│       └── exceptions.py                     # Custom exceptions
├── tests/
│   ├── __init__.py
│   ├── unit/
│   │   ├── test_runner_manager.py
│   │   └── test_api_client.py
│   └── integration/
│       └── test_provision.py                 # Integration tests for installation
├── ansible/
│   ├── requirements.yml                      # Ansible collections
│   ├── install-runner.yml                    # Main playbook
│   ├── update-runner.yml                     # Update playbook (future)
│   ├── inventory.example                     # Optional static inventory (not required)
│   ├── group_vars/
│   │   └── all.yml
│   └── roles/
│       └── gitea-runner/
│           ├── tasks/
│           │   ├── main.yml
│           │   ├── docker.yml                # Install Docker (OS-specific)
│           │   ├── download_act_runner.yml   # Download binary
│           │   ├── validate.yml              # Validate binary
│           │   ├── register.yml              # Register with Gitea
│           │   ├── config.yml                # Create config file
│           │   ├── service.yml               # Systemd service
│           │   ├── prune.yml                 # Docker prune timer
│           │   └── integration_test.yml      # Post-install validation
│           ├── handlers/
│           │   └── main.yml
│           ├── templates/
│           │   ├── act-runner.service.j2
│           │   ├── act-runner-config.toml.j2
│           │   ├── docker-prune.service.j2
│           │   └── docker-prune.timer.j2
│           ├── vars/
│           │   └── main.yml
│           └── molecule/
│               └── default/
│                   ├── molecule.yml
│                   ├── converge.yml
│                   ├── verify.yml
│                   └── prepare.yml
└── .pre-commit-config.yaml                  # Pre-commit and pre-push hooks

5. Python Environment and Dependencies

  • Python version: 3.11.11 (managed by pyenv).
  • Virtual environment: Created automatically by make setup (or manually with python -m venv .venv).

Dependencies (defined in pyproject.toml):

Type Packages
Runtime requests, python-dotenv, click, ansible
Development pytest, pytest-cov, ruff, pyright, molecule, molecule-docker, ansible-lint, pre-commit

All dependencies are installed with make setup or pip install -e .[dev].

6. Makefile (Complete)

The Makefile provides the following targets:

Target Description
setup Full environment setup: checks pyenv, installs Python dependencies, Ansible collections, and pre‑commit hooks.
install Installs a runner on a host. Requires HOST, optional USER, KEY, NAME, TOKEN. Example: make install HOST=192.168.1.10 USER=arch NAME=my-runner
update Updates the act_runner binary on the specified host (future).
lint Runs Python linters (ruff, pyright).
ansible-lint Runs ansible-lint on all playbooks and roles.
lint-all Runs lint and ansible-lint.
test-unit Runs unit tests with coverage.
pytest-cov Runs unit tests with 100% coverage requirement.
molecule Runs Ansible Molecule tests.
test-all Runs pytest-cov and molecule.
clean Removes temporary files and caches.

Example usage:

make setup                 # Initialize development environment

# Install runner on a host (plain IP) with default user (ansible_user in inventory)
make install HOST=192.168.1.10

# With custom user and SSH private key
make install HOST=192.168.1.10 USER=arch KEY=~/.ssh/id_ed25519

# With custom runner name and token (token auto-generated if omitted)
make install HOST=runner.example.com USER=ubuntu NAME=prod-runner

make ansible-lint          # Lint Ansible code
make test-all              # Run all tests (unit + molecule)

7. Pre-commit and Pre-push Hooks

Defined in .pre-commit-config.yaml. Hooks run automatically on git commit and git push.

Hook Stage Purpose
ruff-lint commit Lint Python code
ruff-format commit Format Python code
pyright commit Type‑check Python code
ansible-lint commit Lint Ansible playbooks/roles
detect-secrets commit Prevent committing secrets
pytest-cov push 100% unit test coverage
test-all push Run all tests (unit + molecule)

If any hook fails, the commit or push is blocked.

8. CLI – grm.py

The CLI is built with click and provides the following commands:

# List all registered runners
./grm.py list

# Generate a new registration token
./grm.py token

# Install and configure a runner on a remote host
./grm.py install <host> --user <user> [--key <private_key_path>] [--name <runner_name>] [--token <token>]

# Update runner binary
./grm.py update <host> --user <user> [--key <private_key_path>] [--version <specific_version>]

Options:

  • --user: SSH user (default: from environment or ansible_user in inventory, fallback to root).
  • --key: Path to private SSH key (optional, uses default key if not provided).
  • --name: Runner name (default: hostname).
  • --token: Registration token (auto-generated if not provided).

Environment:

  • Reads .env file if present.
  • Uses GITEA_URL, GITEA_TOKEN, and optionally GITEA_RUNNER_USER, GITEA_RUNNER_KEY from environment.

Implementation (src/gitea_runner_manager/cli.py):

  • list → calls api_client.get_runners().
  • token → calls api_client.create_registration_token().
  • install → generates token (if not provided), builds an Ansible command with -i <host>, and --user <user> and --private-key <key>.
  • update → similar to install but with the update playbook.

Ansible invocation:

ansible-playbook install-runner.yml \
  -i "<host>," \
  -u <user> \
  --private-key <key> \
  --extra-vars "registration_token=<token> runner_name=<name>"

9. Ansible Role – gitea-runner

The role performs the following tasks in order, adapting to the OS distribution using ansible_facts['os_family'] and ansible_distribution.

9.1. docker.yml – OS‑specific Docker installation

  • For Debian/Ubuntu:

    • Install apt-transport-https, ca-certificates, curl.
    • Add Docker GPG key and repository.
    • Install docker-ce, docker-ce-cli, containerd.io, docker-compose-plugin.

  • For Arch Linux:

    • Install docker, docker-compose using pacman.
    • Ensure the docker systemd service is enabled and started.
    • Add the current user to the docker group.

The playbook detects the OS family and executes the appropriate block.

9.2. download_act_runner.yml

  • Fetches the latest (or specified) act_runner binary from Gitea releases.
  • Extracts it to /usr/local/bin/act_runner and sets executable permissions.
  • Uses ansible_architecture to choose the correct binary (linux_amd64).

9.3. validate.yml

  • Checks that /usr/local/bin/act_runner exists and is executable.
  • Runs act_runner --version to ensure it works.
  • Checks Docker connectivity (docker version).
  • Sets runner_validated: true if all checks pass.

9.4. config.yml

  • Creates /etc/act-runner/config.toml with the following content: 
    log.level = "info"
runner.file = ".runner"
container.label = "gitea-runner=true"
  • This ensures all spawned containers are labelled, enabling safe pruning.

9.5. register.yml

  • Ensures work directory (/var/lib/gitea-runner) exists.
  • Runs act_runner register with the provided registration_token, runner_name, labels, and gitea_url.
  • Skips registration if .act_runner already exists (idempotent).

9.6. service.yml

  • Creates systemd service file /etc/systemd/system/act-runner-{{ runner_name }}.service.
  • Points to the config file with --config /etc/act-runner/config.toml.
  • Enables and starts the service.

9.7. prune.yml

  • Creates systemd service and timer for daily Docker prune:
    • docker-prune.service: runs docker system prune and docker volume prune with filters for label=gitea-runner=true and until=24h.
    • docker-prune.timer: triggers daily.
  • Enables and starts the timer.

9.8. integration_test.yml

  • Waits up to 2 minutes for the runner to appear in the Gitea API.
  • Checks that the runner status is "online".
  • Fails the playbook if the runner is not found or not online.
  • This ensures that the runner is fully functional after installation.

10. Integration Tests (Detailed)

After registration and service start, the playbook runs integration_test.yml. It uses the Gitea API to verify the runner is online. The test is written in Ansible and uses the uri module.

Conditions:

  • Retry every 10 seconds for up to 12 attempts (2 minutes total).
  • If the runner is not found or not online, the playbook fails with a clear error message.

Why this matters:

  • Catches registration failures early.
  • Ensures the runner can communicate with Gitea.
  • Prevents deploying a broken runner.

11. Safe Docker Pruning

The runner labels all its containers with gitea-runner=true (via container.label in the config file). The prune service uses --filter "label=gitea-runner=true" to ensure it only removes resources created by the runner. This guarantees that other services on the same host are not affected.

12. Quality Gates

  • 100% unit test coverage (make pytest-cov).
  • All linters pass (ruff, pyright, ansible-lint).
  • Molecule tests pass (role validation in Docker container).
  • Integration tests pass (post‑installation validation).

These gates are enforced by pre‑push hooks.

13. Installation & Usage

13.1. Developer Setup

git clone https://git.oblachno.oblachno.com/oblachno/gitea-runner-manager.git
cd gitea-runner-manager
pyenv install 3.11.11
pyenv local 3.11.11
make setup

13.2. Configure Gitea Credentials

cp .env.example .env
# Edit .env:
# GITEA_URL=https://git.oblachno.oblachno.com
# GITEA_TOKEN=your-personal-access-token
# Optional: GITEA_RUNNER_USER=ubuntu  # default SSH user
# Optional: GITEA_RUNNER_KEY=~/.ssh/id_rsa

The token needs admin:runner scope (or admin for full management).

13.3. Install a Runner

Using the CLI (recommended for flexibility):

./grm.py install 192.168.1.10 --user ubuntu --key ~/.ssh/id_ed25519 --name prod-runner

Using Make:

make install HOST=192.168.1.10 USER=ubuntu KEY=~/.ssh/id_ed25519 NAME=prod-runner

If USER is not provided, the CLI uses the environment variable GITEA_RUNNER_USER or falls back to the current local user's username (which may not exist on the remote host – it's better to always specify).

13.4. Verify Runner

Check Gitea admin UI under Actions → Runners. The runner should appear as Online.

13.5. Update Runner Binary (Future)

./grm.py update 192.168.1.10 --user ubuntu

14. Molecule Tests

The Ansible role is tested with Molecule using a systemd‑enabled Docker container. For Arch Linux, we may use a different Docker image (e.g., archlinux/archlinux). The test suite will include scenarios for Ubuntu, Debian, and Arch Linux.

The default scenario:

  • Verifies Docker installation.
  • Checks that act_runner binary is present and executable.
  • Asserts that the systemd service is enabled and running.
  • Ensures the prune timer is active.
  • Runs the integration_test task with a mock Gitea API (or skips it if runner_register=false).

Molecule tests run as part of make test-all.

15. Logging

  • All Ansible output goes to stdout (visible in CLI).
  • act_runner logs go to journald via the systemd service.
  • To view runner logs: sudo journalctl -u act-runner-<name> -f.

16. Future Extensions (Optional)

  • Uninstall: A playbook to stop the service, remove the binary, and delete the work directory.
  • Version pinning: Allow specifying a particular act_runner version via CLI.
  • Additional distributions: Extend the role to support more distros if needed.

17. Success Criteria

  • make setup configures the development environment.
  • make install HOST=... USER=... provisions a runner on Ubuntu, Debian, and Arch Linux.
  • Integration tests pass after installation; installation fails if they do not.
  • Pre‑commit and pre‑push hooks enforce quality gates (100% coverage, linting).
  • Molecule tests pass for all supported OS.
  • Docker prune only affects runner‑labelled resources.
  • make ansible-lint runs successfully.
  • Documentation is complete and accurate.

18. License

  • GPL‑3.0 – open source, free to use and modify.

This GRM plan is production‑ready, lean, cross‑distribution, and flexible. It supports Arch, Ubuntu, and Debian, and accepts plain IP addresses with configurable SSH user and key. All components are specified, and the make setup command gets a developer from zero to a fully configured environment in minutes.

Reference in New Issue View Git Blame Copy Permalink