grm/README.md

# Gitea Runner Manager (GRM)

A lean command-line tool to automate the installation, configuration, and lifecycle management of Gitea Actions runners on Arch Linux, Ubuntu, and Debian hosts.

By default, runners are deployed as **Docker containers** using the official `gitea/runner` image. A traditional **binary** deployment mode is also available.

> **Pronunciation note:** GRM is short for *Gitea Runner Manager*, but say it like **ГРЪМ** (roughly "GRUM" in Latin letters) — the Bulgarian word for **thunder**. Wherever there are clouds, there may be thunders. This is an open-source project from **Oblachno** (облачно means *cloudy* in Bulgarian).

## Features

- **Simple and focused** — no unnecessary features.
- **Secure** — no hardcoded secrets, uses scoped tokens.
- **Idempotent** — can be run multiple times safely.
- **Flexible** — accepts a plain IP address or hostname, and allows specifying the SSH user and private key.
- **Runner registry** — stores runner connection metadata locally after installation. Subsequent commands need only the runner name.
- **Lifecycle management** — start, stop, enable, disable, status, and remove runners via CLI.
- **Multi-instance** — run multiple isolated runners on the same host, each with its own data directory and systemd service.
- **Systemd-managed** — both Docker and binary modes run under systemd template units (`gitea-runner@<name>.service`).

## Supported Operating Systems

- Arch Linux
- Ubuntu 22.04 / 24.04 / 26.04
- Debian 12 / 13

## Prerequisites

- **SSH key authentication** — The remote host must be reachable via SSH using the user specified with `--user` and the private key specified with `--key`. GRM uses Ansible under the hood, which connects to the target host over SSH to execute all installation and configuration tasks. Without valid SSH credentials, Ansible cannot establish a connection and the deployment will fail.
- **Sudo access** — GRM requires root privileges on the remote host to install packages, create systemd services, and manage Docker. You will be prompted interactively for the sudo password. For automation or uninterrupted workflows, configure passwordless sudo on the remote host.

## Quick Start

### Developer Setup

```bash
git clone https://git.oblachno.oblachno.com/oblachno/gitea-runner-manager.git
cd gitea-runner-manager
pyenv install 3.11.11
pyenv local 3.11.11
make setup
```

### Configure Gitea Credentials

```bash
cp .env.example .env
# Edit .env:
# GITEA_URL=https://git.example.com
# GITEA_REGISTRATION_TOKEN=your-registration-token
```

`GITEA_REGISTRATION_TOKEN` is the runner registration token obtained from your Gitea instance (Admin → Actions → Runners → Create Registration Token).

#### Admin API Token (optional)

Set `GITEA_ADMIN_TOKEN` to enable informational API checks during integration test. This is **optional** — the test primarily verifies the runner by checking:

1. **`.runner` registration file** exists and contains valid JSON (proves successful registration)
2. **Container/service** is running (proves daemon is polling for jobs)

API checks, if enabled, are purely informational and do not affect pass/fail.

### Install a Runner

Using the CLI (you will be prompted for the sudo password interactively):

```bash
# Docker mode (default) — deploys gitea_runner as a container
grm install 192.168.1.10 --user ubuntu --key ~/.ssh/id_ed25519 --name prod-runner --ask-become-pass

# Binary mode — downloads and installs the gitea_runner binary with systemd
grm install 192.168.1.10 --user ubuntu --key ~/.ssh/id_ed25519 --name prod-runner --mode binary --ask-become-pass
```

> **Automation tip:** Configure passwordless sudo on the remote host to skip the password prompt. This is recommended for CI/CD pipelines.

Using Make:

```bash
# Docker mode (default)
make install HOST=192.168.1.10 USER=ubuntu KEY=~/.ssh/id_ed25519 NAME=prod-runner ASK_BECOME_PASS=1

# Binary mode
make install HOST=192.168.1.10 USER=ubuntu KEY=~/.ssh/id_ed25519 NAME=prod-runner MODE=binary ASK_BECOME_PASS=1
```

### Runner Registry

After installation, GRM stores each runner's connection details (host, user, SSH key, mode, Gitea URL) in a local JSON registry at `~/.local/share/grm/runners.json`. This means you rarely need to repeat connection arguments:

```bash
# List all registered runners with live systemd status
grm list
```

### Manage Runner Lifecycle

Once a runner is installed, lifecycle commands work by runner name only:

```bash
# Start a runner
grm start prod-runner

# Stop a runner
grm stop prod-runner

# Enable a runner to start on boot
grm enable prod-runner

# Disable a runner (stops, deregisters, and disables systemd)
grm disable prod-runner --token <token>

# Check runner status
grm status prod-runner

# Remove a runner completely
grm remove prod-runner --token <token>
```

You can override any stored value by passing the corresponding flag:

```bash
grm start prod-runner --host 192.168.1.11 --user root --mode binary
```

> **Automation tip:** If the remote host has passwordless sudo configured, omit `--ask-become-pass`.

### Multiple Instances on the Same Host

Each runner instance is fully isolated with its own data directory and systemd service:

```bash
# Install two runners on the same host
grm install 192.168.1.10 --user ubuntu --name workflow-runner --ask-become-pass
grm install 192.168.1.10 --user ubuntu --name build-runner --mode binary --ask-become-pass

# Manage them independently by name
grm stop workflow-runner
grm status build-runner
```

### Verify Runner

The installer performs an automated integration test that verifies:

1. **`.runner` file exists** with valid JSON containing `id`, `uuid`, `token`, `address` — this proves successful registration with Gitea
2. **Container/service is running** — this proves the daemon is active and polling for jobs

You can also check the Gitea UI under **Actions → Runners** to confirm the runner appears as **Online**.

Optional: If `GITEA_ADMIN_TOKEN` is set, the installer will also query the Gitea API and report whether the runner appears in the admin or repo runners list. This is purely informational.

### View Logs

**GRM application logs** (Python CLI output):

```bash
# Application log file (all messages including DEBUG)
cat ~/.local/state/grm/logs/grm.log

# Enable debug logging in the current session
GRM_LOG_LEVEL=DEBUG grm install 192.168.1.10 --user ubuntu --name prod-runner
```

**Runner logs** (on the remote host):

```bash
# Binary mode logs (via systemd template unit)
sudo journalctl -u gitea-runner@<name> -f

# Docker mode logs
docker logs gitea-runner-<name> -f
```

The GRM application writes two parallel log streams:

| Destination | Level | Content |
|-------------|-------|---------|
| Console (stderr) | `GRM_LOG_LEVEL` (default: INFO) | User-facing messages and operation reports |
| `~/.local/state/grm/logs/grm.log` | DEBUG | All messages with timestamps and severity |

Set `GRM_LOG_LEVEL` to one of `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL` to control console verbosity. The log file always captures everything at DEBUG level.

## Architecture

GRM consists of two layers:

1. **Python CLI** (`src/gitea_runner_manager/`) — built with Click, handles argument parsing, environment loading, i18n translations, and delegates to Ansible via the `ansible-playbook` subprocess.
2. **Ansible Role** (`ansible/roles/gitea-runner/`) — idempotent role that installs Docker, configures the runner (binary or container), creates systemd template units, and registers the runner with Gitea.

```
grm install <host>
  └── RunnerManager.install()
      └── ansible-playbook ansible/install-runner.yml
          └── role: gitea-runner
              ├── docker.yml        (Docker installation)
              ├── docker_mode.yml   (container deployment)
              ├── binary_mode.yml   (binary + systemd deployment)
              ├── service.yml       (systemd template unit)
              ├── deregister.yml    (deregistration from Gitea)
              ├── prune.yml         (Docker prune timer)
              └── integration_test.yml (validate online status)
```

Both Docker and binary modes run under a single systemd template unit (`gitea-runner@.service`), instantiated per runner name (e.g., `gitea-runner@prod-runner.service`). Each instance has fully isolated directories:

- **Data**: `/var/lib/gitea-runner/<name>/`
- **Config**: `/etc/gitea-runner/<name>/` (binary mode)
- **Service**: `gitea-runner@<name>.service`

## Configuration

All tunable values are exposed as Ansible variables in `ansible/roles/gitea-runner/defaults/main.yml`:

| Variable | Default | Description |
|----------|---------|-------------|
| `runner_mode` | `docker` | Deployment mode: `docker` or `binary` |
| `gitea_runner_version` | `1.0.8` | Docker image tag / binary version |
| `gitea_runner_docker_image` | `gitea/runner` | Docker image name |
| `runner_labels` | `ubuntu-latest:docker://runner-images:ubuntu-22.04` | Runner labels |
| `skip_runner_registration` | `false` | Skip API registration (useful for tests) |
| `gitea_runner_base_data_dir` | `/var/lib/gitea-runner` | Base data directory (instance-scoped) |
| `gitea_runner_base_config_dir` | `/etc/gitea-runner` | Base config directory (instance-scoped) |
| `gitea_runner_data_dir` | `{{ base }}/{{ runner_name }}` | Runtime data directory per instance |
| `gitea_runner_config_dir` | `{{ base }}/{{ runner_name }}` | Config directory per instance |
| `gitea_runner_binary_path` | `/usr/local/bin/gitea_runner` | Binary install path |
| `gitea_runner_prune_until` | `24h` | Prune resources older than this |
| `gitea_runner_prune_schedule` | `daily` | systemd timer schedule |
| `gitea_runner_prune_label` | `gitea-runner=true` | Docker label for pruning |
| `gitea_runner_service_restart_sec` | `5` | systemd RestartSec value |
| `gitea_runner_service_user` | `{{ ansible_user \| default('root') }}` | Service user |
| `gitea_runner_log_level` | `info` | Runner log level |
| `gitea_runner_container_label` | `gitea-runner=true` | Container label |
| `docker_gpg_key_path` | `/etc/apt/keyrings/docker.asc` | Docker GPG key path |
| `GRM_LANG` | `en` | CLI language: `en`, `bg`, `de`, `ru`, `zh` |
| `GRM_LOG_LEVEL` | `INFO` | Console verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` |

Override any variable by passing it to the CLI with `--extra-vars` or by setting it in your Ansible inventory.

## Development

### Project Structure

```
.
├── src/gitea_runner_manager/      # Python CLI source
│   ├── cli.py                     # Click commands
│   ├── runner_manager.py          # Ansible orchestration + registry integration
│   ├── executor.py                # Ansible subprocess execution
│   ├── registry.py                # Local JSON runner registry
│   ├── i18n.py                    # Translations (en, bg, de, ru, zh)
│   └── exceptions.py              # Custom exceptions
├── ansible/
│   ├── roles/gitea-runner/        # Main Ansible role
│   │   ├── defaults/main.yml      # Default variables
│   │   ├── tasks/                 # Task files
│   │   ├── templates/             # Jinja2 templates
│   │   └── molecule/              # Test scenarios
│   ├── install-runner.yml         # Install playbook
│   ├── update-runner.yml          # Update playbook
│   ├── start-runner.yml           # Start playbook
│   ├── stop-runner.yml            # Stop playbook
│   ├── enable-runner.yml          # Enable playbook
│   ├── disable-runner.yml         # Disable playbook
│   ├── status-runner.yml          # Status playbook
│   └── remove-runner.yml          # Remove playbook
├── tests/
│   ├── unit/                      # Unit tests
│   └── integration/               # Integration tests
├── Makefile                       # Build & test automation
└── pyproject.toml                # Python project metadata
```

### Setup Development Environment

```bash
make setup        # Creates venv, installs deps, sets up hooks
source .venv/bin/activate
```

### Running Linters

```bash
make lint          # Python (ruff + pyright)
make ansible-lint  # Ansible
make makefile-lint # Makefile
```

## Testing

### Unit Tests

```bash
make test-unit
```

Runs pytest with 100% coverage requirement.

### Molecule Tests

```bash
make molecule
```

Runs four scenarios:

- **default** — Docker mode installation in an Ubuntu 22.04 container
- **binary** — Binary mode installation in an Ubuntu 22.04 container
- **multi-instance** — Two isolated runner instances on the same host
- **lifecycle** — Stop, disable, re-enable, and start sequence

All scenarios test idempotence (second run produces zero changes).

### Integration Tests

```bash
make test-integration
```

Tests the full CLI lifecycle commands end-to-end ( mocked executor boundary).

### Full Test Suite

```bash
make test-all      # Runs unit tests + linters + molecule
```

## Troubleshooting

### "Event loop is closed" warning

This is a harmless cleanup traceback from Molecule's Docker driver when the test process is interrupted. It does not indicate a test failure.

### Runner appears offline after installation

- Check that the `GITEA_URL` and `GITEA_REGISTRATION_TOKEN` environment variables are correct.
- Verify the runner container or service is running: `docker ps` or `systemctl status gitea-runner@<name>`.
- Check logs for registration errors.

### Integration test fails

The test checks two things:

1. **`.runner` file missing or invalid** — Registration failed. Check:
   - `GITEA_URL` and `GITEA_REGISTRATION_TOKEN` are correct
   - Runner logs for registration errors
   - The `.runner` file should exist at `/var/lib/gitea-runner/<name>/.runner`

2. **Container/service not running** — Daemon failed to start. Check:
   - `docker ps` or `systemctl status gitea-runner@<name>`
   - Logs for connection errors

### Docker mode: container won't start

- Ensure Docker is installed and running on the host.
- Verify the Docker socket is accessible: `docker version`.
- Check systemd status: `systemctl status gitea-runner@<name>`.

### Binary mode: systemd service fails

- Check the service status: `systemctl status gitea-runner@<name>`.
- Verify the binary exists at `gitea_runner_binary_path`.
- Ensure the service user is in the `docker` group.
- Check logs: `journalctl -u gitea-runner@<name> -f`.

## Makefile Targets

| Target | Description |
|--------|-------------|
| `setup` | Full environment setup |
| `install` | Installs a runner on a host |
| `update` | Updates a runner on a host |
| `start` | Starts a runner instance |
| `stop` | Stops a runner instance |
| `enable` | Enables a runner to start on boot |
| `disable` | Disables and deregisters a runner |
| `status` | Checks runner status |
| `remove` | Removes a runner completely |
| `list` | Lists registered runners with live status |
| `lint` | Runs Python linters |
| `ansible-lint` | Runs `ansible-lint` |
| `test-unit` | Runs unit tests with coverage |
| `test-integration` | Runs integration tests |
| `molecule` | Runs Ansible Molecule tests |
| `test-all` | Runs all tests |

## License

GPL-3.0