Public Access
DEVX-110: Standardize venv management in devx.mak #167
@@ -34,12 +34,13 @@ permissions:
|
||||
|
||||
You are a CI failure investigator for the devx repo.
|
||||
|
||||
## Working Directory
|
||||
## Working Directory & Virtual Environment
|
||||
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first:
|
||||
```bash
|
||||
cd /home/emo/dev/ideas/oblachno/devx
|
||||
```
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first.
|
||||
|
||||
All Python tools run inside `.venv`. `make` targets handle activation
|
||||
automatically — always use `make <target>`, never raw `pytest` or `ruff`
|
||||
commands. If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## CI Job Dependency Graph
|
||||
|
||||
@@ -131,8 +132,8 @@ make build-images-dry-run # dry-run build
|
||||
|
||||
For doc coverage issues:
|
||||
```bash
|
||||
python3 -m devx.ci.doc_coverage --fail-on-missing
|
||||
python3 -m devx.ci.lint_docs --root .
|
||||
.venv/bin/python -m devx.ci.doc_coverage --fail-on-missing
|
||||
.venv/bin/python -m devx.ci.lint_docs --root .
|
||||
```
|
||||
|
||||
### Step 5: Check for related Vikunja tasks
|
||||
|
||||
@@ -30,10 +30,14 @@ permissions:
|
||||
|
||||
You are a dependency upgrade specialist for the devx repo.
|
||||
|
||||
## Working Directory
|
||||
## Working Directory & Virtual Environment
|
||||
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first.
|
||||
|
||||
All Python tools run inside `.venv`. `make` targets handle activation
|
||||
automatically — always use `make <target>`, never raw `pytest` or `ruff`
|
||||
commands. If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## Dependency Reference Locations
|
||||
|
||||
- **Primary**: `pyproject.toml` — `[project] dependencies` and `[project.optional-dependencies]`
|
||||
@@ -81,8 +85,8 @@ If adding a new dependency without a comment, `check_pyproject_deps` will fail.
|
||||
pip install -e .[dev] # reinstall with new deps
|
||||
make pytest-cov # 100% coverage required
|
||||
make lint-all # ruff + pyright + bandit + actionlint + hadolint
|
||||
python3 -m devx.tools.check_pyproject_deps # verify dep docs
|
||||
python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
|
||||
.venv/bin/python -m devx.tools.check_pyproject_deps # verify dep docs
|
||||
.venv/bin/python -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
|
||||
```
|
||||
|
||||
All must pass. If `check_pyproject_deps` fails, add the missing comment.
|
||||
|
||||
@@ -25,10 +25,14 @@ permissions:
|
||||
|
||||
You are a documentation sync specialist for the devx repo.
|
||||
|
||||
## Working Directory
|
||||
## Working Directory & Virtual Environment
|
||||
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first.
|
||||
|
||||
All Python tools run inside `.venv`. `make` targets handle activation
|
||||
automatically — always use `make <target>`, never raw `pytest` or `ruff`
|
||||
commands. If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
```
|
||||
@@ -56,7 +60,7 @@ docs/
|
||||
|
||||
### Step 1: Check documentation coverage
|
||||
```bash
|
||||
python3 -m devx.ci.doc_coverage --fail-on-missing
|
||||
.venv/bin/python -m devx.ci.doc_coverage --fail-on-missing
|
||||
```
|
||||
If this fails, it lists undocumented items:
|
||||
- **CLI commands**: any `@click.command()` or `@click.group()` without a docs entry
|
||||
@@ -69,7 +73,7 @@ for modules.
|
||||
|
||||
### Step 2: Lint documentation structure
|
||||
```bash
|
||||
python3 -m devx.ci.lint_docs --root .
|
||||
.venv/bin/python -m devx.ci.lint_docs --root .
|
||||
```
|
||||
Common issues:
|
||||
- **Broken internal links**: `[text](page.md)` where `page.md` doesn't exist
|
||||
@@ -88,7 +92,7 @@ exist. If a script/module was renamed or deleted, update all doc references.
|
||||
|
||||
### Step 4: Verify wiki sync (if investigating a sync failure)
|
||||
```bash
|
||||
python3 -m devx.ci.sync_wiki --repo oblachno-oss/devx --strict
|
||||
.venv/bin/python -m devx.ci.sync_wiki --repo oblachno-oss/devx --strict
|
||||
```
|
||||
Common sync failures:
|
||||
- **Content mismatch**: wiki page content doesn't match local docs — usually means a previous sync was interrupted
|
||||
|
||||
@@ -34,10 +34,14 @@ permissions:
|
||||
|
||||
You are a Docker image build specialist for the devx repo.
|
||||
|
||||
## Working Directory
|
||||
## Working Directory & Virtual Environment
|
||||
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first.
|
||||
|
||||
All Python tools run inside `.venv`. `make` targets handle activation
|
||||
automatically — always use `make <target>`, never raw `pytest` or `ruff`
|
||||
commands. If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## Image Architecture
|
||||
|
||||
Three tier images built sequentially (each FROM the previous):
|
||||
@@ -87,7 +91,7 @@ This builds all 3 tiers sequentially and pushes to the Gitea registry.
|
||||
|
||||
If only one tier needs rebuilding:
|
||||
```bash
|
||||
python3 -m devx.tools.build_image \
|
||||
.venv/bin/python -m devx.tools.build_image \
|
||||
--dockerfile docker/ci-quality/Dockerfile \
|
||||
--name oblachno-oss/runner-images/ci-quality \
|
||||
--tag latest \
|
||||
|
||||
@@ -27,10 +27,14 @@ permissions:
|
||||
|
||||
You are a Gitea Actions workflow validator for the devx repo.
|
||||
|
||||
## Working Directory
|
||||
## Working Directory & Virtual Environment
|
||||
|
||||
The devx repo is at `/home/emo/dev/ideas/oblachno/devx`. Always `cd` there first.
|
||||
|
||||
All Python tools run inside `.venv`. `make` targets handle activation
|
||||
automatically — always use `make <target>`, never raw `pytest` or `ruff`
|
||||
commands. If `.venv` doesn't exist, run `make setup` first.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `.gitea/workflows/ci.yml` — PR pipeline (quality, detect-changes, release-dry-run, pr-review, auto-merge)
|
||||
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
PYTHONPATH: src
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
|
||||
python3 -m devx.tools.check_test_speed --max-seconds 6 --max-single-seconds 0.5
|
||||
- name: Documentation coverage check
|
||||
env:
|
||||
PYTHONPATH: src
|
||||
|
||||
@@ -1,5 +1,19 @@
|
||||
# AGENTS.md — Project Conventions for devx
|
||||
|
||||
## Virtual Environment
|
||||
|
||||
All Python tools, tests, and scripts run inside a standard `.venv` directory.
|
||||
Activate it before running any non-`make` command:
|
||||
|
||||
```bash
|
||||
source activate.sh # bash/zsh
|
||||
source activate.fish # fish
|
||||
source activate.zsh # zsh
|
||||
```
|
||||
|
||||
If `.venv` doesn't exist, run `make setup` first. The `make` targets handle
|
||||
venv activation automatically — always prefer `make <target>` over raw commands.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
```bash
|
||||
|
||||
@@ -6,6 +6,30 @@ BIN := $(VENV)/bin
|
||||
|
||||
all: setup
|
||||
|
||||
# --- devx.mak integration ----------------------------------------------------
|
||||
# Include shared targets from the devx package itself (venv management,
|
||||
# workflow-lint, notify-failure, checkmake, lint targets, quality checks, etc.)
|
||||
# Since devx IS the package, we can include its own devx.mak.
|
||||
DEVX_PYTHON := $(BIN)/python
|
||||
DEVX_VENV := $(VENV)
|
||||
DEVX_BIN := $(BIN)
|
||||
DEVX_LINT_PATHS := src/ tests/
|
||||
DEVX_COV_PKG := src/devx
|
||||
DEVX_TEST_PATHS := tests/
|
||||
|
||||
DEVX_MAK := $(shell $(BIN)/python -c \
|
||||
"from pathlib import Path; import devx; print(Path(devx.__file__).parent / 'make' / 'devx.mak')" \
|
||||
2>/dev/null)
|
||||
-include $(DEVX_MAK)
|
||||
|
||||
# venv, .env, and activate-scripts are provided by devx.mak
|
||||
# (devx-venv, devx-env, devx-activate-scripts, $(DEVX_VENV)/bin/activate rule)
|
||||
# Aliases for convenience and backward compatibility:
|
||||
.PHONY: venv activate-scripts
|
||||
venv: devx-venv
|
||||
.env: devx-env
|
||||
activate-scripts: devx-activate-scripts
|
||||
|
||||
# Full setup for local development
|
||||
setup: $(VENV)/bin/activate .env activate-scripts install-tools
|
||||
@$(BIN)/pip install -e '.[dev]' 2>/dev/null; \
|
||||
@@ -35,22 +59,9 @@ setup-release: $(VENV)/bin/activate .env
|
||||
# an older devx.mak that doesn't yet define devx-setup-image. Consumer repos
|
||||
# (grm, infra) can safely alias to devx-setup-image since they install devx from PyPI.
|
||||
setup-image:
|
||||
@if [ -d /opt/venv ]; then ln -sf /opt/venv .venv; . .venv/bin/activate && pip install --no-cache-dir -e . 2>/dev/null; \
|
||||
@if [ -d /opt/venv ]; then ln -sf /opt/venv $(VENV); . $(VENV)/bin/activate && pip install --no-cache-dir -e . 2>/dev/null; \
|
||||
else echo "[setup-image] /opt/venv not found — falling back to setup-ci"; $(MAKE) setup-ci; fi
|
||||
|
||||
.env:
|
||||
@if [ ! -f .env ]; then cp .env.example .env; echo "Created .env from .env.example — please edit it."; fi
|
||||
|
||||
$(VENV)/bin/activate:
|
||||
@python3 -c "import sys; v=sys.version_info; assert v >= (3, 12), f'Python 3.12+ required, found {v.major}.{v.minor}'; print(f'Python {v.major}.{v.minor}.{v.micro} OK')"
|
||||
$(PYTHON) -m venv $(VENV)
|
||||
$(BIN)/pip install --upgrade pip setuptools wheel
|
||||
|
||||
activate-scripts: $(VENV)/bin/activate
|
||||
@test -f activate.sh || (echo '#!/usr/bin/env bash' > activate.sh && echo 'source "$$(cd "$$(dirname "$${BASH_SOURCE[0]}")" && pwd)/.venv/bin/activate"' >> activate.sh && chmod +x activate.sh)
|
||||
@test -f activate.fish || (echo '#!/usr/bin/env fish' > activate.fish && echo 'set -l script_dir (dirname (status --current-filename))' >> activate.fish && echo 'source "$$script_dir/.venv/bin/activate.fish"' >> activate.fish && chmod +x activate.fish)
|
||||
@test -f activate.zsh || (echo '#!/usr/bin/env zsh' > activate.zsh && echo '0="$${ZERO:-$${0:#$$ZSH_ARGZERO}}"' >> activate.zsh && echo '0="$${$${(M)0:#/*}:-$$PWD/$$0}"' >> activate.zsh && echo 'source "$${0:A:h}/.venv/bin/activate"' >> activate.zsh && chmod +x activate.zsh)
|
||||
|
||||
install-hooks:
|
||||
@cp hooks/pre-commit .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit
|
||||
@cp hooks/pre-push .git/hooks/pre-push && chmod +x .git/hooks/pre-push
|
||||
@@ -60,23 +71,13 @@ install-tools: $(VENV)/bin/activate
|
||||
@$(BIN)/pip install -e '.' 2>/dev/null; \
|
||||
$(BIN)/python -m devx.tools.install_tools
|
||||
|
||||
# --- devx.mak integration ----------------------------------------------------
|
||||
# Include shared targets from the devx package itself (workflow-lint,
|
||||
# notify-failure, checkmake, lint targets, quality checks, etc.)
|
||||
# Since devx IS the package, we can include its own devx.mak.
|
||||
DEVX_PYTHON := $(BIN)/python
|
||||
DEVX_VENV := $(VENV)
|
||||
DEVX_BIN := $(BIN)
|
||||
DEVX_LINT_PATHS := src/ tests/
|
||||
DEVX_COV_PKG := src/devx
|
||||
DEVX_TEST_PATHS := tests/
|
||||
|
||||
DEVX_MAK := $(shell $(BIN)/python -c \
|
||||
"from pathlib import Path; import devx; print(Path(devx.__file__).parent / 'make' / 'devx.mak')" \
|
||||
2>/dev/null)
|
||||
-include $(DEVX_MAK)
|
||||
|
||||
# Aliases — project-specific names map to devx.mak targets
|
||||
.PHONY: lint-ruff lint-format typecheck lint-bandit lint-deps lint
|
||||
.PHONY: workflow-lint workflow-dryrun workflow-dryrun-safe workflow-check
|
||||
.PHONY: notify-failure checkmake check-mutable-globals check-dep-docs
|
||||
.PHONY: check-test-speed check-test-coverage check-docs
|
||||
.PHONY: create-task create-pr push-with-pr git-push rebase pr-rebase
|
||||
.PHONY: lint-all lint-dockerfiles
|
||||
lint-ruff: devx-lint-ruff
|
||||
lint-format: devx-lint-format
|
||||
typecheck: devx-typecheck
|
||||
|
||||
+44
-14
@@ -56,17 +56,57 @@ DEVX_DOCKERFILE_PATHS ?= docker
|
||||
# PIP_INSTALL — helper to run pip with Gitea private PyPI registry configured.
|
||||
# Usage: $(DEVX_PIP_INSTALL) install -e '.[ci,lint]'
|
||||
# CI_GITEA_USERNAME can be set in .env, as an env var, or as a Make variable.
|
||||
# Projects can alias: PIP_INSTALL = $(DEVX_PIP_INSTALL)
|
||||
DEVX_PIP_INSTALL := if [ -z "$$CI_GITEA_TOKEN" ]; then . ./.env 2>/dev/null; fi; \
|
||||
CI_GITEA_TOKEN="$$CI_GITEA_TOKEN"; \
|
||||
_PYPI_USER="$${CI_GITEA_USERNAME:-emil}"; \
|
||||
if [ -n "$$CI_GITEA_TOKEN" ] && [ -n "$$_PYPI_USER" ]; then export PIP_EXTRA_INDEX_URL="https://$$_PYPI_USER:$$CI_GITEA_TOKEN@$(DEVX_GITEA_PYPI_HOST)/api/packages/$(DEVX_GITEA_PYPI_ORG)/pypi/simple/"; fi; \
|
||||
$(DEVX_BIN)/pip
|
||||
|
||||
# ── Virtual environment management ────────────────────────────────────────────
|
||||
#
|
||||
# These targets provide a single, consistent venv setup across all
|
||||
# devx-integrated projects (infra, grm, devx). Each project includes
|
||||
# devx.mak and aliases its local targets to these.
|
||||
#
|
||||
# The venv is a standard .venv directory (no pyenv virtualenv dependency).
|
||||
# pyenv can still be used to install Python 3.12+ but the venv itself
|
||||
# is created with `python3 -m venv .venv`.
|
||||
#
|
||||
# Projects should set these variables BEFORE including devx.mak:
|
||||
# DEVX_VENV — venv directory (default: .venv)
|
||||
# DEVX_BIN — venv bin directory (default: $(DEVX_VENV)/bin)
|
||||
# DEVX_PYTHON — Python executable (default: python3; should be $(DEVX_BIN)/python after setup)
|
||||
#
|
||||
# Common aliases in project Makefiles:
|
||||
# PIP_INSTALL = $(DEVX_PIP_INSTALL)
|
||||
# venv: devx-venv
|
||||
# activate-scripts: devx-activate-scripts
|
||||
# .env: devx-env
|
||||
|
||||
# Create .venv with Python version check (3.12+ required)
|
||||
$(DEVX_VENV)/bin/activate:
|
||||
@python3 -c "import sys; v=sys.version_info; assert v >= (3, 12), f'Python 3.12+ required, found {v.major}.{v.minor}'; print(f'Python {v.major}.{v.minor}.{v.micro} OK')"
|
||||
python3 -m venv $(DEVX_VENV)
|
||||
$(DEVX_BIN)/pip install --upgrade pip setuptools wheel
|
||||
|
||||
# Alias: devx-venv creates the venv (delegates to the activate rule)
|
||||
devx-venv: $(DEVX_VENV)/bin/activate
|
||||
|
||||
# Ensure a venv exists — in CI (no pyenv), creates .venv if missing.
|
||||
# Locally, uses the existing .venv (created by `make setup` or `make devx-venv`).
|
||||
devx-ensure-venv:
|
||||
@if [ ! -f $(DEVX_BIN)/python ]; then \
|
||||
echo "[ensure-venv] Creating $(DEVX_VENV) (no venv found)..."; \
|
||||
python3 -m venv $(DEVX_VENV); \
|
||||
$(DEVX_BIN)/pip install --upgrade pip setuptools wheel; \
|
||||
fi
|
||||
|
||||
.PHONY: devx-create-task devx-create-pr devx-push devx-push-with-pr devx-check-config
|
||||
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-pr-review devx-rebase devx-pr-rebase
|
||||
.PHONY: devx-configure-gitea-pypi devx-install-tools devx-install-checkmake devx-checkmake
|
||||
.PHONY: devx-workflow-lint devx-workflow-dryrun devx-workflow-dryrun-safe devx-workflow-check
|
||||
.PHONY: devx-notify-failure devx-install-hooks devx-activate-scripts
|
||||
.PHONY: devx-notify-failure devx-install-hooks devx-activate-scripts devx-venv devx-ensure-venv
|
||||
.PHONY: devx-lint-ruff devx-lint-format devx-typecheck devx-lint-bandit devx-lint-deps devx-lint
|
||||
.PHONY: devx-clean devx-pre-push
|
||||
.PHONY: devx-check-mutable-globals devx-check-dep-docs devx-check-test-coverage devx-check-docs devx-check-test-speed
|
||||
@@ -165,12 +205,6 @@ devx-env:
|
||||
echo "Created .env from .env.example — please edit it with your credentials."; \
|
||||
fi
|
||||
|
||||
# Create Python venv with version check
|
||||
devx-venv:
|
||||
@python3 -c "import sys; v=sys.version_info; assert v >= (3, 12), f'Python 3.12+ required, found {v.major}.{v.minor}'; print(f'Python {v.major}.{v.minor}.{v.micro} OK')"
|
||||
$(DEVX_PYTHON) -m venv $(DEVX_VENV)
|
||||
$(DEVX_BIN)/pip install --upgrade pip setuptools wheel
|
||||
|
||||
# Create activate scripts for shell/fish/zsh
|
||||
devx-activate-scripts:
|
||||
@test -f activate.sh || (echo '#!/usr/bin/env bash' > activate.sh && echo 'source "$$(cd "$$(dirname "$${BASH_SOURCE[0]}")" && pwd)/.venv/bin/activate"' >> activate.sh && chmod +x activate.sh)
|
||||
@@ -266,7 +300,7 @@ devx-lint: devx-lint-ruff devx-lint-format devx-typecheck devx-lint-bandit
|
||||
# ── Testing ───────────────────────────────────────────────────────────────────
|
||||
|
||||
devx-test-unit:
|
||||
@$(DEVX_BIN)/pytest $(DEVX_TEST_PATHS) -v --no-cov
|
||||
@$(DEVX_BIN)/pytest $(DEVX_TEST_PATHS) -q --no-cov
|
||||
|
||||
devx-pytest-cov:
|
||||
@$(DEVX_BIN)/pytest $(DEVX_TEST_PATHS) -v --cov=$(DEVX_COV_PKG) --cov-report=term-missing --cov-fail-under=100
|
||||
@@ -341,12 +375,8 @@ devx-lint-dockerfiles:
|
||||
# devx-setup-ci) — each project defines its own setup-ci target.
|
||||
|
||||
devx-setup-image:
|
||||
@if [ -d /opt/venv ]; then ln -sf /opt/venv $(DEVX_VENV); . $(DEVX_BIN)/activate; \
|
||||
_U="$${CI_GITEA_USERNAME:-emil}"; \
|
||||
if [ -n "$$CI_GITEA_TOKEN" ]; then export PIP_EXTRA_INDEX_URL="https://$$_U:$$CI_GITEA_TOKEN@$(DEVX_GITEA_PYPI_HOST)/api/packages/$(DEVX_GITEA_PYPI_ORG)/pypi/simple/"; fi; \
|
||||
pip install --no-cache-dir -e .$(if $(EXTRAS),[$(EXTRAS)],); \
|
||||
echo "[devx-setup-image] Linked /opt/venv$(if $(EXTRAS), with [$(EXTRAS)],)."; \
|
||||
else echo "[devx-setup-image] /opt/venv not found — falling back to setup-ci"; $(MAKE) setup-ci; fi
|
||||
@/opt/venv/bin/python -m devx.tools.setup_image --venv $(DEVX_VENV) --extras "$(EXTRAS)" \
|
||||
--gitea-host $(DEVX_GITEA_PYPI_HOST) --gitea-org $(DEVX_GITEA_PYPI_ORG)
|
||||
|
||||
# ── Docker image build / push / cleanup ───────────────────────────────────────
|
||||
#
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Set up the project inside a pre-built CI image.
|
||||
|
||||
CI images (e.g. ``ci-quality:latest``) ship with a Python virtualenv at
|
||||
``/opt/venv`` that already contains the runtime dependencies. This tool
|
||||
links that venv to ``.venv`` in the project root and installs the project
|
||||
itself in editable mode, optionally with extras.
|
||||
|
||||
If ``/opt/venv`` does not exist (local development), falls back to
|
||||
``make setup-ci`` via ``subprocess``.
|
||||
|
||||
Usage::
|
||||
|
||||
python3 -m devx.tools.setup_image # runtime deps only
|
||||
python3 -m devx.tools.setup_image --extras lint # runtime + lint deps
|
||||
python3 -m devx.tools.setup_image --extras ci,lint
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess # nosec B404
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
|
||||
DEFAULT_VENV = ".venv"
|
||||
OPT_VENV = "/opt/venv"
|
||||
FALLBACK_TARGET = "setup-ci"
|
||||
|
||||
|
||||
def _build_pip_extra_index_url(
|
||||
gitea_host: str,
|
||||
gitea_org: str,
|
||||
username: str,
|
||||
token: str,
|
||||
) -> str:
|
||||
"""Build the PIP_EXTRA_INDEX_URL for the Gitea PyPI registry.
|
||||
|
||||
Returns a URL of the form:
|
||||
https://<user>:<token>@<host>/api/packages/<org>/pypi/simple/
|
||||
"""
|
||||
return f"https://{username}:{token}@{gitea_host}/api/packages/{gitea_org}/pypi/simple/"
|
||||
|
||||
|
||||
def _install_in_image(
|
||||
venv_link: str,
|
||||
opt_venv: str,
|
||||
extras: str,
|
||||
gitea_host: str,
|
||||
gitea_org: str,
|
||||
) -> None:
|
||||
"""Link /opt/venv to .venv, activate it, and pip install the project.
|
||||
|
||||
Sets ``PIP_EXTRA_INDEX_URL`` when ``CI_GITEA_TOKEN`` is available so
|
||||
that private packages from the Gitea PyPI registry can be installed.
|
||||
"""
|
||||
# Symlink /opt/venv → .venv
|
||||
link = Path(venv_link)
|
||||
if link.exists() or link.is_symlink():
|
||||
link.unlink()
|
||||
link.symlink_to(opt_venv)
|
||||
|
||||
# Build pip install command
|
||||
spec = f".[{extras}]" if extras else "."
|
||||
pip_bin = str(Path(venv_link) / "bin" / "pip")
|
||||
cmd = [pip_bin, "install", "--no-cache-dir", "-e", spec]
|
||||
|
||||
env = os.environ.copy()
|
||||
token = env.get("CI_GITEA_TOKEN", "")
|
||||
if token:
|
||||
username = env.get("CI_GITEA_USERNAME", "emil")
|
||||
env["PIP_EXTRA_INDEX_URL"] = _build_pip_extra_index_url(
|
||||
gitea_host,
|
||||
gitea_org,
|
||||
username,
|
||||
token,
|
||||
)
|
||||
|
||||
click.echo(f"[setup-image] Linked {opt_venv}" + (f" with [{extras}]" if extras else "") + ".")
|
||||
subprocess.run(cmd, check=True, env=env) # nosec B603
|
||||
|
||||
|
||||
def _fallback_to_setup_ci() -> None:
|
||||
"""Fall back to ``make setup-ci`` when /opt/venv is not present."""
|
||||
click.echo(f"[setup-image] {OPT_VENV} not found — falling back to {FALLBACK_TARGET}")
|
||||
subprocess.run( # nosec B603, B607
|
||||
["make", FALLBACK_TARGET],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option(
|
||||
"--venv",
|
||||
default=DEFAULT_VENV,
|
||||
show_default=True,
|
||||
help="Path to the local venv symlink (e.g. .venv).",
|
||||
)
|
||||
@click.option(
|
||||
"--opt-venv",
|
||||
default=OPT_VENV,
|
||||
show_default=True,
|
||||
help="Path to the pre-built venv inside the CI image.",
|
||||
)
|
||||
@click.option(
|
||||
"--extras",
|
||||
default="",
|
||||
help="Comma-separated dependency extras (e.g. 'ci,lint'). Empty for runtime only.",
|
||||
)
|
||||
@click.option(
|
||||
"--gitea-host",
|
||||
default="git.oblachno.oblachno.fyi",
|
||||
show_default=True,
|
||||
help="Gitea host for the PyPI registry.",
|
||||
)
|
||||
@click.option(
|
||||
"--gitea-org",
|
||||
default="oblachno-oss",
|
||||
show_default=True,
|
||||
help="Gitea org for the PyPI registry.",
|
||||
)
|
||||
def cli(
|
||||
venv: str,
|
||||
opt_venv: str,
|
||||
extras: str,
|
||||
gitea_host: str,
|
||||
gitea_org: str,
|
||||
) -> None:
|
||||
"""Set up the project using a pre-built CI image venv."""
|
||||
if Path(opt_venv).is_dir():
|
||||
_install_in_image(venv, opt_venv, extras, gitea_host, gitea_org)
|
||||
else:
|
||||
_fallback_to_setup_ci()
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli() # pragma: no cover
|
||||
@@ -0,0 +1,262 @@
|
||||
"""Unit tests for devx.tools.setup_image."""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.tools.setup_image import (
|
||||
_build_pip_extra_index_url,
|
||||
_fallback_to_setup_ci,
|
||||
_install_in_image,
|
||||
cli,
|
||||
)
|
||||
|
||||
|
||||
class TestBuildPipExtraIndexUrl:
|
||||
def test_basic_url(self) -> None:
|
||||
url = _build_pip_extra_index_url(
|
||||
"git.oblachno.oblachno.fyi",
|
||||
"oblachno-oss",
|
||||
"emil",
|
||||
"tok123",
|
||||
)
|
||||
assert url == "https://emil:tok123@git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple/"
|
||||
|
||||
def test_custom_host_org(self) -> None:
|
||||
url = _build_pip_extra_index_url(
|
||||
"gitea.example.com",
|
||||
"my-org",
|
||||
"user",
|
||||
"secret",
|
||||
)
|
||||
assert url == "https://user:secret@gitea.example.com/api/packages/my-org/pypi/simple/"
|
||||
|
||||
|
||||
class TestInstallInImage:
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_link_and_install_no_token(self, mock_path: MagicMock, mock_run: MagicMock, tmp_path: Path) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "", "host", "org")
|
||||
|
||||
mock_path.return_value.symlink_to.assert_called_once_with("/opt/venv")
|
||||
mock_run.assert_called_once()
|
||||
cmd = mock_run.call_args[0][0]
|
||||
assert "--no-cache-dir" in cmd
|
||||
assert "-e" in cmd
|
||||
assert "." in cmd
|
||||
# No extras → spec is "."
|
||||
assert ".[]" not in " ".join(cmd)
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_link_and_install_with_extras(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "ci,lint", "host", "org")
|
||||
|
||||
cmd = mock_run.call_args[0][0]
|
||||
assert ".[ci,lint]" in cmd
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_install_with_token_sets_pip_extra_index_url(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
{"CI_GITEA_TOKEN": "tok123", "CI_GITEA_USERNAME": "emil"},
|
||||
clear=True,
|
||||
):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "lint", "git.host", "org")
|
||||
|
||||
env = mock_run.call_args[1]["env"]
|
||||
assert "PIP_EXTRA_INDEX_URL" in env
|
||||
assert "emil:tok123@git.host" in env["PIP_EXTRA_INDEX_URL"]
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_install_with_token_defaults_username(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(os.environ, {"CI_GITEA_TOKEN": "tok123"}, clear=True):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "", "host", "org")
|
||||
|
||||
env = mock_run.call_args[1]["env"]
|
||||
assert "emil:tok123@host" in env["PIP_EXTRA_INDEX_URL"]
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_install_removes_existing_link(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = True
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.unlink = MagicMock()
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "", "host", "org")
|
||||
|
||||
mock_path.return_value.unlink.assert_called_once()
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_install_removes_existing_symlink(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = True
|
||||
mock_path.return_value.unlink = MagicMock()
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "", "host", "org")
|
||||
|
||||
mock_path.return_value.unlink.assert_called_once()
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_install_failure_raises(self, mock_path: MagicMock, mock_run: MagicMock, tmp_path: Path) -> None:
|
||||
venv_link = tmp_path / ".venv"
|
||||
mock_path.return_value.exists.return_value = False
|
||||
mock_path.return_value.is_symlink.return_value = False
|
||||
mock_path.return_value.symlink_to = MagicMock()
|
||||
mock_run.side_effect = subprocess.CalledProcessError(1, ["pip"])
|
||||
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
with pytest.raises(subprocess.CalledProcessError):
|
||||
_install_in_image(str(venv_link), "/opt/venv", "", "host", "org")
|
||||
|
||||
|
||||
class TestFallbackToSetupCi:
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
def test_fallback_runs_make_setup_ci(self, mock_run: MagicMock) -> None:
|
||||
_fallback_to_setup_ci()
|
||||
mock_run.assert_called_once_with(["make", "setup-ci"], check=True)
|
||||
|
||||
@patch("devx.tools.setup_image.subprocess.run")
|
||||
def test_fallback_failure_raises(self, mock_run: MagicMock) -> None:
|
||||
mock_run.side_effect = subprocess.CalledProcessError(1, ["make"])
|
||||
with pytest.raises(subprocess.CalledProcessError):
|
||||
_fallback_to_setup_ci()
|
||||
|
||||
|
||||
class TestCli:
|
||||
@patch("devx.tools.setup_image._install_in_image")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_cli_with_opt_venv_present(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_install: MagicMock,
|
||||
) -> None:
|
||||
mock_path.return_value.is_dir.return_value = True
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--extras", "ci,lint"])
|
||||
assert result.exit_code == 0
|
||||
mock_install.assert_called_once()
|
||||
|
||||
@patch("devx.tools.setup_image._fallback_to_setup_ci")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_cli_falls_back_when_no_opt_venv(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_fallback: MagicMock,
|
||||
) -> None:
|
||||
mock_path.return_value.is_dir.return_value = False
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, [])
|
||||
assert result.exit_code == 0
|
||||
mock_fallback.assert_called_once()
|
||||
|
||||
@patch("devx.tools.setup_image._install_in_image")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_cli_default_values(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_install: MagicMock,
|
||||
) -> None:
|
||||
mock_path.return_value.is_dir.return_value = True
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, [])
|
||||
assert result.exit_code == 0
|
||||
call_args = mock_install.call_args[0]
|
||||
assert call_args[0] == ".venv"
|
||||
assert call_args[1] == "/opt/venv"
|
||||
assert call_args[2] == "" # no extras
|
||||
assert call_args[3] == "git.oblachno.oblachno.fyi"
|
||||
assert call_args[4] == "oblachno-oss"
|
||||
|
||||
@patch("devx.tools.setup_image._install_in_image")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_cli_custom_venv_and_gitea(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_install: MagicMock,
|
||||
) -> None:
|
||||
mock_path.return_value.is_dir.return_value = True
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
["--venv", ".custom-venv", "--gitea-host", "gitea.io", "--gitea-org", "myorg"],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
call_args = mock_install.call_args[0]
|
||||
assert call_args[0] == ".custom-venv"
|
||||
assert call_args[3] == "gitea.io"
|
||||
assert call_args[4] == "myorg"
|
||||
|
||||
@patch("devx.tools.setup_image._install_in_image")
|
||||
@patch("devx.tools.setup_image.Path")
|
||||
def test_cli_with_extras(
|
||||
self,
|
||||
mock_path: MagicMock,
|
||||
mock_install: MagicMock,
|
||||
) -> None:
|
||||
mock_path.return_value.is_dir.return_value = True
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--extras", "lint"])
|
||||
assert result.exit_code == 0
|
||||
assert mock_install.call_args[0][2] == "lint"
|
||||
Reference in New Issue
Block a user