devx/src/devx/ci/classify_changes.py

#!/usr/bin/env python3
"""Classify git changes as user-facing or infrastructure.

Determines whether changes between two git refs (e.g., last tag and HEAD)
affect the published package (user-facing) or only the CI/CD infrastructure
(workflow-only).  This is used by:

- **release.py** — skips release when only infrastructure files changed
- **CI workflow** — skips molecule tests and release dry-run when only
  infrastructure files changed

== Design Philosophy ==

**Safe-by-default**: Any file that doesn't match a rule defaults to
user-facing.  This prevents new file types from accidentally skipping
releases — a critical safety property.  When in doubt, release.

**Framework-provided defaults**: The framework ships with
``DEFAULT_INFRASTRUCTURE`` — a curated list of paths that are
infrastructure for ANY Python project (CI workflows, tests, docs,
lint config, etc.).  Projects inherit these automatically and only
need to specify what's *different* about their project.

**Config-driven**: Classification rules are read from ``[tool.devx.classify]``
in ``pyproject.toml``.  No project needs to modify the framework code.
Each project declares its own paths; the framework handles the logic.

**Layered rules** (evaluated in priority order):

  1. **User-facing overrides** (highest priority — safety override)
     Files that match infrastructure patterns but MUST be treated as
     user-facing.  Use this when an infrastructure pattern is too broad.

  2. **Infrastructure overrides**
     Files that would default to user-facing but are actually
     infrastructure (e.g., ``src/pkg/__init__.py`` which only contains
     ``__version__`` — a release artifact, not user-facing code).

  3. **Infrastructure patterns** (deny-list)
     Path globs matching infrastructure files.  This is the union of
     ``DEFAULT_INFRASTRUCTURE`` and the project's ``infrastructure`` list.
     Changes to these don't trigger a release.

  4. **Default**: user-facing (lowest priority — safe default)

**Tag system** (orthogonal to release impact):
  Projects can define custom tags (e.g., ``ansible``, ``docs``) for CI
  conditional execution.  A file can be both infrastructure (no release)
  and tagged ``ansible`` (run molecule tests).  Tags are evaluated
  independently of the user-facing/infrastructure classification.
  The ``--check`` CLI option accepts any tag name defined in the config,
  and ``--github-output`` writes ``<tag>-changed`` for each configured tag.

== Configuration ==

In ``pyproject.toml``::

    [tool.devx.classify]
    # Whether to merge with DEFAULT_INFRASTRUCTURE (default: true).
    # Set to false to specify all patterns explicitly.
    # use_defaults = true

    # Project-specific infrastructure paths (merged with defaults).
    # Only list paths NOT already in DEFAULT_INFRASTRUCTURE.
    infrastructure = [
        "scripts/**",       # e.g., if scripts/ is dev-only tooling
    ]

    # Infrastructure overrides — files that would default to user-facing
    # but are actually infrastructure
    infrastructure_overrides = [
        "src/mypkg/__init__.py",   # only contains __version__
    ]

    # User-facing overrides — safety override for broad infrastructure patterns
    # (empty by default)
    user_facing_overrides = []

    # Tag patterns — additional categories for CI conditional execution
    [tool.devx.classify.tags]
    ansible = ["ansible/**", ".ansible-lint"]

== What counts as "user-facing" ==

A change is user-facing if it affects the behavior of the installed
package.  For a library/CLI tool, this means:

  - Source code in ``src/`` (except ``__init__.py`` which only holds
    ``__version__``)
  - Package metadata (``pyproject.toml`` — dependencies, entry points)
  - Ansible roles, playbooks, templates (if the project ships Ansible)
  - Translation files (user-visible messages)
  - Any file not explicitly classified as infrastructure

A change is infrastructure if it only affects the project's own
development/CI environment:

  - CI/CD workflows (``.gitea/**``, ``.github/**``)
  - Tests (``tests/**``)
  - Documentation (``docs/**``, ``README.md``, ``CHANGELOG.md``)
  - Linting/formatting config (``.ruff.toml``, ``.pre-commit-config.yaml``)
  - Build tooling (``Makefile``, ``cliff.toml``)
  - Git hooks (``hooks/**``)
  - Generated scripts (``activate.sh``, ``activate.fish``, ``activate.zsh``)

== Glob Syntax ==

Patterns support standard glob syntax:

  - ``**`` matches any number of path segments (including zero)
  - ``*`` matches any characters within a single path segment
  - ``?`` matches a single character within a single path segment
  - Everything else is matched literally

Examples:
  - ``.gitea/**`` matches ``.gitea/workflows/ci.yml``, ``.gitea/actionlint.yaml``
  - ``tests/**`` matches ``tests/unit/test_cli.py``, ``tests/conftest.py``
  - ``src/devx/__init__.py`` matches exactly that file
  - ``Makefile`` matches exactly that file

Usage:
  python3 -m devx.ci.classify_changes [--base <ref>] [--head <ref>]
  python3 -m devx.ci.classify_changes --base v0.3.0 --head HEAD
  python3 -m devx.ci.classify_changes --check ansible --quiet
  python3 -m devx.ci.classify_changes --github-output
"""

from __future__ import annotations

import os
import re
import subprocess  # nosec B404
import sys
import tomllib
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

import click

from devx.ci._shared import get_latest_tag
from devx.i18n import _

# ---------------------------------------------------------------------------
# Data structures
# ---------------------------------------------------------------------------


@dataclass(frozen=True)
class FileClassification:
    """Result of classifying a single file.

    Attributes:
        path: The file path relative to repo root.
        is_user_facing: True if changes to this file require a release.
        reason: Human-readable explanation of the classification.
        matched_rule: Which rule matched (e.g., "infrastructure: .gitea/**").
            None if the default rule was used.
        tags: Custom category tags (e.g., {"ansible"}).
    """

    path: str
    is_user_facing: bool
    reason: str
    matched_rule: str | None
    tags: frozenset[str] = frozenset()


@dataclass
class ClassificationResult:
    """Result of classifying a set of changed files.

    Attributes:
        files: Per-file classification details.
        user_facing: List of file paths classified as user-facing.
        infrastructure: List of file paths classified as infrastructure.
        tags: Dict mapping tag name to list of file paths matching that tag.
    """

    files: list[FileClassification] = field(default_factory=list)
    user_facing: list[str] = field(default_factory=list)
    infrastructure: list[str] = field(default_factory=list)
    tags: dict[str, list[str]] = field(default_factory=dict)

    @property
    def has_user_facing(self) -> bool:
        """True if any user-facing files were found."""
        return bool(self.user_facing)

    def has_tag(self, tag: str) -> bool:
        """True if any files matched the given tag."""
        return bool(self.tags.get(tag))


# ---------------------------------------------------------------------------
# Glob matching
# ---------------------------------------------------------------------------


def _glob_to_regex(pattern: str) -> re.Pattern[str]:
    """Convert a glob pattern to a compiled regex.

    Supports:
      - ``**``  -> matches any number of path segments (including zero)
      - ``*``   -> matches any chars within a single path segment
      - ``?``   -> matches a single char within a path segment
      - All other characters are matched literally
    """
    # Handle ** at the end (e.g., ".gitea/**")
    # ** matches anything including slashes
    parts: list[str] = []
    i = 0
    while i < len(pattern):
        c = pattern[i]
        if c == "*" and i + 1 < len(pattern) and pattern[i + 1] == "*":
            parts.append(".*")
            i += 2
            # Skip trailing slash after **
            if i < len(pattern) and pattern[i] == "/":
                i += 1
        elif c == "*":
            parts.append("[^/]*")
            i += 1
        elif c == "?":
            parts.append("[^/]")
            i += 1
        else:
            parts.append(re.escape(c))
            i += 1
    return re.compile("^" + "".join(parts) + "$")


def _matches_glob(file_path: str, pattern: str) -> bool:
    """Check if a file path matches a glob pattern.

    Also supports prefix matching: if the pattern ends with ``/``,
    any file starting with that prefix matches.  This is a convenience
    for patterns like ``.gitea/`` (equivalent to ``.gitea/**``).
    """
    # Prefix matching for patterns ending with /
    if pattern.endswith("/") and (file_path.startswith(pattern) or file_path == pattern.rstrip("/")):
        return True
    return _glob_to_regex(pattern).match(file_path) is not None


# ---------------------------------------------------------------------------
# Classifier
# ---------------------------------------------------------------------------


# ---------------------------------------------------------------------------
# Default infrastructure patterns
# ---------------------------------------------------------------------------

# Common infrastructure paths that apply to ANY Python project using devx.
# Projects inherit these automatically and only need to specify project-specific
# paths in their [tool.devx.classify] section.
#
# Rationale: these files/directories are development tooling, CI/CD config,
# or generated artifacts. Changes to them don't affect the installed package's
# behavior, so they don't warrant a release.
DEFAULT_INFRASTRUCTURE: list[str] = [
    # CI/CD workflow definitions
    ".gitea/**",
    ".github/**",
    # Test files
    "tests/**",
    # Documentation
    "docs/**",
    # Git hooks
    "hooks/**",
    # Build tooling
    "Makefile",
    "cliff.toml",
    "uv.lock",
    # Linting / formatting config
    ".pre-commit-config.yaml",
    ".ruff.toml",
    ".ansible-lint",
    ".checkmake.ini",
    ".editorconfig",
    # Environment templates (not the actual .env which is gitignored)
    ".env.example",
    # Git config
    ".gitignore",
    ".gitattributes",
    # Project-level documentation (not part of the installed package)
    "AGENTS.md",
    "README.md",
    "CHANGELOG.md",
    "TROUBLESHOOTING.md",
    "CONTRIBUTING.md",
    "CODE_OF_CONDUCT.md",
    "REVIEW_CHECKLIST.md",
    # Agent/CI tooling config (not part of the installed package)
    ".devin/**",
    # Generated venv activation scripts (created by `make setup`)
    "activate.sh",
    "activate.fish",
    "activate.zsh",
    # CI task tracking file (written by CI, not by developers)
    ".taskid",
]


@dataclass
class ClassifierConfig:
    """Configuration for the change classifier.

    Loaded from ``[tool.devx.classify]`` in ``pyproject.toml``.

    By default, the framework's ``DEFAULT_INFRASTRUCTURE`` patterns are
    merged with the project's ``infrastructure`` list.  Set
    ``use_defaults = false`` to disable defaults and specify all
    patterns explicitly.

    Attributes:
        infrastructure: Glob patterns for infrastructure paths
            (merged with DEFAULT_INFRASTRUCTURE unless use_defaults is False).
        infrastructure_overrides: Exact paths that are infrastructure
            despite not matching any infrastructure pattern.
        user_facing_overrides: Exact paths that are user-facing
            despite matching an infrastructure pattern (safety override).
        tags: Dict mapping tag name to list of glob patterns.
        use_defaults: If True (default), merge with DEFAULT_INFRASTRUCTURE.
    """

    infrastructure: list[str] = field(default_factory=list)
    infrastructure_overrides: list[str] = field(default_factory=list)
    user_facing_overrides: list[str] = field(default_factory=list)
    tags: dict[str, list[str]] = field(default_factory=dict)
    use_defaults: bool = True

    @classmethod
    def from_pyproject(cls, pyproject_path: str = "pyproject.toml") -> ClassifierConfig:
        """Load classifier config from pyproject.toml.

        Reads the ``[tool.devx.classify]`` section.  If the section or
        file is missing, returns a config with only DEFAULT_INFRASTRUCTURE
        (everything else defaults to user-facing — safe-by-default).
        """
        path = Path(pyproject_path)
        if not path.exists():
            return cls(infrastructure=list(DEFAULT_INFRASTRUCTURE))
        with open(path, "rb") as f:  # noqa: PTH123
            data: dict[str, Any] = tomllib.load(f)
        classify_cfg = data.get("tool", {}).get("devx", {}).get("classify", {})

        use_defaults = classify_cfg.get("use_defaults", True)
        project_infra = list(classify_cfg.get("infrastructure", []))

        if use_defaults:
            # Merge defaults with project-specific patterns (deduplicated)
            merged = list(DEFAULT_INFRASTRUCTURE)
            for p in project_infra:
                if p not in merged:
                    merged.append(p)
            infrastructure = merged
        else:
            infrastructure = project_infra

        return cls(
            infrastructure=infrastructure,
            infrastructure_overrides=list(classify_cfg.get("infrastructure_overrides", [])),
            user_facing_overrides=list(classify_cfg.get("user_facing_overrides", [])),
            tags={k: list(v) for k, v in classify_cfg.get("tags", {}).items()},
            use_defaults=use_defaults,
        )


class ChangeClassifier:
    """Classify changed files as user-facing or infrastructure.

    Uses layered rules with safe-by-default semantics.

    Rule evaluation order (first match wins):
      1. User-facing overrides (safety — highest priority)
      2. Infrastructure overrides
      3. Infrastructure patterns
      4. Default: user-facing (safe)
    """

    def __init__(self, config: ClassifierConfig | None = None) -> None:
        self.config = config or ClassifierConfig.from_pyproject()
        # Pre-compile infrastructure patterns for efficiency
        self._infra_patterns = list(self.config.infrastructure)
        self._infra_overrides = set(self.config.infrastructure_overrides)
        self._user_overrides = set(self.config.user_facing_overrides)

    def classify_file(self, file_path: str) -> FileClassification:
        """Classify a single file path.

        Returns a FileClassification with the decision and reason.
        """
        tags = self._compute_tags(file_path)

        # 1. User-facing overrides (highest priority — safety)
        if file_path in self._user_overrides:
            return FileClassification(
                path=file_path,
                is_user_facing=True,
                reason="User-facing override (safety override)",
                matched_rule="user_facing_overrides",
                tags=tags,
            )

        # 2. Infrastructure overrides
        if file_path in self._infra_overrides:
            return FileClassification(
                path=file_path,
                is_user_facing=False,
                reason="Infrastructure override (explicitly listed)",
                matched_rule="infrastructure_overrides",
                tags=tags,
            )

        # 3. Infrastructure patterns
        for pattern in self._infra_patterns:
            if _matches_glob(file_path, pattern):
                return FileClassification(
                    path=file_path,
                    is_user_facing=False,
                    reason=f"Infrastructure (matches '{pattern}')",
                    matched_rule=f"infrastructure: {pattern}",
                    tags=tags,
                )

        # 4. Default: user-facing (safe-by-default)
        return FileClassification(
            path=file_path,
            is_user_facing=True,
            reason="User-facing (default — not in infrastructure patterns)",
            matched_rule=None,
            tags=tags,
        )

    def classify(self, files: list[str]) -> ClassificationResult:
        """Classify a list of changed files.

        Returns a ClassificationResult with per-file details and
        aggregated lists.
        """
        result = ClassificationResult()
        all_tags: dict[str, list[str]] = {}

        for f in files:
            fc = self.classify_file(f)
            result.files.append(fc)
            if fc.is_user_facing:
                result.user_facing.append(f)
            else:
                result.infrastructure.append(f)
            for tag in fc.tags:
                all_tags.setdefault(tag, []).append(f)

        result.tags = all_tags
        return result

    def _compute_tags(self, file_path: str) -> frozenset[str]:
        """Compute custom category tags for a file path."""
        matched: set[str] = set()
        for tag_name, patterns in self.config.tags.items():
            for pattern in patterns:
                if _matches_glob(file_path, pattern):
                    matched.add(tag_name)
                    break
        return frozenset(matched)


# ---------------------------------------------------------------------------
# Git helpers
# ---------------------------------------------------------------------------


def run_git(args: list[str]) -> str:
    """Run a git command and return stdout."""
    result = subprocess.run(  # nosec B603
        args,
        capture_output=True,
        text=True,
        check=False,
    )
    if result.returncode != 0:
        raise click.ClickException(
            _("git command failed ({cmd}): {stderr}", cmd=" ".join(args), stderr=result.stderr.strip())
        )
    return result.stdout.strip()


def get_changed_files(base: str, head: str) -> list[str]:
    """Get list of files changed between base and head refs."""
    output = run_git(["git", "diff", "--name-only", base, head])
    if not output:
        return []
    return output.split("\n")


# ---------------------------------------------------------------------------
# Backward-compatible API (used by release.py and CI workflows)
# ---------------------------------------------------------------------------

# Singleton classifier — loaded lazily from pyproject.toml
_classifier: ChangeClassifier | None = None


def _get_classifier() -> ChangeClassifier:
    """Get or create the singleton classifier from pyproject.toml."""
    global _classifier  # noqa: PLW0603
    if _classifier is None:
        _classifier = ChangeClassifier()
    return _classifier


def is_workflow_only(
    file_path: str,
    patterns: frozenset[str] | None = None,
) -> bool:
    """Check if a file path is infrastructure (not user-facing).

    Backward-compatible API.  Prefer ``ChangeClassifier.classify_file()``
    for new code.

    Args:
        file_path: Path relative to repo root.
        patterns: Deprecated.  If provided, uses simple prefix matching
            against these patterns instead of the config-driven classifier.
    """
    if patterns is not None:
        # Legacy mode — simple prefix matching
        return any(file_path.startswith(p) or file_path == p for p in patterns)
    return not _get_classifier().classify_file(file_path).is_user_facing


def is_user_facing(
    file_path: str,
    patterns: frozenset[str] | None = None,
) -> bool:
    """Check if a file path is user-facing (affects the released package).

    Inverse of ``is_workflow_only()``.
    """
    return not is_workflow_only(file_path, patterns)


def classify_changes(
    files: list[str],
    patterns: frozenset[str] | None = None,
) -> dict[str, list[str]]:
    """Classify changed files into user-facing and workflow-only.

    Returns a dict with keys "user_facing" and "workflow_only".
    """
    if patterns is not None:
        # Legacy mode
        user_facing: list[str] = []
        workflow_only: list[str] = []
        for f in files:
            if is_user_facing(f, patterns):
                user_facing.append(f)
            else:
                workflow_only.append(f)
        return {"user_facing": user_facing, "workflow_only": workflow_only}

    result = _get_classifier().classify(files)
    return {"user_facing": result.user_facing, "workflow_only": result.infrastructure}


def has_user_facing_changes(
    base: str,
    head: str,
    patterns: frozenset[str] | None = None,
) -> bool:
    """Check if any user-facing files changed between base and head.

    Imported by ``devx.ci.release`` to decide whether a release is needed.
    """
    files = get_changed_files(base, head)
    if patterns is not None:
        return any(is_user_facing(f, patterns) for f in files)
    return _get_classifier().classify(files).has_user_facing


# ---------------------------------------------------------------------------
# Gitea Actions output
# ---------------------------------------------------------------------------


def _write_github_output(key: str, value: str) -> None:
    """Append a key=value line to the $GITHUB_OUTPUT file."""
    gh_output = os.environ.get("GITHUB_OUTPUT")
    if not gh_output:
        raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
    with open(gh_output, "a") as f:  # noqa: PTH123
        f.write(f"{key}={value}\n")


# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------


@click.command()
@click.option("--base", default=None, help="Base ref (default: latest tag).")
@click.option("--head", default="HEAD", help="Head ref (default: HEAD).")
@click.option("--quiet", is_flag=True, default=False, help="Only output true/false.")
@click.option(
    "--check",
    default="all",
    help="Check specific category: 'all' (default), 'user-facing', or any tag name "
    "defined in [tool.devx.classify.tags] (e.g., 'ansible').",
)
@click.option(
    "--github-output",
    "github_output",
    is_flag=True,
    default=False,
    help="Write results to $GITHUB_OUTPUT file (for CI workflow steps). "
    "Outputs 'user-facing-changed' and '<tag>-changed' for each configured tag.",
)
def main(base: str | None, head: str, quiet: bool, check: str, github_output: bool) -> None:
    """Classify git changes and output results."""
    classifier = _get_classifier()
    available_tags = list(classifier.config.tags.keys())

    if base is None:
        base = get_latest_tag()
        if not base:
            if github_output:
                _write_github_output("user-facing-changed", "true")
                for tag in available_tags:
                    _write_github_output(f"{tag}-changed", "true")
                click.echo("No tags found — treating all changes as user-facing.")
                return
            if quiet:
                click.echo("true")
            else:
                click.echo(_("No tags found — treating all changes as user-facing."))
            return

    files = get_changed_files(base, head)
    if not files:
        if github_output:
            _write_github_output("user-facing-changed", "false")
            for tag in available_tags:
                _write_github_output(f"{tag}-changed", "false")
            click.echo(f"No changes between {base} and {head}.")
            return
        if quiet:
            click.echo("false")
        else:
            click.echo(_("No changes between {base} and {head}.", base=base, head=head))
        return

    result = classifier.classify(files)

    if github_output:
        _write_github_output("user-facing-changed", "true" if result.has_user_facing else "false")
        for tag in available_tags:
            _write_github_output(f"{tag}-changed", "true" if result.has_tag(tag) else "false")
        click.echo(f"User-facing files changed: {result.has_user_facing}")
        for tag in available_tags:
            click.echo(f"{tag.capitalize()} files changed: {result.has_tag(tag)}")
        return

    # --check: check a specific tag or user-facing
    if check != "all":
        if check == "user-facing":
            checked_files = result.user_facing
            has_checked = bool(checked_files)
            label = "User-facing"
        elif check in available_tags:
            checked_files = result.tags.get(check, [])
            has_checked = bool(checked_files)
            label = check.capitalize()
        else:
            raise click.ClickException(
                _(
                    "Unknown check category '{check}'. Available: all, user-facing{tags}",
                    check=check,
                    tags=", " + ", ".join(available_tags) if available_tags else "",
                )
            )
        if quiet:
            click.echo("true" if has_checked else "false")
            return
        click.echo(_("\n{label} files changed ({count}):", label=label, count=len(checked_files)))
        for f in checked_files:
            click.echo(f"  {f}")
        click.echo(
            _("\nResult: {status}", status=f"{label} changes detected" if has_checked else f"No {label} changes")
        )
        return

    has_user = result.has_user_facing

    if quiet:
        click.echo("true" if has_user else "false")
        return

    click.echo(_("Comparing {base}..{head} ({count} files changed)", base=base, head=head, count=len(files)))
    click.echo(_("\nUser-facing changes ({count}):", count=len(result.user_facing)))
    for f in result.user_facing:
        click.echo(f"  {f}")
    click.echo(_("\nWorkflow-only changes ({count}):", count=len(result.infrastructure)))
    for f in result.infrastructure:
        click.echo(f"  {f}")
    for tag in available_tags:
        tag_files = result.tags.get(tag, [])
        if tag_files:
            click.echo(_("\n{tag} files ({count}):", tag=tag.capitalize(), count=len(tag_files)))
            for f in tag_files:
                click.echo(f"  {f}")
    if has_user:
        status = "USER-FACING changes detected — release needed"
    else:
        status = "Workflow-only changes — no release needed"
    click.echo(_("\nResult: {status}", status=status))

    if not has_user:
        sys.exit(2)  # Exit code 2 = workflow-only (used by CI to skip release)


if __name__ == "__main__":  # pragma: no cover
    main()