devx/.devin/skills/skill-creation/SKILL.md

skill-creation

How to create, validate, and maintain Devin skills. Skills must be clear, succinct, and actionable — no AI slop.

When to Invoke

Invoke this skill when:

• Creating a new skill
• Amending an existing skill
• Evaluating whether a skill is needed
• Reviewing a PR that adds or modifies skills

Prerequisites

• Skill directory: .devin/skills/<skill-name>/SKILL.md
• Validator: tests/test_skills.py (infra) or reference to it
• Tests: tests/unit/test_skills_validation.py (infra)

When to Create a Skill

Create a skill when:

• An agent struggles with a task repeatedly (branch hygiene, PR order)
• A workflow has non-obvious ordering constraints (deployment coordination)
• A task requires specific tool usage over raw commands (CI monitoring)
• Multiple agents need shared context (dependency graph)

Do NOT create a skill for:

• One-off tasks (use a spec instead)
• Tasks already covered by AGENTS.md
• Tasks that are obvious from the Makefile or README
• Tasks that change frequently (skills should be stable)

Skill Structure

Every skill MUST have:

# <skill-name>

One-line description of what the skill does.

## When to Invoke

2-4 bullet points describing when to use this skill.

## Prerequisites

What must exist before using the skill (venv, .env, tools).

## <Core Content>

The actual guidance. Keep it actionable.

## Verification (if applicable)

How to verify the skill's guidance works.

## Common Mistakes (if applicable)

What agents get wrong without this skill.

Quality Standards

Do

• Be specific. Reference exact make targets, file paths, commands.
• Be concise. Each section should be scannable in under 30 seconds.
• Be actionable. Every paragraph should tell the agent what to DO.
• Use tables for command reference, mappings, and comparisons.
• Use code blocks for commands the agent should run.
• Link to other skills when related (e.g., "See dependency-graph skill").

Don't

• No preamble. Don't start with "This skill helps agents..." — just state what it does.
• No filler. Don't repeat information from AGENTS.md or other skills.
• No vague advice. "Be careful with branches" is useless. "Run git branch --show-current before every commit" is useful.
• No AI slop. Don't write "In this comprehensive guide, we will explore..." — just give the guidance.
• No redundant sections. If "Common Mistakes" would repeat "When to Invoke", skip it.
• No marketing. Don't describe the skill as "powerful" or "comprehensive".

Scope Rules

• One skill per concern. Don't mix branch hygiene with CI monitoring.
• Project-specific, not generic. Skills reference this repo's make targets, file paths, and conventions — not abstract advice.
• Shared skills must be identical across repos. Use SHARED_SKILLS in the validator to enforce this.
• Per-repo skills must reflect that repo's reality. Don't copy infra-specific targets to sso-bridge.

Automated Validation

Every skill must pass the validator (tests/test_skills.py). The validator checks:

1. Structure — H1 title, "When to Invoke" section, "Prerequisites" section
2. Commands — referenced make <target> commands exist in Makefile or devx.mak
3. Paths — referenced file paths exist in the repo
4. Shared skills — identical content across repos (SHA-256 comparison)
5. No drift — no references to nonexistent commands or files

Run the validator:

python3 tests/test_skills.py --repo infra --repo sso-bridge

Effectiveness Evaluation

Static Checks (automated, CI)

The validator runs in CI as part of make pytest-cov. A failing skill test blocks the PR. This catches:

• Missing sections
• Invalid commands
• Broken file references
• Cross-repo drift

Runtime Metrics (manual, periodic)

Track these signals to evaluate skill effectiveness:

• Skill invocation frequency — how often agents invoke the skill
• Success rate when invoked — did the skill prevent the mistake it targets?
• Feedback issues — agents create Gitea issues with feedback label when a skill is unclear or wrong
• Mistake recurrence — if agents still make the mistake the skill targets, the skill needs improvement

Retrospective Review

Periodically (monthly or after major incidents) review skills:

1. List all skills and their last-modified dates
2. Check for feedback issues tagged skill-improvement
3. Verify referenced commands still exist (run validator)
4. Remove skills that are no longer relevant
5. Update skills where mistakes still recur
6. Document lessons in this skill's "Common Mistakes" section

Creating a New Skill — Checklist

• Identify the repeated struggle or non-obvious workflow
• Check no existing skill covers it
• Write the skill following the structure above
• Run python3 tests/test_skills.py — must pass
• Run make pytest-cov — must pass with 100% coverage
• If shared across repos, copy identical content to each repo
• Add the skill to SHARED_SKILLS in the validator if shared
• Create PR, verify CI passes, merge