name: skill_evaluator
router_kit: FullStackKit
description: Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, assess, or audit a skill for quality. Analyzes SKILL.md structure, naming conventions, description quality, content organization, and identifies anti-patterns. Produces actionable improvement recommendations.
metadata:
skillport:
category: skills
tags: [architecture, audit, automation, best practices, clean code, coding, collaboration, compliance, debugging, design patterns, development, documentation, efficiency, git, metrics, optimization, productivity, programming, project management, quality assurance, quality check, refactoring, review, skill evaluator, software engineering, standards, testing, utilities, version control, workflow]

Skill Evaluator (WIP)

Evaluates skills against Anthropic's official best practices for agent skill authoring. Produces structured evaluation reports with scores and actionable recommendations.

Quick Start

1. Read the skill's SKILL.md and understand its purpose
2. Run automated validation: scripts/validate_skill.py <skill-path>
3. Perform manual evaluation against criteria below
4. Generate evaluation report with scores and recommendations

Evaluation Workflow

Step 1: Automated Validation

Run the validation script first:

scripts/validate_skill.py <path/to/skill>

This checks:
- SKILL.md exists with valid YAML frontmatter
- Name follows conventions (lowercase, hyphens, max 64 chars)
- Description is present and under 1024 chars
- Body is under 500 lines
- File references are one-level deep

Step 2: Manual Evaluation

Evaluate each dimension and assign a score (1-5):

A. Naming (Weight: 10%)

Rules: Max 64 chars, lowercase + numbers + hyphens only, no reserved words (anthropic, claude), no XML tags.

Good: processing-pdfs, analyzing-spreadsheets, building-dashboards
Bad: pdf, my-skill, ClaudeHelper, anthropic-tools

B. Description (Weight: 20%)

Must include: What the skill does AND when to use it.
Good: "Extracts text from PDFs. Use when working with PDF documents for text extraction, form parsing, or content analysis."
Bad: "A skill for PDFs." or "Helps with documents."

C. Content Quality (Weight: 30%)

Ask: "Does Claude really need this explanation?" Remove anything Claude already knows.

D. Structure & Organization (Weight: 25%)

Check:
- SKILL.md under 500 lines
- References are one-level deep (no nested chains)
- Long reference files (>100 lines) have table of contents
- Uses forward slashes in all paths

E. Degrees of Freedom (Weight: 10%)

Guideline:
- High freedom (text): Multiple valid approaches, context-dependent
- Medium freedom (parameterized): Preferred pattern exists, some variation OK
- Low freedom (specific scripts): Fragile operations, exact sequence required

F. Anti-Pattern Check (Weight: 5%)

Deduct points for each anti-pattern found:

• [ ] Too many options without clear recommendation (-1)
• [ ] Time-sensitive information with date conditionals (-1)
• [ ] Inconsistent terminology (-1)
• [ ] Windows-style paths (backslashes) (-1)
• [ ] Deeply nested references (more than one level) (-2)
• [ ] Scripts that punt error handling to Claude (-1)
• [ ] Magic numbers without justification (-1)

Step 3: Generate Report

Use this template:

# Skill Evaluation Report: [skill-name]

## Summary
- **Overall Score**: X.X/5.0
- **Recommendation**: [Ready for publication / Needs minor improvements / Needs major revision]

## Dimension Scores
| Dimension          | Score | Weight | Weighted |
| ------------------ | ----- | ------ | -------- |
| Naming             | X/5   | 10%    | X.XX     |
| Description        | X/5   | 20%    | X.XX     |
| Content Quality    | X/5   | 30%    | X.XX     |
| Structure          | X/5   | 25%    | X.XX     |
| Degrees of Freedom | X/5   | 10%    | X.XX     |
| Anti-Patterns      | X/5   | 5%     | X.XX     |
| **Total**          |       | 100%   | **X.XX** |

## Strengths
- [List 2-3 things done well]

## Areas for Improvement
- [List specific issues with actionable fixes]

## Anti-Patterns Found
- [List any anti-patterns detected]

## Recommendations
1. [Priority 1 fix]
2. [Priority 2 fix]
3. [Priority 3 fix]

## Pre-Publication Checklist
- [ ] Description is specific with activation triggers
- [ ] SKILL.md under 500 lines
- [ ] One-level-deep file references
- [ ] Forward slashes in all paths
- [ ] No time-sensitive information
- [ ] Consistent terminology
- [ ] Concrete examples provided
- [ ] Scripts handle errors explicitly
- [ ] All configuration values justified
- [ ] Required packages listed
- [ ] Tested with Haiku, Sonnet, Opus

Score Interpretation

References

• references/evaluation-criteria.md - Detailed evaluation criteria with examples
• references/scoring-rubric.md - Complete scoring rubric and edge cases

Skill Evaluator v1.1 - Enhanced

🔄 Workflow

Source: Google Engineering Practices - Code Review & Anthropic System Prompts

Phase 1: Structural Analysis

• [ ] Compliance: Does file structure (scripts/, references/) comply with standards?
• [ ] Metadata: Is YAML frontmatter (name, description) complete and valid?
• [ ] Modularity: Is the skill too large? Does it need to be split? (Single Responsibility Principle).

Phase 2: Content & Semantic Review

• [ ] Clarity: Are instructions written in imperative mode and clear? Is there ambiguity?
• [ ] Context Efficiency: Is there "unnecessary politeness" or "over-explanation"? Token waste should be prevented.
• [ ] Safety: Does the skill suggest a dangerous operation (file deletion, unauthorized access)?

Phase 3: Functionality Verification

• [ ] Script Audit: Are Python/Bash codes in scripts/ safe and working?
• [ ] Reference Check: Are references/ files really necessary? Or should they be embedded in SKILL.md?
• [ ] Usability: Can a user (or agent) read and use this skill immediately?

Checkpoints