How SkillCheck checks a skill
Every check traces to a published spec, a research paper or documented practitioner experience, not to opinion. When we add one, we document what informed it and why it matters.
Built on: Anthropic guidance · W3C WCAG 2.2 · OWASP Top 10 for Agentic Applications · OpenAI · Google Cloud · the MCP specification · Hasan et al. and Wang et al. (arXiv) · all 22 sources
Scores and grades
Every skill starts at 100 points. Each finding takes points off, depending on how serious it is.
| Finding | Points | What it means |
|---|---|---|
| Critical | −20 | Breaks the skill or makes it unsafe. Fix before shipping. |
| Warning | −5 | A quality gap an agent or a user will notice. Should fix. |
| Suggestion | −1 | Worth improving. Suggestions cost at most 15 points in total, so a long, thorough skill is not punished for ordinary wording. |
| Strength | 0 | Something done well, such as a strong gotchas section. Shown in the report; it does not change the score. |
From score to grade
| Grade | Score | What it means |
|---|---|---|
| Excellent | 90–100 | Clear and safe to depend on. Only small suggestions left. |
| Good | 75–89 | Works well. A few warnings worth fixing. |
| Needs Work | 60–74 | Usable, but with problems an agent or a user will notice. |
| Poor | 0–59 | Fix before sharing. |
A Critical finding caps the grade at Needs Work. One Critical costs 20 points, so an otherwise clean skill would still score 80. The cap stops a report from calling a skill Good while it lists a problem that must be fixed first. The same four grades appear in the command line, in reports and on this site.
How a check works
Most checks match published patterns line by line, so the same skill always gets the same result.
Present or absent
Required fields, file references, secrets, token counts. Exact: pass or fail.
Structure · Body · Security · Token budget · Trigger collision
Named patterns
Filler phrases, density signals, design patterns, governance checklists. Read the rules and you can predict the result.
Naming · Anti-slop · Enterprise · OWASP deterministic · Knowledge density
Reading comprehension
Contradictions, workflow clarity, autonomy boundaries. Scored against a published rubric, so the tolerance is wider.
Semantics · Workflow · Autonomy design · OWASP graders
What we hold the checks to
Standards, not opinions
Every check traces to a published spec, a research paper or documented practitioner experience. We record what informed it and why it matters.
Layers, in order
Structure first, then meaning, then content quality, security and agent readiness.
Reward what is good
A strong gotchas section or clear error handling shows up in the report as a strength, not only as missing penalties.
Where the checks come from
SkillCheck grew in eleven phases. Each one added checks grounded in a new, independent source. Open a phase to see what informed it.
01Build to specv1.0 – v1.4 · Dec 2025Categories 1–10
Foundation checks built directly from Anthropic's official guidance: frontmatter fields, trigger conditions, scope boundaries, testing scenarios. WCAG 2.2 informed accessibility; MCP Best Practices established protocol compliance.
Categories 1–10: Structure · Body · Naming · Semantics · Anti-Slop · Visual · Security · Quality · Token · Enterprise
- Anthropic Claude Code Best PracticesFrontmatter schema, trigger conditions, scope boundaries, testing scenarios
- MCP Protocol MCP Best PracticesProtocol compliance, tool-list format, server registration
- W3C WCAG 2.2Colour contrast thresholds, alt-text guidance, accessibility output checks
02Build from practicev3.10 – v3.14 · Mar 2026Categories 11–18
Practitioner insights from shipping skills at scale. Field observations from the Claude Code team introduced description-as-trigger-condition, railroading detection, and gotchas as a quality signal. Agent Readiness added 6 pillars and 28 checks for multi-agent orchestration.
Categories 11–18: Quality Pro · Workflow · Reference Integrity · Eval Readiness · Orchestration Safety · Autonomy Design · Composability · Observability
- Anthropic Lessons from Building Claude Code (Thariq Shihipar)Description-as-trigger condition, railroading detection, gotchas as quality signal, Agent Readiness pillars
03Build from the fieldv3.16 – v3.17 · Apr 2026Categories 19–21
Cross-lab methodology. OpenAI's systematic eval approach informed the Eval Kit. Google's ADK agent-type hierarchy inspired SkillCheck's design-pattern classification. CI/CD integration brought SkillCheck into GitHub Actions.
Categories 19–21: Design Pattern · Trigger Collision · Eval Kit
- OpenAI Testing Agent Skills Systematically with EvalsShould-trigger / should-NOT-trigger prompt pairs, negative controls per validated skill
- Google 5 Agent Skill Design Patterns (Google Cloud Tech)Reviewer / Generator / Inversion / Pipeline / Tool Wrapper classification
04Build from the ecosystemv3.18.0 · Apr 2026Category 22
Academic research and community tools. Hasan et al. found 97% of 856 MCP tools contained a description smell. Wang et al.'s 18-category taxonomy showed standard-compliant descriptions hit 72% selection probability vs a 20% baseline.
Category 22: Knowledge Density
- Hasan et al. · arXiv MCP Tool Descriptions Are Smelly!Description smell taxonomy, 97% prevalence in 856 public MCP tools
- Wang et al. · arXiv From Docs to Descriptions18-category taxonomy; compliant descriptions → 72% selection vs 20% baseline
- Glama Tool Description Quality Score (TDQS)Community-scale calibration of description quality signals
05Build for the marketplacev3.20.0 · Apr 2026Categories 23–25
Scope widens from SKILL.md to three artifact types. Cat 23 evaluates MCP servers on four axes. Cat 24 checks plugin manifests against the Anthropic schema. A 21-plugin benchmark confirmed calibration: zero criticals, but 0/21 declare maintainers or evals.
Categories 23–25: Agent Integration Readiness · Skill Marketplace Governance · Memory Governance (design)
- Anthropic Building Agents That Reach Production with MCPMCP server governance axes: token efficiency, security, unique unlocks, execution environment
- Sam Morrow MCP vs CLIsCLI literacy checks, when a tool wraps a CLI vs implements its own protocol
- Anthropic anthropics/knowledge-work-pluginsReference schema for plugin manifests; benchmark of 21 real plugins
06Build for agentic safetyv3.21.0 · Jun 2026Category 26
Cat 26 checks a skill against the OWASP Top 10 for Agentic Applications (2026). Eight deterministic risks run free and local, in CI and the browser. Three intent-based risks are Pro graders your own AI judges, with no API key required.
Category 26: OWASP Agentic Top 10: tool over-grant · identity override · unpinned deps · code execution · unbounded loops · ungated destructive actions · disabled safeguards · no audit trail · goal hijacking (Pro) · memory poisoning (Pro) · insecure inter-agent comms (Pro)
- OWASP Top 10 for Agentic Applications 2026All 11 risk categories: 8 mapped to deterministic checks, 3 to AI-judge graders
07Measure lived-in-ness, not just craftv3.22.0 · Jun 2026Category 27
Cat 27 reads a skill's git history to answer what the SKILL.md text can't: is anyone actually using this, or was it written once and abandoned? Commit count, recency, and commit-day spread flag skills written once and never revised. It scores in its own domain, so a one-commit skill can still be excellent.
Category 27: Repo Maturity: commit count · recency · single-sitting · actively-iterated (strength). Pro, git-history.
- Vanishing Gradients Show Us Your (Agent) SkillsSkepticism toward one-off skills: many published skills carry a single commit, never iterated by an author who depends on them
08Read the whole skill, reward restraintv3.24 – v3.26 · Jun–Jul 2026Category 28
Cat 28 rewards skills that say 'keep it simple' AND 'never skip validation, security, accessibility'; lazy is fine, careless is not. Reference-aware checking teaches content checks to follow a skill's references/ files instead of flagging a lean SKILL.md as incomplete: first eval-readiness (v3.25), then contracts, confidence signals, and knowledge density (v3.26). Calibrated against the July 2026 corpus rescan: 2,675 public skills, 1 in 5 shipping references/, 2 in 5 among the most popular.
Category 28: Restraint · reference-aware expansion of Categories 14, 17, 18, 22
- Dietrich Gebert ponytail ('lazy senior dev' skill)Tests showed 'do less' instructions cut code but sometimes dropped a safety guard; restraint needs a carveout
- SkillCheck Full-corpus rescan, July 2026 (117 published reports)references/ adoption at 21% corpus-wide vs 41% among featured skills; validators that stop at SKILL.md tax the converging layout
09Close the slop-coverage gapsv3.27 – v3.28 · Jul 2026Category 5 deepened
No new category: two rounds of deepening Category 5 against external catalogues of AI writing rather than against taste. v3.27 ported cluster-mined patterns: an AI-vocabulary list where a single word family stays a Suggestion but two co-occurring families escalate every hit, plus significance inflation, cross-sentence antithesis, and corporate/internet cliché lists. v3.28 audited coverage against Wikipedia's catalogue, found nine applicable categories with no pattern behind them, and closed four. One of them, chat-UI citation markup pasted into a skill, is the first Category 5 pattern scored Critical rather than Warning: every other pattern has a legitimate use, and this one is verbatim paste evidence. Five categories are deliberately left uncovered and written down as decisions, so a later reader can tell a choice from an oversight.
Category 5 deepened: chat-citation-artifact (Critical) · knowledge-cutoff · collab-communication · copula-inflation · ai-vocabulary
- Wikipedia Signs of AI writingCoverage audit target; the nine applicable categories and the four closed in v3.28, including the chat-citation artifact family
- Adam Dunkels deslop-textCluster-mined vocabulary and cliché families ported into 5.3 and 5.4
10Build for the mountv3.31.0 · Aug 2026Category 29
Cat 29 asks a repository-level question the SKILL.md text cannot answer: will these skills actually be discovered when the repository is mounted into an Anthropic Managed Agents session? The platform loads skills from exactly .claude/skills/<name>/SKILL.md, one level deep at the root, with no review step, and a skill placed one directory too deep never errors or logs, it simply never loads. Anthropic's documentation tells users to audit that directory before mounting a repository that accepts outside contributions, without naming a tool. Cat 29 is that audit: the four silent-load layouts, empty skill directories, symlinks escaping the repo, name mismatches and case collisions run free; a tamper-evident evidence manifest with a SHA-256 per mountable skill is the Pro part, so CI fails on layout regressions and content drift.
Category 29: Skill Distribution Readiness: silent-load layouts · empty skill dirs · escaping symlinks · name mismatches · case collisions · mountable-skill threshold · evidence manifest (Pro)
- Anthropic Managed Agents: Skills (documentation, snapshot 10-08-2026)The discovery contract: exact path, one level deep, scan once at session start, read tool required, audit before mounting
11See what the reviewer cannotv3.32.0 · Aug 2026Category 30
Cat 30 came out of reading a competitor. NVIDIA's SkillSpector is a security scanner for agent skills; most of it scans code, outside SkillCheck's scope, but it exposed one gap that was in scope: nothing in SkillCheck detected instructions hidden in invisible characters. A SKILL.md can carry an entire instruction in Unicode tag characters that no editor and no diff will show and every model will read. Four checks now run free and in the browser: ASCII smuggling in tag characters, with the hidden payload decoded into the finding so the reviewer sees exactly what was smuggled; trojan-source bidirectional overrides; long invisible runs as an encoding channel; and stray zero-width characters. Legitimate emoji are exempt by construction, because a flag emoji is built from the same tag block as the attack.
Category 30: Invisible Unicode: ascii-smuggling (Critical, payload decoded) · bidi-override (Critical) · invisible-run · invisible-char
- NVIDIA SkillEvaluatorInvisible-codepoint categories and run-length thresholds (Apache-2.0)
- NVIDIA SkillSpectorThe in-scope gap that started the category, and the emoji tag-sequence carve-out
- Boucher & Anderson Trojan Source (CVE-2021-42574)Bidirectional-override attacks where displayed text differs from parsed text
All sources
Every source that informed a check, with what it informed and the phase it belongs to.
| Source | What it informed | Phase |
|---|---|---|
| Anthropic Claude Code Best Practices | Frontmatter schema, trigger conditions, scope boundaries, testing scenarios | 01 |
| MCP Protocol MCP Best Practices | Protocol compliance, tool-list format, server registration | 01 |
| W3C WCAG 2.2 | Colour contrast thresholds, alt-text guidance, accessibility output checks | 01 |
| Anthropic Lessons from Building Claude Code (Thariq Shihipar) | Description-as-trigger condition, railroading detection, gotchas as quality signal, Agent Readiness pillars | 02 |
| OpenAI Testing Agent Skills Systematically with Evals | Should-trigger / should-NOT-trigger prompt pairs, negative controls per validated skill | 03 |
| Google 5 Agent Skill Design Patterns (Google Cloud Tech) | Reviewer / Generator / Inversion / Pipeline / Tool Wrapper classification | 03 |
| Hasan et al. · arXiv MCP Tool Descriptions Are Smelly! | Description smell taxonomy, 97% prevalence in 856 public MCP tools | 04 |
| Wang et al. · arXiv From Docs to Descriptions | 18-category taxonomy; compliant descriptions → 72% selection vs 20% baseline | 04 |
| Glama Tool Description Quality Score (TDQS) | Community-scale calibration of description quality signals | 04 |
| Anthropic Building Agents That Reach Production with MCP | MCP server governance axes: token efficiency, security, unique unlocks, execution environment | 05 |
| Sam Morrow MCP vs CLIs | CLI literacy checks, when a tool wraps a CLI vs implements its own protocol | 05 |
| Anthropic anthropics/knowledge-work-plugins | Reference schema for plugin manifests; benchmark of 21 real plugins | 05 |
| OWASP Top 10 for Agentic Applications 2026 | All 11 risk categories: 8 mapped to deterministic checks, 3 to AI-judge graders | 06 |
| Vanishing Gradients Show Us Your (Agent) Skills | Skepticism toward one-off skills: many published skills carry a single commit, never iterated by an author who depends on them | 07 |
| Dietrich Gebert ponytail ('lazy senior dev' skill) | Tests showed 'do less' instructions cut code but sometimes dropped a safety guard; restraint needs a carveout | 08 |
| SkillCheck Full-corpus rescan, July 2026 (117 published reports) | references/ adoption at 21% corpus-wide vs 41% among featured skills; validators that stop at SKILL.md tax the converging layout | 08 |
| Wikipedia Signs of AI writing | Coverage audit target; the nine applicable categories and the four closed in v3.28, including the chat-citation artifact family | 09 |
| Adam Dunkels deslop-text | Cluster-mined vocabulary and cliché families ported into 5.3 and 5.4 | 09 |
| Anthropic Managed Agents: Skills (documentation, snapshot 10-08-2026) | The discovery contract: exact path, one level deep, scan once at session start, read tool required, audit before mounting | 10 |
| NVIDIA SkillEvaluator | Invisible-codepoint categories and run-length thresholds (Apache-2.0) | 11 |
| NVIDIA SkillSpector | The in-scope gap that started the category, and the emoji tag-sequence carve-out | 11 |
| Boucher & Anderson Trojan Source (CVE-2021-42574) | Bidirectional-override attacks where displayed text differs from parsed text | 11 |
Independence
SkillCheck is an independent project, not affiliated with or endorsed by Anthropic, OpenAI, Google or any other AI lab.
Research from those organizations informed specific check categories, as listed in the phases above. The implementation, the scoring and the quality judgments are SkillCheck's own.
Run the checks on your own skill.
The free check runs in your browser. Pro adds security, filler text, accessibility and agent readiness.