NVIDIA/SkillEvaluator
Repository · https://github.com/nvidia/skillevaluator
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
First seen by Oriel on .
AI summary
SkillEvaluator is NVIDIA’s multi-tier CLI and CI framework for validating AI agent skills, detecting overlap, generating evaluation datasets, and measuring live behavioral impact.
- Tier 1 checks safety, structure, harness compatibility, and static quality using scanners and optional LLM analysis.
- Tier 2 uses embeddings and chat to identify repeated or overlapping guidance within and across skills.
- Tier 3 uses Harbor sandboxed agent runs to assess whether a skill improves task outcomes, requiring provider, agent, and sandbox access.
- The bundled text-analyzer needs no API key and scored 86.2/100 (B), while incomplete scanner or evaluation stages can block validation.
- The project supports direct CLI use, GitHub or GitLab CI, and integration with Claude Code, Codex, or other skill-aware agents.
#ai agents #skill evaluation #github actions #llm testing
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Stars
Total repository stars at each observation.
- GitHub Trending: 2,991, latest observation ; +1,683 over observed period since
Stars gained (1d)
Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.
- GitHub Trending: 873, latest observation ; +740 over observed period since
Forks
Total repository forks at each observation.
- GitHub Trending: 423, latest observation ; +269 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| GitHub Trending | nvidia/skillevaluator | https://github.com/NVIDIA/SkillEvaluator Read discussion |