4 Commits

Author	SHA1	Message	Date
Jan Krivanek	05aeb657e6	Add license to agent files (#568)	2026-04-21 12:57:18 +00:00
Copilot	6face98ff6	Change agents and skills fields in plugin.json to array format for Claude Code CLI compatibility (#407) ... * Initial plan * Remove agents string field from plugin.json and handle array format in parser Claude Code CLI validates `agents` as array-of-strings per its plugin manifest schema. Our plugin.json files had `"agents": "./agents/"` (a string), which caused "agents: Invalid input" validation errors on install. Remove the field from the 3 affected plugin.json files — agents are still discovered by convention from the `agents/` directory. Also make ParsePluginJson resilient to both string and array formats for forward-compatibility. Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Remove `agents` string field from plugin.json for Claude Code CLI compatibility Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Change agents field to array format in plugin.json, update validator to read array-first with string fallback Plugin.json files now declare agents as an array of file paths: "agents": ["./agents/foo.agent.md", "./agents/bar.agent.md"] ParsePluginJson reads the array first (preferred, Claude Code schema), falls back to a string path (legacy). PluginInfo gains an AgentPaths field. Validator and discovery use the array when present, else fall back to directory-based convention. Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Simplify agents to directory-path array form ["./agents/"], handle directories in array entries Per Copilot CLI spec, agents field values are "Path(s) to agent directories" — no need to list individual files. Use ["./agents/"] to mirror how skills uses "./skills/". Updated validator and discovery to handle both directory and file entries in the array. Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Delete .nuget/nuget.exe * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Convert skills field to array format in plugin.json, update validator to handle both forms Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Align AgentPaths null checks to use { Count: > 0 } pattern for consistency Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Change `agents` and `skills` fields in plugin.json to array format for Claude Code CLI compatibility Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> * Delete .nuget/nuget.exe * Consolidate SkillsPath/SkillPaths and AgentsPath/AgentPaths into single array fields Remove dual string/array fields from PluginInfo. The parser now normalizes legacy string values into single-element arrays, so all downstream code uses only SkillPaths and AgentPaths (IReadOnlyList<string>). This eliminates all array-vs-string fallback branching. Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com> Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-03-20 10:27:28 +01:00
Dan Moseley	8e50fc8558	Add agent and plugin spec conformance validation (#279) ... * Add agent and plugin spec conformance validation Extend the validation pipeline to check .agent.md files and plugin.json files alongside skills. Previously only SKILL.md files were validated. Agent validation (AgentProfiler): - Frontmatter required (error) - Name and description required (error) - Name format: same rules as skills (lowercase alphanum + hyphens, 64 max) - Name must match filename minus .agent.md suffix - Description max 1024 chars - Body max 500 lines Plugin validation (PluginValidator): - Name required, must match directory name - Version and description required - Description max 1024 chars - Skills path required and must exist - Agents path warned if specified but missing Discovery (SkillDiscovery): - DiscoverAgents() walks up from skill paths to plugin roots, scans agents/ directories for .agent.md files - DiscoverPlugins() finds plugin.json from skill paths Integration (ValidateCommand): - Runs after skill discovery, before evaluation - Errors block with exit code 1 (same as spec_conformance_failure) All 4 existing agents and 2 plugins pass cleanly. 25 new tests (13 agent, 12 plugin). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Code review fixes: spec comments, ValidateNameFormat, kind-specific messages - Refactor ValidateName into ValidateNameFormat (format-only) + ValidateName (format + directory match) so agent/plugin validation doesn't abuse the skill directory-match check by passing name twice. - Warning messages now say 'Agent name' or 'Plugin name' instead of 'Skill name' when called from AgentProfiler/PluginValidator. - Add spec URL comments to AgentProfiler and PluginValidator referencing agentskills.io/specification, VS Code agent-plugins docs, and Claude Code plugins-reference. - Add missing agent name edge-case tests: leading/trailing hyphen, consecutive hyphens, and message-prefix assertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove unused agents path from plugins that have no agents The dotnet, dotnet-data, and dotnet-upgrade plugins declared an agents path but had no agents/ directory, producing validation warnings. Remove the field; it can be added back when agents are actually created. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Promote agent/plugin name validation from warnings to errors The agentskills.io spec uses 'Must' for all name constraints (length, character set, no leading/trailing hyphens, no consecutive hyphens). For a publishing validator these should block, not just warn. Agent filename mismatch is also promoted since the IDE won't discover the agent if the name doesn't match. Skill name validation stays as warnings (lenient loading, not part of this PR). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review feedback - Fix FrontmatterRegex: remove RegexOptions.Multiline so a horizontal rule (---) in the body is not mistaken for frontmatter. Detection and stripping now both anchor to start of string. - Use filename as fallback identifier in AgentProfile when agent.Name is empty, so error output shows [agent:foo.agent.md] not [agent:]. - Add TryGetSafeSubdirectory to PluginValidator: rejects absolute paths and parent-directory traversal in skills/agents path fields. Used by both PluginValidator and SkillDiscovery.DiscoverAgents. - Use DirectoryName as fallback for plugin result name when Name is empty, so error output shows [plugin:my-dir] not [plugin:]. - Add tests for path traversal rejection and name fallback behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Detect orphaned test directories with no matching plugin/skill Add FindOrphanedTestDirectories to SkillDiscovery: checks that every directory under tests/{plugin}/{skill}/ has a corresponding directory under plugins/{plugin}/skills/{skill}/. Plugin-level orphans (tests/ directory with no matching plugins/ directory) are also flagged. Integrated into ValidateCommand after plugin/agent validation so it runs on every validation pass. Errors are blocking (exit 1). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR #279 review feedback - Use case-sensitive (Ordinal) comparison for agent filename matching since agent names are required to be lowercase by spec. - Replace OrdinalIgnoreCase path-under-root check in TryGetSafeSubdirectory with Path.GetRelativePath + '..' prefix check for correct behavior on case-sensitive filesystems. - Let ParsePluginJson throw JsonException on malformed JSON instead of silently returning null. ValidateCommand catches it as a blocking error. - Add test for malformed plugin.json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Increase eval timeouts for dump-collect and dotnet-trace-collect to 120s The with-skill runs for these skills were hitting the 60s scenario timeout in CI, causing the agent to be killed before producing output. Other skills in the same plugin already use 120s. The 30s decline/negative-test scenario is left unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix fixture paths in clr-activation-debugging and analyzing-dotnet-performance evals The source paths referenced tests/dotnet/ but the fixtures live under tests/dotnet-diag/. This caused file-not-found exceptions at runtime, which set hasRejections and caused unconditional exit 1 (bypassing --verdict-warn-only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Surface execution errors in validation summary Previously, skills that threw exceptions (e.g. missing fixture files) were silently excluded from the verdict count. The summary showed '2/4 skills passed' with no indication that 2 other skills crashed entirely. Now the summary shows '2/6 skills passed (2 rejected due to execution errors)' and lists the specific error messages near the summary rather than buried in the middle of the log. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Reduce parallelism for infra-change evaluation runs When infrastructure changes trigger evaluation of all plugins simultaneously, the high default parallelism (5x5x5) causes API rate limits (429s) and scenario timeouts from contention. Reduce to 2x3x3 for infra-change runs only; regular single-plugin PR evaluations keep the default 5x5x5. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-08 15:40:06 +01:00
Dan Moseley	042a9d2b2c	Reorganize plugins into domain-specific groupings (#274) ... * Reorganize plugins per agreed structure (#271) Split the dotnet plugin into domain-specific plugins per discussion in #271: - dotnet: narrowed to common everyday C#/.NET coding tasks - dotnet-diag: perf investigations, debugging, incident analysis - dotnet-data: data access and Entity Framework - dotnet-aspnet, dotnet-wpf, dotnet-winforms, dotnet-maui: scaffolded empty to convey intent for future framework-specific skills Moved skills, tests, and CODEOWNERS entries accordingly. No file content was changed; all skill/test moves are pure renames. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Improve dotnet-diag and dotnet-winforms descriptions - dotnet-diag: add '.NET' qualifier to clarify scope - dotnet-winforms: use official 'Windows Forms (WinForms)' name Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add .NET qualifier to dotnet-data description Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove empty plugins; list them in CONTRIBUTING.md as future options Remove the four empty placeholder plugins (dotnet-aspnet, dotnet-wpf, dotnet-winforms, dotnet-maui) that had only plugin.json and no skills. Add a 'Plugin organization' section to CONTRIBUTING.md that describes the domain-specific plugin groupings introduced in this PR and lists the four removed plugin names as reserved candidates for future contributions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Create dotnet-upgrade plugin with migration/upgrade skills Move migrate-nullable-references, thread-abort-migration, and dotnet-aot-compat from dotnet to the new dotnet-upgrade plugin. Update CODEOWNERS paths (same owners), CONTRIBUTING.md plugin table, and fix the thread-abort-migration SKILL.md link in CONTRIBUTING.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-06 21:01:52 -08:00