Commit Graph

7 Commits

Author SHA1 Message Date
Sayed Ibrahim Hashimi 7d5106946f adding scaffolding skill and test (#329)
* adding scaffolding skill and test

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Routes.razor

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* updating pr based on copilot feedback

* updating pr based on copilot feedback

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Layout/ReconnectModal.razor.css

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Layout/ReconnectModal.razor.css

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Layout/ReconnectModal.razor.css

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update plugins/dotnet-data/skills/scaffold-generate-aspnet/SKILL.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/minimal-api/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fixing warnings in test projects

* fixing ci build issues

* fixing eval.yaml

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix unresolved Copilot review comments on PR #329

- Add CODEOWNERS entries for scaffold-generate-aspnet skill and tests
- Fix navbar toggler aria-controls/id mismatch in razor-pages-crud fixture

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updating code owners for scaffold-generate-aspnet

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Pages/Error.razor

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/eval.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/fixtures/blazor-crud/Components/Pages/Error.razor

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update plugins/dotnet-data/skills/scaffold-generate-aspnet/SKILL.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/eval.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/dotnet-data/scaffold-generate-aspnet/eval.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @webreidi

* Apply suggestion from @webreidi

* Apply suggestion from @webreidi

* Add Blazor CRUD application structure and initial components

- Created Home, NotFound, and Weather pages with basic layout and functionality.
- Implemented routing in the Router component.
- Added models for Department and Employee with data annotations for validation.
- Set up Program.cs for Blazor server-side rendering and configured HTTP request pipeline.
- Included launch settings and appsettings for development environment.
- Established CSS styles for the application and added favicon.
- Created minimal API with TodoItem model and basic weather forecast endpoint.
- Developed Razor Pages CRUD structure with models for Category and Product.
- Implemented error handling and privacy policy pages in Razor Pages.

* Remove SKILL.md documentation for ASP.NET Core scaffolding

* Refactor eval.yaml to enhance scaffolding scenarios and improve grading configurations

* Add scaffolding for Minimal API and MVC with existing DbContext, including models, DbContexts, and configuration files

* Refine evaluation criteria for create-datadriven-aspnetcore skill by updating file paths and adding checks for Razor Pages and DbContext presence

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Dan Moseley <danmose@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Abhitej John <aajohn@microsoft.com>
Co-authored-by: Wendy Breiding <55603905+webreidi@users.noreply.github.com>
Co-authored-by: Wendy Breiding (She/Her) <webreidi@microsoft.com>
2026-07-28 15:56:08 +00:00
Abhitej John ca1b47371f Add automated per-plugin versioning (NBGV) with /version-bump + weekly backstop (#813)
* Add automated per-plugin versioning (NBGV) with /version-bump + weekly backstop

WHY
Tools that surface skills (Copilot CLI, Claude Code, Codex, Cursor) read a
plugin's version directly from its checked-in manifest. With no versioning
discipline, a plugin's behavior can change while its advertised version stays
flat, so clients never learn to re-pull, and there is no human-readable signal
of what changed. We want correct, current versions in the repo with minimal
manual work and without bloating the marketplace clone.

WHAT
- Per-plugin semantic versioning via Nerdbank.GitVersioning (NBGV). Each plugin
  owns a version.json whose pathFilters exclude the generated manifests and the
  version.json itself, so version height tracks real content changes only.
- The computed version is materialized into the checked-in manifests
  (plugin.json and .codex-plugin/plugin.json) so every consumer reads a current
  value with no build step on their side.
- eng/version/Sync-PluginVersions.ps1 is the single workhorse. It resolves the
  set of changed plugins from a git diff, computes each version with nbgv
  (predicting the squash-merge height for PRs), and either reports or stamps.

AUTOMATIONS (two, low-touch by design)
- /version-bump: an admin/maintainer comments the command on a PR and the
  affected plugins are stamped on the PR branch. Gated on collaborator
  permission (admin/write/maintain); forks are rejected before any privileged
  step. No other PRs are auto-modified.
- weekly-version-sync: a Monday backstop (and workflow_dispatch) that stamps any
  drift on main, opens/updates a single bot PR, and explains the per-plugin
  reason. This self-heals anything that merged without a bump.

We deliberately did NOT auto-edit contributor PRs or add a noisy advisory
comment bot; maintainers stay in control and the signal stays clean.

SECURITY (multi-model adversarial review: GPT-5.5 + Gemini 3.1 Pro)
- Supply chain (High, both models): dotnet tool restore would have honored a
  nuget.config authored in the PR tree, letting an attacker remap the nbgv
  package source to a malicious feed and run code in the privileged
  contents:write context. Mitigated with a trusted eng/version/nuget.config
  (clear + nuget.org-only + packageSourceMapping), overlaid from main and used
  via --configfile so PR-supplied configs are ignored. No nuget.config is
  tracked in the repo today, so this path was genuinely exploitable.
- TOCTOU (Medium): /version-bump now checks out the authorized head SHA rather
  than the mutable branch name; a racing push fails non-fast-forward, which is
  the safe outcome.
- Injection: Set-ManifestVersion uses a MatchEvaluator (not a replacement
  string) so a "$"-bearing version cannot re-expand, plus a strict
  major.minor.patch guard that throws on a malformed base, leaving manifests
  untouched.
- A base-only version.json bump (0.1 -> 0.2) is correctly detected and stamped.

VERIFIED
End-to-end against a real NBGV git harness: content-scoped predict, base-only
bump -> x.y.0, docs-only -> [], weekly drift stamping, malformed-base guard,
and --configfile restore (exit 0). actionlint passes on both workflows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Copilot review feedback

- Add missing plugins/dotnet-test-migration/version.json so it participates
  in versioning (it was the only plugin without one; manifests are at 0.1.0).
- CONTRIBUTING: the two manifests are not byte-identical; say the version is
  duplicated across two manifest files instead.
- weekly-version-sync: include version.json in commit attribution so a
  base-only bump is explained rather than showing 'no attributable commits'.
- Get-NbgvInfo: capture nbgv stderr and include it in the thrown error so CI
  failures are diagnosable, while keeping stdout clean for JSON parsing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-13 10:07:45 -07:00
Abhitej John 2ab3501158 Add .codex-plugin/plugin.json manifests for Codex CLI plugin install (#726)
The Codex CLI requires .codex-plugin/plugin.json as the plugin manifest
entry point. Without it, 'codex plugin add' fails with 'missing plugin.json'
even though the marketplace listing works.

This adds .codex-plugin/plugin.json to all 14 plugin directories, with paths
relative to the plugin root per the Codex docs. Also updates the agents
marketplace to use dotnet-aspnetcore (per #711 rename) and adds missing
dotnet-blazor and dotnet11 entries.

Fixes #578
Fixes #724

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-09 09:29:42 -07:00
Jan Krivanek 05aeb657e6 Add license to agent files (#568) 2026-04-21 12:57:18 +00:00
Copilot 6face98ff6 Change agents and skills fields in plugin.json to array format for Claude Code CLI compatibility (#407)
* Initial plan

* Remove agents string field from plugin.json and handle array format in parser

Claude Code CLI validates `agents` as array-of-strings per its plugin manifest
schema. Our plugin.json files had `"agents": "./agents/"` (a string), which
caused "agents: Invalid input" validation errors on install.

Remove the field from the 3 affected plugin.json files — agents are still
discovered by convention from the `agents/` directory. Also make ParsePluginJson
resilient to both string and array formats for forward-compatibility.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Remove `agents` string field from plugin.json for Claude Code CLI compatibility

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Change agents field to array format in plugin.json, update validator to read array-first with string fallback

Plugin.json files now declare agents as an array of file paths:
  "agents": ["./agents/foo.agent.md", "./agents/bar.agent.md"]

ParsePluginJson reads the array first (preferred, Claude Code schema),
falls back to a string path (legacy). PluginInfo gains an AgentPaths
field. Validator and discovery use the array when present, else fall
back to directory-based convention.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Simplify agents to directory-path array form ["./agents/"], handle directories in array entries

Per Copilot CLI spec, agents field values are "Path(s) to agent directories"
— no need to list individual files. Use ["./agents/"] to mirror how
skills uses "./skills/". Updated validator and discovery to handle both
directory and file entries in the array.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Delete .nuget/nuget.exe

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Convert skills field to array format in plugin.json, update validator to handle both forms

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Align AgentPaths null checks to use { Count: > 0 } pattern for consistency

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Change `agents` and `skills` fields in plugin.json to array format for Claude Code CLI compatibility

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Delete .nuget/nuget.exe

* Consolidate SkillsPath/SkillPaths and AgentsPath/AgentPaths into single array fields

Remove dual string/array fields from PluginInfo. The parser now
normalizes legacy string values into single-element arrays, so all
downstream code uses only SkillPaths and AgentPaths (IReadOnlyList<string>).
This eliminates all array-vs-string fallback branching.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-20 10:27:28 +01:00
Dan Moseley 8e50fc8558 Add agent and plugin spec conformance validation (#279)
* Add agent and plugin spec conformance validation

Extend the validation pipeline to check .agent.md files and plugin.json
files alongside skills. Previously only SKILL.md files were validated.

Agent validation (AgentProfiler):
- Frontmatter required (error)
- Name and description required (error)
- Name format: same rules as skills (lowercase alphanum + hyphens, 64 max)
- Name must match filename minus .agent.md suffix
- Description max 1024 chars
- Body max 500 lines

Plugin validation (PluginValidator):
- Name required, must match directory name
- Version and description required
- Description max 1024 chars
- Skills path required and must exist
- Agents path warned if specified but missing

Discovery (SkillDiscovery):
- DiscoverAgents() walks up from skill paths to plugin roots,
  scans agents/ directories for .agent.md files
- DiscoverPlugins() finds plugin.json from skill paths

Integration (ValidateCommand):
- Runs after skill discovery, before evaluation
- Errors block with exit code 1 (same as spec_conformance_failure)

All 4 existing agents and 2 plugins pass cleanly.
25 new tests (13 agent, 12 plugin).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Code review fixes: spec comments, ValidateNameFormat, kind-specific messages

- Refactor ValidateName into ValidateNameFormat (format-only) + ValidateName
  (format + directory match) so agent/plugin validation doesn't abuse the
  skill directory-match check by passing name twice.
- Warning messages now say 'Agent name' or 'Plugin name' instead of
  'Skill name' when called from AgentProfiler/PluginValidator.
- Add spec URL comments to AgentProfiler and PluginValidator referencing
  agentskills.io/specification, VS Code agent-plugins docs, and Claude Code
  plugins-reference.
- Add missing agent name edge-case tests: leading/trailing hyphen,
  consecutive hyphens, and message-prefix assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove unused agents path from plugins that have no agents

The dotnet, dotnet-data, and dotnet-upgrade plugins declared an agents
path but had no agents/ directory, producing validation warnings. Remove
the field; it can be added back when agents are actually created.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Promote agent/plugin name validation from warnings to errors

The agentskills.io spec uses 'Must' for all name constraints (length,
character set, no leading/trailing hyphens, no consecutive hyphens).
For a publishing validator these should block, not just warn. Agent
filename mismatch is also promoted since the IDE won't discover the
agent if the name doesn't match.

Skill name validation stays as warnings (lenient loading, not part
of this PR).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review feedback

- Fix FrontmatterRegex: remove RegexOptions.Multiline so a horizontal
  rule (---) in the body is not mistaken for frontmatter. Detection and
  stripping now both anchor to start of string.
- Use filename as fallback identifier in AgentProfile when agent.Name
  is empty, so error output shows [agent:foo.agent.md] not [agent:].
- Add TryGetSafeSubdirectory to PluginValidator: rejects absolute paths
  and parent-directory traversal in skills/agents path fields. Used by
  both PluginValidator and SkillDiscovery.DiscoverAgents.
- Use DirectoryName as fallback for plugin result name when Name is
  empty, so error output shows [plugin:my-dir] not [plugin:].
- Add tests for path traversal rejection and name fallback behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Detect orphaned test directories with no matching plugin/skill

Add FindOrphanedTestDirectories to SkillDiscovery: checks that every
directory under tests/{plugin}/{skill}/ has a corresponding directory
under plugins/{plugin}/skills/{skill}/. Plugin-level orphans (tests/
directory with no matching plugins/ directory) are also flagged.

Integrated into ValidateCommand after plugin/agent validation so it
runs on every validation pass. Errors are blocking (exit 1).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #279 review feedback

- Use case-sensitive (Ordinal) comparison for agent filename matching
  since agent names are required to be lowercase by spec.
- Replace OrdinalIgnoreCase path-under-root check in TryGetSafeSubdirectory
  with Path.GetRelativePath + '..' prefix check for correct behavior on
  case-sensitive filesystems.
- Let ParsePluginJson throw JsonException on malformed JSON instead of
  silently returning null. ValidateCommand catches it as a blocking error.
- Add test for malformed plugin.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Increase eval timeouts for dump-collect and dotnet-trace-collect to 120s

The with-skill runs for these skills were hitting the 60s scenario timeout
in CI, causing the agent to be killed before producing output. Other skills
in the same plugin already use 120s. The 30s decline/negative-test scenario
is left unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix fixture paths in clr-activation-debugging and analyzing-dotnet-performance evals

The source paths referenced tests/dotnet/ but the fixtures live under
tests/dotnet-diag/. This caused file-not-found exceptions at runtime,
which set hasRejections and caused unconditional exit 1 (bypassing
--verdict-warn-only).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Surface execution errors in validation summary

Previously, skills that threw exceptions (e.g. missing fixture files)
were silently excluded from the verdict count. The summary showed
'2/4 skills passed' with no indication that 2 other skills crashed
entirely. Now the summary shows '2/6 skills passed (2 rejected due
to execution errors)' and lists the specific error messages near
the summary rather than buried in the middle of the log.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Reduce parallelism for infra-change evaluation runs

When infrastructure changes trigger evaluation of all plugins
simultaneously, the high default parallelism (5x5x5) causes API
rate limits (429s) and scenario timeouts from contention. Reduce
to 2x3x3 for infra-change runs only; regular single-plugin PR
evaluations keep the default 5x5x5.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 15:40:06 +01:00
Dan Moseley 042a9d2b2c Reorganize plugins into domain-specific groupings (#274)
* Reorganize plugins per agreed structure (#271)

Split the dotnet plugin into domain-specific plugins per discussion
in #271:

- dotnet: narrowed to common everyday C#/.NET coding tasks
- dotnet-diag: perf investigations, debugging, incident analysis
- dotnet-data: data access and Entity Framework
- dotnet-aspnet, dotnet-wpf, dotnet-winforms, dotnet-maui: scaffolded
  empty to convey intent for future framework-specific skills

Moved skills, tests, and CODEOWNERS entries accordingly.
No file content was changed; all skill/test moves are pure renames.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Improve dotnet-diag and dotnet-winforms descriptions

- dotnet-diag: add '.NET' qualifier to clarify scope
- dotnet-winforms: use official 'Windows Forms (WinForms)' name

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add .NET qualifier to dotnet-data description

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove empty plugins; list them in CONTRIBUTING.md as future options

Remove the four empty placeholder plugins (dotnet-aspnet, dotnet-wpf,
dotnet-winforms, dotnet-maui) that had only plugin.json and no skills.

Add a 'Plugin organization' section to CONTRIBUTING.md that describes the
domain-specific plugin groupings introduced in this PR and lists the four
removed plugin names as reserved candidates for future contributions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Create dotnet-upgrade plugin with migration/upgrade skills

Move migrate-nullable-references, thread-abort-migration, and
dotnet-aot-compat from dotnet to the new dotnet-upgrade plugin.

Update CODEOWNERS paths (same owners), CONTRIBUTING.md plugin table,
and fix the thread-abort-migration SKILL.md link in CONTRIBUTING.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-06 21:01:52 -08:00