skills/dotnet__skills
1
0
Fork 0
mirror of https://github.com/dotnet/skills.git synced 2026-09-20 09:49:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
493 Commits 131 Branches 2 Tags
74c58505e5011268ed6870a699872ea2aeff86fb
Commit Graph

7 Commits

Author SHA1 Message Date
YuliiaKovalova 74c58505e5 Improve dotnet-template-engine plugin: accuracy, dedup, and two new skills (#745) 
* Improve dotnet-template-engine plugin: accuracy, dedup, and two new skills

Fix inaccurate reserved-shortName guidance, consolidate validation rules into
a single skill, expand discovery mappings, add explicit CPM/version steps, and
introduce template-comparison and template-smart-defaults skills.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix .codex-plugin manifest indentation; add evals for new skills

Make .codex-plugin/plugin.json byte-consistent with plugin.json (2-space
indent on the agents line). Add eval.yaml + eval.vally.yaml capability
evals for the new template-comparison and template-smart-defaults skills.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: source reserved list from dotnet new --help, note workload/package availability, tighten version-refresh

- Clarify the reserved shortName set is the current dotnet new subcommands
  (authoritative source: dotnet new --help); create is verified as a real
  subcommand (alias behind dotnet new <template>).
- template-discovery: note that some mapped short names (maui, winui3, aspire,
  func, orleans) need workloads/template packages, with fallback to
  dotnet new list/search.
- template-instantiation: keep template versions by default; if refreshing,
  use dotnet list package --outdated + user confirmation and constrain to
  same major/minor rather than always latest stable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address 2nd review round: enforce eval negatives, split combined assertion, reframe reserved list

- smart-defaults evals: enforce --no-https absence (auth scenario), absence of
  minimal-API flag (controllers scenario), and no newer --framework TFM when
  net8.0 is explicitly required, using output_not_contains/output_not_matches.
- comparison eval: split the combined (auth|aot|docker|controllers) check into
  four separate output_matches assertions so partial comparisons fail.
- template-validation/authoring: reframe the reserved shortName list as
  non-exhaustive examples and source the authoritative set from dotnet new --help;
  drop the specific create-alias assertion in favor of parsing-ambiguity wording.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Make template-comparison evals robust to CI SDK currency

Switch the Blazor comparison scenario from blazorserver (absent in the CI
SDK) to blazor (Blazor Web App) vs blazorwasm, both reliably present in
.NET 8+, and instruct the agent to inspect each via --help. Update the
SKILL.md example reference for currency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add CLI-failure resilience guidance to discovery/comparison skills

The isolated eval runs failed because the agent ran 'dotnet new <t> --help',
hit the template engine's global-mutex/persistence error (common when the
command runs concurrently in a sandbox), and then returned no answer at all.

Instruct both skills to run 'dotnet new' calls sequentially, retry once on a
transient mutex/persistence error, and fall back to the intent/parameter
mapping so a concrete answer is always produced instead of empty output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: accurate flags + robust negative-assertion prompts

- template-smart-defaults SKILL.md: drop the non-existent --publish-aot
  flag. Clarify --aot is a dotnet new flag only on templates that expose it
  (console/worker/grpc, not webapi) and that publish-time AOT is the MSBuild
  PublishAot=true property, not a dotnet new flag.
- template-discovery SKILL.md: replace the hardcoded --enable-docker mapping
  (not a real flag on common templates) with generic 'confirm with --help'.
- smart-defaults evals: tighten the negative-assertion prompts to output only
  the command line and not mention unused flags, so a negated explanation
  can't trip output_not_contains/output_not_matches. Switch the AOT scenario
  from webapi to worker (which actually supports --aot).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review round 3: scope negative checks, de-emphasize stale lists

- smart-defaults evals: anchor the negative assertions to the 'dotnet new'
  command line (same-line regex) instead of whole-output substring/regex, so a
  flag mentioned only in prose can't fail the test.
- template-validation / template-authoring: mark the dotnet new subcommand
  examples as illustrative/version-dependent and tell readers not to hardcode
  them; the live 'dotnet new --help' output is canonical.
- template-comparison: fix the example table's AOT row — webapi/webapp do not
  expose a --aot template flag; native AOT is publish-time via PublishAot.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-12 15:15:56 +02:00
Abhitej John 2ab3501158 Add .codex-plugin/plugin.json manifests for Codex CLI plugin install (#726) 
The Codex CLI requires .codex-plugin/plugin.json as the plugin manifest
entry point. Without it, 'codex plugin add' fails with 'missing plugin.json'
even though the marketplace listing works.

This adds .codex-plugin/plugin.json to all 14 plugin directories, with paths
relative to the plugin root per the Codex docs. Also updates the agents
marketplace to use dotnet-aspnetcore (per #711 rename) and adds missing
dotnet-blazor and dotnet11 entries.

Fixes #578
Fixes #724

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-09 09:29:42 -07:00
Jan Krivanek 05aeb657e6 Add license to agent files (#568) 2026-04-21 12:57:18 +00:00
YuliiaKovalova 59d4485e14 Improve dotnet-template-engine skills and eval suite, add template-validation skill (#480) 
* Improve dotnet-template-engine eval quality and plugin activation

- Replace useless 'Create project with specific framework' test (vanilla=10/10
  in all 30 runs, zero skill delta) with harder scenarios that vanilla cannot
  trivially solve
- Expand eval suite from 5 to 10 scenarios to reduce overfitting signal and
  improve coverage of untested capabilities (dry-run, search, install,
  template creation from existing project, ambiguous intent resolution)
- Add file_not_exists assertion to dry-run scenario to verify projects are
  not actually created during preview
- Add output_matches assertions with specific patterns (e.g. 'individual',
  'blazor') instead of relying solely on output_contains
- Increase all timeouts from 120s to 180s to reduce MCP cold-start timeouts
- Enrich plugin.json description with activation keywords (console app, class
  library, web API, Blazor, MAUI, framework versions) to fix Plugin mode
  activation failures (19/21 not-activated events were from one scenario)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add template-validation skill with eval scenarios

New skill encoding template.json validation rules: required fields, identity
format, shortName conflicts with CLI commands, symbol type/datatype validation,
choice parameter default checking, computed/generated symbol completeness,
parameter prefix collisions, source condition syntax, post-action requirements,
constraint configuration, and tag recommendations.

Two eval scenarios:
- Validate a template with 8+ deliberate errors (missing identity, CLI-
  conflicting shortName, invalid defaults, incomplete symbols, missing actionId)
- Validate a correct template and confirm it's ready to publish

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments

- Add ./test-output to dry-run prompt so file_not_exists assertion checks
  the correct location
- Add missing expect_tools: skill to template-authoring validate scenario
- Add 'using Xunit;' to seeded test file so the setup project is compilable

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Tighten assertions and replace low-delta scenario

- Replace low-delta 'worker service search' (vanilla solves trivially) with
  'create multi-project solution' (two projects + project reference — vanilla
  struggles with multi-step orchestration)
- Fix gRPC search assertion: replace trivial output_contains 'template' with
  specific pattern matching gRPC service/package names
- Fix validation restore assertion: require 'shortName' context near 'restore'
  to prevent false pass from post-action description
- Fix correct-template assertion: replace fragile output_not_contains with
  positive assertion that agent mentioned specific fields it checked
- Fix create-from-existing assertion: replace pass-anything pattern with
  assertion requiring template.json / sourceName / .template.config mention

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments (round 2)

- Change datatype 'integer' to 'int' in eval fixture and SKILL.md to align
  with actual template engine schema
- Tighten correct-template assertion: require unambiguous success signal
  (e.g. 'no errors', 'template is valid') instead of permissive pattern
  that matches 'not ready' or 'pass' in other contexts
- Replace output_not_contains 'missing required' (fails on 'no missing
  required fields') with positive assertion checking field names
- Remove int/integer dual listing from valid datatypes in SKILL.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix eval failures from CI run

4 scenarios failed; fixes:

- Web API + AOT: increase timeout 180s -> 300s (auth scaffolding + NuGet
  restore needs more time)
- Dry-run preview: rephrase prompt to explicitly mention 'template' for
  better skill activation matching
- Multi-project solution: replace with simpler 'create xUnit test project
  with net8.0' — the multi-project scenario gave vanilla a HIGHER score
  (4.7) than skilled (4.3) because the skill added overhead without helping
- Create-from-existing: increase timeout 180s -> 300s (template generation
  burned 256K tokens and timed out)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix dishonest eval adjustments

- Revert dry-run prompt to natural language — don't coach activation by
  injecting 'template' keyword. If skill doesn't activate, that's a real
  signal about activation quality.
- Replace recycled 'xUnit + net8.0' test (same ceiling problem as the
  classlib + net9.0 test we removed) with 'Blazor + interactive rendering'
  which tests parameter knowledge vanilla is less likely to have (render
  mode selection, blazor vs blazorserver vs blazorwasm template choice).
- Keep timeout increases (300s for web API + auth, 300s for create-from-
  existing) as these address real infrastructure constraints.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Move dry-run scenario from template-instantiation to template-discovery

The dry-run scenario failed with 'NOT ACTIVATED' because the platform
couldn't match a preview/dry-run prompt to the template-instantiation
skill. This is correct behavior — previewing what files a template
produces is a discovery operation, not instantiation.

The template-discovery SKILL.md explicitly covers dry-run in Step 4:
'Use dotnet new <template> --dry-run to show what files a template
would create without writing anything to disk.'

This is an objective reclassification, not prompt tuning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix skill content causing eval failures

template-instantiation: Add parameter reference table with common parameter
combinations (auth values, interactivity modes, AOT flag) for webapi, blazor,
grpc, worker templates. The web API + AOT scenario scored 2.0 for both
vanilla and skilled because the skill mentioned --auth and --aot but never
explained what values they accept. Now the agent gets actionable knowledge.

template-authoring: Trim verbose content to reduce token overhead. The
create-from-existing scenario burned 256K tokens (vs 185K vanilla) and
timed out because the agent read packaging/distribution steps it didn't
need. Condensed Step 1 example, removed Step 5 (packaging), compressed
Step 4 (testing). Reduces skill size by ~30%.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments (round 3)

- Web API+AOT: replace output_matches for AOT with file_contains checking
  for PublishAot in the generated .csproj — proves the parameter was
  actually passed, not just mentioned in text
- Validation multi-error: add assertions for bool default (enableLogging/
  yes), int default (maxRetries/abc), computed missing value, generated
  missing generator — now enforces 7 of 8 rubric items via assertions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments (round 4)

- Fix YAML double-quote escaping for regex word boundaries
- Fix worker template example to include --aot flag
- Fix webapi example to show --auth Individual --aot together
- Fix grammar: Available Symbol Generators

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove web API+AOT and Blazor eval scenarios

Web API+AOT consistently degraded quality (run 1: 2.0->2.0, run 2:
3.0->2.7). The skill adds reading overhead on execution tasks without
improving the result.

Blazor hit ceiling (5.0->5.0) — vanilla already creates Blazor apps
perfectly, so the scenario contributes zero signal.

template-instantiation now has 1 honest scenario (console app, +1.0
delta). The skill's value is modest for execution tasks; the PR's
strength is in discovery (+2.3 to +3.7), validation (+1.7 to +2.0),
and authoring (+2.0) where knowledge matters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments (round 5)

- Tighten bool/int validation assertions to require explicit error
  wording (invalid, not valid, must be) alongside field/value — prevents
  passing when agent just echoes the JSON without diagnosing the issue
- Add actionId assertion to enforce detection of missing postAction
  actionId (was in rubric but not in assertions)
- Use single-quoted YAML for patterns with backslash sequences

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-01 14:16:24 +02:00
Viktor Hofer a895c226b3 Fix agents field in plugin.json to use explicit file paths (#419) 
* Fix agents field in plugin.json to use explicit file paths

Claude Code's manifest validator rejects directory paths in the agents
field. Replace directory references with explicit .agent.md file paths
in all affected plugins (dotnet-diag, dotnet-msbuild, dotnet-template-engine).

Fixes https://github.com/dotnet/skills/issues/416

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Validate that agents paths are explicit files, not directories

Claude Code's manifest validator rejects directory paths in the agents
field. Update PluginValidator to error when an agent path resolves to a
directory, guiding authors to use explicit .agent.md file paths instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-03-23 08:36:20 +00:00
Copilot 6face98ff6 Change agents and skills fields in plugin.json to array format for Claude Code CLI compatibility (#407) 
* Initial plan

* Remove agents string field from plugin.json and handle array format in parser

Claude Code CLI validates `agents` as array-of-strings per its plugin manifest
schema. Our plugin.json files had `"agents": "./agents/"` (a string), which
caused "agents: Invalid input" validation errors on install.

Remove the field from the 3 affected plugin.json files — agents are still
discovered by convention from the `agents/` directory. Also make ParsePluginJson
resilient to both string and array formats for forward-compatibility.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Remove `agents` string field from plugin.json for Claude Code CLI compatibility

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Change agents field to array format in plugin.json, update validator to read array-first with string fallback

Plugin.json files now declare agents as an array of file paths:
  "agents": ["./agents/foo.agent.md", "./agents/bar.agent.md"]

ParsePluginJson reads the array first (preferred, Claude Code schema),
falls back to a string path (legacy). PluginInfo gains an AgentPaths
field. Validator and discovery use the array when present, else fall
back to directory-based convention.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Simplify agents to directory-path array form ["./agents/"], handle directories in array entries

Per Copilot CLI spec, agents field values are "Path(s) to agent directories"
— no need to list individual files. Use ["./agents/"] to mirror how
skills uses "./skills/". Updated validator and discovery to handle both
directory and file entries in the array.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Delete .nuget/nuget.exe

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Convert skills field to array format in plugin.json, update validator to handle both forms

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Align AgentPaths null checks to use { Count: > 0 } pattern for consistency

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Change `agents` and `skills` fields in plugin.json to array format for Claude Code CLI compatibility

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

* Delete .nuget/nuget.exe

* Consolidate SkillsPath/SkillPaths and AgentsPath/AgentPaths into single array fields

Remove dual string/array fields from PluginInfo. The parser now
normalizes legacy string values into single-element arrays, so all
downstream code uses only SkillPaths and AgentPaths (IReadOnlyList<string>).
This eliminates all array-vs-string fallback branching.

Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-03-20 10:27:28 +01:00
YuliiaKovalova dcba4f3d8e Add dotnet-template-engine plugin (#341) 
* Add dotnet-template-engine plugin

Add template-engine skills for .NET project scaffolding powered by the
DotnetTemplateMCP MCP server (v1.1.0). Includes three skills and one agent:

Skills:
- template-discovery: find, inspect, and compare templates with intent
  resolution (70+ keyword mappings)
- template-instantiation: create projects with CPM adaptation, latest NuGet
  versions, parameter validation, and multi-template composition
- template-authoring: create templates from existing projects and validate
  template.json for authoring issues

Agent:
- template-engine: expert agent that routes to the appropriate skill based
  on user intent

Also adds CODEOWNERS entries, marketplace.json registration, eval tests,
and .mcp.json for DotnetTemplateMCP server configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: mcpServers in plugin.json, expect_tools, path fix

- Move MCP server config into plugin.json under mcpServers so SkillValidator
  discovers the template_* tools during evaluation runs
- Add expect_tools constraints to all eval scenarios to verify actual MCP
  tool invocation instead of relying on rubric text alone
- Fix frontmatter path: template.config → .template.config (leading dot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update DotnetTemplateMCP to v1.2.0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Include 'dotnet new' in agent description for discoverability

Address review feedback from danmoseley: the frontmatter description
is how the caller decides whether to load the agent tokens, so it
needs to mention 'dotnet new' as a trigger keyword.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove MCP server configuration and references

- Delete .mcp.json MCP server config file
- Remove mcpServers section from plugin.json
- Remove MCP/DotnetTemplateMCP references from agent, skills, and marketplace descriptions
- Skills and eval files remain unchanged

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove ghost tool references and add CLI fallback guidance

- Remove references to 4 non-existent tools: template_compose,
  template_suggest_parameters, template_validate, solution_analyze
- Replace with real alternatives (template_inspect, manual review,
  sequential template_instantiate, direct file inspection)
- Add CLI fallback guidance so LLMs degrade gracefully when
  template tools are unavailable
- Fix authoring eval to not expect template_validate tool

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: CODEOWNERS naming, claude marketplace, README

- Rename CODEOWNERS header to dotnet-template-engine for consistency
- Mirror plugin entry in .claude-plugin/marketplace.json
- Add plugin to README What's Included table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix PR review comments: align eval rubric with expect_tools and sync marketplace.json

- Updated template-discovery eval.yaml rubric to match expect_tools
  (removed 'or template_search' since expect_tools enforces template_from_intent)
- Made .claude-plugin/marketplace.json byte-for-byte identical to
  .github/plugin/marketplace.json (removed extra trailing newline)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review comments: strengthen eval assertions and fix rubrics

- Add exit_success assertions to all eval scenarios
- Add file_exists and file_contains assertions for template-instantiation
- Add output_contains for shortName and datatype in template-authoring
- Fix rubrics to reference actual tools (skill + dotnet new CLI) instead
  of non-existent MCP tools (template_from_intent, template_inspect,
  template_instantiate) which caused the judge to penalize quality scores
- Update expect_tools from non-existent MCP tools to 'skill'
- Change plugin version from 1.2.0 to 0.1.0 for consistency with other plugins
- Remove extra blank line in README.md after plugin table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Rewrite skills and agent to use dotnet new CLI instead of phantom MCP tools

The SKILL.md files and agent.md referenced MCP tools (template_from_intent,
template_inspect, template_instantiate, template_dry_run, etc.) that are
not available at runtime since no MCP server is declared in plugin.json.
This caused the agent to waste tokens attempting to find these tools before
falling back to dotnet new CLI commands.

Changes:
- template-discovery/SKILL.md: Replace MCP tool calls with dotnet new
  search, dotnet new list, dotnet new --help, and dotnet new --dry-run.
  Add intent-to-template keyword mapping table.
- template-instantiation/SKILL.md: Replace template_instantiate with
  dotnet new CLI commands. Add explicit CPM adaptation steps and
  multi-project composition with dotnet sln/dotnet add reference.
- template-authoring/SKILL.md: Replace template_create_from_existing
  with manual template.json creation guidance including example JSON.
  Replace template_dry_run/template_instantiate with CLI equivalents.
- template-engine.agent.md: Replace Available Tools table with CLI
  Commands Reference. Update all workflow steps to use CLI commands.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix file assertion paths: remove ./ prefix for glob matcher compatibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Dan Moseley <danmose@microsoft.com>
 2026-03-18 22:51:20 +01:00