mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
a7af3486a7e23990a2b1c1656e30f4df05c198c1
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Amaury Levé
|
460a01882f |
Define and validate native Codex plugin support (#1164)
* fix: validate native Codex plugin support Separate Codex-compatible plugin components from GitHub Copilot custom agents, validate Codex manifest fields and MCP shapes, and document the native agent installation boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: strengthen Codex plugin verification Validate Codex MCP per-tool settings and add a pinned native smoke lane that installs the marketplace, discovers skills and MCP, and calls binlog_overview. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: avoid restarting Codex MCP smoke client Use the real tool call as the app-server initialization so the Linux smoke test does not start and tear down a redundant status client first. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: harden Codex validation and smoke cleanup Constrain smoke artifacts to a dedicated child directory, validate Codex manifest field shapes and required skill paths, and document the separate Agents SDK handoff model. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: align Codex smoke and hooks parsing Use the clean Codex installation's default provider for the direct MCP call and reject heterogeneous hooks arrays like the runtime parser. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: verify every Codex plugin skill Compare native skill discovery with the full repository inventory, match Codex lexical path rejection, and allow null optional MCP tool settings. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: address Codex compatibility review Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> |
||
|
Abhitej John
|
6cf3a72f3e |
Correct merged plugin release checkpoints
Stamp versions from the latest main first-parent authority: dotnet-test 0.2.22 and dotnet-test-migration 0.1.9. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5d007947-f964-4809-a31c-2d9f457a2ba3 |
||
|
Abhitej John
|
503d090536 |
Merge origin/main into bot/weekly-version-sync
Preserve current main content and advance dotnet-test-migration to the next valid release checkpoint. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5d007947-f964-4809-a31c-2d9f457a2ba3 |
||
|
Amaury Levé
|
e4c942d8de |
Add NUnit to MSTest migration skill (#1140)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> |
||
|
github-actions[bot]
|
85ebaf271d | Weekly plugin version sync | ||
|
Amaury Levé
|
3b670a8276 |
Improve test-gap evals and MSTest migration activation (#1107)
* skill-validator: restore 15K aggregate cap, document it as the real Copilot CLI skill-menu budget The per-plugin aggregate description cap had been raised 15,000 -> 20,000 -> 22,000 under the belief that 15K was 'a local repo policy, NOT a documented Copilot constraint'. That belief was wrong: the GitHub Copilot CLI renders the model-facing <available_skills> menu under a hard 15,000- char budget (the agent SDK's SKILL_CHAR_BUDGET, default 15e3, confirmed in CLI 1.0.36 and 1.0.61). Skills are listed alphabetically and emitted with their full <description> only until the budget is exhausted; every skill past the cut-off collapses to a bare name with no description and can no longer be reliably model-activated. Raising the validator cap merely masked this silent menu truncation — e.g. dotnet-test's run-tests and test-* skills stopped activating in plugin eval runs because they fell into the name-only overflow. Changes: - SkillProfiler.MaxAggregateDescriptionLength: 22,000 -> 15,000, with the comment rewritten to document the real Copilot CLI budget (and correct the prior 'not a documented constraint' claim). - CheckCommand aggregate now excludes skills marked 'disable-model-invocation: true' — the CLI drops those from the menu, so they do not consume the budget. This makes the cap satisfiable by hiding reference / agent-orchestrated primitives rather than only by trimming. - InvestigatingResults.md: document plugin-arm-only non-activation caused by skill-menu budget overflow, and how to fix it. Note: dotnet-test currently exceeds 15K and must be slimmed below it (via disable-model-invocation on reference/primitive skills plus description trims) before this cap can go green repo-wide. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * skill-validator: use source-generated regex for disable-model-invocation check Address review: replace Regex.IsMatch(pattern-string) with a [GeneratedRegex] partial method (AOT-friendly, no per-call cache lookup), matching FrontmatterParser's style. Runs once per skill during checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test-gap-analysis: cover validation safeguards Add focused evaluation scenarios for proportional scope, complete outcome inventory, mutation restoration, demonstrated recommendations, and canonical verifier usage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: harden mutation verifier eval Anchor verifier paths to the script directory and require the eval response to report the supplied canonical verifier result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: distinguish verifier failures Require Python 3, preflight the test runner, build mutants explicitly, and treat only the MTP test-failure exit code as a killed mutation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: tighten eval evidence Challenge proportional scope with an unrelated fixture and require affirmative canonical-verifier success wording. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: name MTP failure exit code Document the Microsoft.Testing.Platform test-failure convention used by the mutation verifier. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: require affirmative verifier result Constrain the output grader to a direct successful result for verify_mutations.sh so negative mutation summaries cannot pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * Improve MSTest migration skill activation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9ea64d8-cced-442a-b558-2bb43579c316 * test-gap-analysis: cover proportional scope validation Name the validation outcome directly and prove the focused review excludes an unrelated AccessControl fixture. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: classify killed mutants by test evidence Capture MTP output and require an xUnit execution summary with failed tests, avoiding runner-specific exit-code assumptions while preserving infrastructure diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 * test-gap-analysis: preserve mutant build diagnostics Capture MSBuild output and print it when mutant compilation fails so verifier infrastructure failures remain actionable in CI. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 81d0f63d-89cb-4802-a1da-82a9c4e5c846 Copilot-Session: c9ea64d8-cced-442a-b558-2bb43579c316 |
||
|
Abhitej John
|
ef5f7cada6 |
Merge pull request #1095 from dotnet/bot/weekly-version-sync
Weekly plugin version sync |
||
|
Amaury Levé
|
f775756a79 |
Improve cross-model test migration outcomes (#1096)
* Improve test migration skill outcomes Address cross-model routing and execution failures found in retained evaluation evidence, and align the xUnit companion-package rubric with feed-compatible versions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Cover MTP method filter routing Include the method-filter token in the VSTest-to-MTP skill description so method-specific migration requests activate consistently. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Clarify MSTest SDK v4 discovery guidance Scope the missing Test SDK reference to MSTest.Sdk v4 and remove a fixed package-version example in favor of feed-resolved and CPM-aware guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Clarify xUnit MTP runner preservation State that the migration preserves the MTP runner for projects currently using the YTest shim, rather than preserving the shim package itself. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Avoid xUnit package major assumptions Resolve companion compatibility from package constraints and configured feeds instead of examples that conflate the xUnit product generation with NuGet major versions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * List all MSTest SDK discovery paths Align the CI response contract with the three documented choices for transitional VSTest discovery and MTP-native execution. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix xUnit v3 evaluation evidence Replace a contradictory async-void output grader with a file-state check, strengthen deterministic BeforeAfter signature checks, and require explicit final evidence for both migration paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Align MSTest SDK discovery guidance Correct the behavioral symptom table to describe the v4-only removal and all three documented migration paths consistently. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Make BeforeAfter eval name agnostic Validate the required xUnit v3 override and base-call shapes without hardcoding local parameter names. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Strengthen Luna migration outcomes Add complete advisory output contracts, correct focused-fix execution decisions, require concrete MTP package and filter guidance, and make xUnit package and source evidence explicit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Relax BeforeAfter source formatting Keep deterministic IXunitTest source validation while relying on compilation and test execution for qualified and multiline signature correctness. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> |
||
|
Amaury Levé
|
250089d2dc |
Cover xUnit-to-MSTest Parallelize eval pattern (#1098)
* Cover xUnit migration parallelization eval Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5b0aa9a5-aa8f-445f-8db0-0cc6795d03ac * Improve xUnit migration consistency Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5b0aa9a5-aa8f-445f-8db0-0cc6795d03ac * Preserve xUnit migration semantics Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5b0aa9a5-aa8f-445f-8db0-0cc6795d03ac * Make migration evidence explicit Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5b0aa9a5-aa8f-445f-8db0-0cc6795d03ac --------- Copilot-Session: 5b0aa9a5-aa8f-445f-8db0-0cc6795d03ac |
||
|
github-actions[bot]
|
6cc9e1e1fc | Weekly plugin version sync | ||
|
Amaury Levé
|
2b9056bd91 |
Separate dormancy evidence and improve affected evals (#1079)
* Separate dormancy from preference evidence Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Preserve legacy dashboard preference semantics Default scenarios without schema v4 preference eligibility metadata to eligible so historical schema v3 records remain consistent with their original gate evidence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f * Cover activation contract reporting precedence Exercise the combined activation-contract failure and underpowered preference state so consolidation keeps the routing failure as the headline result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f * Tighten dormancy scalar parsing Require complete YAML false tokens so prefix values cannot alter preference eligibility, and keep activation-contract summary grammar correct. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f * Preserve legacy dashboard gate labels Mark only schema v4 dashboard evidence with dormancy exclusions so historical schema v3 records retain their original all-stimulus vote labeling. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f * Improve skills from evaluation evidence Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f * Harden dormancy evaluation contracts Match PyYAML dormancy parsing, preserve unmatched annotations, prioritize gate-eligible report evidence, and make the MTP Boolean grader exact without case sensitivity. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 808c8105-e596-42c6-9dca-4c62cfcd490f |
||
|
Amaury Levé
|
47dd914532 |
Add Claude plugin manifests (#1071)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> |
||
|
github-actions[bot]
|
2d00e600ac | Weekly plugin version sync | ||
|
github-actions[bot]
|
62ff166f2b | Weekly plugin version sync | ||
|
Amaury Levé
|
030493de5a |
Fix migrate-mstest-v1v2-to-v3 activation and test-skill eval quality (#974)
* Fix migrate-mstest-v1v2-to-v3 skill activation The frontmatter description said DO NOT USE FOR: ... projects already on MSTest v3+, which blocked the skill on every scenario where the packages had already been bumped to 3.x and only the source or settings still needed the v1/v2-to-v3 fixes (Assert object overloads, DataRow strict typing, .testsettings -> .runsettings). It also gated the whole skill behind "the user asks to upgrade MSTest", so a standalone .testsettings conversion never matched. - Rewrite the description around both entry points (pre-upgrade migration and post-upgrade breaking-change fixes) and add the concrete trigger keywords those prompts contain: CS1501/CS1503/CS0121, MSTEST0014, LegacySettings, DeploymentEnabled, per-test TestTimeout, net5.0. Note that the current runner is preserved so "migrate to v3 but keep VSTest" isn't poached by migrate-vstest-to-mtp. - Narrow migrate-mstest-v3-to-v4, which claimed the generic "tests don't compile after upgrading MSTest" phrasing and competed for the same prompts. - Widen the Boundary Gate: a 3.x package version alone no longer ends the migration when v1/v2-era settings or errors remain, so the skill actually performs the requested edits instead of reporting "already migrated". - Add a routing row to the test-migration agent for the same case. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ad6ff32a-d441-4a7b-b474-2bfaee764740 * Clear the eval-quality gate's test-skill findings The gate reported three classes of debt against the dotnet-test and dotnet-test-migration plugins. All three are addressed here; the four ERRORS it also reports are dotnet-maui allowlist lines and are untouched. Underpowered evals (5). Below five trials the pass gate's sign test cannot reach p <= 0.05 at any effect size, so these five evals could never return a verdict. Each is now at or above the floor and its allowlist line is deleted in the same change, as the ledger's shrink-only rule requires: - coverage-analysis 3 -> 5: adds a refactoring-safety question (the "is this safe to change?" use case named in the skill's Purpose but never exercised) and a branch-vs-line coverage question. Both reuse the existing partial-coverage fixture. - find-untested-sources 4 -> 5: adds a mixed C#/TypeScript repository, which is the only case that exercises the documented engine choice - polyglot tree-sitter rather than the C#-only Roslyn engine. Composed from the two existing fixtures. - generate-testability-wrappers 4 -> 5: adds the ambient-context path (Step 5) for a project with no DI container, where AsyncLocal<T> and scoped disposal are the distinguishing content. - grade-tests 4 -> 5: adds a C# case with the production code present. Every prior C# scenario hides it, so "Unverified" was never tested as a negative, and the D band and the swallowed-exception F were never graded at all. New production-available fixture. - code-testing-agent 3 -> 6 via defaults.runs=2. Scenarios are preferred over runs, but each of these drives a full generate-build-test pipeline (npm ci plus two Vitest runs, pip install plus pytest, a dotnet test build) under a 60m budget, which is the documented case for buying trials with runs. Orphaned fixtures (5). v3-sealed-timeout, mtp-mstest-sdk9, mtp-mstest-sdk10, mtp-mstest-hotreload-installed and vstest-mstest are all superseded first- generation copies: their per-scenario successors differ only in whitespace, a dropped rollForward, or a package version. Both evals are already well above the floor, so wiring them up would add no power. Deleted. Skills with no eval (2 of 4). platform-detection and filter-syntax carry real checkable rules that nothing measured, and several are counterintuitive enough that a baseline is likely to get them wrong - global.json test.runner outranking TestingPlatformDotnetTestSupport on .NET 10+, Microsoft.NET.Test.Sdk not being a VSTest signal, MTP properties living in Directory.Build.props, xUnit v3 dropping VSTest --filter while MSTest on MTP keeps it. Both get a 5-scenario eval with small fixtures and no build step. code-testing-extensions and test-analysis-extensions are left flagged on purpose: their bodies are tables of paths to extension files, so a head-to-head eval would score path recall rather than user value. The content those files hold is already exercised through code-testing-agent's three-language pipeline. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ad6ff32a-d441-4a7b-b474-2bfaee764740 * Fix the two skill defects behind the v1v2-to-v3 eval losses The first eval run reached 7W/2T/2L, p=0.090, short of the p<=0.05 gate. Both losses trace to skill content that actively misled the agent, and the session transcripts show exactly how. Loss 1 -- 'Migrate MSTest v1 project with assembly reference', skilled scored 0.00 against a 4.17 baseline. The transcript shows the skill loading correctly and the agent then replying, in full: 'To give you specific migration steps, I need to see your project file. Could you share the path to your .csproj?' The project was already in the working directory. Cause: the Inputs table marked 'Project or solution path' as Required=Yes, which reads as a precondition the agent must obtain before doing anything. This is the worst kind of failure for a real user - they describe their project in prose and get a question back instead of an answer. Path is now optional and discovered by globbing, Step 1 leads with locating the project, and a note forbids opening with a request for the path. The same Required=Yes trap was present in migrate-mstest-v3-to-v4 and migrate-vstest-to-mtp, so both are corrected too. Loss 2 -- 'Fix DataRow type mismatch errors', skilled 3.96 against a 5.00 baseline. The skill's breaking-change table said the 16-argument DataRow cap was 'fixed in later v3 versions' and suggested 'refactor test / wrap extra params in array'. On a project already at MSTest 3.8, the agent concluded the valid 17-argument row exceeded the limit and rewrote it - first as new object[] { 17 }, which failed, then second-guessing itself mid-run ('let me check if the latest 3.x actually fixed the 16-arg limit'), finally settling on a (object)17 cast. Churn plus wasted turns on code that was already correct. The vague wording was the problem, so it is replaced with the fact: the cap was introduced in 3.0.1 and removed again in 3.0.3 (microsoft/testfx#1554 and the maintainer's 'please feel free to update to 3.0.3'). On 3.0.3+ a longer row is valid and must be left alone. A general guideline is added alongside it - confirm the diagnostic before editing, because rewriting valid code to dodge a limit the project is not subject to is a defect rather than caution. Both fixes are about what the skill tells a real user, not about the graders; no eval prompt, fixture, or grader is touched. Skill grows ~480 tokens and stays in the 'standard' tier, below the 5,000-token warning threshold. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ad6ff32a-d441-4a7b-b474-2bfaee764740 * Correct the Assert/DataRow facts and make the fixtures reproduce them The two eval runs on this PR compared byte-identical skill content: run 1 ( |
||
|
Amaury Levé
|
5fff8f1022 |
Improve dotnet-test-migration skill signal (#929)
* Improve migration skill eval signal Trim unnecessary xUnit-to-MSTest workflow turns and make MSTest v1/v2 migration preserve the detected test platform. Add focused cost and activation-boundary eval coverage for issue #900. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 860bd5ec-3bee-44ec-8fad-1bdcc745b9e1 * Strengthen migration evaluation signal Use the first evaluation's per-trial evidence to remove an inert scenario, cover semantic assertion migration, broaden MSTest risk guidance, and make testsettings migration objective. Also fix the MSTest 4.1/Test SDK compatibility rule and add a fast path to reduce retry-heavy migrations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 860bd5ec-3bee-44ec-8fad-1bdcc745b9e1 * Fix migration eval review findings Move the sequence grader into its matching stimulus, positively require element-wise comparison, and make runsettings XML checks tolerant of casing and whitespace. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 860bd5ec-3bee-44ec-8fad-1bdcc745b9e1 * Strengthen migration eval execution signal Convert weak advice-only scenarios into verified migrations, require test discovery and complete framework state, and correct misleading migration mappings found in eval replay. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 06f10e04-4c49-40c3-b663-69be4525e4cb --------- Copilot-Session: 860bd5ec-3bee-44ec-8fad-1bdcc745b9e1 Copilot-Session: 06f10e04-4c49-40c3-b663-69be4525e4cb |
||
|
Amaury Levé
|
4df4da469a |
Upgrade agentic workflows and fix stale PR cleanup (#916)
* chore: upgrade gh-aw runtime * fix: paginate stale pull request cleanup Upgrade the generated agentic workflow assets and ensure stale PR discovery includes every result page and draft pull requests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 968a22c2-327f-4d26-8f86-1c59bcd323ea * Improve dotnet-test eval coverage and efficiency Address remaining high-confidence items from #899 by bounding the code-testing pipeline and adding eval coverage for grade-tests and find-untested-sources. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e430fee9-d3df-4ef5-85a4-745ae4b17046 * fix: address agentic workflow review Pin the Copilot setup checkout action and include the cutoff date in stale PR search results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 968a22c2-327f-4d26-8f86-1c59bcd323ea * Improve test migration skill guidance Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5e19d263-02a6-45b1-9cfb-424fa4d10863 * fix: add fixture namespace imports Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a464e6e4-3e45-41fe-b17d-887c8cb8a448 * test: assert GetOrderById grade Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a464e6e4-3e45-41fe-b17d-887c8cb8a448 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Abhitej John
|
ca1b47371f |
Add automated per-plugin versioning (NBGV) with /version-bump + weekly backstop (#813)
* Add automated per-plugin versioning (NBGV) with /version-bump + weekly backstop WHY Tools that surface skills (Copilot CLI, Claude Code, Codex, Cursor) read a plugin's version directly from its checked-in manifest. With no versioning discipline, a plugin's behavior can change while its advertised version stays flat, so clients never learn to re-pull, and there is no human-readable signal of what changed. We want correct, current versions in the repo with minimal manual work and without bloating the marketplace clone. WHAT - Per-plugin semantic versioning via Nerdbank.GitVersioning (NBGV). Each plugin owns a version.json whose pathFilters exclude the generated manifests and the version.json itself, so version height tracks real content changes only. - The computed version is materialized into the checked-in manifests (plugin.json and .codex-plugin/plugin.json) so every consumer reads a current value with no build step on their side. - eng/version/Sync-PluginVersions.ps1 is the single workhorse. It resolves the set of changed plugins from a git diff, computes each version with nbgv (predicting the squash-merge height for PRs), and either reports or stamps. AUTOMATIONS (two, low-touch by design) - /version-bump: an admin/maintainer comments the command on a PR and the affected plugins are stamped on the PR branch. Gated on collaborator permission (admin/write/maintain); forks are rejected before any privileged step. No other PRs are auto-modified. - weekly-version-sync: a Monday backstop (and workflow_dispatch) that stamps any drift on main, opens/updates a single bot PR, and explains the per-plugin reason. This self-heals anything that merged without a bump. We deliberately did NOT auto-edit contributor PRs or add a noisy advisory comment bot; maintainers stay in control and the signal stays clean. SECURITY (multi-model adversarial review: GPT-5.5 + Gemini 3.1 Pro) - Supply chain (High, both models): dotnet tool restore would have honored a nuget.config authored in the PR tree, letting an attacker remap the nbgv package source to a malicious feed and run code in the privileged contents:write context. Mitigated with a trusted eng/version/nuget.config (clear + nuget.org-only + packageSourceMapping), overlaid from main and used via --configfile so PR-supplied configs are ignored. No nuget.config is tracked in the repo today, so this path was genuinely exploitable. - TOCTOU (Medium): /version-bump now checks out the authorized head SHA rather than the mutable branch name; a racing push fails non-fast-forward, which is the safe outcome. - Injection: Set-ManifestVersion uses a MatchEvaluator (not a replacement string) so a "$"-bearing version cannot re-expand, plus a strict major.minor.patch guard that throws on a malformed base, leaving manifests untouched. - A base-only version.json bump (0.1 -> 0.2) is correctly detected and stamped. VERIFIED End-to-end against a real NBGV git harness: content-scoped predict, base-only bump -> x.y.0, docs-only -> [], weekly drift stamping, malformed-base guard, and --configfile restore (exit 0). actionlint passes on both workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address Copilot review feedback - Add missing plugins/dotnet-test-migration/version.json so it participates in versioning (it was the only plugin without one; manifests are at 0.1.0). - CONTRIBUTING: the two manifests are not byte-identical; say the version is duplicated across two manifest files instead. - weekly-version-sync: include version.json in commit attribution so a base-only bump is explained rather than showing 'no attributable commits'. - Get-NbgvInfo: capture nbgv stderr and include it in the thrown error so CI failures are diagnosable, while keeping stdout clean for JSON parsing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Amaury Levé
|
cbb6121e76 |
dotnet-test: trim skill descriptions to fit skill-menu under 15K (#816)
The plugin's rendered skill-menu was 15,428 chars, over the 15,000 limit. Compressed the verbose per-framework parentheticals in the longest polyglot descriptions and dropped a few redundant clauses, preserving all routing keywords. Aggregate name+description length now ~14.5K. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Amaury Levé
|
35ace775b3 |
Add dotnet-test-migration plugin and move migration skills there (#808)
Move the .NET test framework/platform migration skills (migrate-mstest-v1v2-to-v3, migrate-mstest-v3-to-v4, migrate-vstest-to-mtp, migrate-xunit-to-mstest, migrate-xunit-to-xunit-v3) and the test-migration orchestrator agent out of dotnet-test into a new dedicated dotnet-test-migration plugin, along with their evals. Update all marketplace manifests, READMEs, CODEOWNERS, and .vally.yaml accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |