Abhitej John
ca1b47371f
* Add automated per-plugin versioning (NBGV) with /version-bump + weekly backstop WHY Tools that surface skills (Copilot CLI, Claude Code, Codex, Cursor) read a plugin's version directly from its checked-in manifest. With no versioning discipline, a plugin's behavior can change while its advertised version stays flat, so clients never learn to re-pull, and there is no human-readable signal of what changed. We want correct, current versions in the repo with minimal manual work and without bloating the marketplace clone. WHAT - Per-plugin semantic versioning via Nerdbank.GitVersioning (NBGV). Each plugin owns a version.json whose pathFilters exclude the generated manifests and the version.json itself, so version height tracks real content changes only. - The computed version is materialized into the checked-in manifests (plugin.json and .codex-plugin/plugin.json) so every consumer reads a current value with no build step on their side. - eng/version/Sync-PluginVersions.ps1 is the single workhorse. It resolves the set of changed plugins from a git diff, computes each version with nbgv (predicting the squash-merge height for PRs), and either reports or stamps. AUTOMATIONS (two, low-touch by design) - /version-bump: an admin/maintainer comments the command on a PR and the affected plugins are stamped on the PR branch. Gated on collaborator permission (admin/write/maintain); forks are rejected before any privileged step. No other PRs are auto-modified. - weekly-version-sync: a Monday backstop (and workflow_dispatch) that stamps any drift on main, opens/updates a single bot PR, and explains the per-plugin reason. This self-heals anything that merged without a bump. We deliberately did NOT auto-edit contributor PRs or add a noisy advisory comment bot; maintainers stay in control and the signal stays clean. SECURITY (multi-model adversarial review: GPT-5.5 + Gemini 3.1 Pro) - Supply chain (High, both models): dotnet tool restore would have honored a nuget.config authored in the PR tree, letting an attacker remap the nbgv package source to a malicious feed and run code in the privileged contents:write context. Mitigated with a trusted eng/version/nuget.config (clear + nuget.org-only + packageSourceMapping), overlaid from main and used via --configfile so PR-supplied configs are ignored. No nuget.config is tracked in the repo today, so this path was genuinely exploitable. - TOCTOU (Medium): /version-bump now checks out the authorized head SHA rather than the mutable branch name; a racing push fails non-fast-forward, which is the safe outcome. - Injection: Set-ManifestVersion uses a MatchEvaluator (not a replacement string) so a "$"-bearing version cannot re-expand, plus a strict major.minor.patch guard that throws on a malformed base, leaving manifests untouched. - A base-only version.json bump (0.1 -> 0.2) is correctly detected and stamped. VERIFIED End-to-end against a real NBGV git harness: content-scoped predict, base-only bump -> x.y.0, docs-only -> [], weekly drift stamping, malformed-base guard, and --configfile restore (exit 0). actionlint passes on both workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address Copilot review feedback - Add missing plugins/dotnet-test-migration/version.json so it participates in versioning (it was the only plugin without one; manifests are at 0.1.0). - CONTRIBUTING: the two manifests are not byte-identical; say the version is duplicated across two manifest files instead. - weekly-version-sync: include version.json in commit attribution so a base-only bump is explained rather than showing 'no attributable commits'. - Get-NbgvInfo: capture nbgv stderr and include it in the thrown error so CI failures are diagnosable, while keeping stdout clean for JSON parsing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dotnet-test
Skills and agents for running, generating, analyzing, and improving tests. Originally built for .NET (MSTest, xUnit, NUnit, TUnit) and platforms (VSTest, Microsoft.Testing.Platform); the test-generation pipeline and the six test-analysis skills (anti-patterns, smells, assertion quality, gap analysis, tagging, grade tests) plus the test-quality-auditor agent are polyglot and also work with Python (pytest/unittest), TypeScript/JavaScript (Jest/Vitest/Mocha/Jasmine/node:test), Java (JUnit 4/5/TestNG), Go (testing/testify), Ruby (RSpec/Minitest), Rust (built-in/proptest), Swift (XCTest/Swift Testing), Kotlin (JUnit/Kotest), PowerShell (Pester), and C++ (GoogleTest/Catch2/doctest/Boost.Test).
Test framework/platform migration (MSTest/xUnit upgrades, xUnit → MSTest, VSTest → Microsoft.Testing.Platform) lives in the separate
dotnet-test-migrationplugin.
When to use this plugin
- Run tests (.NET only) — execute
dotnet testwith automatic platform/framework detection and filter syntax - Generate tests (polyglot) — scaffold comprehensive unit tests for any language via a multi-agent pipeline
- Migrate tests (.NET only) — see the separate
dotnet-test-migrationplugin (MSTest v1/v2 → v3 → v4, xUnit v2 → v3, xUnit → MSTest, VSTest → Microsoft.Testing.Platform) - Audit test quality (polyglot) — detect anti-patterns, test smells, assertion gaps, and (for .NET) coverage risks
- Improve testability (.NET only) — find static dependencies, generate wrappers, and migrate call sites to injectable abstractions
- Measure coverage (.NET only) — collect code coverage, compute CRAP scores, and surface risk hotspots
Skills
Test execution
| Skill | Description |
|---|---|
| run-tests | Run .NET tests via dotnet test with platform/framework auto-detection and filter support |
| mtp-hot-reload | Rapid test-fix iteration using MTP hot reload (edit code → re-run without rebuilding) |
Test generation
| Skill | Description |
|---|---|
| code-testing-agent | Multi-agent pipeline (Research → Plan → Implement → Build → Test → Fix → Lint) that generates tests for any language |
| writing-mstest-tests | Best practices and modern APIs for writing MSTest 3.x/4.x tests |
Test migration
Moved to the dotnet-test-migration plugin (migrate-mstest-v1v2-to-v3, migrate-mstest-v3-to-v4, migrate-xunit-to-xunit-v3, migrate-xunit-to-mstest, migrate-vstest-to-mtp, and the test-migration orchestrator agent).
Test quality & analysis (polyglot)
These six skills are all polyglot. They work across all supported languages by loading a per-language reference file from test-analysis-extensions. grade-tests additionally embeds its own scoring rubric (sub-grades, weighting, anti-pattern catalog) so the per-test grades stay consistent across calls.
| Skill | Description |
|---|---|
| test-anti-patterns | Quick pragmatic scan for common test quality issues with severity ranking (any language) |
| test-smell-detection | Deep formal audit using academic test smell taxonomy (19 smell types, any language) |
| assertion-quality | Measure assertion variety and depth — find shallow tests that barely verify anything (any language) |
| test-gap-analysis | Pseudo-mutation analysis to find test blind spots that coverage numbers miss (any language) |
| test-tagging | Tag tests with standardized traits (smoke, regression, boundary, critical-path, etc.); auto-edits where the framework has canonical syntax, report-only otherwise |
| grade-tests | Grade a curated list of test methods individually and produce a compact, PR-comment-friendly table of letter grades (A–F), score bands, and one-line notes — designed for per-PR test-quality feedback (any language) |
Coverage & risk (.NET only)
| Skill | Description |
|---|---|
| coverage-analysis | Project-wide code coverage collection with CRAP score computation and risk hotspot reporting |
| crap-score | Calculate CRAP (Change Risk Anti-Patterns) scores for individual methods, classes, or files |
For non-.NET languages, use the native coverage tool: coverage.py/pytest-cov (Python), jest --coverage/c8/nyc/vitest --coverage (JS/TS), JaCoCo (Java), go test -coverprofile (Go), SimpleCov (Ruby), cargo-tarpaulin/cargo-llvm-cov (Rust), xcrun llvm-cov (Swift), Kover (Kotlin), Pester's built-in code coverage (PowerShell), gcov/llvm-cov (C++).
Testability improvement (.NET only)
| Skill | Description |
|---|---|
| detect-static-dependencies | Scan C# code for hard-to-test statics (DateTime.Now, File.*, HttpClient, etc.) |
| generate-testability-wrappers | Generate wrapper interfaces or guide adoption of built-in abstractions (TimeProvider, IFileSystem) |
| migrate-static-to-wrapper | Bulk-replace static call sites with injected wrapper calls and add constructor injection |
Reference data (loaded by other skills)
| Skill | Description |
|---|---|
| code-testing-extensions | Language-specific guidance loaded by the code-testing pipeline (test generation) |
| test-analysis-extensions | Language-specific guidance loaded by the polyglot analysis skills (test markers, assertion APIs, sleeps, skips, mystery-guest indicators, integration markers, tag-support capability) |
| platform-detection (.NET) | Detect VSTest vs MTP and identify the test framework from project files |
| filter-syntax (.NET) | Test filter syntax reference for VSTest and MTP across all frameworks |
Agents
User-facing agents
These are the entry-point agents you invoke directly:
| Agent | Purpose |
|---|---|
| code-testing-generator | Orchestrates the full test generation pipeline (research → plan → implement → build → test → fix → lint) |
| test-quality-auditor | Runs multi-skill audit pipelines for comprehensive test suite assessment |
| testability-migration | End-to-end testability improvement: detect → generate wrappers → migrate call sites |
Test framework/platform migration is handled by the
test-migrationagent in the separatedotnet-test-migrationplugin.
Internal subagents
These are pipeline stages invoked automatically by the agents above (user-invocable: false). You do not need to call them directly:
| Agent | Called by | Purpose |
|---|---|---|
| code-testing-researcher | code-testing-generator | Analyzes codebase structure, testing patterns, and testability |
| code-testing-planner | code-testing-generator | Creates phased test implementation plans from research findings |
| code-testing-implementer | code-testing-generator | Implements one phase from the plan, runs build-test-fix cycles |
| code-testing-builder | code-testing-implementer | Runs build/compile commands and reports results |
| code-testing-tester | code-testing-implementer | Runs test commands and reports pass/fail results |
| code-testing-fixer | code-testing-implementer | Fixes compilation errors in source or test files |
| code-testing-linter | code-testing-implementer | Runs code formatting and linting |
VS Code — enabling full multi-level fan-out: The pipeline delegates in two levels:
code-testing-generator→ researcher / planner / implementer, andcode-testing-implementer→ builder / tester / fixer / linter. VS Code gates nested delegation (a subagent spawning its own subagents) behind a setting that is off by default, so the first level runs out of the box but the second one does not. For large scopes — many files or modules, where parallel build/test/fix/lint workers help — enable it in your VS Code settings:"chat.subagents.allowInvocationsFromSubagents": trueWithout it,
code-testing-implementerstill builds, tests, fixes, and lints — it just does that work inline instead of delegating to the worker subagents, so results are unaffected. The GitHub Copilot CLI has no such gate and always fans out.
Prerequisites
For polyglot skills and agents
The test-generation pipeline (code-testing-generator and friends) and the six test-analysis skills (test-anti-patterns, test-smell-detection, assertion-quality, test-gap-analysis, test-tagging, grade-tests) plus the test-quality-auditor agent work with any of the supported languages above. You just need a working test runtime for the language you're targeting (e.g., python + pytest, node + npm test, mvn / gradle, go, bundle exec rspec, cargo test, swift test, pwsh + Pester, cmake + your C++ test runner). The skills will detect the framework automatically.
For .NET-only skills and agents
- .NET SDK installed (
dotneton PATH) - A project with an existing test framework (MSTest, xUnit, NUnit, or TUnit) for execution, migration, coverage, CRAP, testability, and the experimental
dotnet-experimentalskills.