skills/dotnet__skills
1
0
Fork 0
mirror of https://github.com/dotnet/skills.git synced 2026-09-20 09:49:54 +08:00
Code Issues Packages Projects Releases Wiki Activity

Improve cross-model test migration outcomes (#1096)

Browse Source 
* Improve test migration skill outcomes

Address cross-model routing and execution failures found in retained evaluation evidence, and align the xUnit companion-package rubric with feed-compatible versions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Cover MTP method filter routing

Include the method-filter token in the VSTest-to-MTP skill description so method-specific migration requests activate consistently.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Clarify MSTest SDK v4 discovery guidance

Scope the missing Test SDK reference to MSTest.Sdk v4 and remove a fixed package-version example in favor of feed-resolved and CPM-aware guidance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Clarify xUnit MTP runner preservation

State that the migration preserves the MTP runner for projects currently using the YTest shim, rather than preserving the shim package itself.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Avoid xUnit package major assumptions

Resolve companion compatibility from package constraints and configured feeds instead of examples that conflate the xUnit product generation with NuGet major versions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* List all MSTest SDK discovery paths

Align the CI response contract with the three documented choices for transitional VSTest discovery and MTP-native execution.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Fix xUnit v3 evaluation evidence

Replace a contradictory async-void output grader with a file-state check, strengthen deterministic BeforeAfter signature checks, and require explicit final evidence for both migration paths.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Align MSTest SDK discovery guidance

Correct the behavioral symptom table to describe the v4-only removal and all three documented migration paths consistently.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Make BeforeAfter eval name agnostic

Validate the required xUnit v3 override and base-call shapes without hardcoding local parameter names.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Strengthen Luna migration outcomes

Add complete advisory output contracts, correct focused-fix execution decisions, require concrete MTP package and filter guidance, and make xUnit package and source evidence explicit.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Relax BeforeAfter source formatting

Keep deterministic IXunitTest source validation while relying on compilation and test execution for qualified and multiline signature correctness.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Amaury Levé
2026-08-31 19:15:52 +02:00
committed by GitHub
parent a7a6170880
commit f775756a79
6 changed files with 258 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plugins/dotnet-test-migration/skills/migrate-mstest-v1v2-to-v3/SKILL.md
+31 -17
View File
@@ -1,21 +1,15 @@
---
name: migrate-mstest-v1v2-to-v3
description: >
Migrate MSTest v1/v2 projects to MSTest v3, and fix v1/v2-to-v3 breaking
changes that surface after the packages are already at 3.x.
USE FOR: removing v1
Microsoft.VisualStudio.QualityTools.UnitTestFramework assembly references;
moving MSTest.TestFramework/TestAdapter 1.x-2.x to 3.x, the MSTest
metapackage, or MSTest.Sdk; tests that broke after a 2.x-to-3.x bump --
CS0411/CS1503 on Assert.AreEqual/AreNotEqual/AreSame once the object
overloads became generic, and DataRow strict type matching (1L vs 1) that
builds with MSTEST0014 but fails at run time; .testsettings/LegacySettings
to .runsettings (DeploymentEnabled, per-test MSTest TestTimeout); v3 timeout
behavior; TFMs v3 dropped (net5.0, .NET Fx below 4.6.2, netstandard1.0).
Applies even when the project already references MSTest 3.x, if a v1/v2-era
setting or error remains. Keeps the current runner.
DO NOT USE FOR: MSTest v4 (use migrate-mstest-v3-to-v4 next), clean v3
projects with no v1/v2 leftovers, other test frameworks, or VSTest-to-MTP.
Upgrade, compare, or repair MSTest v1/v2 projects during migration to v3.
Use for QualityTools assembly references; MSTest.TestFramework/TestAdapter
1.x-2.x; choosing the MSTest metapackage or MSTest.Sdk; "what breaking
changes should I expect?"; CS0411/CS1503 after a v3 package bump; DataRow
type mismatch or MSTEST0014; .testsettings/LegacySettings to .runsettings;
timeout changes; and dropped v3 TFMs such as net5.0. Also use when packages
already say 3.x but v1/v2 source or settings remain, and when asked whether
v1 and v2 migration steps differ. Preserve VSTest/MTP. Do not use for clean
v3 projects, v3-to-v4, another test framework, or runner-only migration.
license: MIT
---
@@ -70,6 +64,14 @@ This overrides all steps below.
> while a search is still reporting project files, and never scaffold a
> substitute project from the prose description as a workaround.
## Execution Contract
- Skill activation is not a stopping point. Continue with workspace discovery and the requested work in the same task.
- The skill directory contains guidance, not the staged project. Search the current working directory, open the literal paths returned by the search, and retry with another available reader/editor if one tool rejects a valid path.
- Never ask for a path while a glob or directory search can discover it. Ask only after an exhaustive current-workspace search finds no project, or multiple projects make the target genuinely ambiguous.
- Classify the requested deliverable, not isolated verbs: "make the edits", "update this project", or "then build and run" means execute; "what do I need to change?", "what should I expect?", "are the steps the same?", or "show me" means answer, even if the prompt also says upgrade or migrate.
- After changing files, name the detected MSTest version and runner, the files changed, the exact source/settings decisions, and the clean test result. Do not claim VSTest preservation, a build, or passing tests without evidence from the project.
## Breaking Changes Summary
MSTest v3 introduces these breaking changes from v1/v2. Address only the ones relevant to the project:
@@ -100,6 +102,7 @@ MSTest v3 introduces these breaking changes from v1/v2. Address only the ones re
- **Change nothing on suspicion -- confirm the error first**: When you believe a construct is unsupported, build and read the actual diagnostic before editing it. If it compiles, this version supports it and it needs no change. Rewriting valid code to dodge a limit the project is not subject to is a defect, not caution.
- **Specific feature migration** (user asks about one aspect like .testsettings, DataRow, or assertions): Address only that feature, but handle every active setting or affected usage in the supplied files. For `.testsettings`, put all MSTest settings under one `<MSTest>` element, map requested deployment, per-test timeout, data collector, and other active configuration, and do not add a session-wide timeout. Do not walk through unrelated breaking changes.
- **"What to expect" questions** (user asks about breaking changes before upgrading): First state the concrete package update needed to reach v3, then summarize every category in the Breaking Changes Summary, marking which ones directly apply to the visible project. Keep each item to one line and do not expand into release-note history.
- **Required shape for "what to expect"**: Use an `Applies / Watch / No change` table grounded in the visible project and cover every row in the Breaking Changes Summary. This completeness is the value of the skill; do not omit runtime-only categories merely to be concise.
- **Full migration requests** (user wants complete migration): Follow the complete workflow below.
- **Comparison questions** (user asks about v1 vs v2 differences): Explain concisely -- v1 uses assembly references and requires removing them first; v2 uses NuGet and just needs a version bump. Both converge on the same v3 packages and breaking changes.
- **Keep execution project-specific**: For fixes and full migrations, change only patterns found in the visible code/configuration. Broader coverage is reserved for explicit "what should I expect?" questions.
@@ -152,7 +155,11 @@ Keep `Microsoft.NET.Test.Sdk` when the project remains on VSTest, but update it
Change `<Project Sdk="Microsoft.NET.Sdk">` to `<Project Sdk="MSTest.Sdk/3.8.0">`. MSTest.Sdk automatically provides the MSTest framework, adapter, and analyzers.
> **Important**: MSTest.Sdk defaults to Microsoft.Testing.Platform (MTP). When preserving VSTest, set `<UseVSTest>true</UseVSTest>`; the SDK then supplies the required `Microsoft.NET.Test.Sdk` reference. Do not switch runners merely as a side effect of the framework upgrade.
> **Important**: MSTest.Sdk defaults to Microsoft.Testing.Platform (MTP). When the
> project itself must remain on VSTest, set `<UseVSTest>true</UseVSTest>`. MSTest.Sdk
> v3 also supplies `Microsoft.NET.Test.Sdk` in MTP mode, so a separate transitional
> `vstest.console` invocation does not by itself require changing the primary runner.
> Do not switch runners merely as a side effect of the framework upgrade.
When switching to MSTest.Sdk, remove these (SDK provides them automatically):
@@ -229,6 +236,13 @@ Key mappings:
> **Important**: Map timeout to `<MSTest><TestTimeout>` (per-test), **not** `<TestSessionTimeout>` (session-wide). Remove `<LegacySettings>` entirely.
Update every project, CI command, or IDE setting that explicitly selected the old
`.testsettings` path to select the new `.runsettings` path. When a VSTest project
must preserve behavior but the legacy file was never selected, make the new file
effective with `RunSettingsFilePath`. For MTP, use the framework-supported
`--settings` path or existing MTP configuration instead of assuming the VSTest
MSBuild property is honored.
### Step 7: Verify
1. Run the same test command, filter, and configuration used for the baseline. `dotnet test` builds by default; run a separate build only to isolate a compilation failure.
@@ -255,4 +269,4 @@ After v3 migration, use `migrate-mstest-v3-to-v4` for MSTest v4.
| "No project on disk" right after a search listed project files | The path was rebuilt under the skill's base directory. Reopen using the literal search result; never scaffold a replacement project |
| Rewriting a `DataRow` with more than 16 arguments | Valid on 3.0.3+, which is every current 3.x. Only 3.0.1/3.0.2 ever rejected it |
| Non-MSTest.Sdk VSTest project missing `Microsoft.NET.Test.Sdk` | Add the package reference for VSTest discovery |
| MSTest.Sdk tests not found by `vstest.console` | Set `<UseVSTest>true</UseVSTest>`; MSTest.Sdk then supplies `Microsoft.NET.Test.Sdk` |
| MSTest.Sdk v3 project must use VSTest as its primary runner | Set `<UseVSTest>true</UseVSTest>`; do not flip the runner merely because a transitional `vstest.console` job also exists |
plugins/dotnet-test-migration/skills/migrate-mstest-v3-to-v4/SKILL.md
+68 -22
View File
@@ -1,20 +1,15 @@
---
name: migrate-mstest-v3-to-v4
description: >
Fix build errors and breaking changes after upgrading MSTest v3 to v4, or
plan a complete v3-to-v4 migration. Use when user says "upgrade to
MSTest v4", "MSTest 4 migration", "MSTest v4 breaking changes", "tests don't
compile after upgrading MSTest 3.x to 4.x", or hits CS0507, CS0103, CS1061, CS1615 after updating MSTest packages to 4.x.
USE FOR: Execute to ExecuteAsync, CallerInfo ctor on TestMethodAttribute,
sealed custom attributes, ClassCleanupBehavior removal, TestContext.Properties
Contains to ContainsKey, Assert.ThrowsException to ThrowsExactly,
Assert.IsInstanceOfType out param removal, ExpectedExceptionAttribute
removal, TestTimeout enum removal, [TestMethod("name")] to DisplayName syntax,
TreatDiscoveryWarningsAsErrors, TestContext.TestName in ClassInitialize,
MSTest.Sdk MTP changes, dropped TFMs (net6.0/net7.0 to net8.0+).
DO NOT USE FOR: MSTest v1/v2 to v3, or errors left from a 2.x-to-3.x upgrade
(use migrate-mstest-v1v2-to-v3 first); test framework conversions; general
.NET upgrades.
Upgrade MSTest 3.x projects to v4 or fix v4 migration failures. Use for
"MSTest v4 breaking changes", tests that stop compiling or behave differently
after 3.x-to-4.x, CS0507/CS0103/CS1061/CS1615, ExecuteAsync or CallerInfo in
custom TestMethodAttribute, DisplayName, ClassCleanupBehavior, ContainsKey,
ThrowsExactly/ExpectedException, IsInstanceOfType out parameters, TestTimeout,
ManagedType, net6/net7 compatibility, TestCase.Id history, TestName in
ClassInitialize, TreatDiscoveryWarningsAsErrors, and MSTest.Sdk/MTP or
vstest.console discovery changes. Do not use for v1/v2-to-v3 leftovers,
framework conversion, runner-only migration, or a general .NET upgrade.
license: MIT
---
@@ -45,20 +40,34 @@ Migrate a test project from MSTest v3 to MSTest v4. The outcome is a project usi
| Build command | No | How to build (e.g., `dotnet build`, a repo build script). Auto-detect if not provided |
| Test command | No | How to run tests (e.g., `dotnet test`). Auto-detect if not provided |
## Decisions That Change the Result
| Detected request or state | Required action |
|---|---|
| Files are supplied in the current workspace | Search there and open the literal returned paths. The skill directory is not the project directory. If one tool rejects a valid path, retry with another available reader/editor; do not ask the user for a path you can discover. |
| User asks to apply changes to supplied files: "fix my project/files", "please update this source", "make the changes", or "then build and run" | Edit every affected occurrence. Run the narrowest meaningful build/test command against the actual package version; skill activation is not a reason to stop at advice. |
| User asks "what should I expect?", "how do I fix these changes?", for compatibility advice, or for a plan | Answer directly from the actual project state even when source is visible. Keep a single-symptom answer focused; include only adjacent risks that change the decision. |
| Unsupported TFM in a full migration | Update the TFM first, then update MSTest packages, then fix source breaks. Do not bury this order in a release-note inventory. |
| Custom `TestMethodAttribute` subclass | Treat `ExecuteAsync`, CallerInfo propagation, display-name handling, and the subclass's retry/result semantics as one coupled migration. Fix the actual class, not a placeholder example. |
| `MSTest.Sdk` v4 plus `vstest.console` | This is a v4 change: MTP mode no longer brings `Microsoft.NET.Test.Sdk`. Keep MTP and add that package for transitional VSTest discovery, opt into `UseVSTest`, or migrate CI to `dotnet test`; state which runner the choice preserves. |
## Response Guidelines
- **Always identify the current version first**: Before recommending any migration steps, explicitly state the current MSTest version detected in the project (e.g., "Your project uses MSTest v3 (3.8.0)"). This confirms you've read the project files and grounds the migration advice.
- **Resolve, do not assume, the target version**: When the user asks for "latest", query the project's configured package source and select the latest stable MSTest v4 version available at execution time. Never copy the example version from this skill into the result without checking it. Keep all MSTest packages on the same resolved version.
- **Focused fix requests** (user has specific compilation errors after upgrading): Address only the relevant breaking changes from Step 3. When project files are available and the user asks to fix or update them, edit every affected occurrence and run the narrowest relevant test command; do not stop at generic advice. **Always provide concrete fixed code** using the user's actual types and method names — show a complete, copy-pasteable code snippet, not just a description of what to change. For custom `TestMethodAttribute` subclasses, show the full fixed class including CallerInfo propagation to the base constructor. Mention any related analyzer that could have caught this earlier (e.g., MSTEST0006 for ExpectedException). Do not walk through the entire migration workflow.
- **Focused fix requests** (user has specific compilation errors after upgrading): Address only the relevant breaking changes from Step 3. Make edits only when the requested deliverable is a source change; "how do I fix these?" remains an answer request. **Always provide concrete fixed code** using the user's actual types and method names. If the fixture still references v3, do not claim that a green v3 build verifies v4 compatibility; either update packages when requested or state the verification boundary. For custom `TestMethodAttribute` subclasses, show the full fixed class including CallerInfo propagation to the base constructor. Mention any related analyzer that could have caught this earlier (e.g., MSTEST0006 for ExpectedException). Do not walk through the entire migration workflow.
- **"What to expect" questions** (user asks about breaking changes before upgrading): Present ALL major breaking changes from the Step 3 quick-lookup table -- not just the ones visible in the current code. For each, provide a one-line fix summary. Also mention key behavioral changes from Step 4 (especially TestCase.Id history impact and TreatDiscoveryWarningsAsErrors default). If project code is available, highlight which changes apply directly.
- **Full migration requests** (user wants complete migration): Follow the complete workflow below.
- **Behavioral/runtime symptom reports** (user describes test execution differences without build errors): Match described symptoms to the behavioral changes table in Step 4. Provide targeted, symptom-specific advice. Mention other behavioral changes the user should watch for. Do not walk through source breaking changes unless the user also has build errors.
- **CI/test-discovery issues** (tests not discovered, vstest.console stopped working, CI pipeline failures after upgrading): Focus on 4.5 (MSTest.Sdk defaults to MTP mode, which does not include Microsoft.NET.Test.Sdk -- needed for vstest.console) and 4.4 (TreatDiscoveryWarningsAsErrors). Explain the root cause clearly and give both fix options (add Microsoft.NET.Test.Sdk package or switch to `dotnet test`). Do not walk through the full migration workflow.
- **CI/test-discovery issues** (tests not discovered, vstest.console stopped working, CI pipeline failures after upgrading): Focus on 4.5 (MSTest.Sdk v4 no longer includes Microsoft.NET.Test.Sdk in its default MTP mode -- it is still needed for vstest.console) and 4.4 (TreatDiscoveryWarningsAsErrors). Explain the root cause clearly and give all three paths: add Microsoft.NET.Test.Sdk while preserving MTP, set `UseVSTest` to switch the project runner, or switch CI to MTP-native `dotnet test`. Do not walk through the full migration workflow.
- **Explanatory questions** (user asks "is this a known change?", "what else should I watch out for?"): Explain the relevant changes and advise. Mention related changes the user might encounter next. Do not prescribe a full migration procedure.
- **Result proof**: End implementation work with the detected v3 version, resolved v4 version, runner choice, files changed, and actual build/test counts. Never report a build, VSTest compatibility, discovery, or passing tests from inference.
## Workflow
> **Commit strategy:** Commit at each logical boundary -- after updating packages (Step 2), after resolving source breaking changes (Step 3), after addressing behavioral changes (Step 4). This keeps each commit focused and reviewable.
> **Commit strategy:** Do not create commits unless the user asks. Keep package,
> source, and behavioral changes logically separable in the diff, but finish and
> verify the requested migration.
### Step 1: Assess the project
@@ -158,6 +167,24 @@ public class MyTestMethodAttribute : TestMethodAttribute
}
```
If the subclass has its own display-name constructor, do not pass that string to
the v4 base constructor. Propagate only caller information and assign the
`DisplayName` property:
```csharp
public sealed class NamedTestMethodAttribute : TestMethodAttribute
{
public NamedTestMethodAttribute(
string displayName,
[CallerFilePath] string callerFilePath = "",
[CallerLineNumber] int callerLineNumber = -1)
: base(callerFilePath, callerLineNumber)
{
DisplayName = displayName;
}
}
```
**If you use `[TestMethodAttribute("Custom display name")]`**, switch to the named parameter syntax:
```csharp
@@ -364,7 +391,7 @@ These changes won't cause build errors but may affect test runtime behavior.
| `TestContext.TestName` throws in `[ClassInitialize]` | v4 enforces lifecycle scope (4.2) | Move access to `[TestInitialize]` or test methods |
| Tests not discovered / discovery failures | `TreatDiscoveryWarningsAsErrors` now true (4.4) | Fix warnings, or set to false in .runsettings |
| Tests hang that didn't before | AppDomain disabled by default (4.1) | Set `DisableAppDomain` to false in .runsettings `RunConfiguration` |
| vstest.console can't find tests with MSTest.Sdk | MSTest.Sdk defaults to MTP; `Microsoft.NET.Test.Sdk` only added in VSTest mode (4.5) | Add explicit package reference or switch to `dotnet test` |
| vstest.console can't find tests with MSTest.Sdk after the v4 upgrade | MSTest.Sdk defaults to MTP; v4 stopped adding `Microsoft.NET.Test.Sdk` in MTP mode (4.5) | Add an explicit package while preserving MTP, set `UseVSTest`, or switch CI to `dotnet test` |
| New warnings from analyzers | Analyzer severities upgraded (4.6) | Fix warnings or suppress in .editorconfig |
#### 4.1 DisableAppDomain defaults to true
@@ -408,9 +435,26 @@ v4 uses stricter defaults. Discovery warnings are now treated as errors, which m
#### 4.5 MSTest.Sdk and vstest.console compatibility
MSTest.Sdk defaults to Microsoft.Testing.Platform (MTP) mode. In MTP mode, MSTest.Sdk does **not** add a reference to `Microsoft.NET.Test.Sdk` -- it only adds it in VSTest mode. This is not a v4-specific change; it applies to MSTest.Sdk v3 as well. Without `Microsoft.NET.Test.Sdk`, `vstest.console` cannot discover or run tests and will silently find zero tests. This commonly surfaces during migration when a CI pipeline uses `vstest.console` but the project uses MSTest.Sdk in its default MTP mode.
MSTest.Sdk defaults to Microsoft.Testing.Platform (MTP) mode. MSTest.Sdk v3
still added `Microsoft.NET.Test.Sdk` in that mode; v4 removes the unnecessary
reference. A CI pipeline that separately invokes `vstest.console` can therefore
drop to zero discovered tests immediately after the v4 upgrade.
**Option A -- Switch to VSTest mode**: Set the `UseVSTest` property. MSTest.Sdk will then automatically add `Microsoft.NET.Test.Sdk`:
**Option A -- Preserve MTP and transitional VSTest discovery**: Add the exact
compatible `Microsoft.NET.Test.Sdk` package explicitly. This is the least
disruptive fix when MTP remains the primary runner but an existing
`vstest.console` job cannot be removed yet:
Use a direct `PackageReference` with the exact compatible version resolved from
the configured feed. Under Central Package Management, add or update the
`Microsoft.NET.Test.Sdk` `PackageVersion` in `Directory.Packages.props` and keep
the project reference versionless. Do not copy a fixed example version.
Verify with the actual `vstest.console` command; a passing `dotnet test` MTP run
does not prove VSTest discovery.
**Option B -- Switch the project to VSTest mode**: Set the `UseVSTest` property.
MSTest.Sdk then adds `Microsoft.NET.Test.Sdk`:
```xml
<PropertyGroup>
@@ -421,9 +465,11 @@ MSTest.Sdk defaults to Microsoft.Testing.Platform (MTP) mode. In MTP mode, MSTes
Keep the resolved exact `MSTest.Sdk` v4 pin from Step 2; this option changes the runner, not the
selected MSTest version or target framework.
**Option B -- Switch CI to `dotnet test`**: Replace `vstest.console` invocations in your CI pipeline with `dotnet test`. This works natively with MTP and is the recommended long-term approach for MSTest.Sdk projects.
**Option C -- Switch CI to `dotnet test`**: Replace `vstest.console` invocations in your CI pipeline with `dotnet test`. This works natively with MTP and is the recommended long-term approach for MSTest.Sdk projects.
If you need VSTest during a transition period, Option A works without changing CI pipelines.
Do not say this behavior predates v4: removal of the transitive
`Microsoft.NET.Test.Sdk` reference in MTP mode is one of the v4 behavioral
breaking changes.
#### 4.6 Analyzer severity changes
plugins/dotnet-test-migration/skills/migrate-vstest-to-mtp/SKILL.md
+48 -30
View File
@@ -1,22 +1,16 @@
---
name: migrate-vstest-to-mtp
description: >
Migrates .NET test projects from VSTest to Microsoft.Testing.Platform (MTP).
Use when user asks to "migrate to MTP", "switch from VSTest", "enable
Microsoft.Testing.Platform", "use MTP runner", set OutputType=Exe only for
test projects in Directory.Build.props, or mentions EnableMSTestRunner,
EnableNUnitRunner, or UseMicrosoftTestingPlatformRunner.
USE FOR: MTP behavioral differences vs VSTest (exit code 8, zero tests
discovered, --ignore-exit-code, TESTINGPLATFORM_EXITCODE_IGNORE);
centralizing MTP properties and OutputType=Exe on test projects via
MSBuildProjectName, not IsTestProject.
Supports MSTest, NUnit, xUnit.net v2 (via YTest.MTP.XUnit2), and
xUnit.net v3. Covers runner enablement, CLI argument and filter
translation (--filter-class/--filter-trait/--filter-query),
global.json config, CI/CD updates, and extension packages.
DO NOT USE FOR: migrating between test frameworks (MSTest/xUnit/NUnit),
xUnit.net v2 to v3 API migration, MSTest version upgrades, TFM upgrades,
or UWP/WinUI test projects.
Migrate .NET test projects and CI from VSTest to Microsoft.Testing.Platform
(MTP), or fix MTP migration behavior. Use for "switch from VSTest", MTP
runner enablement for MSTest/NUnit/xUnit, OutputType=Exe conditions in
Directory.Build.props, EnableMSTestRunner/EnableNUnitRunner/
UseMicrosoftTestingPlatformRunner, YTest.MTP.XUnit2, .NET 10 global.json
test.runner, --filter-class/--filter-method/--filter-trait/--filter-query translation,
VSTest@3 replacement, TRX/coverage/dump extensions, and exit code 8 or zero
tests. Supports xUnit v3 MTP filter work even during a v2-to-v3 upgrade. Do
not use for test-framework conversion, xUnit v2-to-v3 API migration, TFM,
UWP, or WinUI migration.
license: MIT
---
@@ -49,9 +43,21 @@ Migrate a .NET test solution from VSTest to Microsoft.Testing.Platform (MTP). Th
|-------|----------|-------------|
| Project or solution path | No | The `.csproj`, `.sln`, or `.slnx` entry point containing test projects. **Discover it yourself** by globbing the working directory; ask only when nothing is found or the choice is genuinely ambiguous |
| Test framework | No | MSTest, NUnit, xUnit.net v2, or xUnit.net v3. Auto-detected from package references |
| .NET SDK version | No | Determines `dotnet test` integration mode. Auto-detected via `dotnet --version` |
| .NET SDK version | No | Determines `dotnet test` integration mode. Prefer the repository's `global.json` or explicitly stated CI SDK; use host `dotnet --version` only when the repository does not pin or state one |
| CI/CD pipeline files | No | Paths to pipeline definitions that invoke `vstest.console` or `dotnet test` |
## Execution and Answer Contract
- Discover project, props, `global.json`, and pipeline files in the current working directory. Open literal search results; the skill directory is not the user's repository. Continue after skill activation and do not ask for a discoverable path.
- For an implementation request, edit the files, then validate the effective MSBuild properties, the translated command, test counts, and requested artifacts. For a question, provide one exact command/configuration for the detected SDK and framework rather than a menu of near-equivalents.
- Preserve all build arguments and avoid introducing `--no-build`, new settings files, or unrelated package upgrades. Never retain `--settings <file>` unless that file exists.
- For .NET 9 and earlier, show the `--` separator. For .NET 10 native MTP mode, explicitly say to remove it.
- When suppressing exit code 8 is intentional, warn that broad suppression can hide an accidental empty run caused by a bad filter; scope it to the known zero-test project/configuration.
- For an exit-code-8 question, show all three concrete forms: `--ignore-exit-code 8`, `TestingPlatformCommandLineArguments`, and `TESTINGPLATFORM_EXITCODE_IGNORE=8`.
- For xUnit v3 filters, give the directly usable `--filter-class`/`--filter-method`/`--filter-trait` command and explain AND behavior. When the resolved xUnit version supports it, include `--filter-query` path syntax for complex expressions and link the xUnit query-filter documentation.
- When translating reporters, coverage, or dumps, name each required package in the answer; a correct-looking option without its owning extension package is incomplete.
- Final results must name the framework runner opt-in, repository-selected SDK integration mode, exact translated command, extension packages, and verification evidence.
## Workflow
### Step 1: Assess the solution
@@ -60,7 +66,7 @@ Migrate a .NET test solution from VSTest to Microsoft.Testing.Platform (MTP). Th
- **MSTest**: References `MSTest` or `MSTest.TestAdapter`, or uses `MSTest.Sdk` (with `<IsTestApplication>` not set to `false`). Note: `MSTest.TestFramework` alone is a library dependency, not a test project.
- **NUnit**: References `NUnit3TestAdapter`
- **xUnit.net**: References `xunit` and `xunit.runner.visualstudio`
2. Check the .NET SDK version (`dotnet --version`) -- this determines how `dotnet test` integrates with MTP
2. Resolve the SDK used by the repository/CI from `global.json` or explicit user context. Fall back to `dotnet --version` only when neither exists; the agent host SDK must not silently override a stated .NET 8/9 CI target.
3. Check whether a `Directory.Build.props` file exists at the solution or repo root -- all MTP properties should go there for consistency
4. Check for `vstest.console.exe` usage in CI scripts or pipeline definitions
5. Check for VSTest-specific `dotnet test` arguments in CI scripts: `--filter`, `--logger`, `--collect`, `--settings`, `--blame*`
@@ -80,6 +86,11 @@ Migrate a .NET test solution from VSTest to Microsoft.Testing.Platform (MTP). Th
> ```
>
> Adjust the condition (e.g., `.EndsWith('Tests')`, `.Contains('.Test')`) to match the test project naming convention used in the repository.
>
> Put every applicable central property in that same condition: `OutputType`,
> the framework runner opt-in, and `TestingPlatformDotnetTestSupport` on .NET 9
> and earlier. Do not leave an unconditional runner property that still affects
> production projects.
### Step 3: Enable the framework-specific MTP runner
@@ -247,6 +258,12 @@ For complex expressions, use `--filter-query` with a path-segment syntax:
Each segment matches against: assembly name, namespace, class name, method name. Use `*` for "match all" in any segment. Documentation: <https://xunit.net/docs/query-filter-language>
Prefer the directly corresponding `--filter-class` / `--filter-method` /
`--filter-trait` flags when they express the original filter. Use
`--filter-query` only after validating the query with `--list-tests`; a
syntactically accepted query that selects zero tests is not a successful
translation.
#### Translation example
```shell
@@ -264,22 +281,23 @@ dotnet test -- --filter-query "/*/*/*IntegrationTests*/*[Category=Smoke]"
### Step 6: Install MTP extension packages (if needed)
If CI scripts use TRX reporting, crash dumps, or hang dumps, add the corresponding NuGet packages:
If CI scripts use TRX reporting, crash dumps, hang dumps, or coverage, add the
owning packages. Under Central Package Management, the project references are:
```xml
<!-- TRX report generation (replaces --logger trx) -->
<PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="1.6.2" />
<!-- Crash dump collection (replaces --blame-crash) -->
<PackageReference Include="Microsoft.Testing.Extensions.CrashDump" Version="1.6.2" />
<!-- Hang dump collection (replaces --blame-hang) -->
<PackageReference Include="Microsoft.Testing.Extensions.HangDump" Version="1.6.2" />
<!-- Code coverage (replaces --collect "Code Coverage") -->
<PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" Version="17.13.0" />
<ItemGroup>
<PackageReference Include="Microsoft.Testing.Extensions.TrxReport" />
<PackageReference Include="Microsoft.Testing.Extensions.CrashDump" />
<PackageReference Include="Microsoft.Testing.Extensions.HangDump" />
<PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" />
</ItemGroup>
```
Add matching `PackageVersion` entries in `Directory.Packages.props`. Without
Central Package Management, put exact versions resolved from the configured
feed on these references. Keep the versions compatible with the selected MTP
stack; do not copy fixed versions from migration guidance.
### Step 7: Update CI/CD pipelines
#### Azure DevOps
plugins/dotnet-test-migration/skills/migrate-xunit-to-mstest/SKILL.md
+24 -7
View File
@@ -1,12 +1,13 @@
---
name: migrate-xunit-to-mstest
description: >
Convert .NET test projects from xUnit.net v2 or v3 to MSTest v4. Use for
replacing xunit packages, [Fact]/[Theory], xUnit assertions, fixtures,
ITestOutputHelper, traits, skips, and xUnit parallelization with MSTest
equivalents while preserving the current VSTest or MTP runner.
DO NOT USE FOR: xUnit v2 to v3 upgrades, MSTest version upgrades, migrations
from NUnit/TUnit, or runner-only VSTest to MTP migrations.
Convert .NET tests from xUnit.net v2/v3 to MSTest v4 while preserving VSTest
or MTP. Use for replacing xunit packages, Fact/Theory/InlineData/MemberData,
assertions, IClassFixture/ICollectionFixture, ITestOutputHelper, TestContext
cancellation, traits/Owner, skips, timeouts, and xUnit parallelization. Also
use when a "convert xUnit to MSTest" request may already be migrated: inspect
and report the no-op. Do not use for xUnit v2-to-v3, MSTest upgrades,
NUnit/TUnit conversion, or runner-only VSTest-to-MTP migration.
license: MIT
---
@@ -20,6 +21,22 @@ Use this skill only when the project contains xUnit packages or source and the u
Do not combine this framework conversion with a target-framework upgrade or VSTest/MTP migration. Complete and verify one migration before starting another.
## Workspace Contract
- Continue after skill activation. Search the current working directory for the
staged project and source; never look for user files under this skill's base
directory.
- Open the literal paths returned by glob/search. If one reader or patch tool
rejects a path that was just found, retry with another available tool. Do not
ask the user for a path until current-workspace discovery is exhausted.
- Classify by the requested deliverable: "convert this project" means edit,
build, and test; "give me a plan" or "how would I convert it?" means answer.
Do not replace execution with "please provide the files" when files are present.
- The final response must state the source xUnit version, preserved runner,
changed files, each high-risk semantic mapping applied, and actual test
counts. Assertions about fixture lifetime, Owner mapping, cancellation, or
parallelization must be visible in the resulting source, not only prose.
## Response Mode
- **Full migration request:** inspect the project, make the edits, build, and run tests. Do not stop after giving a plan.
@@ -35,7 +52,7 @@ Apply these before the mechanical mapping:
| No xUnit package, namespace, attribute, or fixture remains | Stop. Make no file changes, report that migration is unnecessary, and run the existing `dotnet test` command once to prove the already-MSTest project is healthy. |
| Source uses VSTest | Keep the existing VSTest property/configuration. Prefer retaining and updating a source project's explicit `Microsoft.NET.Test.Sdk` pin; a repository that intentionally relies on the MSTest metapackage's transitive dependency may keep that convention. Do not introduce MTP properties. |
| Source uses MTP | Replace xUnit-specific MTP selection with MSTest MTP configuration. Prefer `MSTest.Sdk`; with the metapackage, set `EnableMSTestRunner=true` and `OutputType=Exe`. Preserve native-versus-bridged command integration, and do not add `<UseVSTest>true</UseVSTest>` or other VSTest-only configuration. |
| Source relies on xUnit's default parallelization | Add `[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]` when the current project has at least two independently runnable test classes. In a one-class project with no explicit parallel setting, omit it because class-level concurrency is not observable. Translate explicit `CollectionBehavior` or `xunit.runner.json` settings regardless of current class count. |
| Source relies on xUnit's default parallelization | Add `[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]` to a compiled `.cs` file when the current project has at least two independently runnable test classes. In a one-class project with no explicit parallel setting, omit it because class-level concurrency is not observable. Translate explicit `CollectionBehavior` or `xunit.runner.json` settings regardless of current class count. Before reporting completion, read the changed file back and name it in the result. |
For detailed mappings and examples, search [`references/mapping-cheatsheet.md`](references/mapping-cheatsheet.md) for constructs actually present in the project and read only the matching sections. Do not load or reproduce the whole reference.
plugins/dotnet-test-migration/skills/migrate-xunit-to-xunit-v3/SKILL.md
+78 -14
View File
@@ -1,13 +1,15 @@
---
name: migrate-xunit-to-xunit-v3
description: >
Migrates .NET test projects from xUnit.net v2 to xUnit.net v3.
USE FOR: upgrading xunit to xunit.v3.
DO NOT USE FOR: migrating between test frameworks (MSTest/NUnit to
xUnit.net), migrating from VSTest to Microsoft.Testing.Platform
(use migrate-vstest-to-mtp). For xUnit v3 MTP filter syntax
(--filter-class, --filter-trait, --filter-query), also load
migrate-vstest-to-mtp.
Migrate .NET test projects from xUnit.net v2 to xunit.v3 and fix v3 breaks.
Use for package/CPM conversion, OutputType=Exe, preserving the VSTest or MTP
runner (including projects currently using YTest.MTP.XUnit2), incompatible
TFMs, async void tests,
string-to-Type attributes, custom Fact/Theory/BeforeAfterTest attributes,
Xunit.SkippableFact, xunit.abstractions/extensibility consolidation, and
Xunit.Combinatorial/StaFact compatibility. Do not use for framework
conversion or a runner-only migration. For xUnit v3 MTP filter syntax, also
use migrate-vstest-to-mtp.
license: MIT
---
@@ -30,11 +32,32 @@ Migrate .NET test projects from xUnit.net v2 to xUnit.net v3. The outcome is a s
| Input | Required | Description |
|-------|----------|-------------|
| Test project or solution | Yes | The .NET project or solution containing xUnit.net v2 test projects |
| Test project or solution | No | Discover `.csproj`, `.sln`, `.slnx`, central props, and source in the current working directory; ask only if none are found or the target is ambiguous |
## Workspace and Completion Contract
- Skill activation is not completion. For migrate/fix/update requests, inspect
the staged files, edit them, and run tests in the same task.
- The skill base directory contains only guidance. Search the current working
directory and open paths exactly as returned. If a tool rejects a path just
found by search, retry with another available reader/editor instead of
concluding that files are missing.
- Do not ask the user to provide a path while workspace discovery can find it.
- Inventory project/central package files and all affected source in one pass.
A package-only migration is incomplete when v2-only APIs remain.
- End with the detected source version and runner, exact package compatibility
set, files changed, discovered/passed/failed/skipped counts, and any
platform-specific result. A build without test discovery is not success.
- Package validity is empirical: record the configured feed query or resolved
package graph and a successful restore. In the final result, say how the
selected exact versions were proven available; do not merely name a version
and leave a judge or user to infer whether it exists.
## Workflow
> **Commit strategy:** Commit after each major step so the migration is reviewable and bisectable. Separate project file changes from code changes.
> **Commit strategy:** Do not create commits unless the user asks. Keep project
> configuration and source edits logically separable in the diff, but finish
> and verify the whole requested migration.
> **Prioritization:** Steps 1-5 are required for every migration. Steps 6-12 are conditional — only apply the ones relevant to the project's code patterns. Skip steps that don't apply.
@@ -48,10 +71,19 @@ Run this preflight before editing:
| xUnit v2 uses `YTest.MTP.XUnit2` | Preserve MTP: remove that shim, set `UseMicrosoftTestingPlatformRunner=true`, and do not add `xunit.runner.visualstudio` or `IsTestingPlatformApplication=false`. |
| xUnit v2 does not use the MTP shim | Preserve VSTest: keep/update `xunit.runner.visualstudio` and set `IsTestingPlatformApplication=false`. |
| A custom type derives from `BeforeAfterTestAttribute` | Preserve that inheritance and its behavior. Add the `IXunitTest` parameter to both overrides and pass it to `base.Before`/`base.After`; do not replace the subclass with a direct interface implementation. |
| A Type-based collection/orderer attribute points to a custom type | Migrate both the attribute syntax and the referenced type's v3 contract. For a collection factory, implement the required xUnit v3 `IXunitTestCollectionFactory` behavior; compiling the attribute while leaving an empty factory is not a complete migration. |
| Companion packages are present | Resolve `xunit.v3`, Xunit.Combinatorial, and Xunit.StaFact as one compatible set from configured feeds. If the newest xunit.v3 major has no compatible stable companion on those feeds, select the newest compatible xunit.v3 major and explain the pin. Validate discovery, not just compilation. |
| `OutputType=Exe` makes a `net*-windows` project fail on a non-Windows host | Add `EnableWindowsTargeting=true` when cross-building is intended, then rerun. Do not dismiss this migration-induced failure as pre-existing. |
Resolve package versions from the configured package source. Do not infer a package version from
the product's "v3" name, invent a `4.0.0`, or update unrelated packages. Change only files that
contain a package, property, or source construct required by the applicable rule.
Resolve package versions from the configured package source. Do not guess a
version from the product's "v3" name or update unrelated packages. Change only
files that contain a package, property, or source construct required by the
applicable rule.
After editing a Central Package Management project, read back both
`Directory.Packages.props` and the project file. Confirm that `PackageVersion`
owns the version, the renamed `PackageReference` is versionless, and
`OutputType=Exe` is effective.
### Step 1: Identify xUnit.net projects and verify compatibility
@@ -116,6 +148,11 @@ Find any `using Xunit.Abstractions;` directives in C# files and remove them comp
In xUnit.net v3, `async void` test methods are no longer supported and will fail to compile. Search for any test methods declared with `async void` and change them to `async Task`. Test methods can be identified via the `[Fact]` or `[Theory]` attributes or other test attributes.
In the final result, state why the source changed: xUnit.net v3 rejects
`async void` tests, so each affected method now returns `Task`. Do not report
only the mechanical replacement. Also state how the exact package versions were
resolved from the configured feed.
### Step 7: Address breaking change of attributes (if applicable)
In xUnit.net v3, some attributes were updated so that they accept a `System.Type` instead of two strings (fully qualified type name and assembly name). These attributes are:
@@ -154,6 +191,12 @@ internal sealed class MyFactAttribute : FactAttribute
}
```
Before reporting completion, read back every affected `FactAttribute`- and
`TheoryAttribute`-derived constructor. Name each type and confirm that both
caller-info parameters and the corresponding `base(sourceFilePath,
sourceLineNumber)` forwarding are present. A passing test run alone does not
prove source information was propagated.
### Step 9: Inheriting from BeforeAfterTestAttribute (if applicable)
Identify if there are any custom attributes that inherit from `BeforeAfterTestAttribute`. These custom user-defined attributes must update their method signatures. Previously, they would have `Before`/`After` overrides that look like this:
@@ -197,6 +240,12 @@ existing base calls and their ordering relative to custom logic. Implementing
`IBeforeAfterTestAttribute` directly may compile, but it is not the mechanical v2-to-v3 migration
and can discard base-class behavior.
Before reporting completion, read the resulting attribute file and quote the
actual `Before(MethodInfo, IXunitTest)` and `After(MethodInfo, IXunitTest)`
signatures. Explicitly confirm that both `base.Before` and `base.After` receive
the same `IXunitTest` argument; a generic claim that the overrides were updated
is insufficient evidence.
### Step 10: Address new xUnit analyzer warnings (if applicable)
xunit.v3 introduced new analyzer warnings. The most notable is xUnit1051 (use `TestContext.Current.CancellationToken` for methods accepting `CancellationToken`). Address these if present.
@@ -212,6 +261,11 @@ Then, follow these steps to eliminate usages of APIs coming from the removed pac
- Change `Skip.If` method calls to `Assert.SkipWhen`.
- Change `Skip.IfNot` method calls to `Assert.SkipUnless`.
Verify both branches when the fixture makes that practical: the default
condition should report the intended skip reason, and the enabled condition
should execute and pass. A grep plus a generic passing run does not prove the
runtime skip semantics were preserved.
Limit this conversion to the existing project/central-package file and the source files containing
these APIs. Do not create a new `Directory.Build.props` merely to perform this companion-package
migration; any required runner property belongs in the existing test project when no shared props
@@ -219,8 +273,18 @@ file already exists.
### Step 12: Update companion packages (if applicable)
- `Xunit.Combinatorial` 1.x → latest 2.x
- `Xunit.StaFact` 1.x → latest 3.x
- Query the configured feeds for a mutually compatible set instead of resolving
each package independently. `Xunit.Combinatorial` 1.x moves to 2.x or later,
and `Xunit.StaFact` 1.x moves to a line compatible with the selected
`xunit.v3` major.
- Do not infer companion compatibility from the product name or matching major
numbers. Use package dependency constraints and the versions available on the
configured feeds, then prove the selected set through test discovery.
- On `net*-windows` projects built from Linux/macOS after switching to
executable output, set `EnableWindowsTargeting=true` if cross-targeting is
intended.
- Run tests and confirm expected platform skips (such as STA tests on Linux)
separately from failures.
### Step 13: Build and verify
tests/dotnet-test-migration/migrate-xunit-to-xunit-v3/eval.yaml
+9 -3
View File
@@ -73,9 +73,10 @@ stimuli:
- type: output-matches
config:
pattern: async Task
- type: output-not-matches
- type: file-not-contains
config:
pattern: async void
path: AsyncTests.cs
value: async void
- type: run-command
config:
command: dotnet test -p:TreatWarningsAsErrors=true
@@ -86,6 +87,7 @@ stimuli:
- Changes 'async void' to 'async Task' for test methods decorated with [Fact] or [Theory]
- Applies the change to both GetUser_ReturnsExpectedName and ProcessItem_Completes methods
- Explains that xUnit.net v3 no longer supports async void test methods and they will fail to compile
- States the exact xunit.v3 version selected and that configured-feed restore succeeded
- name: Convert string-based attribute constructors to typeof syntax
prompt: |
Migrate this test project from xUnit.net v2 to xUnit.net v3. Pay special
@@ -177,6 +179,10 @@ stimuli:
config:
path: DatabaseSetupAttribute.cs
value: base.After(
- type: file-contains
config:
path: DatabaseSetupAttribute.cs
value: IXunitTest
- type: run-command
config:
command: dotnet test -p:TreatWarningsAsErrors=true
@@ -389,6 +395,6 @@ stimuli:
- type: prompt
rubric:
- Updates Xunit.Combinatorial from 1.x to the latest compatible 2.x version
- Updates Xunit.StaFact from 1.x to the latest compatible 3.x version
- Updates Xunit.StaFact from 1.x to a stable version compatible with the selected xunit.v3 major available on the configured feed
- Also migrates the core xunit package to xunit.v3
- Sets OutputType to Exe