mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Jakub Florkowski
cf626334ee
The gate keeps flapping on these three, and the artifacts show why: with 4 scenarios, ONE tie is mathematically disqualifying. 4 scenarios, 3W/1T/0L -> mean +30%, CI [-1.8%, +61.8%] FAIL 5 scenarios, 4W/1T/0L -> mean +32%, CI [+9.8%, +54.2%] PASS maui-collectionview already demonstrates this: it scored 4W/1T/0L on the last run and PASSED, because the regression-guard scenario added earlier gave it a 5th data point. The other three had no such margin. That the flapping is sampling noise rather than content is now directly evidenced - verified with git diff that the content was byte-identical across these runs: maui-theming714b5975✅ 4W/0T/0L ->6d1201c3❌ 3W/1T/0L (identical) maui-shell-nav1db8796d✅ 4W/0T/0L ->714b5975❌ 3W/1T/0L (identical) maui-dependency-inj1db8796d✅ 4W/0T/0L ->714b5975❌ 3W/1T/0L (identical) and the tie drifts between scenarios run to run rather than sticking to one. So this commit adds a 5th scenario to each, each one guarding a real bug this PR fixed - the same pattern @AbhitejJohn asked for on maui-collectionview: - maui-theming, "Swap theme dictionaries without destroying app styles": requires the Remove-based swap. Guards the MergedDictionaries.Clear() bug (verified at runtime: Clear() drops the merged count 2 -> 1 and the template's "Primary" resource stops resolving). - maui-shell-navigation, "Stable routes for deep linking into tabs": requires an explicit Route= on ShellContent. Guards the auto-generated D_FAULT_{TypeName}{n} names (verified at runtime: D_FAULT_ShellContent2 and D_FAULT_ShellContent5 - non-sequential and order-dependent). - maui-dependency-injection, "Diagnose a page whose injected dependencies are missing": output-not-matches on the "silently null / no exception is thrown" claim. Guards the misconception corrected earlier in this PR. Two things I got wrong and corrected while building these: - The theming guard first used output-not-matches on Clear(). That failed the SKILLED arm 0/3 - because the correct answer legitimately *names* Clear() when diagnosing the cause. Replaced with a positive requirement for .Remove(, which is the reliable signal. - The theming prompt first described the symptom ("every button lost its styling"), which handed baseline the answer; it tied 0/0/0. Reworded to just ask for the switching code, so the skill's knowledge is what differentiates. Measured after both corrections (n=3/scenario, executor claude-opus-4.6, judge gpt-5.5): maui-dependency-injection 15W/0T/0L mean +0.60 maui-shell-navigation 13W/2T/0L mean +0.55 maui-theming 13W/2T/0L mean +0.39 [+0.26, +0.51] Zero losses across all 45 trials. All three new scenarios pass in the skilled arm; the DI and shell guards fail in the baseline arm, which is the point. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 58513e0c-4086-4112-9ea1-95dbd8e8e469