- Updated the skill description to clarify usage and restrictions for refactoring requests.
- Added new test cases for preserving serialized contracts in CustomerProfile.
- Introduced CustomerProfile class to support new test scenarios.
The Vally file-contains and file-not-contains graders require the config key
'value' (per the schema and all main eval.yaml files); 'substring' is rejected
with [invalid-grader-config]. Verified with `vally lint --eval-spec`: both evals
now lint with 0 errors, matching main's passing evals (only the benign
scoring-defaults + config-deprecation warnings that main evals also emit).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
Adapts both eval.yaml files from the legacy scenarios/assertions format to
the stimuli/graders/config schema introduced by #877 (Migrate LLM evals to
the Vally harness). Prompts and rubrics are preserved verbatim; assertions
map 1:1 to graders (file_contains->file-contains, run_command_and_assert->
run-command, output_matches->output-matches) and copy_test_files->
environment.files. Validated with `vally experiment run --dry-run`: both evals
now resolve and match the experiment filter (previously matched none).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
Rely on the SDK-default language version per pinned TFMs (net8.0;net10.0) so fixture compile behavior does not drift with the installed SDK. Both fixtures build clean (0 warnings/errors) without it. Addresses reviewer feedback.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
The Vally file-contains and file-not-contains graders require the config key
'value' (per the schema and all main eval.yaml files); 'substring' is rejected
with [invalid-grader-config]. Verified with `vally lint --eval-spec`: both evals
now lint with 0 errors, matching main's passing evals (only the benign
scoring-defaults + config-deprecation warnings that main evals also emit).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
Adapts both eval.yaml files from the legacy scenarios/assertions format to
the stimuli/graders/config schema introduced by #877 (Migrate LLM evals to
the Vally harness). Prompts and rubrics are preserved verbatim; assertions
map 1:1 to graders (file_contains->file-contains, run_command_and_assert->
run-command, output_matches->output-matches) and copy_test_files->
environment.files. Validated with `vally experiment run --dry-run`: both evals
now resolve and match the experiment filter (previously matched none).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
Rely on the SDK-default language version per pinned TFMs (net8.0;net10.0) so fixture compile behavior does not drift with the installed SDK. Both fixtures build clean (0 warnings/errors) without it. Addresses reviewer feedback.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
Wendy Breiding (She/Her)