Complete eval correctness hardening

Add Vally 0.13 all-error compatibility, a practical net-win floor, objective completion criteria, and CI fault-injection coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 1c6716a2-172b-461a-adb5-127b1ba7e96b
This commit is contained in:
Abhitej John
2026-08-17 17:46:33 -07:00
parent 983a803141
commit 9faa838da9
8 changed files with 422 additions and 62 deletions
@@ -8,6 +8,7 @@ on:
- ".github/workflows/evaluation-workflow-tests.yml"
- "eng/evaluation-tools/**"
- "eng/evaluation/test_token_failover.py"
- "eng/vally-adapter/**"
push:
branches: [main]
paths:
@@ -16,12 +17,30 @@ on:
- ".github/workflows/evaluation-workflow-tests.yml"
- "eng/evaluation-tools/**"
- "eng/evaluation/test_token_failover.py"
- "eng/vally-adapter/**"
workflow_dispatch:
permissions:
contents: read
jobs:
vally-adapter:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v6
with:
persist-credentials: false
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22
- name: Run adapter fault-injection and report tests
run: node --test eng/vally-adapter/*.test.mjs
evaluation-tools:
runs-on: ubuntu-latest
timeout-minutes: 10