fix(benchmarks): make agentic fixture path portable via PONYTAIL_TMPL (#206)

tasks.py hardcoded _TMPL to a personal absolute Windows path (D:\dev\...), which the agentic benchmark's Reproduce flow depends on. Resolve it via a PONYTAIL_TMPL env override with a relative fallback, mirroring the existing PONYTAIL_PLUGIN_DIR convention in run.py; run.py's loader resolves the relative name under fixtures/. selftest skips the open tasks that consume _TMPL, so instrument validation is unaffected (verified: all instruments valid).
This commit is contained in:
Hamza Ali Shahjahan
2026-07-09 06:13:52 +05:00
committed by GitHub
parent a3e0169c3d
commit fb72987fb5
2 changed files with 5 additions and 2 deletions
+2 -1
View File
@@ -112,7 +112,8 @@ python complete.py --run runs/<stamp> # completeness-score every workspace
## Reproduce
Needs the `claude` CLI (this is the harness, no SDK), Python 3, an authenticated Claude Code, and a
clone of the template at the pinned commit (point `_TMPL` in `tasks.py` at it):
clone of the template at the pinned commit (set `PONYTAIL_TMPL` to its path, or drop it at
`fixtures/full-stack-fastapi-template`):
```bash
git clone https://github.com/fastapi/full-stack-fastapi-template
+3 -1
View File
@@ -25,7 +25,9 @@ from pathlib import Path
# Real-repo fixture: tiangolo/full-stack-fastapi-template @ cd83fc1 (v0.10.0, MIT), cloned locally.
# Reproduce: git clone https://github.com/tiangolo/full-stack-fastapi-template && git -C ... checkout cd83fc1
_TMPL = r"D:\dev\fullstack-fastapi-template"
# Point PONYTAIL_TMPL at your local clone, or drop it at fixtures/full-stack-fastapi-template
# (run.py resolves a relative name under fixtures/). Mirrors the PONYTAIL_PLUGIN_DIR override.
_TMPL = os.environ.get("PONYTAIL_TMPL", "full-stack-fastapi-template")
# --- helpers ---
_imp_n = 0