skills/ruvnet__ruflo
1
0
Fork 0
mirror of https://github.com/ruvnet/ruflo.git synced 2026-09-14 14:01:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

darwin: iter 30 — arguana dense=2.0 flat regression; weight axis exhausted

Browse Source
This commit is contained in:
ruv
2026-06-26 19:29:12 -04:00
parent e61dde18ea
commit bf10046c4e
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/darwin/log.jsonl
+1
View File
@@ -28,3 +28,4 @@
{"iter":27,"ts":"2026-06-26T23:00:51.427Z","dimension":"beir-arguana","change":"validate-stacked-iters-1-26-on-arguana","measured":{"ndcg10":0.4386,"baseline":0.4324},"deltaScore":0.0062,"action":"measured-improved","commit":"","note":"Arguana first measurement with full stack: 0.4324 → 0.4386 (+0.0062, +1.4% rel). Rank still 5/11 but now TIED with GTR-XL. Gap to SOTA 0.55: 0.118 → 0.111 (6% closed). Halt counter for arguana: 0/3. NOTE: surprisingly modest gain despite weights match nfcorpus/scifact recipe — arguana symmetric queries may favor different ratio."}
{"iter":28,"ts":"2026-06-26T23:11:07.116Z","dimension":"beir-arguana","change":"arguana-bm25-weight-0.2-to-0.0","measured":{"ndcg10":0.4311,"prior":0.4386},"deltaScore":-0.0075,"action":"noImprovement","commit":"","note":"Pure dense REJECTED for arguana. Unlike nfcorpus/scifact, arguana symmetric retrieval benefits from BM25 lexical signal at 0.2 weight. Reverted to dense=1.0/bm25=0.2 (champion). Latest file restored. Arguana halt: 1/3. Hypothesis: scientific lexical overlap (citations, technical terms) carries information here that pure dense embeddings miss."}
{"iter":29,"ts":"2026-06-26T23:19:53.666Z","dimension":"beir-arguana","change":"arguana-bm25-weight-0.2-to-0.5","measured":{"ndcg10":0.4311,"prior":0.4386},"deltaScore":-0.0075,"action":"noImprovement","commit":"","note":"Both directions from 0.2 regress. arguana 0.4386 (bm25=0.2) is a local optimum on the BM25-weight axis. Halt: 2/3 for arguana."}
{"iter":30,"ts":"2026-06-26T23:29:12.728Z","dimension":"beir-arguana","change":"arguana-dense-weight-1.0-to-2.0","measured":{"ndcg10":0.4384,"prior":0.4386},"deltaScore":-0.0002,"action":"noImprovement","commit":"","note":"Boosting dense from 1.0 to 2.0 (keeping bm25=0.2) regressed -0.02% (essentially flat). dense=1.0 confirmed as RRF optimum for arguana. Arguana halt: 3/3 — would trigger halt on this dim per plan, but global halt requires noImprovement across ALL dims in 3 consecutive ticks. Other dims still active."}