mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
fix(task11-12): simplify review findings — 5 fixes + 1 new test
Reviewers flagged 3 must-fix + 3 minor items on commits365826d+7124342. Important: - check_prisma_trAIce_freshness.py: catch yaml.YAMLError (previously unhandled — malformed YAML would surface as a traceback rather than a clean ERROR: line). Added test_malformed_yaml_fails_cleanly covering this path (26 tests total, all green). - Script's upstream_source fallback string was the repo slug 'cqh4046/PRISMA-trAIce'; the real frontmatter stores the full URL, and the fallback should match that shape for user clarity. - PERFORMANCE.md attribution was imprecise: "ARS does not persist session state inside Claude Code" conflates ARS with Claude Code's own session mechanism. Rewrite as "ARS does not maintain its own orchestrator state between sessions." Minor: - Remove dead 'import subprocess' in test file (unused; run_script handles subprocess). - Remove opaque '# Non-blocking: exit 0 per E6 in spec' comment; the module docstring already states non-blocking semantics. - PERFORMANCE.md v3.4.0 cost table: align column headers with the main cost table above (Skill / Mode, Input Tokens, Output Tokens, Estimated Cost — title-case) and add the "+" prefix to deltas + cross-reference the 15K-word / 60-ref estimate basis.
This commit is contained in:
+6
-6
@@ -36,16 +36,16 @@
|
||||
The full academic pipeline is designed for human-in-the-loop execution, with mandatory user confirmation at every stage. In practice, a full run often spans hours to days — longer than Anthropic's prompt cache TTL (5 minutes). Two consequences:
|
||||
|
||||
1. **Cache misses between checkpoints are normal.** When a stage checkpoint pauses longer than 5 minutes, the next stage reads its context uncached. This is an unavoidable cost of human-paced pipelines.
|
||||
2. **Cross-session resume relies on Material Passport.** ARS does not persist session state inside Claude Code. To resume in a new session, paste your Material Passport YAML back; the orchestrator reads `compliance_history[]` and stage completion markers to locate your breakpoint.
|
||||
2. **Cross-session resume relies on Material Passport.** ARS does not maintain its own orchestrator state between sessions. To resume in a new session, paste your Material Passport YAML back; the orchestrator reads `compliance_history[]` and stage completion markers to locate your breakpoint.
|
||||
|
||||
### v3.4.0 compliance agent cost
|
||||
|
||||
Adding the mode-aware `compliance_agent` to Stage 2.5 and Stage 4.5 increases full-pipeline SR tokens by approximately:
|
||||
|
||||
| Skill / mode | Input token delta | Output token delta | Estimated cost delta |
|
||||
| Skill / Mode | Input Tokens | Output Tokens | Estimated Cost |
|
||||
|---|---|---|---|
|
||||
| `deep-research systematic-review` (2.5 only) | ~5–8K | ~3–5K | ~$0.15 |
|
||||
| Full pipeline SR (2.5 + 4.5) | ~10–15K | ~5–8K | ~$0.30 |
|
||||
| `academic-paper full` (pre-finalize) | ~3–5K | ~2–3K | ~$0.08 |
|
||||
| `deep-research systematic-review` (2.5 only) | +~5–8K | +~3–5K | +~$0.15 |
|
||||
| Full pipeline SR (2.5 + 4.5) | +~10–15K | +~5–8K | +~$0.30 |
|
||||
| `academic-paper full` (pre-finalize) | +~3–5K | +~2–3K | +~$0.08 |
|
||||
|
||||
These are on top of the existing per-skill costs in the table above. Cross-model verification costs (if enabled) are unchanged.
|
||||
These are on top of the existing per-skill costs in the table above (same 15,000-word / 60-reference basis; see footnote on line 23). Cross-model verification costs (if enabled) are unchanged.
|
||||
|
||||
@@ -36,16 +36,16 @@
|
||||
完整 pipeline 設計為 human-in-the-loop,每個階段都需使用者確認。實務上一次完整執行會跨越數小時到數天,遠長於 Anthropic 的 prompt cache TTL(5 分鐘)。兩項結果:
|
||||
|
||||
1. **階段間 cache miss 是常態。** 當 stage checkpoint 停留超過 5 分鐘,下一階段會以未快取狀態讀取 context。這是 human-paced pipeline 不可避免的成本。
|
||||
2. **跨 session 續跑依賴 Material Passport。** ARS 不在 Claude Code 內保留 session state;要在新 session 續跑,把 Material Passport YAML 貼回即可。orchestrator 讀取 `compliance_history[]` 與階段完成標記定位中斷點。
|
||||
2. **跨 session 續跑依賴 Material Passport。** ARS 本身不跨 session 保留 orchestrator 狀態。要在新 session 續跑,把 Material Passport YAML 貼回即可;orchestrator 讀取 `compliance_history[]` 與階段完成標記定位中斷點。
|
||||
|
||||
### v3.4.0 compliance agent 成本
|
||||
|
||||
在 Stage 2.5 與 Stage 4.5 加上 mode-aware `compliance_agent` 會讓 SR 全 pipeline token 多出:
|
||||
|
||||
| Skill / mode | Input token 增量 | Output token 增量 | 成本增量 |
|
||||
| Skill / 模式 | 輸入 Token | 輸出 Token | 估算費用 |
|
||||
|---|---|---|---|
|
||||
| `deep-research systematic-review`(僅 2.5)| ~5–8K | ~3–5K | ~$0.15 |
|
||||
| 全 pipeline SR(2.5 + 4.5)| ~10–15K | ~5–8K | ~$0.30 |
|
||||
| `academic-paper full`(pre-finalize)| ~3–5K | ~2–3K | ~$0.08 |
|
||||
| `deep-research systematic-review`(僅 2.5)| +~5–8K | +~3–5K | +~$0.15 |
|
||||
| 全 pipeline SR(2.5 + 4.5)| +~10–15K | +~5–8K | +~$0.30 |
|
||||
| `academic-paper full`(pre-finalize)| +~3–5K | +~2–3K | +~$0.08 |
|
||||
|
||||
以上是在既有 per-skill 成本之上額外產生。Cross-model verification 成本(若啟用)維持不變。
|
||||
以上為既有 per-skill 成本之上的額外增量(與上表共用 15,000 字 / 60 篇引用基準,見上表下方 footnote)。跨模型驗證成本(若啟用)維持不變。
|
||||
|
||||
@@ -62,7 +62,7 @@ def main() -> int:
|
||||
text = args.path.read_text(encoding="utf-8")
|
||||
fm = extract_frontmatter(text)
|
||||
snapshot = parse_snapshot_date(fm)
|
||||
except (FileNotFoundError, ValueError) as exc:
|
||||
except (FileNotFoundError, ValueError, yaml.YAMLError) as exc:
|
||||
print(f"ERROR: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
@@ -71,11 +71,10 @@ def main() -> int:
|
||||
print(
|
||||
f"WARNING: prisma_trAIce_protocol.md snapshot is {age_days} days old "
|
||||
f"(threshold {STALE_THRESHOLD_DAYS}). Upstream may have updated — "
|
||||
f"please review {fm.get('upstream_source', 'cqh4046/PRISMA-trAIce')} "
|
||||
f"please review {fm.get('upstream_source', 'https://github.com/cqh4046/PRISMA-trAIce')} "
|
||||
f"and re-sync if needed. (STALE status surfaced; non-blocking.)",
|
||||
file=sys.stderr,
|
||||
)
|
||||
# Non-blocking: exit 0 per E6 in spec
|
||||
else:
|
||||
print(f"OK: snapshot is {age_days} days old (current)")
|
||||
return 0
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
"""Unit tests for check_prisma_trAIce_freshness.py."""
|
||||
import subprocess
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
@@ -53,6 +52,17 @@ class TestFreshnessCheck(unittest.TestCase):
|
||||
result = run_script(SCRIPT, str(p))
|
||||
self.assertEqual(result.returncode, 1)
|
||||
|
||||
def test_malformed_yaml_fails_cleanly(self) -> None:
|
||||
with TemporaryDirectory() as tmp:
|
||||
p = Path(tmp) / "prisma_trAIce_protocol.md"
|
||||
p.write_text(
|
||||
'---\nsnapshot_date: "2026-03-01\nunclosed_quote: "yes\n---\n# body\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
result = run_script(SCRIPT, str(p))
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("ERROR", result.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user