fix(deps): un-comment jieba so default installs get word-level CJK tokenization

BM25Index already supports jieba and falls back to per-character
tokenization when it is missing. Because the dependency was commented
out, a default 'pip install -r requirements.txt' silently ran the
degraded path, so the wiki search quality users actually get is well
below what the code (and our published eval numbers in #107) assume.

Measured impact on the 60-question eval (with vs without jieba):
- direct-question recall@5: 0.80 -> 0.53
- paraphrased-question recall@5: 0.20 -> 0.10 (no top-1 hit at all)
- exact-fact recall@10: 1.00 -> 0.80

One-line change, no code changes: the graceful fallback stays for
anyone who deliberately removes it.

Refs: #107
This commit is contained in:
RIC
2026-09-03 17:13:12 +08:00
parent 969b800677
commit 28b635177a
+1 -1
View File
@@ -60,7 +60,7 @@ mcp>=1.0.0
# ── Wiki 知识库中文分词(可选,提升搜索质量)──
# 不安装会自动回退为单字分词,不影响基本功能
# jieba>=0.42
jieba>=0.42