mirror of
https://github.com/countbot-ai/CountBot.git
synced 2026-09-14 20:46:47 +08:00
fix(deps): un-comment jieba so default installs get word-level CJK tokenization
BM25Index already supports jieba and falls back to per-character tokenization when it is missing. Because the dependency was commented out, a default 'pip install -r requirements.txt' silently ran the degraded path, so the wiki search quality users actually get is well below what the code (and our published eval numbers in #107) assume. Measured impact on the 60-question eval (with vs without jieba): - direct-question recall@5: 0.80 -> 0.53 - paraphrased-question recall@5: 0.20 -> 0.10 (no top-1 hit at all) - exact-fact recall@10: 1.00 -> 0.80 One-line change, no code changes: the graceful fallback stays for anyone who deliberately removes it. Refs: #107
This commit is contained in:
+1
-1
@@ -60,7 +60,7 @@ mcp>=1.0.0
|
||||
|
||||
# ── Wiki 知识库中文分词(可选,提升搜索质量)──
|
||||
# 不安装会自动回退为单字分词,不影响基本功能
|
||||
# jieba>=0.42
|
||||
jieba>=0.42
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user