mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-05 15:20:30 +08:00
310275208b
## Summary Follow-up to #17835 (merged). The OVER_CAP / UNDER_CAP merge strategy was threaded through `mergeDecision` and `mergeByTokenSizeFromJSON` as an inlined `allowBoundaryOverflow bool` derived from `!c.param.UnderCap` at three call sites. This replaces that with a named `schema.MergeStrategy` enum. ## Why - The `!c.param.UnderCap` inversion was hand-written in three places, so a future strategy addition could silently drift between the JSON path (`invokeTextPayload` / `invokeJSONPayload`) and the text path (`mergeByTokenSize`) — no compile error, and the existing tests don't cover all three sites with both strategies. - The strategy concept was never named; `allowBoundaryOverflow` (true = OVER_CAP) is a double-negation of `UnderCap` and reads opaquely at the 5th positional argument. ## What changed - Add `schema.MergeStrategy` (`MergeOverCap` / `MergeUnderCap`) mirroring Python's `rag/nlp/__init__.py` `MergeStrategy`, so Go and Python stay on the same vocabulary. - Expose `TokenChunkerParam.MergeStrategy()` derived from the wire-facing `UnderCap bool` (existing `"under_cap"` configs keep working — no schema break). - `mergeDecision` and `mergeByTokenSizeFromJSON` now take `schema.MergeStrategy` instead of `allowBoundaryOverflow bool`; the three call sites pass `c.param.MergeStrategy()` (no `!`). - Tests updated to pass the enum; added a guard test for the `UnderCap` -> `MergeStrategy` mapping and an end-to-end test for UNDER_CAP on the JSON path. No behavior change: default remains OVER_CAP, `under_cap=true` still selects UNDER_CAP. ## Test plan `bash build.sh --test ./internal/ingestion/component/chunker/... ./internal/ingestion/component/schema/...` — all green, including `TestMergeByTokenSizeFromJSON_UnderCapNoOverflow`, `TestMergeByTokenSize_UnderCapNoOverflow`, `TestInvokeJSONPayload_UnderCapEndToEnd`, and `TestTokenChunkerParamMergeStrategy`. ## Related issues - Relates to #17835 — wired UNDER_CAP as a tested merge-strategy seam (merged) - Relates to #17799 — contract doc for token-chunker cap/delimiter alignment - Relates to #17808 — related chunker alignment work --------- Co-authored-by: CodeBuddy <noreply@cnb.cool>