Commit Graph

1746 Commits

Author SHA1 Message Date
Sevenzuo
f1e6b22c4b fix: honor minimum_should_match in GaussDB search (#18456)
### Summary

GaussDB DocEngine could return no chunks for conversational queries even
when relevant content was available. `Dealer.search()` supplies
`minimum_should_match` (30%, then 10% on retry), but the GaussDB adapter
discarded it and built a single `plainto_tsquery` from every token. This
effectively required all conversational filler terms to match.
2026-08-18 21:05:49 +08:00
Jack
ebd6be09c7 feat(title-chunker): configurable token-count cap with sentence-boundary re-split (#18455)
## Summary

The TitleChunker (both `hierarchy` and `group` methods) had no
token-size ceiling, so a long section without sub-headings became one
giant chunk. This adds a configurable `chunk_token_cap` that guarantees
every text chunk stays within a token budget.
2026-08-18 21:03:51 +08:00
Harsh Kashyap
b3d9e02d03 fix(model_meta): Base._get_api_key handles JSON-dict api_key for the model-list verify path (#18386) 2026-08-18 20:16:19 +08:00
Harsh Kashyap
5d164085f5 fix(model_meta): LocalAI._get_api_key handles JSON-dict api_key for the verify path (#18314)
## Summary

Closes #17757.
2026-08-18 20:15:18 +08:00
qinling0210
2aaa6baf0c fix(agentic-rag): raise max_parallel_agents for high/ultra to 4, use web search after locate fails repeatedly (#18430)
### Summary

1. It changes the fallback semantics of the locate phase. When no chunks
are found, the system stays in locate. If the same claim has two
consecutive locate rounds with neither evidence chunks nor newly routed
document scope, web_search is admitted to the candidate tool set on the
next locate round as an external fallback.

2. It makes locate_empty_streak claim-scoped instead of shared in the
global context. This prevents one claim’s empty locate rounds from
affecting sibling claims running in parallel.

3. On the config side, it only raises max_parallel_agents for high /
ultra to 4, without changing max_agent_cycles. This increases parallel
claim execution without deepening per-claim search.
2026-08-18 19:40:07 +08:00
Wang Qi
858f882d21 Fix task time is incorrect (#18453) 2026-08-18 19:29:25 +08:00
Lynn
eb4e1aa2a4 Fix: set enable_thinking to True for qwen3.8-2.4t-a95b (#18442) 2026-08-18 19:27:31 +08:00
ump45nose
15aad78518 fix: isolate extract html request headers (#18322) 2026-08-18 16:49:17 +08:00
Wang Qi
2a132d281d Fix parent chunk with extra newline (#18426) 2026-08-18 16:06:30 +08:00
Wang Qi
75f737de67 Refactor: make page number readable (#18409) 2026-08-18 13:27:04 +08:00
Lynn
344bd4d57a Fix: correct chunk order for non‑PDF documents (#18412) 2026-08-18 13:06:13 +08:00
Lynn
2368c54689 Fix: handle title and truncate document preview by sentence (#18407) 2026-08-18 11:54:55 +08:00
zws202606
d02723ca76 feat: Gaussdb adaptation (#17703) 2026-08-18 09:53:11 +08:00
Harsh Kashyap
4593d08821 fix(model_meta): Ollama._get_api_key handles JSON-dict api_key for the verify path (#18321) 2026-08-17 17:14:04 +08:00
qinling0210
f8a16ecef2 Fix _routed_docs leak across parallel claims in agentic research (#18348)
### Summary

Fix _routed_docs leak across parallel claims in agentic research
2026-08-17 12:28:55 +08:00
S
c6ba54bc72 fix: stop three recurring errors on doc delete (#17685) (#17686)
Follow-up to #17526 ("Refactor: merge dataset scope graph"), which introduced two code paths that touch Infinity columns the deployed schema does not declare. This PR makes the runtime robust against the old schema while also adding the new column to the new schema so freshly created tables are correct.
2026-08-16 09:25:51 +08:00
qinling0210
554fb1133a Search in navigation tree & store graph content in nav_doc (#18303)
### Summary

Search in navigation tree & store graph content in nav_doc during parse
2026-08-14 22:26:57 +08:00
buua436
024c35f085 fix: track wiki chunk changes incrementally (#18285) 2026-08-14 20:11:55 +08:00
Lynn
471070c2c8 Fix: return error when no valid speech text in intranscription response (#18277) 2026-08-14 16:27:34 +08:00
buua436
15a63bc15c fix: exclude disabled documents from wiki compilation (#18261) 2026-08-14 13:45:14 +08:00
Jack
423c8489b5 fix(chunker): carry overlap-head PDF positions into new chunk (#18148) (#18227)
When `TokenChunker` starts a fresh chunk with an overlap prefix (Go `computeOverlapPrefix` / Python visible-text cut), the previous chunk's **tail PDF coordinates were dropped**. As a result, the overlap head of a PDF chunk is displayed but **not highlighted** — the highlight box is shifted/truncated relative to the displayed span (infiniflow/ragflow#18148).
2026-08-13 22:18:33 +08:00
qinling0210
c5ff2bced5 Refine agentic RAG phase logging (#18228)
### Summary

Refine agentic RAG phase logging

example:
```
2026-08-13 17:25:06,733 INFO     2434554 [Agentic RAG] LLM usage by phase:                                                             phase            llm_calls prompt_tok   output_tok  total_tok    time(s)
  formalize              1        389          151        540        1.7
  route                  1        287          234        521        2.7
  planner                1      11857          832      12689        6.8
  orchestrator round 1       0          0            0          0      171.5
    claim_research (2)       8      82519        23146     105665      131.0
    sufficiency            2      10122         1709      11831       15.0
    grounded               1       5846         3232       9078       24.7
  finalize               2       9945         2228      12173       19.9
  total: 16 LLM calls, 152497 tokens
```
2026-08-13 20:15:24 +08:00
buua436
a4e819504c feat: support entity and topic wiki modes (#18216) 2026-08-13 19:03:24 +08:00
qinling0210
a6dcc0f665 Add dataset navigation search for agentic RAG (#18218)
### Summary

Add dataset navigation search for agentic RAG.

dataset_navigation_search() calls
dataset_api_service.search_dataset_layers with mode as "nav_doc"
2026-08-13 17:35:24 +08:00
Jack
913777d965 fix(book): preserve PDF coordinate tags in the naive merge branch (#18203)
This PR fixes **#18193** — the Python `rag/app/book.py` naive-branch `split("@")` bug that destroys PDF coordinate (`@@`) tags, so chunks lose their clickable page highlight.
2026-08-13 17:00:34 +08:00
Wang Qi
bedc395472 Fix invalid query string (e.g. ?) will error search (#18217) 2026-08-13 16:43:15 +08:00
Ziyang Guo
9dae986d75 fix(tests): isolate token chunker imports (#18018) 2026-08-13 15:43:11 +08:00
Wang Qi
7e536c7614 Fix agentic chat low/medium cannot answer the table content (#18196) 2026-08-13 14:40:31 +08:00
iridescentWen
a0cc2be2dc docs: document the tenant_id parameter in resume parsing (#18169)
Second ragflow PR (#17639 merged 2026-07-31). Docstrings only, no
behavior change.

---

🤖 Written with [Claude Code](https://claude.com/claude-code). Each site
was opened and read, and
each description was checked against the line that consumes the value.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:21:26 +08:00
buua436
649b631cac fix: improve incremental wiki compilation (#18164) 2026-08-12 19:12:39 +08:00
euvre
a530a3a170 fix: clean up wiki blueprint default instruction text (#17893) 2026-08-12 14:39:36 +08:00
Wang Qi
3e4a1fa887 Add doc_name for compilation generated chunk (#18126) 2026-08-12 09:37:25 +08:00
Charles
add00924e7 fix: replace editdistance with rapidfuzz for python 3.13 wheel support (#18132)
### Summary

Fixes #18107.

`editdistance==0.8.1` (the only recent release on PyPI) has no cp313
wheels for any platform. Since this project requires exactly Python
3.13, `uv`/`pip`/`poetry` fall back to building it from source (Cython),
which fails on Windows for anyone without a working C build toolchain —
that's the PEP 517 build error in the issue.

Swapped `editdistance` for `rapidfuzz`, which ships full cp313 wheels
(win32/win_amd64/win_arm64 included) and has no build-from-source step
on any of our target platforms. The only call site was
`EntityResolution.is_similarity` in `rag/graphrag/entity_resolution.py`,
using `editdistance.eval(a, b)` to get the unweighted Levenshtein
distance between two entity names.
`rapidfuzz.distance.Levenshtein.distance(a, b)` computes the same thing
(verified identical output on several string pairs) and is used as a
direct replacement.
2026-08-11 21:44:36 +08:00
Ziyang Guo
75363af54b fix(mistral): honor dataset language in figure prompts (#18021)
### Summary

Refs #17885.

Mistral figure enrichment now receives the dataset language through the
production parsing path. `by_mistral_ocr` forwards `lang` to
`MistralParser.parse_pdf`; the parser stores the normalized language and
passes it to the figure-description prompt. Empty or missing values
still fall back to English.
2026-08-11 20:38:24 +08:00
buua436
0cfd8f41e4 fix: improve incremental wiki compilation (#18130)
### What problem does this PR solve?

Incremental Wiki compilation could lose provenance for claim-light
entities, produce unstable page groups across embedding models, route
entities to unrelated pages, and assign topics without sufficient
page-level context. Document removals and page membership changes could
also leave stale Wiki state.

This PR:

- preserves source document and chunk provenance throughout entity
matching, reduction, page generation, and deletion;
- uses embeddings to retrieve candidates and the LLM to make final page
grouping and incremental routing decisions;
- batches embedding and LLM operations with bounded concurrency and
deterministic fallbacks;
- selects source-scoped topic candidates with embeddings before the page
LLM chooses the final topic;
- rebuilds Wiki state when the compilation mode or embedding model
changes;
- normalizes Wiki array fields returned by the API and retains entities
without relations in graph responses.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-08-11 20:13:04 +08:00
nikminer
8bd5768ebc Integrate MWS model with API support and enhance chat functionality (#17959)
## What

This pull request adds **MWS GPT Model Hub** as a built-in model
provider in RAGFlow.

The integration allows users to configure an MWS project endpoint and
token, discover the models available to that project, and use supported
MWS models for chat completion, embeddings, and reranking.

Co-authored-by: ilarionov_n <ilarionov_n@promis.ru>
2026-08-11 19:12:42 +08:00
Lynn
cd6996b301 Fix: xinference asr (#18110) 2026-08-11 19:07:50 +08:00
Yingfeng
fbcb8656ca Revert "Refine agentic search & orchestration loop" (#18108) 2026-08-11 18:53:49 +08:00
qinling0210
d49af7f218 Generate navigation, navigation search (#18096)
### Summary
2 API

POST /api/v1/datasets/{dataset_id}/navigation

GET
/api/v1/datasets/{dataset_id}/navigation/search?q={query}&mode={mode}&top_k={topk}


2 cli

uv run --no-sync python3 admin/client/ragflow_cli.py -h 127.0.0.1 -p
9380 -t user

ragflow> GENERATE NAVIGATION OF DATASET 'frames tree';

ragflow> NAVIGATION SEARCH 'Christie introduced blockchain-based digital
passports' IN DATASET 'frames tree' MODE 'all' topk 20;

(mode: chunk, nav_cluster, nav_doc, navigation_tree, all)
2026-08-11 17:48:24 +08:00
Yingfeng
f1641228e2 Refine agentic search & orchestration loop (#18057)
## Summary

This PR improves the RAGFlow agentic-search path in three areas: it
stops the outer agent from re-looping over the same rag call, lets the
medium thinking mode discover and follow new sub-claims mid-loop, and
strengthens retrieval by having the LLM emit synonym-rich queries with
time/date/number terms boosted.

1. Avoid the outer re-loop — keep all multi-hop cycles inside agentic
RAG

2. Dynamic claims in medium mode — keep querying newly discovered
sub-questions
medium now enables allows_dynamic_claims. During orchestration, when
claim analysis discovers a new required sub-question
(discovered_claims), the loop spawns it as a new ClaimTarget and
continues searching it in subsequent cycles (bounded by the
dynamic-claim budget) instead of stopping. Also added:

3. Stronger query strategy — synonym-rich queries + time/date/number
weighting

LLM-generated synonyms: the claim-analysis prompt now instructs the
model to write each next_queries entry as a retrieval-boosted query that
actively folds in entity aliases, DATE/TIME synonyms (e.g. 1994 → 1994,
66th Academy Awards), and number/unit variants (e.g. 1.95 m → 6 ft 5
in).

Time/date/number boosting: query.py boosts numeric/date tokens to a high
weight (_NUM_DATE_TOKEN_RE).
2026-08-11 13:40:11 +08:00
Lynn
15ab2b3ee2 Fix: update tree when delete skill node (#18063) 2026-08-11 09:49:22 +08:00
Lynn
7ac59d1a36 Fix: set column type priority (#18054) 2026-08-10 20:08:06 +08:00
Ziyang Guo
eb184c839c chore(wiki): remove stray debug print (#18020)
### Summary

Remove a stray `print()` from the Wiki source-context fallback.
2026-08-10 18:22:44 +08:00
buua436
6f50e478e0 fix: exclude disabled documents from dataset structures (#18041) 2026-08-10 17:50:34 +08:00
黑墨水鱼
4eaad55efe fix: align LocalAI vision constructor arguments (#16920)
## What problem does this PR solve?

`TenantLLMService.model_instance` constructs vision providers with
`lang` as the third positional argument and `base_url` as a keyword
argument.

`LocalAICV` declared `base_url` as its third parameter, causing:

```text
TypeError: LocalAICV.__init__() got multiple values for argument 'base_url'
```

This prevents LocalAI vision models from being used during document
parsing.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-10 15:59:06 +08:00
buua436
c0582b8e18 fix: handle disabled documents in dataset structures (#18033) 2026-08-10 13:52:49 +08:00
Yingfeng
b5bffa0fa3 Refine sufficient check using LLM draft (#18028) 2026-08-10 11:44:47 +08:00
Wang Qi
3d41ebdded Fix naive ask report error (#18030) 2026-08-10 11:21:23 +08:00
S
99110c2df0 chore(rag/app): remove stray debug print() calls (#17943)
chore(rag/app): remove stray debug print() calls

Two hot-path debug print() calls were leaking content/error text to
stdout in production code paths.

* rag/app/naive.py: TxtParser branch in chunk() was printing the entire
  parsed sections list (formatted via repr()) wrapped in 150-char banner
  lines. For large text documents (e.g. a 1000+-page book ingest) this
  dumped tens of thousands of lines per ingest into the docker logs.
  Replaced with a structured
  `logging.info("TxtParser produced %d sections for %s", len(sections),
  filename)` so the parse count is still observable without the content
  leak.

* rag/app/presentation.py: Pdf.position parsing had a debug
  `print(f"Error parsing position: {e}")` inside an except clause in the
  ingest hot path. Replaced with
  `logging.warning(f"Error parsing position in {filename}: {e}")` to
  match the file's existing logging pattern and add filename context.

Both call sites already had logging imported; no new imports added.
logging was used throughout the surrounding code in the same
logging.{info,warning,error}(...) style.
2026-08-08 15:53:16 +08:00
Yingfeng
4cc2dbc067 More stable sufficient check for agentic search (#17962) 2026-08-07 22:10:22 +08:00