1939 Commits

Author SHA1 Message Date
Jin Hai
cf13082a1a Revert "feat: Go knowledge compiler with scheduler-driven dataset compilation" (#17897)
Reverts infiniflow/ragflow#17881
2026-08-05 21:50:28 +08:00
Mage Lu
d9d75690de fix: correct Tongyi-Qianwen international endpoint typo (compatible-model -> compatible-mode) (#17887)
## Summary
- `api/apps/services/provider_api_service.py` hardcoded the DashScope
international base URL for Tongyi-Qianwen as `.../compatible-model/v1`
instead of `.../compatible-mode/v1`, in two places (`list_providers`,
lines ~93 and ~116).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 20:37:45 +08:00
Zhichang Yu
14b943a04a feat: Go knowledge compiler with scheduler-driven dataset compilation (#17881)
Ports the dataset knowledge compilation (wiki/graph/tree/mindmap) to the
Go scheduler with a status contract, aligns wiki storage/retrieval with
Python, and sizes prompts by content_length.
2026-08-05 20:00:42 +08:00
buua436
38435255f9 fix: filter wiki artifacts by keywords (#17871) 2026-08-05 19:40:27 +08:00
chanx
2dfb34f7ab feat(agent-list): support multi-select canvas category filtering (#17843) 2026-08-05 19:32:45 +08:00
buua436
cd6e73bcb5 fix: stream agentic reasoning and answers correctly (#17849) 2026-08-05 15:59:19 +08:00
Lynn
02923bfc46 Fix: display embedding model name in list dataset api response (#17832) 2026-08-05 14:51:07 +08:00
Wang Qi
f057544e31 Fix: SIGTERM not handle well in ragflow server chat channel (#17828) 2026-08-05 11:52:53 +08:00
Yingfeng
302e611a43 Fix: throw exception when switching ingestion pipeline (#17833)
### Summary

Throw exception when switching pipeline :

OperationalError('Attempting to close database while transaction is open.!)
2026-08-05 11:44:15 +08:00
deadtrickster
f063cfdb19 fix(api): decrement knowledgebase counters on SDK re-parse / stop-parse (#17236) 2026-08-05 10:28:46 +08:00
EthanZhang
bdcd8aadde feat(chat): add Querit web search provider (#17813) 2026-08-05 09:54:46 +08:00
Wang Qi
f707cca074 Fix: Let delete dataset/document to a dedicated thread to avoid blocking othe APIs (#17800) 2026-08-04 19:04:16 +08:00
buua436
4d997bc740 fix: allow team members to update shared datasets (#17798) 2026-08-04 17:24:23 +08:00
Wang Qi
7e67b71b81 Let DELETE /datasets to a dedicated thread to avoid blocking other APIs (#17796) 2026-08-04 16:19:56 +08:00
deadtrickster
8c8c1e7df7 fix(api): default delimiter must be a real newline, not the r"\n" escape (#17591) 2026-08-04 16:16:34 +08:00
deadtrickster
197b142cef feat(serenedb): add SereneDB doc-store engine (Go + Python connectors) (#17375)
## What

Adds [**SereneDB**](https://serenedb.com) as a selectable doc-store
engine on **both** RAGFlow paths:
- the **Go** `DocEngine` (`internal/engine/serenedb`), alongside
Elasticsearch and Infinity;
- the **Python** `DocStoreConnection` (`rag/utils/serenedb_conn.py`) +
`DOC_ENGINE=serenedb` registration.

SereneDB is a PostgreSQL-wire engine (DuckDB execution) whose single
inverted index carries **both** a scored text column (`@@`, BM25) and an
IVF vector column (`<#>`, inner product), so hybrid search is one SQL
statement. The Go engine connects with `database/sql` + `lib/pq`
(already a dependency, no new module); the Python connector uses
psycopg2 (already a dependency).

## Storage model

One table per tenant with `kb_id` as a filter column - the
**Elasticsearch / OceanBase** model, not Infinity's per-dataset tables.
This keeps BM25 statistics (IDF, avgdl) computed over the whole tenant
corpus (global IDF). Both connectors use this identical layout, so they
are storage- and retrieval-compatible: `hybrid` proxy routing and
Python↔Go switching are safe. On the Python side the connector is wired
as OceanBase's plain-SQL sibling (chunk_data JSON metadata, inline chunk
vectors, verbatim ES field names); the ES tokenizer path is unchanged.
Metadata stays one table per tenant (`ragflow_doc_meta_<tenant>`).

The query shapes mirror the Python connector, including the five
empirically-found landmines: the scored dictionary needs `frequency +
norm` (else `BM25()` silently returns 0.0), the `@@` query is the
tokenized query, the scored lexical branch matches one column, vectors
use an L2-normalized shadow column with `ip`/`sq8`, and the similarity
threshold goes directly in the ANN scan's `WHERE`. **Minimum engine
version: SereneDB 26.07.4.**

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-04 14:16:39 +08:00
buua436
3e11914144 fix: normalize legacy parser configuration (#17761) 2026-08-04 13:49:44 +08:00
Wang Qi
ffa168ccca fix: search sessions by id and conversation (#808) (#17764)
Co-authored-by: buua436 <sz_buua@foxmail.com>
2026-08-04 11:34:32 +08:00
Wang Qi
d236fc4437 Let agentic rag search honor metadata filter (#17731) 2026-08-04 10:35:21 +08:00
Lynn
11b2dfcfdd Fix: handle llm_setting switch (#17745) 2026-08-04 10:02:14 +08:00
Jin Hai
86021932ae Go: fix warnings (#17738)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-03 21:30:01 +08:00
buua436
1d141aff18 fix: optimize dataflow indexing and logs (#17737) 2026-08-03 19:15:44 +08:00
buua436
3e7cfbe052 refa: simplify RAPTOR tree clustering and configuration (#17614) 2026-08-03 17:46:50 +08:00
buua436
8ac047d11a fix: synchronize chat model configuration (#17717) 2026-08-03 16:43:53 +08:00
Kevin Hu
3b4a96d421 Refactor: refine wiki plan procedure. (#17579)
### Summary

Refine wiki plan procedure.

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
Co-authored-by: buua436 <sz_buua@foxmail.com>
2026-08-03 16:03:17 +08:00
Wang Qi
a1a79ab98c Fix: add doc_type_kwd back (#17658) 2026-08-03 13:21:22 +08:00
Lynn
2e0dda59fc Fix: handle html tags in empty_response (#17657) 2026-08-03 11:14:18 +08:00
Wang Qi
9db2e2314a Refine knownledge compilation REST APIs (#17624) 2026-07-31 17:05:57 +08:00
Lynn
03b7b04e2c Fix: Add missing parameter key for LLM calls (#17628) 2026-07-31 17:05:40 +08:00
Wang Qi
4b619f699b Refactor: consolidate page and page_size (#17597) 2026-07-31 14:08:44 +08:00
buua436
44e13d1cb6 feat: add LLM-guided semantic rechunking for knowledge compilation (#17546) 2026-07-31 13:33:30 +08:00
Wang Qi
0969c04eca Feature: GET /datasets to support ids array (#17606) 2026-07-31 13:23:02 +08:00
Wang Qi
2c6da4113d Fix: multiple model chat pass-in session_id, and do not save session. (#17581) 2026-07-31 11:25:29 +08:00
balibabu
569a29500f Fix: The save interface is continuously called without any operation being performed on the agent page. (#17576) 2026-07-31 11:23:12 +08:00
Lynn
78da321fb0 Fix: return model.extra for response display (#17583) 2026-07-31 09:52:22 +08:00
euvre
8fc20dd9ca Test: release Go-proxy RESTful contract tests verified passing in Go mode (#17468)
### Summary

Aligns Go and Python error codes/messages so both backends honor the
same RESTful API contract, removing implementation-specific error leaks
(MySQL errors, `ValueError`, `AttributeError`, Gin validator format) in
favor of clean business error codes.

**Chat list** — invalid `orderby` now returns code 101 (was: raw Python
`AttributeError` code 100); invalid `page`/`page_size` values fall back
to defaults (was: raw `ValueError`/`ProgrammingError` code 100).

**Dataset create/update/delete** — adds UUID validation (101),
extra-field rejection (101), duplicate-id detection (101), content-type
/ JSON-syntax / object-shape checks (101), and "lacks permission" for
nonexistent datasets (IDOR). Create auto-deduplicates dataset names.
Pagerank updates tolerate a missing ES index. List response includes
`parser_config` and `pagerank`.

**Session list/update** — adds filtering, sorting, and pagination
support. Empty payloads are valid no-ops. Authorization errors map to
code 109.

**Chunk list** — doc object uses Python key names (`chunk_count`,
`dataset_id`, `chunk_method`, run text status). Add validates list
element types.

**Document update** — adds `chunk_method` alias, pydantic-style Field
error messages, metadata index auto-create with refresh, and "These
documents do not belong to dataset" messages. List validates
`metadata_condition` and reports ownership errors for unmatched name/id
filters.

**Search completion** — `kb_ids` ownership failure returns code 102
instead of 109.

Released 31 contract tests from `GO_ONLY_SKIPS` (all verified passing on
both Go and Python backends with real LLM keys).
2026-07-30 19:58:49 +08:00
Kevin Hu
3f8a3dfcff Feat: Compilation benefit naive rag. (#17555)
### Summary

Compilation benifit naive rag

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-07-30 18:39:01 +08:00
Wang Qi
5d76b0b96c Fix multiple model chat, the content overwrite the current chat session (#17566) 2026-07-30 16:50:13 +08:00
Wang Qi
4592d06a67 Fix deepcopy the chat model (#17560) 2026-07-30 14:29:24 +08:00
Wang Qi
6e0e495929 Fix: upload document might be hung the whole ragflow (#17537) 2026-07-30 13:54:07 +08:00
Kevin Hu
9bb037271e Refact: Rename "artifact_" to "wiki_". (#17553) 2026-07-30 11:06:16 +08:00
Haruko386
c7b1b3a4d2 feat: make chat-channel and implement WhatsApp bot (#17518) 2026-07-29 18:55:22 +08:00
Kevin Hu
48a3280eac Refactor: merge dataset scope graph. (#17526)
### Summary

merge dataset scope graph.

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-07-29 18:23:51 +08:00
balibabu
1140db3b8c Fix: The multi-model comparison page cannot output messages. (#17507) 2026-07-29 15:16:40 +08:00
Wang Qi
3c59b707c2 Refactor: delete llm_app.py (#17517) 2026-07-29 14:54:32 +08:00
Wang Qi
3297b5c756 Fix: drop and add index for mysql (#17513)
Drop complile template not used index
2026-07-29 14:54:20 +08:00
Sbaaoui Idriss
e9637c9f94 fix: list model function for nvidia on python/go not returning current models (#17501)
### Summary

fix the list model logic for nvidia models
2026-07-29 13:15:34 +08:00
Wang Qi
989529c00a Fix filter dataset by owner_ids not working (#17499) 2026-07-29 11:48:40 +08:00
Harsh Kashyap
404a5cc33b fix(admin,api): honor STORAGE_IMPL when surfacing file_store backend (closes #17294) (#17441)
## Summary

When `STORAGE_IMPL=AWS_S3`, the Admin Service status page keeps showing
MinIO. Three things conspire:

1. `admin/server/config.py::load_configurations` only knows the
   `minio` and `minio_0` config keys. An `s3` block lands on the
   `case _:` branch and logs `Unknown configuration key: s3`
   (issue #17294).
2. `admin/server/services.py::ServiceMgr.get_all_services` filters
   retrieval services by `DOC_ENGINE` but has no equivalent filter
   for `file_store` services by `STORAGE_IMPL`. A stale MinIO block
   is returned regardless of the active backend.
3. The wired health check is hardcoded to `check_minio_alive`, which
   calls `settings.MINIO['host']`. With `STORAGE_IMPL=AWS_S3` that
   block is uninitialized, so the check always times out.


Fixes #17294
2026-07-29 11:13:20 +08:00
Lynn
8d1eff6ea2 Fix: resolve model_id to model before judge parse method (#17493) 2026-07-29 09:34:52 +08:00