This PR modularizes the **Extractor** component configuration with dedicated feature subtabs, adds independent system prompt configuration, fixes multi-node execution determinism and parameter persistence across save and page refresh, and ensures backward compatibility with legacy flat fields.
### Summary
Pregel nodes created with `AddNodeWithOptions` retain a node-level
`RetryPolicy`, but task construction dropped that policy. When a task
had no policy, execution also bypassed the engine-level policy
configured with `WithRetryPolicy` and instantiated the default policy
directly.
Fixes the defect where selecting many files (e.g. 20+) in one dataset and starting parsing at once leaves most of them stuck in `RUNNING` forever: a few parse, the rest never do.
- The Go DLA client (`inference.Client.DLA`) kept `footer`/`header`/`reference` regions whose detection confidence was below 0.4.
- Python's production path (`parse_into_bboxes` -> `LayoutRecognizer.__call__`, `deepdoc/vision/layout_recognizer.py:97` and `:379`) drops such regions via a 0.4 garbage gate over
`garbage_layouts=['footer','header','reference']`.
- The shared `/predict/dla` backend runs `LayoutRecognizer.forward`, which does **not** apply that gate, so the Go client was the only place the filter was missing — a real Go<->Python parity gap.
## Problem
In the Go PDF pipeline, `processTablesWithReplacements`
(`internal/deepdoc/parser/pdf/table/table_post.go`) marked **every** box
overlapping a DLA table region for removal, then `insertTableBoxes`
re-inserted an HTML box only when `buildTableHTMLs` produced one.
When DLA over-labels a **text** box as a table but TSR produces **no
cells** (so `buildTableHTMLs` skips the table and `htmls[ti]` is unset),
the box was removed with nothing re-inserted — the original text was
**silently dropped**.
Fixes Go ingestion progress reporting and pipeline selection:
- Add timestamps to document progress logs.
- Keep document duration and status updated during parsing.
- Start frontend polling immediately after parsing begins.
- Prevent documents explicitly using General from inheriting an old
dataset pipeline.
- Populate missing pipeline operation log fields.
- Remove stale component progress logs between retries.
- Prevent progress values greater than `1`.
Supersedes / folds in #18305. The score-based layer-2 rotation selection from #18305 now lives here, on top of `WarpCrop` (layer 1), applied to **all three** Go OCR paths, together with the Python score plumbing the Go side depends on. #18305 is closed in favor of this PR.
Aligns three Go DLA / PDF post-processing behaviors with the Python `deepdoc` reference so the Go PDF pipeline matches Python's DLA region / annotation semantics.
Follow-up to #17526 ("Refactor: merge dataset scope graph"), which introduced two code paths that touch Infinity columns the deployed schema does not declare. This PR makes the runtime robust against the old schema while also adding the new column to the new schema so freshly created tables are correct.
Add `ManualChunker`, the Go port of Python's `manual` doc-type chunk method (`rag/app/manual.py`). Like `GroupTitleChunker` it merges adjacent text records into heading-bounded groups, but it first re-sorts the records into physical reading order before grouping.
- Add Go `EmailParser` support for Outlook `.msg` (OLE2/CFB) files via
the `gomsg` library, in addition to the existing `.eml` (RFC 5322)
support. The `.msg` hard-error is gone; emails with `.msg` attachments
are now ingested end-to-end.
- Re-chunk email attachments into retrievable text (user-oriented). Each
attachment is re-parsed by its file extension through the shared parser
registry and folded back into the same document, so attachment content
becomes searchable. This mirrors Python's legacy `rag/app/email.py`.
Binary attachments (images/audio/video/folders) are skipped by design.
- Restore a corrupted `sample.msg` test fixture and add guards so binary
fixtures are never mangled again (`.gitattributes` marks `*.msg` binary;
`check_files.py` skips NUL-byte files). Also made `check_files.py`
ruff-clean.
When `TokenChunker` starts a fresh chunk with an overlap prefix (Go `computeOverlapPrefix` / Python visible-text cut), the previous chunk's **tail PDF coordinates were dropped**. As a result, the overlap head of a PDF chunk is displayed but **not highlighted** — the highlight box is shifted/truncated relative to the displayed span (infiniflow/ragflow#18148).
