fix(llm): retry hollow responses instead of bisecting them (#2880)

A response that parses but carries no symbols (a "hollow" reply) was routed into the
truncation-bisection path, which just re-split a chunk the model had already answered
emptily, wasting calls. Give hollow its own finish_reason and route it through a bounded
same-chunk retry with backoff instead; on persistent hollow, give up loudly and mark the
files partial. GRAPHIFY_MAX_RETRY_DEPTH=0 now disables the hollow retry too, so a chunk
costs exactly one call. The #2866 timeout and the truncation paths still bisect.
This commit is contained in:
rajarshidattapy
2026-08-20 15:45:18 +01:00
committed by safishamsi
parent e8bef863f0
commit 69e2c0deae
3 changed files with 312 additions and 75 deletions
+1
View File
@@ -524,6 +524,7 @@ These are only needed for **headless / CI extraction** (`graphify extract`). Whe
| `GRAPHIFY_MAX_OUTPUT_TOKENS` | Raise output cap for dense corpora | optional — e.g. `32768` for large files |
| `GRAPHIFY_API_TIMEOUT` | Per-call timeout in seconds for HTTP, claude-cli, Anthropic SDK, and Bedrock backends (default: 600) | optional — also `--api-timeout` flag |
| `GRAPHIFY_MAX_RETRIES` | How many times to retry a rate-limited (429) request before giving up (default: 6; honors `Retry-After`) | optional — raise for strict per-org limits (e.g. kimi); `0` disables |
| `GRAPHIFY_MAX_RETRY_DEPTH` | How deep a truncated chunk may be bisected and re-extracted (default: 3, so up to 8x sub-calls for one chunk) | optional — lower it to cap worst-case spend; `0` disables every retry (no bisection, no hollow-response retry), so a chunk costs exactly one call |
| `GRAPHIFY_FORCE` | Force graph rebuild even with fewer nodes | optional — also `--force` flag |
| `GRAPHIFY_GOOGLE_WORKSPACE` | Auto-enable Google Workspace export | optional — set to `1` |
| `GRAPHIFY_TRIAGE_BACKEND` | Backend for `graphify prs --triage` | optional — auto-detected from available keys |