mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
31a8c52db5
Production code now talks HTTP through aiohttp only, and only asynchronously. httpx's async client is 5-12x slower per request than aiohttp at the concurrency we run embeddings/rerank/LLM calls with (local stub: 1.3k vs 6.8k req/s at c=8, 527 vs 6.3k at c=64), and the sync paths needed a thread per in-flight request. - engine/aiohttp_session.py: LoopLocal / LoopLocalSession (one session per event loop, created lazily, closed-loop entries released), per_phase_timeout (httpx-style per-phase timeouts), raise_for_status -> UpstreamHTTPError (keeps body + status_code for remote_retry), close_loop_sessions() called from MemoryEngine.close(). - Embeddings interface is async (encode/encode_query/encode_documents); remote providers use aiohttp or the SDK's async client, bounded fan-out via a per-loop semaphore instead of a thread pool. Local models stay in a worker thread. - Rerankers, Codex/Nous/xAI OAuth providers (async token refresh with a per-loop asyncio.Lock + non-blocking flock), Fireworks, Ollama-native, llama.cpp, LlamaParse and Iris parsers moved to aiohttp. - Webhook SSRF guard re-implemented as an aiohttp resolver that only returns validated addresses (no DNS cache, no redirects, no env proxies), with IP literals checked before sending. - hindsight-embed probes, supabase-tenant extension and the litellm integration's async hooks moved to aiohttp. - ruff TID251 bans httpx/requests/urllib.request/urllib3/http.client in hindsight-api-slim and hindsight-embed production code; code-review skill documents the rule (tests exempt).