8 Commits

Author	SHA1	Message	Date
Rustem Kamalov	3d5eddfeed	fix(engines): better classify captcha/blocked/no-results pages instead of returning empty results, also enable it to <engine>/parse endpoints. Enable allow_request_proxy_url by default	2026-07-01 01:38:03 +03:00
Rustem Kamalov	15bab0295f	refactor: extract shared RankState, unify rod/goquery parse paths	2026-06-27 18:04:25 +03:00
Rustem Kamalov	9cc69da758	feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs ... - Add structured `search [engine] [query]` CLI: --limit/--lang/--region/--site/--file, --format (json|text|markdown|ndjson), --extract N, --search-timeout; Envelope and route logs to stderr with a --quiet default (fixes stdout pollution) - Unify engines behind a single engineSpec registry (CLI + serve share it) - Unify the extract knob to bool-or-int `extract=N` (drop extract_top); CLI and HTTP share core batch extraction, raw/rendered fetch, and clamp helpers - Engines: Ecosia CF captcha detection (raw + browser), Yandex progressive-result wait, Google PAA poll + Has() existence probes, Bing title/desc attribute fallbacks - Proxy: rotate challenged proxies out of the tag pool for one retry (X-Proxy-Attempts); browser health-ping skip window; opt-in WaitStable	2026-06-16 03:51:37 +03:00
Rustem Kamalov	2b96a8330f	SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements	2026-05-28 01:36:52 +03:00
Rustem Kamalov	82abe57aae	Improve SERP result classification and ad positioning	2026-05-19 00:32:11 +03:00
Rustem Kamalov	b3ca4e0803	feat(api): v2 response shape - absolute rank, ad/organic split, slimmer fields	2026-05-13 01:30:54 +03:00
Rustem Kamalov	471e5ad7fe	refactor: speed up SERP parsing and centralize selectors	2026-05-06 04:12:15 +03:00
Rustem Kamalov	ed23af7acc	feat: add POST /{engine}/parse HTML parser endpoints. One source of truth for browser and raw mode selectors	2026-05-05 05:07:59 +03:00