Rustem Kamalov 9cc69da758 feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs ...

- Add structured `search [engine] [query]` CLI: --limit/--lang/--region/--site/--file, --format (json|text|markdown|ndjson), --extract N, --search-timeout; Envelope and route logs to stderr with a --quiet default (fixes stdout pollution)
- Unify engines behind a single engineSpec registry (CLI + serve share it)
- Unify the extract knob to bool-or-int `extract=N` (drop extract_top); CLI and HTTP share core batch extraction, raw/rendered fetch, and clamp helpers
- Engines: Ecosia CF captcha detection (raw + browser), Yandex progressive-result wait, Google PAA poll + Has() existence probes, Bing title/desc attribute fallbacks
- Proxy: rotate challenged proxies out of the tag pool for one retry (X-Proxy-Attempts); browser health-ping skip window; opt-in WaitStable

2026-06-16 03:51:37 +03:00

..

browser

Rework browser profiles. Add x-use-profile header. Selectors fixes

2026-04-30 23:32:39 +03:00

fpcheck

Rework browser profiles. Add x-use-profile header. Selectors fixes

2026-04-30 23:32:39 +03:00

region

extract region/UULE resolution into dependency-free core/region subpackage

2026-06-02 05:02:51 +03:00

browser_panic_test.go

Enhance error handling and logging across search engines

2026-04-21 03:48:52 +03:00

browser_profile_usage.go

Add browser profile ID in logs and response headers

2026-04-30 20:46:08 +03:00

browser_resource_blocking_test.go

fix: browser request blocking cleanup panic

2026-04-29 05:03:30 +03:00

browser_test.go

feat(proxy): support X-Proxy-URL via per-process browser pool

2026-04-28 03:46:08 +03:00

browser_unit_test.go

Rework browser profiles. Add x-use-profile header. Selectors fixes

2026-04-30 23:32:39 +03:00

browser.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

cache_test.go

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

cache.go

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

captcha_test.go

fix(captcha): exit non-zero on misconfig, tighten solver gate tests

2026-04-20 20:12:19 +03:00

captcha_unit_test.go

fix(captcha): exit non-zero on misconfig, tighten solver gate tests

2026-04-20 20:12:19 +03:00

captcha.go

fix(captcha): exit non-zero on misconfig, tighten solver gate tests

2026-04-20 20:12:19 +03:00

circuit_breaker_test.go

feat(mega): add mode fast/any/balanced with CB latency-based fast path

2026-05-02 16:30:56 +03:00

circuit_breaker.go

feat(mega): add mode fast/any/balanced with CB latency-based fast path

2026-05-02 16:30:56 +03:00

clusters.go

fix api response format issues

2026-04-26 02:58:13 +03:00

common_test.go

fix default search limits

2026-06-02 21:44:58 +03:00

common.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

context_test.go

feat(query): add region parameter for per-engine geo targeting (google, yandex, bing, duck)

2026-05-21 20:45:10 +03:00

context.go

feat(query): add region parameter for per-engine geo targeting (google, yandex, bing, duck)

2026-05-21 20:45:10 +03:00

enrichment_domain.go

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

enrichment_domains.yaml

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

errors.go

feat(proxy): support X-Proxy-URL via per-process browser pool

2026-04-28 03:46:08 +03:00

feature_selectors.go

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

format_markdown.go

Add new URL extraction endpoint and search feature

2026-06-04 02:22:38 +03:00

format_text.go

Add new URL extraction endpoint and search feature

2026-06-04 02:22:38 +03:00

html_parser.go

feat: add POST /{engine}/parse HTML parser endpoints. One source of truth for browser and raw mode selectors

2026-05-05 05:07:59 +03:00

http_client_test.go

refactor: Speed up Baidu. Fix raw client + reuse HTML parsers for raw search

2026-05-08 00:00:33 +03:00

http_client.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

locale_test.go

feat(query): add region parameter for per-engine geo targeting (google, yandex, bing, duck)

2026-05-21 20:45:10 +03:00

locale.go

extract region/UULE resolution into dependency-free core/region subpackage

2026-06-02 05:02:51 +03:00

logger.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

middleware_test.go

Add browser profile ID in logs and response headers

2026-04-30 20:46:08 +03:00

middleware.go

fix: derive a per-request deadline from the retry budget.

2026-06-12 04:40:31 +03:00

network_guard_test.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

network_guard.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

network_usage.go

feat: add network bytes telemetry, mega timeout, and per-engine error details

2026-04-29 02:45:16 +03:00

page_helpers.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

profile_context.go

Rework browser profiles. Add x-use-profile header. Selectors fixes

2026-04-30 23:32:39 +03:00

proxy_context.go

feat(proxy): support X-Proxy-URL via per-process browser pool

2026-04-28 03:46:08 +03:00

proxy_integration_test.go

feat: propagate request context across search, retry, limiter, and browser navigation

2026-04-20 05:53:13 +03:00

proxy_lane_test.go

feat(proxy): support X-Proxy-URL via per-process browser pool

2026-04-28 03:46:08 +03:00

proxy_lane.go

feat(proxy): support X-Proxy-URL via per-process browser pool

2026-04-28 03:46:08 +03:00

proxy_per_context_test.go

fix integration tests, dedupe engine smoke tests

2026-06-10 15:42:26 +03:00

proxy_rotation_test.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

proxy_test.go

refactor: Speed up Baidu. Fix raw client + reuse HTML parsers for raw search

2026-05-08 00:00:33 +03:00

proxy.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

query_extract_param_test.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

rate_limiter_test.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

region_test.go

extract region/UULE resolution into dependency-free core/region subpackage

2026-06-02 05:02:51 +03:00

region.go

extract region/UULE resolution into dependency-free core/region subpackage

2026-06-02 05:02:51 +03:00

resilient_context_test.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

resilient.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

response_builder.go

SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements

2026-05-28 01:36:52 +03:00

response.go

dedup feature rendering, align default limit, simplify CSS guard

2026-06-02 04:21:56 +03:00

result_rank_test.go

Improve SERP result classification and ad positioning

2026-05-19 00:32:11 +03:00

result.go

Add new URL extraction endpoint and search feature

2026-06-04 02:22:38 +03:00

retry_test.go

Enhance error handling and logging across search engines

2026-04-21 03:48:52 +03:00

retry.go

fix: derive a per-request deadline from the retry budget.

2026-06-12 04:40:31 +03:00

serp_features_test.go

dedup feature rendering, align default limit, simplify CSS guard

2026-06-02 04:21:56 +03:00

server_extract_test.go

fix(core): harden extract and stabilize request protection

2026-06-12 18:09:40 +03:00

server_extract.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00

server_panic_test.go

fix: centralize engine panic recovery in resilient layer

2026-06-12 04:24:06 +03:00

server_parse_test.go

refactor: speed up SERP parsing and centralize selectors

2026-05-06 04:12:15 +03:00

server_test.go

fix default search limits

2026-06-02 21:44:58 +03:00

server_timeout_test.go

fix: derive a per-request deadline from the retry budget.

2026-06-12 04:40:31 +03:00

server.go

feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs

2026-06-16 03:51:37 +03:00