Rustem Kamalov
9cc69da758
feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs
...
- Add structured `search [engine] [query]` CLI: --limit/--lang/--region/--site/--file, --format (json|text|markdown|ndjson), --extract N, --search-timeout; Envelope and route logs to stderr with a --quiet default (fixes stdout pollution)
- Unify engines behind a single engineSpec registry (CLI + serve share it)
- Unify the extract knob to bool-or-int `extract=N` (drop extract_top); CLI and HTTP share core batch extraction, raw/rendered fetch, and clamp helpers
- Engines: Ecosia CF captcha detection (raw + browser), Yandex progressive-result wait, Google PAA poll + Has() existence probes, Bing title/desc attribute fallbacks
- Proxy: rotate challenged proxies out of the tag pool for one retry (X-Proxy-Attempts); browser health-ping skip window; opt-in WaitStable
2026-06-16 03:51:37 +03:00
Rustem Kamalov
45287112df
fix docs + google: gate organic results by matched selector, harden answer-box parse
2026-06-10 15:15:42 +03:00
Rustem Kamalov
b31e84316c
Add contributor onboarding kit
2026-06-10 14:54:45 +03:00
Rustem Kamalov
aa397eb2c1
Add new URL extraction endpoint and search feature
2026-06-04 02:22:38 +03:00
Rustem Kamalov
2b4a80fcb2
fix default search limits
2026-06-02 21:44:58 +03:00
Rustem Kamalov
7ed0780192
Add google uule query param support for regional search
2026-06-02 03:54:55 +03:00
Rustem Kamalov
2b96a8330f
SERP features: AI summary, answer boxes, related searches/questions, etc - for supporting engines + other fixesa and improvements
2026-05-28 01:36:52 +03:00
Rustem Kamalov
25f4ff942b
feat(query): add region parameter for per-engine geo targeting (google, yandex, bing, duck)
2026-05-21 20:45:10 +03:00
Rustem Kamalov
82abe57aae
Improve SERP result classification and ad positioning
2026-05-19 00:32:11 +03:00
Rustem Kamalov
b3ca4e0803
feat(api): v2 response shape - absolute rank, ad/organic split, slimmer fields
2026-05-13 01:30:54 +03:00
Rustem Kamalov
ed23af7acc
feat: add POST /{engine}/parse HTML parser endpoints. One source of truth for browser and raw mode selectors
2026-05-05 05:07:59 +03:00
Rustem Kamalov
3eaa30118e
fix(ecosia): wire proxy policy, validation, raw mode, and OpenAPI docs
2026-05-05 02:47:47 +03:00
Rustem Kamalov
5d48690eb3
feat(mega): add mode fast/any/balanced with CB latency-based fast path
2026-05-02 16:30:56 +03:00
Rustem Kamalov
063fb74f23
Add browser profile ID in logs and response headers
2026-04-30 20:46:08 +03:00
Rustem Kamalov
f9890ae10f
feat: add network bytes telemetry, mega timeout, and per-engine error details
2026-04-29 02:45:16 +03:00
Rustem Kamalov
e8fb0c24fa
feat(proxy): support X-Proxy-URL via per-process browser pool
2026-04-28 03:46:08 +03:00
Rustem Kamalov
94ef0b20b4
docs: update readme to match new api format and features, update architecture doc
2026-04-26 03:50:48 +03:00
Rustem Kamalov
d47bf08523
fix api response format issues
2026-04-26 02:58:13 +03:00
Rustem Kamalov
1bdb103b23
docs: rewrite OpenAPI spec to v1 schema
...
Envelopes replace bare arrays on all search endpoints. New schemas for
QueryEcho, ResponseMeta, Pagination, Result (with Position, DomainInfo,
Classification), Cluster, and image variants. ErrorResponse gains reason
field. limit parameter gets min/max bounds. FormatQuery parameter and
X-Request-ID header documented.
2026-04-26 02:58:13 +03:00
Rustem Kamalov
d052c30bd5
feat(browser): replace stealth with custom profile functionality
2026-04-23 03:06:49 +03:00
Rustem Kamalov
47b5ad3b09
feat: graceful shutdown with drain signal and browser cleanup
2026-04-22 04:25:56 +03:00
Rustem Kamalov
0b7fbd3a45
docs: add godoc for public api and readme response examples
2026-04-20 01:20:52 +03:00
Rustem Kamalov
a9986fd8fe
docs: add architecture, OpenAPI spec, Swagger UI, and contributing guide
2026-04-15 03:25:06 +03:00
