zlei6/openserp
1
0
Fork 0
mirror of https://github.com/karust/openserp.git synced 2026-08-11 11:31:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
151 Commits 4 Branches 16 Tags
aec0d9c42a685a02027a52985768bc7b1d8b61af
Commit Graph

8 Commits

Author SHA1 Message Date
Rustem Kamalov
9cc69da758 feat(cli): clean search, add extract/format flags; harden engines & proxy rotation; fix bugs; update docs 
- Add structured `search [engine] [query]` CLI: --limit/--lang/--region/--site/--file, --format (json|text|markdown|ndjson), --extract N, --search-timeout; Envelope and route logs to stderr with a --quiet default (fixes stdout pollution)
- Unify engines behind a single engineSpec registry (CLI + serve share it)
- Unify the extract knob to bool-or-int `extract=N` (drop extract_top); CLI and HTTP share core batch extraction, raw/rendered fetch, and clamp helpers
- Engines: Ecosia CF captcha detection (raw + browser), Yandex progressive-result wait, Google PAA poll + Has() existence probes, Bing title/desc attribute fallbacks
- Proxy: rotate challenged proxies out of the tag pool for one retry (X-Proxy-Attempts); browser health-ping skip window; opt-in WaitStable
 2026-06-16 03:51:37 +03:00
Rustem Kamalov
e8fb0c24fa feat(proxy): support X-Proxy-URL via per-process browser pool 2026-04-28 03:46:08 +03:00
Rustem Kamalov
0ccb13c36e fix(proxy): isolate proxy health from captcha/parser errors, replace re-enable thrash with quarantine 2026-04-24 04:33:41 +03:00
Rustem Kamalov
5d15335501 Fix CI 2026-04-22 05:32:59 +03:00
Rustem Kamalov
0d4e2da960 Improve logging: add structured logs, enrich logs with useful info. 2026-04-22 03:39:30 +03:00
Rustem Kamalov
3c07c7789a Runtime proxy override 2026-04-01 17:38:13 +03:00
Rustem Kamalov
3daa48e6a3 Proxy policy wiring and fail-closed execution 
- replace proxy wiring with global and per-engine tag policies
- split stats endpoints and lock fail-closed proxy behavior with tests
 2026-04-01 00:54:21 +03:00
Rustem Kamalov
8bec5578c0 Implement proxy support with configuration, pooling, and integration tests 
- Added ProxyConfig and ProxyPool structures to manage proxy settings and rotation.
- Implemented normalization functions for proxy URLs and configurations.
- Created a new ProxyPool that supports failure tracking and round-robin selection.
- Integrated proxy handling into the ResilientSearcher for search queries.
- Added integration tests for various proxy scenarios including SOCKS5 and HTTP proxies.
- Updated server and resilience stats to include proxy information.
- Refactored search functions to utilize the new proxy client.
- Removed direct proxy handling from search_raw.go and yandex/search_raw.go, using the new core proxy client instead.
- Added unit tests for proxy configuration and pool behavior.
 2026-03-30 22:46:03 +03:00