mirror of
https://github.com/karust/openserp.git
synced 2026-08-05 16:53:54 +08:00
docs: add godoc for public api and readme response examples
This commit is contained in:
Rustem Kamalov
@@ -175,33 +175,33 @@ Defaults below are the shipped defaults in `config.yaml` (if present). If the co
|
||||
|
||||
### `server`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `server.host` | `0.0.0.0` | API bind host |
|
||||
| `server.port` | `7000` | API bind port |
|
||||
| `server.debug` | `false` | Debug mode, forces headful browser |
|
||||
| `server.verbose` | `true` | Info-level request logs |
|
||||
| `server.raw_requests` | `false` | `true` = raw HTTP mode |
|
||||
| `server.insecure` | `true` | Allow insecure TLS connections |
|
||||
| Key | Default | Description |
|
||||
| --------------------- | --------- | ---------------------------------- |
|
||||
| `server.host` | `0.0.0.0` | API bind host |
|
||||
| `server.port` | `7000` | API bind port |
|
||||
| `server.debug` | `false` | Debug mode, forces headful browser |
|
||||
| `server.verbose` | `true` | Info-level request logs |
|
||||
| `server.raw_requests` | `false` | `true` = raw HTTP mode |
|
||||
| `server.insecure` | `true` | Allow insecure TLS connections |
|
||||
|
||||
### `app`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `app.timeout` | `15` | Request timeout in seconds |
|
||||
| `app.browser_path` | `""` | Custom browser binary path |
|
||||
| `app.head` | `false` | Headful browser UI |
|
||||
| `app.leakless` | `false` | Force browser process cleanup |
|
||||
| `app.leave_head` | `false` | Keep browser tabs open |
|
||||
| `app.stealth` | `false` | Enable stealth plugin |
|
||||
| Key | Default | Description |
|
||||
| ------------------ | ------- | ----------------------------- |
|
||||
| `app.timeout` | `15` | Request timeout in seconds |
|
||||
| `app.browser_path` | `""` | Custom browser binary path |
|
||||
| `app.head` | `false` | Headful browser UI |
|
||||
| `app.leakless` | `false` | Force browser process cleanup |
|
||||
| `app.leave_head` | `false` | Keep browser tabs open |
|
||||
| `app.stealth` | `false` | Enable stealth plugin |
|
||||
|
||||
### `proxies`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `proxies.global` | unset | Force single proxy for all engines |
|
||||
| `proxies.entries[]` | empty | Tagged proxy pool entries (`url`, `tags`) |
|
||||
| `proxies.health.failure_threshold` | `3` | Disable proxy after N failures |
|
||||
| Key | Default | Description |
|
||||
| ---------------------------------- | ------- | ----------------------------------------- |
|
||||
| `proxies.global` | unset | Force single proxy for all engines |
|
||||
| `proxies.entries[]` | empty | Tagged proxy pool entries (`url`, `tags`) |
|
||||
| `proxies.health.failure_threshold` | `3` | Disable proxy after N failures |
|
||||
|
||||
Per-engine optional proxy tag:
|
||||
|
||||
@@ -213,52 +213,52 @@ Per-engine optional proxy tag:
|
||||
|
||||
### `cache`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `cache.ttl_seconds` | `60` | Response cache TTL (0 disables cache) |
|
||||
| `cache.max_size` | `1000` | Max cached entries |
|
||||
| Key | Default | Description |
|
||||
| ------------------- | ------- | ------------------------------------- |
|
||||
| `cache.ttl_seconds` | `60` | Response cache TTL (0 disables cache) |
|
||||
| `cache.max_size` | `1000` | Max cached entries |
|
||||
|
||||
### `resilience`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `resilience.max_retries` | `2` | Retry attempts per request |
|
||||
| Key | Default | Description |
|
||||
| ------------------------------------ | ------- | ------------------------------------------------------ |
|
||||
| `resilience.max_retries` | `2` | Retry attempts per request |
|
||||
| `resilience.allow_endpoint_fallback` | `false` | Allow dedicated endpoints to fallback to other engines |
|
||||
|
||||
### `circuit_breaker`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `circuit_breaker.failures` | `5` | Failures before opening circuit |
|
||||
| `circuit_breaker.recovery_seconds` | `60` | Open -> half-open wait time |
|
||||
| `circuit_breaker.successes` | `2` | Half-open successes to close circuit |
|
||||
| Key | Default | Description |
|
||||
| ---------------------------------- | ------- | ------------------------------------ |
|
||||
| `circuit_breaker.failures` | `5` | Failures before opening circuit |
|
||||
| `circuit_breaker.recovery_seconds` | `60` | Open -> half-open wait time |
|
||||
| `circuit_breaker.successes` | `2` | Half-open successes to close circuit |
|
||||
|
||||
### `cors`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `cors.enabled` | `true` | Enable CORS middleware |
|
||||
| `cors.allow_origins` | `"*"` | Allowed origins |
|
||||
| `cors.allow_methods` | `"GET, POST, OPTIONS"` | Allowed methods |
|
||||
| `cors.allow_headers` | `"Origin, Content-Type, Accept, Authorization, X-Use-Proxy"` | Allowed headers |
|
||||
| `cors.max_age` | `86400` | Preflight cache max age (seconds) |
|
||||
| Key | Default | Description |
|
||||
| -------------------- | ------------------------------------------------------------ | --------------------------------- |
|
||||
| `cors.enabled` | `true` | Enable CORS middleware |
|
||||
| `cors.allow_origins` | `"*"` | Allowed origins |
|
||||
| `cors.allow_methods` | `"GET, POST, OPTIONS"` | Allowed methods |
|
||||
| `cors.allow_headers` | `"Origin, Content-Type, Accept, Authorization, X-Use-Proxy"` | Allowed headers |
|
||||
| `cors.max_age` | `86400` | Preflight cache max age (seconds) |
|
||||
|
||||
### `2captcha`
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `2captcha.apikey` | unset | Optional captcha solver key |
|
||||
| Key | Default | Description |
|
||||
| ----------------- | ------- | --------------------------- |
|
||||
| `2captcha.apikey` | unset | Optional captcha solver key |
|
||||
|
||||
### Engine rate-limit defaults
|
||||
|
||||
For each engine (`google`, `yandex`, `baidu`, `bing`, `duckduckgo`):
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `<engine>.rate_requests` | `4` | Average requests per minute |
|
||||
| `<engine>.rate_burst` | `2` | Burst capacity |
|
||||
| `<engine>.rate_seconds` | `60` (implicit) | Rate window seconds |
|
||||
| `<engine>.selector_timeout` | `5` (implicit) | Selector wait timeout seconds |
|
||||
| Key | Default | Description |
|
||||
| --------------------------- | --------------- | ----------------------------- |
|
||||
| `<engine>.rate_requests` | `4` | Average requests per minute |
|
||||
| `<engine>.rate_burst` | `2` | Burst capacity |
|
||||
| `<engine>.rate_seconds` | `60` (implicit) | Rate window seconds |
|
||||
| `<engine>.selector_timeout` | `5` (implicit) | Selector wait timeout seconds |
|
||||
|
||||
Google-only additional toggle:
|
||||
|
||||
|
||||
120
docs/CONTRIBUTING.md
Normal file
120
docs/CONTRIBUTING.md
Normal file
@@ -0,0 +1,120 @@
|
||||
# Contributing to OpenSERP
|
||||
|
||||
## Development Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.24+
|
||||
- Chromium/Chrome (only required for browser-mode work and integration tests)
|
||||
- Optional: Docker
|
||||
|
||||
### Clone, build, run
|
||||
|
||||
```bash
|
||||
git clone https://github.com/karust/openserp.git
|
||||
cd openserp
|
||||
go build -o openserp .
|
||||
./openserp serve
|
||||
```
|
||||
|
||||
### Test commands
|
||||
|
||||
Unit tests (default, no browser/network assumptions):
|
||||
|
||||
```bash
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
Integration tests (explicitly enabled):
|
||||
|
||||
```bash
|
||||
OPENSERP_INTEGRATION_TESTS=1 go test -race -timeout=120s ./...
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Integration tests are gated by `testutil.RequireIntegration(t)`.
|
||||
- Do not create browser instances in `init()` or package-level variables.
|
||||
|
||||
## Adding a New Search Engine
|
||||
|
||||
### 1) Create engine package
|
||||
|
||||
Create a new folder (example: `myengine/`) with:
|
||||
|
||||
- `myengine/url.go` (`BuildURL`, and `BuildImageURL` when image support exists)
|
||||
- `myengine/search.go` (browser mode implementation)
|
||||
- `myengine/search_raw.go` (optional raw mode implementation)
|
||||
|
||||
### 2) Implement `core.SearchEngine`
|
||||
|
||||
Your engine type must implement:
|
||||
|
||||
- `Search(core.Query) ([]core.SearchResult, error)`
|
||||
- `SearchImage(core.Query) ([]core.SearchResult, error)`
|
||||
- `IsInitialized() bool`
|
||||
- `Name() string`
|
||||
- `GetRateLimiter() *rate.Limiter`
|
||||
|
||||
Use the existing engines (for example `google/`) as the reference pattern.
|
||||
|
||||
### 3) Register the engine in server wiring
|
||||
|
||||
Update [`cmd/serve.go`](cmd/serve.go):
|
||||
|
||||
- Add engine spec in `browserEngineSpecs()`
|
||||
- Add raw-mode handling if raw support exists
|
||||
|
||||
### 4) Add config block
|
||||
|
||||
Update [`config.yaml`](config.yaml) with your engine section:
|
||||
|
||||
- `rate_requests`
|
||||
- `rate_burst`
|
||||
- optional `proxy` tag
|
||||
- optional engine-specific fields
|
||||
|
||||
### 5) Add tests
|
||||
|
||||
- URL builder tests (table-driven)
|
||||
- Parser tests (prefer deterministic fixtures in `testdata/`)
|
||||
- Integration tests guarded by `testutil.RequireIntegration(t)`
|
||||
|
||||
## Code Style and Quality Checks
|
||||
|
||||
Run these before opening a PR:
|
||||
|
||||
```bash
|
||||
gofmt -w .
|
||||
go vet ./...
|
||||
golangci-lint run
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
Guidelines:
|
||||
|
||||
- Return `error` values instead of panicking in library code.
|
||||
- Reuse existing patterns in `core/` and existing engines.
|
||||
- Add comments only for non-obvious decisions (why, not what).
|
||||
|
||||
## Test Categories
|
||||
|
||||
- Unit tests: deterministic tests that run with `go test ./...` and do not require browser/network.
|
||||
- Integration tests: live/browser/network dependent tests gated by `OPENSERP_INTEGRATION_TESTS=1`.
|
||||
|
||||
When adding tests, keep unit and integration behavior clearly separated.
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
For each PR:
|
||||
|
||||
1. Describe what changed and why.
|
||||
2. Link the related issue (if available).
|
||||
3. Include or update tests for behavior changes.
|
||||
4. Include updated docs when API/config/contracts change.
|
||||
|
||||
If you change API behavior, update:
|
||||
|
||||
- [`docs/openapi.yaml`](docs/openapi.yaml)
|
||||
- [`README.md`](README.md)
|
||||
- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) when flow/design changes
|
||||
Reference in New Issue
Block a user