Add Serping API as an optional SERP provider (serp-analyzer, keyword-research, competitor-analysis) (#14)

Co-authored-by: Musab Gultekin <3749407+musabgultekin@users.noreply.github.com>
Co-authored-by: Quanlai Li <liquanlai1995@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Musab Gultekin
2026-09-18 08:57:18 +03:00
committed by GitHub
parent 828134ce0d
commit e537860372
5 changed files with 115 additions and 2 deletions
+3
View File
@@ -229,6 +229,9 @@ SERPAPI_API_KEY=your_key_here
DATAFORSEO_LOGIN=your_login
DATAFORSEO_PASSWORD=your_password
# Serping API (serp-analyzer, keyword-research, competitor-analysis)
SERPINGAPI_API_KEY=your_key_here
# Glasser (glasser-research; interactive users can run `glasser login` instead)
GLASSER_API_KEY=your_key_here
+2 -1
View File
@@ -36,7 +36,7 @@ After installing, open Claude Code and use any skill as a slash command:
| Skill | Description |
|-------|-------------|
| `seo-audit` | Full technical + on-page SEO audit with actionable fixes |
| `keyword-research` | Keyword research via SemRush, DataForSEO, SerpAPI |
| `keyword-research` | Keyword research via SemRush, DataForSEO, SerpAPI, Serping API |
| `serp-analyzer` | Analyze SERP results and ranking factors |
| `backlink-audit` | Audit backlink profile via SemRush or Ahrefs |
| `schema-markup` | Generate and validate Schema.org structured data |
@@ -108,6 +108,7 @@ RESEND_API_KEY= # Send emails (email-sequence, email-subject-lines)
SEMRUSH_API_KEY= # SEO data (keyword-research, backlink-audit, competitor-analysis)
AHREFS_API_KEY= # Backlink data (backlink-audit)
SERPAPI_API_KEY= # SERP data (serp-analyzer, keyword-research)
SERPINGAPI_API_KEY= # SERP data, free tier (serp-analyzer, keyword-research, competitor-analysis)
BRANDDEV_API_KEY= # Brand monitoring (brand-monitor)
UNSPLASH_CLIENT_ID= # Stock images (write-blog, social-content)
GOOGLE_CLIENT_ID= # Google APIs (analytics, search-console, ads)
+20 -1
View File
@@ -13,6 +13,7 @@ The following API keys enable richer data collection. All are optional -- the fr
- `SEMRUSH_API_KEY` - Domain overview, organic keywords, competitor discovery, traffic estimates
- `SERPAPI_API_KEY` - Real-time SERP competitive analysis, ad copy extraction
- `SERPINGAPI_API_KEY` - Real-time organic SERP positions and SERP features (free tier: 1,000 searches/month, no card)
- `SCRAPINGBEE_API_KEY` - Scrape competitor pages that block direct fetching
### SemRush API (if SEMRUSH_API_KEY available)
@@ -64,6 +65,24 @@ curl -s "https://serpapi.com/search.json?q={commercial_keyword}&api_key=${SERPAP
```
The response `ads` array contains: `position`, `title`, `link`, `displayed_link`, `tracking_link`, `description`, `sitelinks`. This reveals competitor ad copy, landing pages, and messaging.
### Serping API (if SERPINGAPI_API_KEY available)
**SERP Competitive Analysis** - Same use as SerpAPI above, for organic results and SERP features (web search only — no `ads` or `shopping_results`). Free tier is 1,000 searches/month with no card, so it works when no SerpAPI key is configured:
```bash
# Real-time SERP for competitive keywords
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 20}'
```
Use this to:
- Identify which competitors dominate organic results for target keywords (parse `organic`: `position`, `title`, `link`, `snippet`)
- Discover related competitor keywords from `relatedSearches` (`.query`)
- See competitor presence in SERP features: `knowledgeGraph`, `answerBox`, `peopleAlsoAsk`
- Add `"tbs": "qdr:m"` to see who ranked in the last month, or `"location"` for a local SERP
Errors are `{"error": {"code", "message"}}`; on `429 quota_exceeded` tell the user the monthly quota is spent and fall back to WebSearch.
### ScrapingBee (if SCRAPINGBEE_API_KEY available)
Use ScrapingBee to scrape competitor pages that block direct fetching via WebFetch (e.g., JavaScript-heavy pages, bot-protected sites, pricing pages):
@@ -97,7 +116,7 @@ Google "[category]" (ads + organic top 10), G2/Capterra "Compare" pages, Reddit/
## Step 3: SEO Analysis
If `SEMRUSH_API_KEY` is available, use the Domain Overview and Organic Keywords endpoints (see Optional API Integrations above) to populate the profile below with real data. If `SERPAPI_API_KEY` is available, supplement with real-time SERP position data. Otherwise, use WebSearch and public tools to estimate.
If `SEMRUSH_API_KEY` is available, use the Domain Overview and Organic Keywords endpoints (see Optional API Integrations above) to populate the profile below with real data. If `SERPAPI_API_KEY` or `SERPINGAPI_API_KEY` is available, supplement with real-time SERP position data. Otherwise, use WebSearch and public tools to estimate.
For each competitor:
+44
View File
@@ -351,3 +351,47 @@ curl -s "https://serpapi.com/search.json?q={keyword}&api_key=${SERPAPI_API_KEY}&
- `device=desktop` or `device=mobile` - Desktop vs mobile SERPs (mobile may show different PAA questions)
**Note:** SerpAPI charges per search. Use it strategically for your highest-priority keywords rather than for bulk research. Pair it with SemRush for volume data and DataForSEO for bulk lookups.
### Serping API (People Also Ask & Related Searches — free tier)
If `SERPINGAPI_API_KEY` is available, use Serping API (https://serpingapi.com) for the same "People Also Ask" and related-searches data from live Google SERPs. The free Reader plan includes 1,000 searches per month with no credit card, so it is a good default when no SerpAPI key is set. Paid plans are flat monthly (from $49 for 20,000 searches).
**Search Endpoint:**
```bash
# Get SERP data including People Also Ask and related searches
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 10}'
```
**Parsing People Also Ask:**
```bash
# Extract People Also Ask questions
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" -H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 10}' | \
jq -r '.peopleAlsoAsk[] | {question, snippet, link, title}'
```
Key response sections (present only when Google shows them):
- **`peopleAlsoAsk`** - Array of "People Also Ask" questions with snippets and source URLs
- **`relatedSearches`** - Array of `{ "query": ... }` objects that Google suggests
- **`organic`** - Organic results with `position`, `title`, `link`, `snippet`
- **`answerBox`** / **`knowledgeGraph`** - Featured snippet and knowledge panel, when present
**Parsing Related Searches:**
```bash
# Extract related searches for content ideation
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" -H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 10}' | \
jq -r '.relatedSearches[] | .query'
```
Use the data exactly as described for SerpAPI above (FAQ headings, gap discovery, keyword expansion, intent validation, clustering).
**Additional Serping API parameters:** `location` (e.g. `"Seattle, Washington, United States"`), `page` (starting at 1), `tbs` time filter (`qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`). Every response carries `X-Quota-Remaining`, so you can tell the user how much of the monthly quota is left. Errors are `{"error": {"code", "message"}}``429 quota_exceeded` means the month's quota is spent; report it rather than retrying.
+46
View File
@@ -13,6 +13,7 @@ Optional API keys for enriched data (the skill can work without any of them usin
- `SEMRUSH_API_KEY` - for keyword and organic results data
- `SERPAPI_API_KEY` - for real-time Google SERP data including SERP features
- `DATAFORSEO_LOGIN` and `DATAFORSEO_PASSWORD` - for advanced SERP data
- `SERPINGAPI_API_KEY` - for real-time Google SERP data (free tier: 1,000 searches/month, no card)
## Analysis Process
@@ -95,6 +96,51 @@ The response provides:
Location codes: 2840 = US, 2826 = UK, 2124 = Canada, 2036 = Australia. Change `location_code` for geo-targeted analysis.
**Method F: Serping API (if SERPINGAPI_API_KEY available)**
Real-time Google SERP data as Serper-style JSON from a single endpoint. The free Reader plan includes 1,000 searches per month with no credit card, so this works for a first run without a deposit. Get a key at https://serpingapi.com and set `SERPINGAPI_API_KEY`.
```bash
# Real-time Google SERP data via Serping API (POST, JSON body)
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 20}'
```
The JSON response includes (sections appear only when Google returns them):
- `organic` - Array of organic listings with `position`, `title`, `link`, `snippet` (sometimes `sitelinks`, `date`, `rating`)
- `peopleAlsoAsk` - People Also Ask questions with `question`, `snippet`, `title`, `link`
- `answerBox` - Featured snippet / direct answer
- `knowledgeGraph` - Knowledge panel data with `title`, `type`, `description`, `attributes`
- `relatedSearches` - Related search queries as `{ "query": ... }`
- `searchParameters` - Echo of the parameters the search ran with
Parse example:
```bash
# Extract organic results
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" -H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 20}' | \
jq '.organic[] | {position, title, link, snippet}'
# Extract People Also Ask questions
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" -H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 20}' | \
jq '.peopleAlsoAsk[] | {question, snippet}'
# Check for featured snippet / knowledge graph
curl -s -X POST "https://api.serpingapi.com/v1/search" \
-H "X-API-Key: ${SERPINGAPI_API_KEY}" -H "Content-Type: application/json" \
-d '{"q": "{keyword}", "gl": "us", "hl": "en", "num": 20}' | \
jq '{answerBox, knowledgeGraph: (.knowledgeGraph | {title, type, description})}'
```
Optional parameters: `location` (e.g. `"Seattle, Washington, United States"`), `page` (starting at 1), `tbs` for a time filter (`qdr:d` day, `qdr:w` week, `qdr:m` month, `qdr:y` year). Web search only — no ads, shopping, or local pack sections.
Errors come back as `{"error": {"code": "...", "message": "..."}}`: `401 invalid_api_key` means the key is wrong or revoked; `429 quota_exceeded` means the monthly quota is used up (resets on the 1st, UTC). In either case tell the user the specific error and fall back to Method B.
### Step 2: Map SERP Features
Document every SERP feature present for this keyword: