docs: add a page for the agent skill and restructure the website sections to clearer

This commit is contained in:
Karim shoair
2026-08-22 19:32:11 +03:00
parent 6e6373c997
commit 6815ca4f4f
4 changed files with 46 additions and 6 deletions
+37
View File
@@ -0,0 +1,37 @@
# Scrapling Agent Skill
While the [MCP server](mcp-server.md) gives AI chatbots tools to scrape websites for you, the **Scrapling Agent Skill** teaches coding agents the library itself, so the code they write with Scrapling matches the current API instead of guessing from outdated training data.
The skill follows the [AgentSkills](https://agentskills.io/specification) specification, so it's readable by [Claude Code](https://claude.com/product/claude-code), [OpenClaw](https://github.com/openclaw/openclaw), and other agentic tools. It packages almost all of this documentation website's content as Markdown references that the agent loads on demand, together with ready-to-run examples, so it can answer most Scrapling questions and write correct scraping code without leaving your editor.
The skill lives in the [agent-skill](https://github.com/D4Vinci/Scrapling/tree/main/agent-skill) directory of the repository under the name `scrapling-official`, and its version always tracks the library version.
## Installation
### Skills.sh
Install the skill from the repository with the [skills.sh](https://skills.sh) CLI:
```bash
npx skills add D4Vinci/Scrapling --skill scrapling-official
```
The CLI detects the agents you have installed and adds the skill to them.
### Clawhub
If you use OpenClaw, you can install the skill through [Clawhub](https://docs.openclaw.ai/tools/clawhub):
```bash
clawhub install scrapling-official
```
Or install it from its [Clawhub page](https://clawhub.ai/D4Vinci/scrapling-official).
### Manual download
Download the skill's ZIP file [directly from the repository](https://github.com/D4Vinci/Scrapling/raw/refs/heads/main/agent-skill/Scrapling-Skill.zip) and extract it into your agent's skills directory (for Claude Code, that's `~/.claude/skills/`).
## What's inside
- **`SKILL.md`**: The entry point the agent reads first. It covers setup, which fetcher to choose, and the safety rules the agent must follow, like always passing `--ai-targeted` to the CLI commands for prompt injection protection.
- **`references/`**: Markdown mirrors of the documentation pages (parsing, fetching, spiders, CLI, MCP server, and integrations), loaded on demand so they don't consume the agent's context until needed.
- **`examples/`**: Ready-to-run scripts the agent can copy from.
We tested the skill on OpenClaw and Claude Code. If you face any issues with it, open a [ticket](https://github.com/D4Vinci/Scrapling/issues/new/choose) or reach out on our [Discord server](https://discord.gg/EMgGbDceNQ).
Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 138 KiB

+1 -1
View File
@@ -1,4 +1,4 @@
# Scrapy
# Integrating Scrapling into Scrapy
If you have an existing Scrapy project, you don't need to rewrite it to enjoy Scrapling's parsing API. The Scrapy integration converts Scrapy responses to Scrapling [Response](../fetching/choosing.md#response-object) objects right inside your spider callbacks, so Scrapy keeps handling the crawling while Scrapling handles the parsing.
+8 -5
View File
@@ -44,14 +44,17 @@ nav = [
{"Interactive shell" = "cli/interactive-shell.md"},
{"Extract commands" = "cli/extract-commands.md"}
]},
{Integrations = [
{"AI MCP server" = "ai/mcp-server.md"},
{Scrapy = "integrations/scrapy.md"}
{"Using with AI" = [
{"MCP server" = "ai/mcp-server.md"},
{"Agent skill" = "ai/agent-skill.md"}
]},
{"Integrations and migrations" = [
{"Integrating into Scrapy" = "integrations/scrapy.md"},
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"}
]}
]},
{Tutorials = [
{"A Free Alternative to AI for Robust Web Scraping" = "tutorials/replacing_ai.md"},
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"}
{"A Free Alternative to AI for Robust Web Scraping" = "tutorials/replacing_ai.md"}
]},
{Development = [
{"API Reference" = [