mirror of
https://github.com/apify/agent-skills.git
synced 2026-09-14 20:00:06 +08:00
chore: align docs with Apify writing style guide
Apply sentence case to headings, capitalize Actor as a noun, drop the
article before "Apify Console" / "Apify Store", add the article before
"the Apify platform", and fix a few typos ("confirms" -> "conforms",
"javascript" -> "JavaScript"). Code blocks, CLI commands, file paths,
and JSON values are unchanged.
This commit is contained in:
Dusan Vystrcil
@@ -8,7 +8,7 @@ A collection of AI agent skills for web scraping, data extraction, and Actor dev
|
||||
|
||||
### Scraping
|
||||
|
||||
- **[Ultimate scraper](skills/apify-ultimate-scraper/)** (`apify-ultimate-scraper`) — AI-powered web scraper for 55+ platforms including Instagram, Facebook, TikTok, YouTube, Google Maps, Amazon, Walmart, eBay, Booking.com, TripAdvisor, and more. Can also search the [Apify Store](https://apify.com/store) to find the right Actor for any platform not listed here.
|
||||
- **[Ultimate scraper](skills/apify-ultimate-scraper/)** (`apify-ultimate-scraper`) — AI-powered web scraper for 55+ platforms including Instagram, Facebook, TikTok, YouTube, Google Maps, Amazon, Walmart, eBay, Booking.com, TripAdvisor, and more. Can also search [Apify Store](https://apify.com/store) to find the right Actor for any platform not listed here.
|
||||
|
||||
### Development
|
||||
|
||||
|
||||
+23
-23
@@ -1,13 +1,13 @@
|
||||
---
|
||||
description: Guided Apify Actor development with best practices and systematic workflow
|
||||
argument-hint: Optional actor description
|
||||
argument-hint: Optional Actor description
|
||||
---
|
||||
|
||||
# Actor Development
|
||||
# Actor development
|
||||
|
||||
You are helping a developer create an Apify Actor - a serverless cloud program for web scraping, automation, and data processing. Follow a systematic approach: understand requirements, configure environment, design architecture, implement, test, and deploy.
|
||||
|
||||
## Core Principles
|
||||
## Core principles
|
||||
|
||||
- **Ask clarifying questions**: Identify target websites, data requirements, edge cases, and constraints before implementation
|
||||
- **Follow Apify best practices**: Use appropriate crawlers (Cheerio vs Playwright), implement proper error handling, respect rate limits
|
||||
@@ -19,14 +19,14 @@ You are helping a developer create an Apify Actor - a serverless cloud program f
|
||||
|
||||
## Phase 1: Discovery
|
||||
|
||||
**Goal**: Understand what actor needs to be built
|
||||
**Goal**: Understand what Actor needs to be built
|
||||
|
||||
Initial request: $ARGUMENTS
|
||||
|
||||
**Actions**:
|
||||
1. Create todo list with all phases
|
||||
2. Ask user for clarification if needed:
|
||||
- What is the actor's primary purpose? (web scraping, automation, data processing)
|
||||
- What is the Actor's primary purpose? (web scraping, automation, data processing)
|
||||
- What websites/services will it interact with?
|
||||
- What data should it extract or what actions should it perform?
|
||||
- Any specific requirements or constraints?
|
||||
@@ -34,7 +34,7 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Environment Setup
|
||||
## Phase 2: Environment setup
|
||||
|
||||
**Goal**: Verify Apify CLI is installed and authenticated
|
||||
|
||||
@@ -51,7 +51,7 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Language Selection
|
||||
## Phase 3: Language selection
|
||||
|
||||
**Goal**: Choose programming language and template
|
||||
|
||||
@@ -64,13 +64,13 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Requirements & Architecture Design
|
||||
## Phase 4: Requirements and architecture design
|
||||
|
||||
**Goal**: Define input/output schemas and implementation approach
|
||||
|
||||
**Actions**:
|
||||
1. Clarify detailed requirements:
|
||||
- What input parameters should the actor accept?
|
||||
- What input parameters should the Actor accept?
|
||||
- What output format is needed? (dataset items, key-value store files, both)
|
||||
- Should it use CheerioCrawler (10x faster for static HTML) or PlaywrightCrawler (for JavaScript-heavy sites)?
|
||||
- Concurrency settings? (HTTP: 10-50, Browser: 1-5)
|
||||
@@ -86,9 +86,9 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Actor Creation
|
||||
## Phase 5: Actor creation
|
||||
|
||||
**Goal**: Create actor from template and configure schemas
|
||||
**Goal**: Create Actor from template and configure schemas
|
||||
|
||||
**DO NOT START WITHOUT USER APPROVAL**
|
||||
|
||||
@@ -96,7 +96,7 @@ Initial request: $ARGUMENTS
|
||||
1. Wait for explicit user approval
|
||||
2. Copy appropriate language template from `skills/apify-actor-development/references/` directory
|
||||
3. Update `.actor/actor.json`:
|
||||
- Set actor name and version
|
||||
- Set Actor name and version
|
||||
- **IMPORTANT**: Fill in `generatedBy` property with current model name
|
||||
- Configure runtime, memory, timeout
|
||||
- Set `usesStandbyMode` if applicable
|
||||
@@ -117,12 +117,12 @@ Initial request: $ARGUMENTS
|
||||
|
||||
## Phase 6: Implementation
|
||||
|
||||
**Goal**: Implement actor logic following best practices
|
||||
**Goal**: Implement Actor logic following best practices
|
||||
|
||||
**Actions**:
|
||||
1. Implement actor code in `src/main.py`, `src/main.js`, or `src/main.ts`
|
||||
1. Implement Actor code in `src/main.py`, `src/main.js`, or `src/main.ts`
|
||||
2. Follow best practices:
|
||||
- ✓ Use Apify SDK (`apify`) for code running on Apify platform
|
||||
- ✓ Use Apify SDK (`apify`) for code running on the Apify platform
|
||||
- ✓ Validate input early with proper error handling
|
||||
- ✓ Use CheerioCrawler for static HTML (10x faster)
|
||||
- ✓ Use PlaywrightCrawler only for JavaScript-heavy sites
|
||||
@@ -148,7 +148,7 @@ Initial request: $ARGUMENTS
|
||||
|
||||
**Actions**:
|
||||
1. Create README.md with:
|
||||
- Clear description of what the actor does
|
||||
- Clear description of what the Actor does
|
||||
- Input parameters with examples
|
||||
- Output format with examples
|
||||
- Usage instructions
|
||||
@@ -159,16 +159,16 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Local Testing
|
||||
## Phase 8: Local testing
|
||||
|
||||
**Goal**: Test actor locally before deployment
|
||||
**Goal**: Test Actor locally before deployment
|
||||
|
||||
**Actions**:
|
||||
1. Install dependencies:
|
||||
- JavaScript/TypeScript: `npm install`
|
||||
- Python: `pip install -r requirements.txt`
|
||||
2. Create test input file at `storage/key_value_stores/default/INPUT.json` with sample parameters
|
||||
3. Run actor locally: `apify run`
|
||||
3. Run Actor locally: `apify run`
|
||||
4. Verify:
|
||||
- Input is parsed correctly
|
||||
- Actor completes successfully
|
||||
@@ -182,7 +182,7 @@ Initial request: $ARGUMENTS
|
||||
|
||||
## Phase 9: Deployment
|
||||
|
||||
**Goal**: Deploy actor to Apify platform
|
||||
**Goal**: Deploy Actor to the Apify platform
|
||||
|
||||
**DO NOT DEPLOY WITHOUT USER APPROVAL**
|
||||
|
||||
@@ -192,7 +192,7 @@ Initial request: $ARGUMENTS
|
||||
3. Actor will be deployed with name from `.actor/actor.json`
|
||||
4. Provide user with:
|
||||
- Deployment confirmation
|
||||
- Actor URL on Apify platform
|
||||
- Actor URL on the Apify platform
|
||||
- Instructions for running on platform
|
||||
|
||||
---
|
||||
@@ -204,7 +204,7 @@ Initial request: $ARGUMENTS
|
||||
**Actions**:
|
||||
1. Mark all todos complete
|
||||
2. Summarize:
|
||||
- What actor was built
|
||||
- What Actor was built
|
||||
- Key features and capabilities
|
||||
- Input/output schemas
|
||||
- Files created/modified
|
||||
@@ -213,7 +213,7 @@ Initial request: $ARGUMENTS
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
## Additional resources
|
||||
|
||||
**MCP Tools** (if configured):
|
||||
- `search-apify-docs` - Search documentation
|
||||
|
||||
@@ -3,7 +3,7 @@ name: apify-actor-development
|
||||
description: Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Actor code.
|
||||
---
|
||||
|
||||
# Apify Actor Development
|
||||
# Apify Actor development
|
||||
|
||||
**Important:** Before you begin, fill in the `generatedBy` property in the meta section of `.actor/actor.json`. Replace it with the tool and model you're currently using, such as "Claude Code with Claude Sonnet 4.5". This helps Apify monitor and improve AGENTS.md for specific AI tools and models.
|
||||
|
||||
@@ -18,9 +18,9 @@ Actors are serverless programs inspired by the UNIX philosophy - programs that d
|
||||
- Can run from seconds to hours or even indefinitely
|
||||
- Persist state and can be restarted
|
||||
|
||||
## Prerequisites & Setup (MANDATORY)
|
||||
## Prerequisites and setup (mandatory)
|
||||
|
||||
Before creating or modifying actors, verify that `apify` CLI is installed `apify --help`.
|
||||
Before creating or modifying Actors, verify that `apify` CLI is installed `apify --help`.
|
||||
|
||||
If it is not installed, use one of these methods (listed in order of preference):
|
||||
|
||||
@@ -54,27 +54,27 @@ If browser login isn't available (headless environment or CI), the CLI automatic
|
||||
> Never log, print, or embed `APIFY_TOKEN` in source code or configuration files.
|
||||
> Use a token with the minimum required permissions (scoped token) and rotate it periodically.
|
||||
|
||||
## Template Selection
|
||||
## Template selection
|
||||
|
||||
**IMPORTANT:** Before starting actor development, always ask the user which programming language they prefer:
|
||||
**IMPORTANT:** Before starting Actor development, always ask the user which programming language they prefer:
|
||||
- **JavaScript** - Use `apify create <actor-name> -t project_empty`
|
||||
- **TypeScript** - Use `apify create <actor-name> -t ts_empty`
|
||||
- **Python** - Use `apify create <actor-name> -t python-empty`
|
||||
|
||||
Use the appropriate CLI command based on the user's language choice. Additional packages (Crawlee, Playwright, etc.) can be installed later as needed.
|
||||
|
||||
## Quick Start Workflow
|
||||
## Quick start workflow
|
||||
|
||||
1. **Create actor project** - Run the appropriate `apify create` command based on user's language preference (see Template Selection above)
|
||||
1. **Create Actor project** - Run the appropriate `apify create` command based on user's language preference (see Template selection above)
|
||||
2. **Install dependencies** (verify package names match intended packages before installing)
|
||||
- JavaScript/TypeScript: `npm install` (uses `package-lock.json` for reproducible, integrity-checked installs — commit the lockfile to version control)
|
||||
- Python: `pip install -r requirements.txt` (pin exact versions in `requirements.txt`, e.g. `crawlee==1.2.3`, and commit the file to version control)
|
||||
3. **Implement logic** - Write the actor code in `src/main.py`, `src/main.js`, or `src/main.ts`
|
||||
3. **Implement logic** - Write the Actor code in `src/main.py`, `src/main.js`, or `src/main.ts`
|
||||
4. **Configure schemas** - Update input/output schemas in `.actor/input_schema.json`, `.actor/output_schema.json`, `.actor/dataset_schema.json`
|
||||
5. **Configure platform settings** - Update `.actor/actor.json` with actor metadata (see [references/actor-json.md](references/actor-json.md))
|
||||
5. **Configure platform settings** - Update `.actor/actor.json` with Actor metadata (see [references/actor-json.md](references/actor-json.md))
|
||||
6. **Write documentation** - Create comprehensive README.md for the marketplace (see [references/actor-readme.md](references/actor-readme.md) — this is mandatory, not optional)
|
||||
7. **Test locally** - Run `apify run` to verify functionality (see Local Testing section below)
|
||||
8. **Deploy** - Run `apify push` to deploy the actor on the Apify platform (actor name is defined in `.actor/actor.json`)
|
||||
7. **Test locally** - Run `apify run` to verify functionality (see Local testing section below)
|
||||
8. **Deploy** - Run `apify push` to deploy the Actor on the Apify platform (Actor name is defined in `.actor/actor.json`)
|
||||
|
||||
## Security
|
||||
|
||||
@@ -87,11 +87,11 @@ Use the appropriate CLI command based on the user's language choice. Additional
|
||||
- **Review dependencies before installing** — When adding packages with `npm install` or `pip install`, verify the package name and publisher. Typosquatting is a common supply-chain attack vector. Prefer well-known, actively maintained packages.
|
||||
- **Pin versions and use lockfiles** — Always commit `package-lock.json` (Node.js) or pin exact versions in `requirements.txt` (Python). Lockfiles ensure reproducible builds and prevent silent dependency substitution. Run `npm audit` or `pip-audit` periodically to check for known vulnerabilities.
|
||||
|
||||
## Best Practices
|
||||
## Best practices
|
||||
|
||||
**✓ Do:**
|
||||
- Use `apify run` to test actors locally (configures Apify environment and storage)
|
||||
- Use Apify SDK (`apify`) for code running ON Apify platform
|
||||
- Use `apify run` to test Actors locally (configures Apify environment and storage)
|
||||
- Use Apify SDK (`apify`) for code running on the Apify platform
|
||||
- Validate input early with proper error handling and fail gracefully
|
||||
- Use CheerioCrawler for static HTML (10x faster than browsers)
|
||||
- Use PlaywrightCrawler only for JavaScript-heavy sites
|
||||
@@ -107,8 +107,8 @@ Use the appropriate CLI command based on the user's language choice. Additional
|
||||
- Implement readiness probe handler (required if your Actor uses standby mode)
|
||||
|
||||
**✗ Don't:**
|
||||
- Use `npm start`, `npm run start`, `npx apify run`, or similar commands to run actors (use `apify run` instead)
|
||||
- Assume local storage from `apify run` is pushed to or visible in the Apify Console — it is local-only; deploy with `apify push` and run on the platform to see results in the Console
|
||||
- Use `npm start`, `npm run start`, `npx apify run`, or similar commands to run Actors (use `apify run` instead)
|
||||
- Assume local storage from `apify run` is pushed to or visible in Apify Console — it is local-only; deploy with `apify push` and run on the platform to see results in Apify Console
|
||||
- Rely on `Dataset.getInfo()` for final counts on Cloud
|
||||
- Use browser crawlers when HTTP/Cheerio works
|
||||
- Hard code values that should be in input schema or environment variables
|
||||
@@ -135,11 +135,11 @@ apify push # Deploy to Apify platform (uses name from .actor/actor.json)
|
||||
apify help # List all commands
|
||||
```
|
||||
|
||||
**IMPORTANT:** Always use `apify run` to test actors locally. Do not use `npm run start`, `npm start`, `yarn start`, or other package manager commands - these will not properly configure the Apify environment and storage.
|
||||
**IMPORTANT:** Always use `apify run` to test Actors locally. Do not use `npm run start`, `npm start`, `yarn start`, or other package manager commands - these will not properly configure the Apify environment and storage.
|
||||
|
||||
## Local Testing
|
||||
## Local testing
|
||||
|
||||
When testing an actor locally with `apify run`, provide input data by creating a JSON file at:
|
||||
When testing an Actor locally with `apify run`, provide input data by creating a JSON file at:
|
||||
|
||||
```
|
||||
storage/key_value_stores/default/INPUT.json
|
||||
@@ -147,13 +147,13 @@ storage/key_value_stores/default/INPUT.json
|
||||
|
||||
This file should contain the input parameters defined in your `.actor/input_schema.json`. The actor will read this input when running locally, mirroring how it receives input on the Apify platform.
|
||||
|
||||
**IMPORTANT - Local storage is NOT synced to the Apify Console:**
|
||||
**IMPORTANT - Local storage is NOT synced to Apify Console:**
|
||||
- Running `apify run` stores all data (datasets, key-value stores, request queues) **only on your local filesystem** in the `storage/` directory.
|
||||
- This data is **never** automatically uploaded or pushed to the Apify platform. It exists only on your machine.
|
||||
- To verify results on the Apify Console, you must deploy the Actor with `apify push` and then run it on the platform.
|
||||
- Do **not** rely on checking the Apify Console to verify results from local runs — instead, inspect the local `storage/` directory or check the Actor's log output.
|
||||
- To verify results on Apify Console, you must deploy the Actor with `apify push` and then run it on the platform.
|
||||
- Do **not** rely on checking Apify Console to verify results from local runs — instead, inspect the local `storage/` directory or check the Actor's log output.
|
||||
|
||||
## Standby Mode
|
||||
## Standby mode
|
||||
|
||||
Standby mode enables Actors to work as API servers - they remain ready in the background to handle HTTP requests.
|
||||
|
||||
@@ -161,7 +161,7 @@ Standby mode enables Actors to work as API servers - they remain ready in the ba
|
||||
|
||||
When building a Standby Actor, set `usesStandbyMode: true` in `.actor/actor.json` and implement an HTTP server. See [references/standby-mode.md](references/standby-mode.md) for configuration, environment variables, complete code examples, and operational limits.
|
||||
|
||||
## Project Structure
|
||||
## Project structure
|
||||
|
||||
```
|
||||
.actor/
|
||||
@@ -177,23 +177,23 @@ storage/ # Local-only storage (NOT synced to Apify Console)
|
||||
Dockerfile # Container image definition
|
||||
```
|
||||
|
||||
## Actor Configuration
|
||||
## Actor configuration
|
||||
|
||||
See [references/actor-json.md](references/actor-json.md) for complete actor.json structure and configuration options.
|
||||
|
||||
## Input Schema
|
||||
## Input schema
|
||||
|
||||
See [references/input-schema.md](references/input-schema.md) for input schema structure and examples.
|
||||
|
||||
## Output Schema
|
||||
## Output schema
|
||||
|
||||
See [references/output-schema.md](references/output-schema.md) for output schema structure, examples, and template variables.
|
||||
|
||||
## Dataset Schema
|
||||
## Dataset schema
|
||||
|
||||
See [references/dataset-schema.md](references/dataset-schema.md) for dataset schema structure, configuration, and display properties.
|
||||
|
||||
## Key-Value Store Schema
|
||||
## Key-value store schema
|
||||
|
||||
See [references/key-value-store-schema.md](references/key-value-store-schema.md) for key-value store schema structure, collections, and configuration.
|
||||
|
||||
@@ -206,7 +206,7 @@ See [references/actor-readme.md](references/actor-readme.md) for the required st
|
||||
- [Instagram Scraper](https://apify.com/apify/instagram-scraper)
|
||||
- [Google Maps Scraper](https://apify.com/compass/crawler-google-places)
|
||||
|
||||
## Apify MCP Tools
|
||||
## Apify MCP tools
|
||||
|
||||
If MCP server is configured, use these tools for documentation:
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Actor Configuration (actor.json)
|
||||
# Actor configuration (actor.json)
|
||||
|
||||
The `.actor/actor.json` file contains the Actor's configuration including metadata, schema references, and platform settings.
|
||||
|
||||
@@ -30,8 +30,8 @@ The `.actor/actor.json` file contains the Actor's configuration including metada
|
||||
{
|
||||
"actorSpecification": 1,
|
||||
"name": "project-cheerio-crawler-javascript",
|
||||
"title": "Project Cheerio Crawler Javascript",
|
||||
"description": "Crawlee and Cheerio project in javascript.",
|
||||
"title": "Project Cheerio Crawler JavaScript",
|
||||
"description": "Crawlee and Cheerio project in JavaScript.",
|
||||
"version": "0.0",
|
||||
"meta": {
|
||||
"templateId": "js-crawlee-cheerio",
|
||||
@@ -48,14 +48,14 @@ The `.actor/actor.json` file contains the Actor's configuration including metada
|
||||
|
||||
## Properties
|
||||
|
||||
- `actorSpecification` (integer, required) - Version of actor specification (currently 1)
|
||||
- `actorSpecification` (integer, required) - Version of Actor specification (currently 1)
|
||||
- `name` (string, required) - Actor identifier (lowercase, hyphens allowed)
|
||||
- `title` (string, required) - Human-readable title displayed in UI
|
||||
- `description` (string, optional) - Actor description for marketplace
|
||||
- `version` (string, required) - Semantic version number
|
||||
- `meta` (object, optional) - Metadata about actor generation
|
||||
- `templateId` (string) - ID of template used to create the actor
|
||||
- `generatedBy` (string) - Tool and model name that generated/modified the actor (e.g., "Claude Code with Claude Sonnet 4.5")
|
||||
- `meta` (object, optional) - Metadata about Actor generation
|
||||
- `templateId` (string) - ID of template used to create the Actor
|
||||
- `generatedBy` (string) - Tool and model name that generated/modified the Actor (e.g., "Claude Code with Claude Sonnet 4.5")
|
||||
- `input` (string, optional) - Path to input schema file
|
||||
- `output` (string, optional) - Path to output schema file
|
||||
- `storages` (object, optional) - Storage schema references
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Actor README Guidelines
|
||||
# Actor README guidelines
|
||||
|
||||
The README is the Actor's landing page on Apify Store. It serves as SEO content, first impression, usage guide, and support resource. **Always generate a README.md when creating or deploying an Actor.**
|
||||
|
||||
## Required Structure
|
||||
## Required structure
|
||||
|
||||
Write in Markdown. Use H2 (`##`) for main sections (these form the table of contents) and H3 (`###`) for subsections. Do not use H1 — the Actor name is automatically used as H1.
|
||||
|
||||
@@ -17,7 +17,7 @@ Write in Markdown. Use H2 (`##`) for main sections (these form the table of cont
|
||||
|
||||
- Business use cases and benefits
|
||||
- List main features and capabilities
|
||||
- Highlight Apify platform advantages: scheduling, API access, integrations, proxy rotation, monitoring
|
||||
- Highlight the Apify platform advantages: scheduling, API access, integrations, proxy rotation, and monitoring
|
||||
|
||||
### 3. What data can [Actor name] extract?
|
||||
|
||||
@@ -62,7 +62,7 @@ Write in Markdown. Use H2 (`##`) for main sections (these form the table of cont
|
||||
- Link to API tab for programmatic access
|
||||
- Use cases for the extracted data
|
||||
|
||||
## SEO Best Practices
|
||||
## SEO best practices
|
||||
|
||||
- Include keywords naturally in H2/H3 headings (e.g., "How to scrape Instagram" not just "How to use")
|
||||
- Target "People Also Ask" style questions as H3 headings
|
||||
@@ -79,12 +79,12 @@ Write in Markdown. Use H2 (`##`) for main sections (these form the table of cont
|
||||
|
||||
## Reference Actors
|
||||
|
||||
Before writing a README, review these top Actors on the Apify Store for best practices on structure, tone, and content:
|
||||
Before writing a README, review these top Actors on Apify Store for best practices on structure, tone, and content:
|
||||
|
||||
- [Instagram Scraper](https://apify.com/apify/instagram-scraper)
|
||||
- [Google Maps Scraper](https://apify.com/compass/crawler-google-places)
|
||||
|
||||
## Key Rules
|
||||
## Key rules
|
||||
|
||||
- Always write the README as part of Actor development — do not skip this step
|
||||
- The first 25% of the README is what most visitors read — put the most important info there
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Dataset Schema Reference
|
||||
# Dataset schema reference
|
||||
|
||||
The dataset schema defines how your Actor's output data is structured, transformed, and displayed in the Output tab in the Apify Console.
|
||||
The dataset schema defines how your Actor's output data is structured, transformed, and displayed in the Output tab in Apify Console.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -176,20 +176,20 @@ Then create the dataset schema in `.actor/dataset_schema.json`:
|
||||
|
||||
## Properties
|
||||
|
||||
### Dataset Schema Properties
|
||||
### Dataset schema properties
|
||||
|
||||
- `actorSpecification` (integer, required) - Specifies the version of dataset schema structure document (currently only version 1)
|
||||
- `fields` (JSONSchema object, required) - Schema of one dataset object (use JsonSchema Draft 2020-12 or compatible)
|
||||
- `views` (DatasetView object, required) - Object with API and UI views description
|
||||
|
||||
### DatasetView Properties
|
||||
### DatasetView properties
|
||||
|
||||
- `title` (string, required) - Visible in UI Output tab and API
|
||||
- `description` (string, optional) - Only available in API response
|
||||
- `transformation` (ViewTransformation object, required) - Data transformation applied when loading from Dataset API
|
||||
- `display` (ViewDisplay object, required) - Output tab UI visualization definition
|
||||
|
||||
### ViewTransformation Properties
|
||||
### ViewTransformation properties
|
||||
|
||||
- `fields` (string[], required) - Fields to present in output (order matches column order)
|
||||
- `unwind` (string[], optional) - Deconstructs nested children into parent object
|
||||
@@ -198,12 +198,12 @@ Then create the dataset schema in `.actor/dataset_schema.json`:
|
||||
- `limit` (integer, optional) - Maximum number of results (default: all)
|
||||
- `desc` (boolean, optional) - Sort order (true = newest first)
|
||||
|
||||
### ViewDisplay Properties
|
||||
### ViewDisplay properties
|
||||
|
||||
- `component` (string, required) - Only `table` is available
|
||||
- `properties` (Object, optional) - Keys matching `transformation.fields` with ViewDisplayProperty values
|
||||
|
||||
### ViewDisplayProperty Properties
|
||||
### ViewDisplayProperty properties
|
||||
|
||||
- `label` (string, optional) - Table column header
|
||||
- `format` (string, optional) - One of: `text`, `number`, `date`, `link`, `boolean`, `image`, `array`, `object`
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Input Schema Reference
|
||||
# Input schema reference
|
||||
|
||||
The input schema defines the input parameters for an Actor. It's a JSON object comprising various field types supported by the Apify platform.
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Key-Value Store Schema Reference
|
||||
# Key-value store schema reference
|
||||
|
||||
The key-value store schema organizes keys into logical groups called collections for easier data management.
|
||||
|
||||
@@ -110,14 +110,14 @@ Then create the key-value store schema in `.actor/key_value_store_schema.json`:
|
||||
|
||||
## Properties
|
||||
|
||||
### Key-Value Store Schema Properties
|
||||
### Key-value store schema properties
|
||||
|
||||
- `actorKeyValueStoreSchemaVersion` (integer, required) - Version of key-value store schema structure document (currently only version 1)
|
||||
- `title` (string, required) - Title of the schema
|
||||
- `description` (string, optional) - Description of the schema
|
||||
- `collections` (Object, required) - Object where each key is a collection ID and value is a Collection object
|
||||
|
||||
### Collection Properties
|
||||
### Collection properties
|
||||
|
||||
- `title` (string, required) - Collection title shown in UI tabs
|
||||
- `description` (string, optional) - Description appearing in UI tooltips
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Actor Logging Reference
|
||||
# Actor logging reference
|
||||
|
||||
## JavaScript and TypeScript
|
||||
|
||||
**ALWAYS use the `apify/log` package for logging** - This package contains critical security logic including censoring sensitive data (Apify tokens, API keys, credentials) to prevent accidental exposure in logs.
|
||||
|
||||
### Available Log Levels in `apify/log`
|
||||
### Available log levels in `apify/log`
|
||||
|
||||
The Apify log package provides the following methods for logging:
|
||||
|
||||
@@ -19,7 +19,7 @@ The Apify log package provides the following methods for logging:
|
||||
- `log.softFail()` - Soft failure logs (non-critical failures that don't stop execution, e.g., input validation errors, skipped items)
|
||||
- `log.internal()` - Internal level logs (internal/system messages)
|
||||
|
||||
### Best Practices
|
||||
### Best practices
|
||||
|
||||
- Use `log.debug()` for detailed operation-level diagnostics (inside functions)
|
||||
- Use `log.info()` for general informational messages (API requests, successful operations)
|
||||
@@ -31,7 +31,7 @@ The Apify log package provides the following methods for logging:
|
||||
|
||||
**ALWAYS use `Actor.log` for logging** - This logger contains critical security logic including censoring sensitive data (Apify tokens, API keys, credentials) to prevent accidental exposure in logs.
|
||||
|
||||
### Available Log Levels
|
||||
### Available log levels
|
||||
|
||||
The Apify Actor logger provides the following methods for logging:
|
||||
|
||||
@@ -41,7 +41,7 @@ The Apify Actor logger provides the following methods for logging:
|
||||
- `Actor.log.error()` - Error level logs (error messages for failures)
|
||||
- `Actor.log.exception()` - Exception level logs (for exceptions with stack traces)
|
||||
|
||||
### Best Practices
|
||||
### Best practices
|
||||
|
||||
- Use `Actor.log.debug()` for detailed operation-level diagnostics (inside functions)
|
||||
- Use `Actor.log.info()` for general informational messages (API requests, successful operations)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Output Schema Reference
|
||||
# Output schema reference
|
||||
|
||||
The Actor output schema builds upon the schemas for the dataset and key-value store. It specifies where an Actor stores its output and defines templates for accessing that output. Apify Console uses these output definitions to display run results.
|
||||
|
||||
@@ -35,7 +35,7 @@ The Actor output schema builds upon the schemas for the dataset and key-value st
|
||||
}
|
||||
```
|
||||
|
||||
## Output Schema Template Variables
|
||||
## Output schema template variables
|
||||
|
||||
- `links` (object) - Contains quick links to most commonly used URLs
|
||||
- `links.publicRunUrl` (string) - Public run url in format `https://console.apify.com/view/runs/:runId`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Actor Standby Mode Reference
|
||||
# Actor Standby mode reference
|
||||
|
||||
## When to Use Standby Mode
|
||||
## When to use Standby mode
|
||||
|
||||
Use Standby when the Actor must handle interactive, real-time HTTP requests — API endpoints, webhook receivers, real-time data lookups, MCP servers, or scraping APIs serving on-demand single-URL requests.
|
||||
|
||||
@@ -27,13 +27,13 @@ Set `usesStandbyMode: true` in `.actor/actor.json`:
|
||||
|
||||
### OpenAPI Schema (`webServerSchema`)
|
||||
|
||||
Define an OpenAPI v3 schema describing the Actor's HTTP endpoints. This can be a file path (e.g., `"./openapi.json"`) or an inline object in `actor.json`. Ensure that the schema confirms to the OpenAPI spec.
|
||||
Define an OpenAPI v3 schema describing the Actor's HTTP endpoints. This can be a file path (e.g., `"./openapi.json"`) or an inline object in `actor.json`. Ensure that the schema conforms to the OpenAPI spec.
|
||||
|
||||
**Why:** The schema is rendered as Swagger UI in the Standby tab of Apify Console and on the Actor's Store page. This lets users browse endpoint documentation and try out the API directly from the browser.
|
||||
**Why:** The schema is rendered as Swagger UI in the Standby tab of Apify Console and on the Actor's Apify Store page. This lets users browse endpoint documentation and try out the API directly from the browser.
|
||||
|
||||
The presence of `webServerSchema` also counts as a quality metric for Actor publication.
|
||||
|
||||
### Environment Variables
|
||||
### Environment variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
@@ -41,7 +41,7 @@ The presence of `webServerSchema` also counts as a quality metric for Actor publ
|
||||
| `ACTOR_STANDBY_URL` | The public Standby URL (stable across runs, format: `https://<username>--<actor-name>.apify.actor`) |
|
||||
| `APIFY_META_ORIGIN` | Set to `STANDBY` when the Actor was launched in Standby mode |
|
||||
|
||||
## Readiness Probe
|
||||
## Readiness probe
|
||||
|
||||
The platform sends `GET /` requests with the header `x-apify-container-server-readiness-probe` to check server readiness. You MUST respond with HTTP 200. Keep the response lightweight.
|
||||
|
||||
@@ -51,11 +51,11 @@ Callers authenticate to Standby URLs via:
|
||||
- **Bearer token** (recommended): `Authorization: Bearer <APIFY_TOKEN>`
|
||||
- **Query parameter** (fallback): `?token=<APIFY_TOKEN>`
|
||||
|
||||
## Input Handling
|
||||
## Input handling
|
||||
|
||||
Standby Actors receive per-request input via HTTP query parameters or request body — NOT via `INPUT.json`. The traditional input schema (`input_schema.json`) is used for Actor initialization/configuration only, not per-request data.
|
||||
|
||||
## Complete Examples
|
||||
## Complete examples
|
||||
|
||||
### JavaScript / TypeScript (Express)
|
||||
|
||||
@@ -136,13 +136,13 @@ if __name__ == '__main__':
|
||||
2. Test the readiness probe: `curl -H "x-apify-container-server-readiness-probe: true" http://localhost:<port>/`
|
||||
3. Send requests with curl/httpie to verify endpoints
|
||||
|
||||
## Standby vs. Container Web Server
|
||||
## Standby vs. container web server
|
||||
|
||||
Do not confuse these:
|
||||
- **Container web server** (`ACTOR_WEB_SERVER_URL`): per-run unique URL, no load balancing, no auto-scaling. Useful for live view UIs during a run.
|
||||
- **Standby mode** (`ACTOR_STANDBY_URL`): stable hostname, load-balanced across runs, auto-scaled based on traffic. Use this for production APIs.
|
||||
|
||||
## Further Reading
|
||||
## Further reading
|
||||
|
||||
- [Developing Actors using Standby mode](https://docs.apify.com/platform/actors/development/programming-interface/standby#developing-actors-using-standby-mode)
|
||||
- [Running Actors in Standby mode](https://docs.apify.com/platform/actors/running/standby)
|
||||
|
||||
@@ -7,7 +7,7 @@ description: Convert existing projects into Apify Actors - serverless cloud prog
|
||||
|
||||
Actorization converts existing software into reusable serverless applications compatible with the Apify platform. Actors are programs packaged as Docker images that accept well-defined JSON input, perform an action, and optionally produce structured JSON output.
|
||||
|
||||
## Quick Start
|
||||
## Quick start
|
||||
|
||||
1. Run `apify init` in project root
|
||||
2. Wrap code with SDK lifecycle (see language-specific section below)
|
||||
@@ -15,9 +15,9 @@ Actorization converts existing software into reusable serverless applications co
|
||||
4. Test with `apify run --input '{"key": "value"}'`
|
||||
5. Deploy with `apify push`
|
||||
|
||||
## When to Use This Skill
|
||||
## When to use this skill
|
||||
|
||||
- Converting an existing project to run on Apify platform
|
||||
- Converting an existing project to run on the Apify platform
|
||||
- Adding Apify SDK integration to a project
|
||||
- Wrapping a CLI tool or script as an Actor
|
||||
- Migrating a Crawlee project to Apify
|
||||
@@ -62,7 +62,7 @@ If browser login isn't available (headless environment or CI), ensure the `APIFY
|
||||
> Never log, print, or embed `APIFY_TOKEN` in source code or configuration files.
|
||||
> Use a token with the minimum required permissions (scoped token) and rotate it periodically.
|
||||
|
||||
## Actorization Checklist
|
||||
## Actorization checklist
|
||||
|
||||
Copy this checklist to track progress:
|
||||
|
||||
@@ -72,11 +72,11 @@ Copy this checklist to track progress:
|
||||
- [ ] Step 4: Configure `.actor/input_schema.json`
|
||||
- [ ] Step 5: Configure `.actor/output_schema.json` (if applicable)
|
||||
- [ ] Step 6: Update `.actor/actor.json` metadata
|
||||
- [ ] Step 7: Write README.md for the Apify Store listing
|
||||
- [ ] Step 7: Write README.md for Apify Store listing
|
||||
- [ ] Step 8: Test locally with `apify run`
|
||||
- [ ] Step 9: Deploy with `apify push`
|
||||
|
||||
## Step 1: Analyze the Project
|
||||
## Step 1: Analyze the project
|
||||
|
||||
Before making changes, understand the project:
|
||||
|
||||
@@ -86,7 +86,7 @@ Before making changes, understand the project:
|
||||
4. **Identify outputs** - Files, console output, API responses
|
||||
5. **Check for state** - Does it need to persist data between runs?
|
||||
|
||||
## Step 2: Initialize Actor Structure
|
||||
## Step 2: Initialize Actor structure
|
||||
|
||||
Run in the project root:
|
||||
|
||||
@@ -96,10 +96,10 @@ apify init
|
||||
|
||||
This creates:
|
||||
- `.actor/actor.json` - Actor configuration and metadata
|
||||
- `.actor/input_schema.json` - Input definition for the Apify Console
|
||||
- `.actor/input_schema.json` - Input definition for Apify Console
|
||||
- `Dockerfile` (if not present) - Container image definition
|
||||
|
||||
## Step 3: Apply Language-Specific Changes
|
||||
## Step 3: Apply language-specific changes
|
||||
|
||||
Choose based on your project's language:
|
||||
|
||||
@@ -107,7 +107,7 @@ Choose based on your project's language:
|
||||
- **Python**: See [python-actorization.md](references/python-actorization.md)
|
||||
- **Other Languages (CLI-based)**: See [cli-actorization.md](references/cli-actorization.md)
|
||||
|
||||
### Quick Reference
|
||||
### Quick reference
|
||||
|
||||
| Language | Install | Wrap Code |
|
||||
|----------|---------|-----------|
|
||||
@@ -115,7 +115,7 @@ Choose based on your project's language:
|
||||
| Python | `pip install apify` | `async with Actor:` |
|
||||
| Other | Use CLI in wrapper script | `apify actor:get-input` / `apify actor:push-data` |
|
||||
|
||||
## Steps 4-6: Configure Schemas
|
||||
## Steps 4-6: Configure schemas
|
||||
|
||||
See [schemas-and-output.md](references/schemas-and-output.md) for detailed configuration of:
|
||||
- Input schema (`.actor/input_schema.json`)
|
||||
@@ -134,9 +134,9 @@ See the Actor README guidelines at `skills/apify-actor-development/references/ac
|
||||
- [Instagram Scraper](https://apify.com/apify/instagram-scraper)
|
||||
- [Google Maps Scraper](https://apify.com/compass/crawler-google-places)
|
||||
|
||||
## Step 8: Test Locally
|
||||
## Step 8: Test locally
|
||||
|
||||
Run the actor with inline input (for JS/TS and Python actors):
|
||||
Run the Actor with inline input (for JS/TS and Python Actors):
|
||||
|
||||
```bash
|
||||
apify run --input '{"startUrl": "https://example.com", "maxItems": 10}'
|
||||
@@ -156,17 +156,17 @@ apify run --input-file ./test-input.json
|
||||
apify push
|
||||
```
|
||||
|
||||
This uploads and builds your actor on the Apify platform.
|
||||
This uploads and builds your Actor on the Apify platform.
|
||||
|
||||
## Monetization (Optional)
|
||||
## Monetization (optional)
|
||||
|
||||
After deploying, you can monetize your actor in the Apify Store. The recommended model is **Pay Per Event (PPE)**:
|
||||
After deploying, you can monetize your Actor in Apify Store. The recommended model is **Pay Per Event (PPE)**:
|
||||
|
||||
- Per result/item scraped
|
||||
- Per page processed
|
||||
- Per API call made
|
||||
|
||||
Configure PPE in the Apify Console under Actor > Monetization. Charge for events in your code with `await Actor.charge('result')`.
|
||||
Configure PPE in Apify Console under Actor > Monetization. Charge for events in your code with `await Actor.charge('result')`.
|
||||
|
||||
Other options: **Rental** (monthly subscription) or **Free** (open source).
|
||||
|
||||
@@ -181,7 +181,7 @@ Other options: **Rental** (monthly subscription) or **Free** (open source).
|
||||
- **Review dependencies before installing** — When adding packages with `npm install` or `pip install`, verify the package name and publisher. Typosquatting is a common supply-chain attack vector. Prefer well-known, actively maintained packages.
|
||||
- **Pin versions and use lockfiles** — Always commit `package-lock.json` (Node.js) or pin exact versions in `requirements.txt` (Python). Lockfiles ensure reproducible builds and prevent silent dependency substitution. Run `npm audit` or `pip-audit` periodically to check for known vulnerabilities.
|
||||
|
||||
## Pre-Deployment Checklist
|
||||
## Pre-deployment checklist
|
||||
|
||||
- [ ] `.actor/actor.json` exists with correct name and description
|
||||
- [ ] `.actor/actor.json` validates against `@apify/json_schemas` (`actor.schema.json`)
|
||||
@@ -198,7 +198,7 @@ Other options: **Rental** (monthly subscription) or **Free** (open source).
|
||||
- [ ] `README.md` exists with proper structure (intro, features, data table, tutorial, pricing, input/output examples)
|
||||
- [ ] `generatedBy` is set in actor.json meta section
|
||||
|
||||
## Apify MCP Tools
|
||||
## Apify MCP tools
|
||||
|
||||
If MCP server is configured, use these tools for documentation:
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# CLI-Based Actorization
|
||||
# CLI-based Actorization
|
||||
|
||||
For languages without an SDK (Go, Rust, Java, etc.), create a wrapper script that uses the Apify CLI.
|
||||
|
||||
## Create Wrapper Script
|
||||
## Create wrapper script
|
||||
|
||||
Create `start.sh` in project root:
|
||||
|
||||
@@ -57,9 +57,9 @@ RUN chmod +x start.sh
|
||||
CMD ["./start.sh"]
|
||||
```
|
||||
|
||||
## Testing CLI-Based Actors
|
||||
## Testing CLI-based Actors
|
||||
|
||||
For CLI-based actors (shell wrapper scripts), you may need to test the underlying application directly with mock input, as `apify run` requires a Node.js or Python entry point.
|
||||
For CLI-based Actors (shell wrapper scripts), you may need to test the underlying application directly with mock input, as `apify run` requires a Node.js or Python entry point.
|
||||
|
||||
Test your wrapper script locally:
|
||||
|
||||
@@ -71,7 +71,7 @@ export INPUT='{"myParam": "test-value"}'
|
||||
./start.sh
|
||||
```
|
||||
|
||||
## CLI Commands Reference
|
||||
## CLI commands reference
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
npm install apify
|
||||
```
|
||||
|
||||
## Wrap Main Code with Actor Lifecycle
|
||||
## Wrap main code with Actor lifecycle
|
||||
|
||||
```javascript
|
||||
import { Actor } from 'apify';
|
||||
@@ -37,14 +37,14 @@ console.log('Input:', input);
|
||||
await Actor.exit();
|
||||
```
|
||||
|
||||
## Key Points
|
||||
## Key points
|
||||
|
||||
- `Actor.init()` configures storage to use Apify API when running on platform
|
||||
- `Actor.exit()` handles graceful shutdown and cleanup
|
||||
- Both calls must be awaited
|
||||
- Local execution remains unchanged - the SDK automatically detects the environment
|
||||
|
||||
## Crawlee Projects
|
||||
## Crawlee projects
|
||||
|
||||
Crawlee projects require minimal changes - just wrap with Actor lifecycle:
|
||||
|
||||
@@ -78,7 +78,7 @@ await crawler.run([startUrl]);
|
||||
await Actor.exit();
|
||||
```
|
||||
|
||||
## Express/HTTP Servers
|
||||
## Express/HTTP servers
|
||||
|
||||
For web servers, use standby mode in actor.json:
|
||||
|
||||
@@ -92,7 +92,7 @@ For web servers, use standby mode in actor.json:
|
||||
|
||||
Then implement readiness probe. See [standby-mode.md](../../apify-actor-development/references/standby-mode.md).
|
||||
|
||||
## Batch Processing Scripts
|
||||
## Batch processing scripts
|
||||
|
||||
```javascript
|
||||
import { Actor } from 'apify';
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
pip install apify
|
||||
```
|
||||
|
||||
## Wrap Main Function with Actor Context Manager
|
||||
## Wrap main function with Actor context manager
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
@@ -37,13 +37,13 @@ if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Key Points
|
||||
## Key points
|
||||
|
||||
- `async with Actor:` handles both initialization and cleanup
|
||||
- Automatically manages platform event listeners and graceful shutdown
|
||||
- Local execution remains unchanged - the SDK automatically detects the environment
|
||||
|
||||
## Crawlee Python Projects
|
||||
## Crawlee Python projects
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
@@ -75,7 +75,7 @@ if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Batch Processing Scripts
|
||||
## Batch processing scripts
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Schemas and Output Configuration
|
||||
# Schemas and output configuration
|
||||
|
||||
## Input Schema
|
||||
## Input schema
|
||||
|
||||
Map your application's inputs to `.actor/input_schema.json`. Validate against the JSON Schema from the `@apify/json_schemas` npm package (`input.schema.json`).
|
||||
|
||||
@@ -29,23 +29,23 @@ Map your application's inputs to `.actor/input_schema.json`. Validate against th
|
||||
}
|
||||
```
|
||||
|
||||
### Mapping Guidelines
|
||||
### Mapping guidelines
|
||||
|
||||
- Command-line arguments → input schema properties
|
||||
- Environment variables → input schema or Actor env vars in actor.json
|
||||
- Config files → input schema with object/array types
|
||||
- Flatten deeply nested structures for better UX
|
||||
|
||||
## Output Schema
|
||||
## Output schema
|
||||
|
||||
Define output structure in `.actor/output_schema.json`. Validate against the JSON Schema from the `@apify/json_schemas` npm package (`output.schema.json`).
|
||||
|
||||
### For Table-Like Data (Multiple Items)
|
||||
### For table-like data (multiple items)
|
||||
|
||||
- Use `Actor.pushData()` (JS) or `Actor.push_data()` (Python)
|
||||
- Each item becomes a row in the dataset
|
||||
|
||||
### For Single Files or Blobs
|
||||
### For single files or blobs
|
||||
|
||||
- Use key-value store: `Actor.setValue()` / `Actor.set_value()`
|
||||
- Get the public URL and include it in the dataset:
|
||||
@@ -62,7 +62,7 @@ const publicUrl = `https://api.apify.com/v2/key-value-stores/${storeInfo.id}/rec
|
||||
await Actor.pushData({ reportUrl: publicUrl });
|
||||
```
|
||||
|
||||
### For Multiple Files with a Common Prefix (Collections)
|
||||
### For multiple files with a common prefix (collections)
|
||||
|
||||
```javascript
|
||||
// Store multiple files with a prefix
|
||||
@@ -72,7 +72,7 @@ for (const [name, data] of files) {
|
||||
// Files are accessible at: .../records/screenshots%2F{name}
|
||||
```
|
||||
|
||||
## Actor Configuration (actor.json)
|
||||
## Actor configuration (actor.json)
|
||||
|
||||
Configure `.actor/actor.json`. Validate against the JSON Schema from the `@apify/json_schemas` npm package (`actor.schema.json`).
|
||||
|
||||
@@ -81,7 +81,7 @@ Configure `.actor/actor.json`. Validate against the JSON Schema from the `@apify
|
||||
"actorSpecification": 1,
|
||||
"name": "my-actor",
|
||||
"title": "My Actor",
|
||||
"description": "Brief description of what the actor does",
|
||||
"description": "Brief description of what the Actor does",
|
||||
"version": "1.0.0",
|
||||
"meta": {
|
||||
"templateId": "ts_empty",
|
||||
@@ -94,9 +94,9 @@ Configure `.actor/actor.json`. Validate against the JSON Schema from the `@apify
|
||||
|
||||
**Important:** Fill in the `generatedBy` property with the tool/model used.
|
||||
|
||||
## State Management
|
||||
## State management
|
||||
|
||||
### Request Queue - For Pausable Task Processing
|
||||
### Request queue - for pausable task processing
|
||||
|
||||
The request queue works for any task processing, not just web scraping. Use a dummy URL with custom `uniqueKey` and `userData` for non-URL tasks:
|
||||
|
||||
@@ -129,7 +129,7 @@ while ((request = await requestQueue.fetchNextRequest())) {
|
||||
}
|
||||
```
|
||||
|
||||
### Key-Value Store - For Checkpoint State
|
||||
### Key-value store - for checkpoint state
|
||||
|
||||
```javascript
|
||||
// Save state
|
||||
|
||||
@@ -3,11 +3,11 @@ name: apify-generate-output-schema
|
||||
description: Generate output schemas (dataset_schema.json, output_schema.json, key_value_store_schema.json) for an Apify Actor by analyzing its source code. Use when creating or updating Actor output schemas.
|
||||
---
|
||||
|
||||
# Generate Actor Output Schema
|
||||
# Generate Actor output schema
|
||||
|
||||
You are generating output schema files for an Apify Actor. The output schema tells Apify Console how to display run results. You will analyze the Actor's source code, create `dataset_schema.json`, `output_schema.json`, and `key_value_store_schema.json` (if the Actor uses key-value store), and update `actor.json`.
|
||||
|
||||
## Core Principles
|
||||
## Core principles
|
||||
|
||||
- **Analyze code first**: Read the Actor's source to understand what data it actually pushes to the dataset — never guess
|
||||
- **Every field is nullable**: APIs and websites are unpredictable — always set `"nullable": true`
|
||||
@@ -18,7 +18,7 @@ You are generating output schema files for an Apify Actor. The output schema tel
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Discover Actor Structure
|
||||
## Phase 1: Discover Actor structure
|
||||
|
||||
**Goal**: Locate the Actor and understand its output
|
||||
|
||||
@@ -377,7 +377,7 @@ If `key_value_store_schema.json` was generated in Phase 3, add a second property
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Review and Validate
|
||||
## Phase 6: Review and validate
|
||||
|
||||
**Goal**: Ensure correctness and completeness
|
||||
|
||||
|
||||
Reference in New Issue
Block a user