feat(redis-query-engine): add spec-compliant skill for RQE indexing and search

Introduces skills/redis-query-engine/ covering the six rqe-* rules from skills/redis-development/rules/ in agentskills.io spec layout: rqe-dialect → references/dialect.md rqe-field-types → references/field-types.md rqe-index-creation → references/index-creation.md rqe-index-management → references/index-management.md rqe-query-optimization → references/query-optimization.md rqe-skip-initial-scan → references/skip-initial-scan.md SKILL.md carries decision-oriented summaries (field-type table, DIALECT 2 default, schema/prefix rule, alias-based zero-downtime updates, SKIPINITIALSCAN guidance, query-optimization levers) with pointers into references/ for full code samples. Additive only: the source rqe-*.md rules under skills/redis-development/ remain in place so the legacy compiled AGENTS.md continues to serve existing plugin consumers unchanged. They are removed in the final cleanup PR alongside the rest of the rules/ tree. Validation: - skill-validator check skills/redis-query-engine → passed (0 warnings) - npm run validate → rules + plugin validators green Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-09-19 01:25:14 +08:00 · 2026-05-22 15:35:19 +03:00
parent 9032f7e8df
commit 77369035cb
7 changed files with 455 additions and 0 deletions
@@ -0,0 +1,132 @@
+---
+name: redis-query-engine
+description: Redis Query Engine (RQE) guidance covering FT.CREATE schema design, field type selection (TEXT, TAG, NUMERIC, GEO, GEOSHAPE, VECTOR), DIALECT 2 query syntax, efficient FT.SEARCH and FT.AGGREGATE queries, zero-downtime index updates via aliases, and the SKIPINITIALSCAN option. Use when defining a search index on Hash or JSON documents, picking between TEXT and TAG for filtering, writing FT.SEARCH queries with filters and SORTBY, managing or swapping indexes in production, or troubleshooting slow searches with FT.PROFILE.
+license: MIT
+metadata:
+  author: Redis, Inc.
+  version: "0.1.0"
+---
+
+# Redis Query Engine
+
+Guidance for using the Redis Query Engine (RQE) to index and search Hash or JSON documents. Covers schema design with `FT.CREATE`, field-type choices, query syntax, index lifecycle management, and the most common performance pitfalls.
+
+## When to apply
+
+- Creating, modifying, or reviewing an RQE index (`FT.CREATE`, `FT.ALTER`).
+- Writing or optimizing `FT.SEARCH` / `FT.AGGREGATE` queries.
+- Deciding between `TEXT`, `TAG`, `NUMERIC`, `GEO`, `GEOSHAPE`, or `VECTOR` for a field.
+- Rolling out a new index schema without downtime.
+- Spinning up an index that should only cover newly written keys.
+
+## 1. Use DIALECT 2 (the modern default)
+
+`DIALECT 2` is the baseline. Other dialects (1, 3, 4) are deprecated as of Redis 8. Most modern client libraries already default to it — but specify it explicitly in raw commands for portability.
+
+```
+FT.SEARCH idx:products "@name:laptop" DIALECT 2
+```
+
+`DIALECT 2` is **required** for vector search queries. It also handles special characters and NULLs predictably.
+
+See [references/dialect.md](references/dialect.md).
+
+## 2. Pick the right field type
+
+The field type decides both what you can query and how fast that query is. Use the narrowest type that supports your access pattern.
+
+| Field type | Use when | Notes |
+|---|---|---|
+| `TEXT` | Full-text search needed | Tokenized + stemmed; **not** for exact match |
+| `TAG` | Exact match / filtering | Add `SORTABLE UNF` for fastest tag queries |
+| `NUMERIC` | Range queries, sorting | Prices, counts, timestamps |
+| `GEO` | Lat/long point queries | Single points (stores, users) |
+| `GEOSHAPE` | Polygon / area queries | Delivery zones, regions |
+| `VECTOR` | Similarity search | HNSW or FLAT; see redis-vector-search |
+
+The classic mistake is using `TEXT` for a category or status field because "it's a string." `TAG` is 10× faster for those.
+
+See [references/field-types.md](references/field-types.md).
+
+## 3. Index only what you query — and always set a prefix
+
+`FT.CREATE` without a `PREFIX` indexes **every** matching key in the database; with a wide schema it can blow up index size and write latency.
+
+```
+FT.CREATE idx:products ON HASH PREFIX 1 product:
+    SCHEMA
+        name TEXT WEIGHT 2.0
+        category TAG SORTABLE
+        price NUMERIC SORTABLE
+        location GEO
+```
+
+Rules of thumb:
+
+- Start with the minimum schema. Add fields as new query patterns emerge.
+- Always set `PREFIX` (or filter via `FILTER` expression).
+- Use `FT.INFO idx:<name>` to monitor index size after adding fields.
+- Use `SORTABLE` only on fields you actually sort by; it has a memory cost.
+
+See [references/index-creation.md](references/index-creation.md).
+
+## 4. Zero-downtime index updates — use aliases
+
+For schema changes in production, keep application queries pointed at an alias and swap the underlying index.
+
+```
+FT.CREATE idx:products_v2 ON HASH PREFIX 1 product: SCHEMA ...
+FT.ALIASUPDATE products idx:products_v2
+
+# App queries are stable:
+FT.SEARCH products "@category:{electronics}"
+```
+
+Useful management commands: `FT.INFO`, `FT.DROPINDEX`, `FT._LIST`, `FT.ALIASADD/UPDATE/DEL`.
+
+See [references/index-management.md](references/index-management.md).
+
+## 5. SKIPINITIALSCAN — only when historical data is irrelevant
+
+By default `FT.CREATE` walks all existing keys that match the prefix and indexes them. Use `SKIPINITIALSCAN` only when:
+
+- You're standing up the index for a *new* feature and existing data shouldn't be queryable.
+- Existing data is too large to scan synchronously.
+- You're indexing event streams where only future events matter.
+
+For most schema migrations, the default (scan everything) is what you want.
+
+See [references/skip-initial-scan.md](references/skip-initial-scan.md).
+
+## 6. Write specific queries, not `*`
+
+Narrow the result set with filters before paging or aggregating.
+
+```
+# Good — specific filter, limited fields returned
+FT.SEARCH idx:products "@category:{electronics} @price:[100 500]"
+    LIMIT 0 20
+    RETURN 3 name price category
+```
+
+```
+# Bad — full scan plus unbounded LIMIT
+FT.SEARCH idx:products "*" LIMIT 0 10000
+```
+
+Other levers:
+
+- `SORTBY` requires `SORTABLE` on the sort field. Without it, sort is slow.
+- `LIMIT` early; the engine still processes everything above the limit if you don't.
+- `RETURN` specific fields — don't fetch the whole document if you only need a few.
+- Profile with `FT.PROFILE idx:<name> SEARCH QUERY "<query>"` when a query is slow.
+
+See [references/query-optimization.md](references/query-optimization.md).
+
+## References
+
+- [Redis: Query Engine — Indexing](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/)
+- [Redis: Query syntax](https://redis.io/docs/latest/develop/interact/search-and-query/query/)
+- [Redis: Query dialects](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/dialects/)
+- [Redis: Administration (aliases, dropindex)](https://redis.io/docs/latest/develop/interact/search-and-query/administration/)
+- [FT.CREATE](https://redis.io/docs/latest/commands/ft.create/)
@@ -0,0 +1,38 @@
+# Use DIALECT 2 for Query Syntax
+
+Use DIALECT 2 for consistent query behavior. Many Redis client libraries now default to DIALECT 2, and other dialects (1, 3, 4) are deprecated as of Redis 8.
+
+**Correct:** Use DIALECT 2 explicitly or rely on modern client defaults.
+
+```python
+from redis import Redis
+
+r = Redis()
+
+# Modern redis-py (6.0+) defaults to DIALECT 2
+# You can also set it explicitly
+results = r.ft("idx:products").search(
+    "@name:laptop",
+    dialect=2
+)
+```
+
+```
+# In raw commands, specify DIALECT 2
+FT.SEARCH idx:products "@name:laptop" DIALECT 2
+
+FT.AGGREGATE idx:products "@category:{electronics}"
+    GROUPBY 1 @category
+    REDUCE COUNT 0 AS count
+    DIALECT 2
+```
+
+**Note:** DIALECT 2 is required for vector search queries. Most modern client libraries (redis-py 6.0+, go-redis, Lettuce) now use DIALECT 2 by default.
+
+**Why DIALECT 2:**
+- Consistent handling of special characters
+- Better NULL value handling
+- More predictable query parsing
+- Required for vector search
+
+Reference: [Query Dialects](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/dialects/)
@@ -0,0 +1,72 @@
+# Choose the Correct Field Type
+
+Each field type has different capabilities and performance characteristics.
+
+| Field Type | Use When | Notes |
+|------------|----------|-------|
+| TEXT | Full-text search needed | Tokenized, stemmed |
+| TAG | Exact match, filtering | Faster than TEXT for filtering |
+| NUMERIC | Range queries, sorting | Use for prices, counts, timestamps |
+| GEO | Point location queries | Lat/long coordinates (single points) |
+| GEOSHAPE | Area/region queries | Polygons, circles, rectangles |
+| VECTOR | Similarity search | HNSW or FLAT algorithm |
+
+**Correct:** Use TAG for exact matching.
+
+```
+# Good: TAG for exact category matching
+FT.CREATE idx:products ON HASH PREFIX 1 product:
+    SCHEMA
+        category TAG SORTABLE
+        status TAG
+```
+
+**Java** (Jedis):
+```java
+import redis.clients.jedis.search.*;
+
+Schema schema = new Schema()
+    .addTextField("name", 1)
+    .addTagField("categories");  // TAG for exact matching
+
+IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH);
+
+jedis.ftCreate("idx", IndexOptions.defaultOptions().setDefinition(def), schema);
+
+// Query with TAG syntax
+SearchResult result = jedis.ftSearch("idx", "@categories:{chef|runner}");
+```
+
+**Incorrect:** Using TEXT when you don't need full-text features.
+
+```
+# Overkill: TEXT for category adds unnecessary tokenization
+FT.CREATE idx:products ON HASH PREFIX 1 product:
+    SCHEMA
+        category TEXT
+        status TEXT
+```
+
+**Java** (Jedis):
+```java
+// Bad: TEXT for categories adds unnecessary overhead
+Schema schema = new Schema()
+    .addTextField("name", 1)
+    .addTextField("categories", 1);  // Overkill for exact matching
+```
+
+**Correct:** Use GEO for points, GEOSHAPE for areas.
+
+```
+# GEO for point locations (stores, users)
+FT.CREATE idx:stores ON HASH PREFIX 1 store:
+    SCHEMA
+        location GEO
+
+# GEOSHAPE for areas (delivery zones, boundaries)
+FT.CREATE idx:zones ON JSON PREFIX 1 zone:
+    SCHEMA
+        $.boundary AS boundary GEOSHAPE
+```
+
+Reference: [Redis Search Field Types](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/geoindex/)
@@ -0,0 +1,63 @@
+# Index Only Fields You Query
+
+Create indexes with only the fields you need to search, filter, or sort on.
+
+**Correct:** Index specific fields and use prefixes.
+
+```
+FT.CREATE idx:products ON HASH PREFIX 1 product:
+    SCHEMA
+        name TEXT WEIGHT 2.0
+        description TEXT
+        category TAG SORTABLE
+        price NUMERIC SORTABLE
+        location GEO
+```
+
+**Java** (Jedis):
+```java
+import redis.clients.jedis.search.*;
+
+Schema schema = new Schema()
+    .addTextField("name", 1)
+    .addTagField("categories");
+
+// Good: Specify prefix to index only matching keys
+IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH)
+    .setPrefixes("person:");
+
+jedis.ftCreate("idx", IndexOptions.defaultOptions().setDefinition(def), schema);
+```
+
+**Incorrect:** Over-indexing or indexing unused fields.
+
+```
+# Bad: Indexing every field "just in case"
+FT.CREATE idx:products ON HASH PREFIX 1 product:
+    SCHEMA
+        name TEXT
+        description TEXT
+        category TEXT
+        subcategory TEXT
+        brand TEXT
+        sku TEXT
+        price NUMERIC
+        cost NUMERIC
+        margin NUMERIC
+        ...
+```
+
+**Java** (Jedis):
+```java
+// Bad: No prefix means all hashes get indexed
+IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH);
+// This will index every hash in the database!
+```
+
+**Tips:**
+- Start with the minimum required fields
+- Add fields as query patterns emerge
+- Use `FT.INFO` to monitor index size
+- Always specify a prefix to avoid indexing unrelated keys
+
+Reference: [Redis Search Indexing](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/)
@@ -0,0 +1,39 @@
+# Manage Indexes for Zero-Downtime Updates
+
+Use aliases to swap indexes without application changes.
+
+**Correct:** Use aliases for production indexes.
+
+```
+# Create versioned index
+FT.CREATE idx:products_v2 ON HASH PREFIX 1 product:
+    SCHEMA
+        name TEXT
+        category TAG SORTABLE
+        price NUMERIC SORTABLE
+
+# Point alias to new index
+FT.ALIASADD products idx:products_v2
+
+# Application queries use alias
+FT.SEARCH products "@category:{electronics}"
+
+# Later, swap to new version
+FT.ALIASUPDATE products idx:products_v3
+```
+
+**Useful management commands:**
+
+```
+# Check index info
+FT.INFO idx:products
+
+# Drop and recreate (non-blocking)
+FT.DROPINDEX idx:products
+FT.CREATE idx:products ...
+
+# List all indexes
+FT._LIST
+```
+
+Reference: [Redis Search Index Management](https://redis.io/docs/latest/develop/interact/search-and-query/administration/)
@@ -0,0 +1,39 @@
+# Write Efficient Queries
+
+Be specific and use filters to reduce the result set early.
+
+**Correct:** Use specific filters and limit results.
+
+```
+# Good: Specific query with filters
+FT.SEARCH idx:products "@category:{electronics} @price:[100 500]"
+    LIMIT 0 20
+    RETURN 3 name price category
+
+# Good: Use SORTBY and LIMIT
+FT.SEARCH idx:products "@name:laptop"
+    SORTBY price ASC
+    LIMIT 0 10
+```
+
+**Incorrect:** Broad queries returning large result sets.
+
+```
+# Bad: Wildcard prefix scans entire index
+FT.SEARCH idx:products "*" LIMIT 0 10000
+
+# Bad: Loading all fields from source document
+FT.AGGREGATE idx:products "*" LOAD *
+```
+
+**Performance tips:**
+- Add `SORTABLE` to fields used in `SORTBY`
+- Use `TAG SORTABLE UNF` for best performance on tag fields
+- Use `NOSTEM` if you don't need stemming
+- Profile queries with `FT.PROFILE`
+
+```
+FT.PROFILE idx:products SEARCH QUERY "@category:{electronics}"
+```
+
+Reference: [Redis Search Query Syntax](https://redis.io/docs/latest/develop/interact/search-and-query/query/)
@@ -0,0 +1,72 @@
+# Use SKIPINITIALSCAN for New Data Only Indexes
+
+Enable the `SKIPINITIALSCAN` option when creating an index if you only want to include items that are added after the index is created. This makes index creation faster and avoids indexing existing data that you don't need to search.
+
+**Correct:** Use SKIPINITIALSCAN when you only need to index new data.
+
+**Python** (redis-py):
+```python
+import redis
+from redis.commands.search.field import TextField, TagField
+from redis.commands.search.indexDefinition import IndexDefinition, IndexType
+
+client = redis.Redis(host='localhost', port=6379)
+
+# Create index that only indexes new documents
+schema = (
+    TextField("name"),
+    TagField("categories")
+)
+
+definition = IndexDefinition(
+    prefix=["person:"],
+    index_type=IndexType.HASH
+)
+
+# SKIPINITIALSCAN - only index documents added after creation
+client.ft("idx").create_index(
+    schema,
+    definition=definition,
+    skip_initial_scan=True
+)
+```
+
+**Java** (Jedis):
+```java
+import redis.clients.jedis.UnifiedJedis;
+import redis.clients.jedis.search.FTCreateParams;
+import redis.clients.jedis.search.IndexDataType;
+import redis.clients.jedis.search.schemafields.SchemaField;
+import redis.clients.jedis.search.schemafields.TagField;
+import redis.clients.jedis.search.schemafields.TextField;
+
+try (UnifiedJedis jedis = new UnifiedJedis("redis://localhost:6379")) {
+    FTCreateParams params = new FTCreateParams()
+        .on(IndexDataType.HASH)
+        .skipInitialScan();  // Only index new documents
+
+    jedis.ftCreate(
+        "idx",
+        params,
+        new SchemaField[]{
+            new TextField("name"),
+            new TagField("categories")
+        }
+    );
+}
+```
+
+**When to use SKIPINITIALSCAN:**
+- Creating an index for a new feature where existing data is irrelevant
+- Setting up indexes in advance before data arrives
+- When existing data would be too large to scan during index creation
+- Event-driven architectures where you only care about new events
+
+**When NOT to use (default behavior is correct):**
+- You need to search existing data immediately after index creation
+- Migrating to a new index schema and need all data indexed
+- Most typical use cases where historical data matters
+
+**Note:** The default behavior (without SKIPINITIALSCAN) indexes all existing matching keys, which is usually what you want.
+
+Reference: [FT.CREATE SKIPINITIALSCAN](https://redis.io/docs/latest/commands/ft.create/)