align aiconfigs with docs (#49)

* align aiconfigs with docs

skills.json

@@ -26,7 +26,7 @@
     },
     {
       "name": "aiconfig-migrate",
-      "description": "Migrate an application with hardcoded LLM prompts to a full LaunchDarkly AI Configs implementation in five stages: extract prompts, wrap in the AI SDK, add tools, add tracking, add evals/judges. Use when the user wants to externalize model/prompt configuration, move from direct provider calls (OpenAI, Anthropic, Bedrock, Gemini) to a managed AI Config, or stage a full hardcoded-to-LaunchDarkly migration.",
+      "description": "Migrate an application with hardcoded LLM prompts to a full LaunchDarkly AI Configs implementation in five stages: extract prompts, wrap in the AI SDK, add tools, add tracking, add evals/judges. Use when the user wants to externalize model/prompt configuration, move from direct provider calls (OpenAI, Anthropic, Bedrock, Gemini, Strands) to a managed AI Config, or stage a full hardcoded-to-LaunchDarkly migration.",
       "path": "skills/ai-configs/aiconfig-migrate",
       "version": "0.1.0",
       "license": "Apache-2.0",

skills/ai-configs/aiconfig-ai-metrics/SKILL.md

@@ -34,7 +34,7 @@ A call to `track_openai_metrics` / `trackOpenAIMetrics` / `track_bedrock_convers
 Before picking a tier, find the provider call and answer these questions:
 
 - [ ] **Shape?** Is it a chat loop (history + turn-based), a one-shot completion, an agent step, or something else? → drives Tier 1 vs 2.
-- [ ] **Framework?** Raw provider SDK? LangChain / LangGraph? Vercel AI SDK? CrewAI? → drives which Tier-2 provider package (if any) applies.
+- [ ] **Framework?** Raw provider SDK? LangChain / LangGraph? Vercel AI SDK? CrewAI? Strands? → drives which Tier-2 provider package (if any) applies.
 - [ ] **Provider?** OpenAI, Anthropic, Bedrock, Gemini, Azure, custom HTTP? → cross-reference with the package availability matrix below.
 - [ ] **Streaming?** If yes, you'll need TTFT tracking, which means Tier 4 for the TTFT part even if the rest is Tier 2.
 - [ ] **Language?** Python or Node? Provider-package coverage differs between them.
@@ -47,11 +47,12 @@ Use this matrix to decide whether Tier 2 (provider package) is available for you
 | Framework / provider | Python provider package | Node provider package | Reference |
 |---|---|---|---|
 | OpenAI (direct SDK) | `launchdarkly-server-sdk-ai-openai` | `@launchdarkly/server-sdk-ai-openai` | [openai-tracking.md](references/openai-tracking.md) |
-| LangChain / LangGraph | `launchdarkly-server-sdk-ai-langchain` | `@launchdarkly/server-sdk-ai-langchain` | (use the LangChain provider docs) |
+| LangChain / LangGraph | `launchdarkly-server-sdk-ai-langchain` | `@launchdarkly/server-sdk-ai-langchain` | [langchain-tracking.md](references/langchain-tracking.md) |
 | Vercel AI SDK | — | `@launchdarkly/server-sdk-ai-vercel` | (use the Vercel provider docs) |
 | AWS Bedrock (Converse or InvokeModel) | — (use LangChain-aws or custom extractor) | — (use LangChain-aws or custom extractor) | [bedrock-tracking.md](references/bedrock-tracking.md) |
 | Anthropic direct SDK | — | — | [anthropic-tracking.md](references/anthropic-tracking.md) |
-| Gemini / Google GenAI | — | — | Tier 3 custom extractor |
+| Gemini / Google GenAI | — | — | [gemini-tracking.md](references/gemini-tracking.md) |
+| Strands Agents | — (Tier 3 custom extractor) | — (Tier 3 custom extractor) | [strands-tracking.md](references/strands-tracking.md) |
 | Cohere, Mistral, custom HTTP | — | — | Tier 3 custom extractor |
 | **Any provider, streaming + TTFT** | — (Tier 4 only) | `trackStreamMetricsOf` (no TTFT) + manual TTFT | [streaming-tracking.md](references/streaming-tracking.md) |
 
@@ -64,7 +65,7 @@ Guardrails that apply to every tier:
 1. **Always check `config.enabled`** before making the tracked call. A disabled config means the user has flagged the feature off — you should short-circuit to whatever fallback the app uses (cached response, error, degraded path) rather than making the provider call at all.
 2. **Wrap the existing call, don't rewrite it.** Tier 2 and Tier 3 are designed to slot around an unmodified provider call. If you find yourself rewriting the call to fit the tracker, you're at the wrong tier — drop down one.
 3. **Errors go through the tracker too.** `trackMetricsOf` handles the success path; errors still need an explicit `tracker.trackError()` in the catch block (or a try/except around the whole thing). Tier 1 handles both paths automatically.
-4. **Flush in short-lived processes.** In serverless, cron jobs, CLI scripts — anything that exits quickly — call `ldClient.flush()` (sync or await) before the process terminates, or the tracker events never leave the machine.
+4. **Always flush before close.** Call `ldClient.flush()` (Python: `ldclient.get().flush()`; Node: `await ldClient.flush()`) before closing the client. Trailing events are at risk of being lost otherwise — in short-lived scripts and long-running services alike. In Node, `ldClient.close()` returns a Promise; await it.
 
 ### 4. Verify
 

skills/ai-configs/aiconfig-ai-metrics/references/gemini-tracking.md

@@ -0,0 +1,212 @@
+# Gemini Metrics Tracking
+
+**There is no LaunchDarkly provider package for Gemini today** (neither Python nor Node). The canonical path is Tier 3: a small custom extractor composed with `trackMetricsOf`. The Gemini response shape is stable — `response.usage_metadata` / `response.usageMetadata` carries `prompt_token_count` / `promptTokenCount`, `candidates_token_count` / `candidatesTokenCount`, and `total_token_count` / `totalTokenCount` — so the extractor is three lines.
+
+## Tier 1 is not available
+
+`ManagedModel` / `TrackedChat` do not currently ship a Gemini provider. If you need Tier 1 for a chat app, route via the LangChain provider package (`ChatGoogleGenerativeAI` under the hood), which restores the zero-tracker-call experience. See [langchain-tracking.md](langchain-tracking.md).
+
+## Tier 3 — Custom extractor + `trackMetricsOf` (primary)
+
+Gemini's API diverges from OpenAI's in three places that matter for a wrapper:
+
+1. **System messages are a top-level field.** `GenerateContentConfig.system_instruction` / `systemInstruction` carries the system prompt; the `contents` array only holds `user` and `model` turns. You cannot put a `role: "system"` item in `contents`.
+2. **Assistant messages use role `model`.** Convert `role: "assistant"` → `role: "model"` when mapping LD messages into Gemini's `contents`.
+3. **Parameter names differ.** `max_tokens` on a LaunchDarkly variation (the snake_case key shown in the LD UI) becomes `max_output_tokens` on Python's `GenerateContentConfig`, or `maxOutputTokens` in Node. Other LD parameter names (`temperature`, `top_p`, `top_k`) either pass through or map with the same helper.
+
+Two helpers absorb the divergence — a message splitter and a parameter remapper — and the metrics extractor sits on top.
+
+**Python** — `google-genai`:
+
+```python
+from google import genai
+from google.genai.types import Content, Part, GenerateContentConfig
+from ldai.providers.types import LDAIMetrics, TokenUsage
+
+gemini_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
+
+def gemini_metrics(response) -> LDAIMetrics:
+    usage = response.usage_metadata
+    return LDAIMetrics(
+        success=True,
+        usage=TokenUsage(
+            total=usage.total_token_count or 0,
+            input=usage.prompt_token_count or 0,
+            output=usage.candidates_token_count or 0,
+        ) if usage else None,
+    )
+
+def map_to_gemini_messages(ld_messages):
+    """Split LD messages into (system_instruction, contents) for google-genai.
+    System messages concatenate into the top-level system_instruction; user and
+    assistant messages become Content items with role 'user' or 'model'."""
+    system_parts: list[str] = []
+    contents: list[Content] = []
+    for m in ld_messages or []:
+        if m.role == "system":
+            system_parts.append(m.content)
+        elif m.role == "user":
+            contents.append(Content(role="user", parts=[Part(text=m.content)]))
+        elif m.role == "assistant":
+            contents.append(Content(role="model", parts=[Part(text=m.content)]))
+    return (" ".join(system_parts) or None), contents
+
+def gemini_config_kwargs(params):
+    """Map AI Config parameter names to google-genai's GenerateContentConfig.
+    LaunchDarkly stores max_tokens (snake_case, matching the LD UI); Gemini's
+    Python SDK expects max_output_tokens. Drop `tools` — they go on
+    GenerateContentConfig.tools directly; leaving them here would double-pass."""
+    mapping = {"max_tokens": "max_output_tokens"}
+    return {mapping.get(k, k): v for k, v in (params or {}).items() if k != "tools"}
+
+def call_with_tracking(ai_config, user_prompt: str) -> str | None:
+    if not ai_config.enabled:
+        return None
+
+    system_instruction, contents = map_to_gemini_messages(ai_config.messages or [])
+    contents.append(Content(role="user", parts=[Part(text=user_prompt)]))
+
+    params = (ai_config.model.to_dict().get("parameters") if ai_config.model else None) or {}
+
+    def call_gemini():
+        return gemini_client.models.generate_content(
+            model=ai_config.model.name,
+            contents=contents,
+            config=GenerateContentConfig(
+                system_instruction=system_instruction,
+                **gemini_config_kwargs(params),
+            ),
+        )
+
+    try:
+        response = ai_config.tracker.track_metrics_of(call_gemini, gemini_metrics)
+        return response.text
+    except Exception:
+        ai_config.tracker.track_error()
+        raise
+```
+
+**Node** — `@google/genai`:
+
+```typescript
+import { GoogleGenAI, type Content } from '@google/genai';
+import type { LDAIMetrics } from '@launchdarkly/server-sdk-ai';
+
+const genAI = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY! });
+
+const geminiMetrics = (response: any): LDAIMetrics => {
+  const usage = response.usageMetadata;
+  return {
+    success: true,
+    usage: usage
+      ? {
+          total: usage.totalTokenCount ?? 0,
+          input: usage.promptTokenCount ?? 0,
+          output: usage.candidatesTokenCount ?? 0,
+        }
+      : undefined,
+  };
+};
+
+function mapToGeminiMessages(
+  ldMessages?: Array<{ role: string; content: string }>,
+): { systemInstruction: string | undefined; contents: Content[] } {
+  const contents: Content[] = [];
+  const systemParts: string[] = [];
+  for (const m of ldMessages ?? []) {
+    if (m.role === 'system') systemParts.push(m.content);
+    else if (m.role === 'user') contents.push({ role: 'user', parts: [{ text: m.content }] });
+    else if (m.role === 'assistant') contents.push({ role: 'model', parts: [{ text: m.content }] });
+  }
+  return {
+    systemInstruction: systemParts.length ? systemParts.join(' ') : undefined,
+    contents,
+  };
+}
+
+// Map AI Config parameter names to @google/genai's GenerateContentConfig keys.
+// LaunchDarkly stores max_tokens (snake_case, matching the LD UI); @google/genai
+// expects maxOutputTokens. Drop `tools` — they go on GenerateContentConfig.tools
+// directly; leaving them here would double-pass.
+function geminiConfigFields(params: Record<string, unknown>): Record<string, unknown> {
+  const mapping: Record<string, string> = { max_tokens: 'maxOutputTokens' };
+  return Object.fromEntries(
+    Object.entries(params ?? {})
+      .filter(([k]) => k !== 'tools')
+      .map(([k, v]) => [mapping[k] ?? k, v]),
+  );
+}
+
+async function callWithTracking(
+  aiConfig: LDAICompletionConfig,
+  userPrompt: string,
+): Promise<string | null> {
+  if (!aiConfig.enabled) return null;
+
+  const { systemInstruction, contents } = mapToGeminiMessages(aiConfig.messages);
+  contents.push({ role: 'user', parts: [{ text: userPrompt }] });
+
+  const params = (aiConfig.model?.parameters ?? {}) as Record<string, unknown>;
+
+  try {
+    const response = await aiConfig.tracker.trackMetricsOf(
+      geminiMetrics,
+      () => genAI.models.generateContent({
+        model: aiConfig.model!.name,
+        contents,
+        config: {
+          systemInstruction,
+          ...geminiConfigFields(params),
+        },
+      }),
+    );
+    return response.text ?? null;
+  } catch (err) {
+    aiConfig.tracker.trackError();
+    throw err;
+  }
+}
+```
+
+Notes on the extractor shape:
+
+- Gemini uses `snake_case` in Python (`prompt_token_count`) and `camelCase` in Node (`promptTokenCount`). The LD `TokenUsage` / `LDAIMetrics` type is the same in both.
+- `total_token_count` already includes input + output from Google; do not recompute it.
+- `success: true` in the extractor is not a lie — `trackMetricsOf` only calls the extractor on the success path. Errors go through the catch block and `trackError()`.
+
+## Tools
+
+LaunchDarkly stores attached tools on `ai_config.model.parameters.tools` in the flat `{type, name, description, parameters}` shape. Gemini's `GenerateContentConfig.tools` expects a list of `{function_declarations: [{name, description, parameters}]}` blocks (Python) or `{functionDeclarations: [...]}` (Node), so convert at runtime:
+
+```python
+ld_tools = (params.get("tools") or [])
+gemini_tools = [
+    {
+        "function_declarations": [
+            {
+                "name": t["name"],
+                "description": t.get("description", ""),
+                "parameters": t.get("parameters", {"type": "object", "properties": {}}),
+            }
+            for t in ld_tools
+        ],
+    }
+] if ld_tools else []
+```
+
+Tool handlers stay in your application code — LaunchDarkly stores the schema, your application owns the behavior. For the full agent loop pattern (`MAX_STEPS`, `functionCalls` handling, `tracker.track_tool_call`), see the agent-mode section of `aiconfig-tools`.
+
+## Tier 2 option — route via LangChain
+
+If the app can adopt LangChain, the LangChain provider package handles Gemini (via `@langchain/google-genai` / `langchain-google-genai`) through the standard `trackMetricsOf(LangChainProvider.getAIMetricsFromResponse, ...)` pattern. The provider package handles LaunchDarkly→LangChain provider-name mapping (for example, `"gemini"` → `"google_genai"`) and forwards all variation parameters automatically, so you do not need your own mapping helper. See [langchain-tracking.md](langchain-tracking.md).
+
+## Tier 4 — Manual (streaming only)
+
+Streaming Gemini needs manual TTFT tracking; the pattern is identical to OpenAI streaming. See [streaming-tracking.md](streaming-tracking.md).
+
+## What NOT to do
+
+- **Do not look for a `track_gemini_metrics` helper** — it does not exist. Gemini support lives in the extractor above.
+- **Do not invent a provider package** like `@launchdarkly/server-sdk-ai-gemini` or `launchdarkly-server-sdk-ai-gemini`. Neither exists on npm or PyPI. Check [ai-providers in js-core](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers) and [python-server-sdk-ai/packages/ai-providers](https://github.com/launchdarkly/python-server-sdk-ai/tree/main/packages/ai-providers) before recommending one.
+- **Do not put `role: "system"` items inside `contents`.** Gemini will either ignore them or error. The system prompt goes on `system_instruction` / `systemInstruction`.
+- **Do not assume LaunchDarkly stores `maxTokens` (camelCase) as the parameter key.** The UI and the stored variation use `max_tokens`. The mapping helper renames it to `max_output_tokens` / `maxOutputTokens` for Gemini's SDK.

skills/ai-configs/aiconfig-ai-metrics/references/langchain-tracking.md

@@ -0,0 +1,201 @@
+# LangChain & LangGraph Metrics Tracking
+
+LangChain is covered by a first-class LaunchDarkly provider package in both Python and Node. The same package is what LangGraph rides on — there is no separate LangGraph helper.
+
+- Python: `launchdarkly-server-sdk-ai-langchain` (imported as `ldai_langchain`)
+- Node: `@launchdarkly/server-sdk-ai-langchain` (exports `LangChainProvider`)
+
+Two helpers do the heavy lifting. Use both — skipping either silently drops value that the provider package would otherwise give you.
+
+| Helper | Purpose |
+|---|---|
+| `create_langchain_model(config)` (Python) / `LangChainProvider.createLangChainModel(config)` (Node) | Build a LangChain chat model from the AI Config. Forwards **all** variation parameters (temperature, max_tokens, top_p, and so on), picks the correct LangChain chat class based on `config.provider.name`, and handles provider-name mapping internally (for example, LaunchDarkly's `"gemini"` → LangChain's `"google_genai"`). |
+| `get_ai_metrics_from_response` (top-level import) / `LangChainProvider.getAIMetricsFromResponse` (Node class method) | Extract token usage from a LangChain response. Pass as the extractor argument to `track_metrics_of` / `trackMetricsOf`. Both import forms are supported in Node; the top-level import is how Python exposes it. |
+
+## Tier 2 — LangChain (single model, not a graph)
+
+The common case: a one-shot LangChain call (ChatOpenAI, ChatAnthropic, ChatGoogleGenerativeAI, ChatBedrockConverse, etc.) against an AI Config in completion mode.
+
+**Python:**
+
+```python
+from ldai_langchain import (
+    create_langchain_model,
+    convert_messages_to_langchain,
+    get_ai_metrics_from_response,
+)
+from langchain_core.messages import HumanMessage
+
+config = ai_client.completion_config("my-config-key", context)
+if not config.enabled:
+    return None
+
+# create_langchain_model reads config.model.name + parameters and picks the
+# right chat class (ChatOpenAI, ChatAnthropic, …) with no per-provider branching.
+llm = create_langchain_model(config)
+
+messages = convert_messages_to_langchain(config.messages or [])
+messages.append(HumanMessage(content=user_prompt))
+
+try:
+    completion = await config.tracker.track_metrics_of_async(
+        lambda: llm.ainvoke(messages),
+        get_ai_metrics_from_response,
+    )
+    return completion.content
+except Exception:
+    config.tracker.track_error()
+    raise
+```
+
+**Node:**
+
+```typescript
+import { LangChainProvider } from '@launchdarkly/server-sdk-ai-langchain';
+import { HumanMessage } from '@langchain/core/messages';
+
+const aiConfig = await aiClient.completionConfig('my-config-key', context);
+if (!aiConfig.enabled) return null;
+
+// createLangChainModel picks the right chat class (ChatOpenAI, ChatAnthropic, …)
+// and forwards all variation parameters.
+const llm = await LangChainProvider.createLangChainModel(aiConfig);
+
+const messages = LangChainProvider.convertMessagesToLangChain(aiConfig.messages ?? []);
+messages.push(new HumanMessage(userPrompt));
+
+try {
+  const completion = await aiConfig.tracker.trackMetricsOf(
+    LangChainProvider.getAIMetricsFromResponse,
+    () => llm.invoke(messages),
+  );
+  return completion.content;
+} catch (err) {
+  aiConfig.tracker.trackError();
+  throw err;
+}
+```
+
+Both `create_langchain_model` and `LangChainProvider.createLangChainModel` raise at model-creation time if the matching LangChain provider integration is not installed. For example, if the variation's `provider.name` is `anthropic`, your environment needs `langchain-anthropic` (Python) or `@langchain/anthropic` (Node). The error surface is LangChain's, not LaunchDarkly's — install the missing integration and re-run.
+
+### Why not `init_chat_model` + a custom provider-name mapping helper?
+
+You will see examples in the wild that build the model by hand with `init_chat_model(model=config.model.name, model_provider=map_provider_to_langchain(config.provider.name))`. Do not do this. It **silently drops every parameter** set on the variation (temperature, max_tokens, top_p, stop sequences, and any new field LaunchDarkly adds later), because `init_chat_model` only receives the name and provider. `create_langchain_model` forwards the whole parameter dict.
+
+## Tier 2 — LangGraph (agent workflows)
+
+LangGraph's `create_react_agent` takes a `model`, `tools`, and `prompt`. Build the model the same way as the single-LangChain case — `create_langchain_model` — and pass it in. The tracker wraps the whole agent invocation, and the extractor aggregates token usage across every message the agent produced.
+
+**Python** — agent mode with a `MemorySaver` checkpointer:
+
+```python
+from ldai.tracker import TokenUsage
+from ldai_langchain import create_langchain_model, get_ai_metrics_from_response
+from langgraph.prebuilt import create_react_agent
+from langgraph.checkpoint.memory import MemorySaver
+
+agent_config = ai_client.agent_config("my-agent-key", context)
+if not agent_config.enabled:
+    return None
+
+llm = create_langchain_model(agent_config)
+
+# MemorySaver gives the ReAct agent short-term memory per thread_id.
+checkpointer = MemorySaver()
+agent = create_react_agent(
+    llm,
+    tools=[...],                     # application-owned tool handlers
+    prompt=agent_config.instructions,
+    checkpointer=checkpointer,
+)
+
+async def track_langgraph_metrics(tracker, func):
+    """Aggregate token usage across every message the agent produced.
+    wraps track_duration_of + manual success/tokens/error tracking."""
+    try:
+        result = await tracker.track_duration_of(func)
+        tracker.track_success()
+        total_in = total_out = total = 0
+        for message in result.get("messages", []):
+            metrics = get_ai_metrics_from_response(message)
+            if metrics.usage:
+                total_in += metrics.usage.input
+                total_out += metrics.usage.output
+                total += metrics.usage.total
+        if total > 0:
+            tracker.track_tokens(TokenUsage(input=total_in, output=total_out, total=total))
+        return result
+    except Exception:
+        tracker.track_error()
+        raise
+
+result = await track_langgraph_metrics(
+    agent_config.tracker,
+    lambda: agent.ainvoke(
+        {"messages": [{"role": "user", "content": user_prompt}]},
+        config={"configurable": {"thread_id": thread_id}},
+    ),
+)
+```
+
+**Node** — same pattern with `trackMetricsOf` + a custom aggregator:
+
+```typescript
+import { LangChainProvider } from '@launchdarkly/server-sdk-ai-langchain';
+import type { LDAIMetrics } from '@launchdarkly/server-sdk-ai';
+import { createReactAgent } from '@langchain/langgraph/prebuilt';
+import { MemorySaver } from '@langchain/langgraph';
+
+const agentConfig = await aiClient.agentConfig('my-agent-key', context);
+if (!agentConfig.enabled) return null;
+
+const llm = await LangChainProvider.createLangChainModel(agentConfig);
+const checkpointer = new MemorySaver();
+const agent = createReactAgent({
+  llm,
+  tools: [/* ... */],
+  prompt: agentConfig.instructions,
+  checkpointer,
+});
+
+// Aggregate tokens across every message the agent produced.
+const langgraphMetrics = (result: any): LDAIMetrics => {
+  let input = 0, output = 0, total = 0;
+  for (const message of result.messages ?? []) {
+    const m = LangChainProvider.getAIMetricsFromResponse(message);
+    if (m.usage) {
+      input += m.usage.input ?? 0;
+      output += m.usage.output ?? 0;
+      total += m.usage.total ?? 0;
+    }
+  }
+  return { success: true, usage: total > 0 ? { input, output, total } : undefined };
+};
+
+const result = await agentConfig.tracker.trackMetricsOf(
+  langgraphMetrics,
+  () => agent.invoke(
+    { messages: [{ role: 'user', content: userPrompt }] },
+    { configurable: { thread_id: threadId } },
+  ),
+);
+```
+
+### Why aggregate per message
+
+`get_ai_metrics_from_response` / `getAIMetricsFromResponse` is defined on a single LangChain `AIMessage`. A LangGraph run produces N messages (model turn, tool result, model turn, tool result, final). If you pass the whole `result` to the extractor, you miss most of the token usage. Iterating and summing is deliberate — it's the same pattern the LaunchDarkly LangGraph guide uses.
+
+## Tier 3 — fall through to a custom extractor
+
+You will not usually need Tier 3 for LangChain or LangGraph — `get_ai_metrics_from_response` normalizes the response shape across providers. If the variation points at a model whose LangChain integration does not populate `usage_metadata` (rare, usually a custom integration), write a small extractor that reads whatever field the integration exposes and returns `LDAIMetrics`. This is the same fallback documented in [openai-tracking.md](openai-tracking.md) and [anthropic-tracking.md](anthropic-tracking.md).
+
+## Tier 4 — Manual (streaming only)
+
+LangChain streaming with TTFT tracking uses the same manual pattern as direct-SDK streaming. See [streaming-tracking.md](streaming-tracking.md).
+
+## What NOT to do
+
+- **Do not build the model with `init_chat_model` + a hand-rolled provider-name mapping.** The helper forwards all variation parameters; the hand-rolled version silently drops them.
+- **Do not pass the full LangGraph `result` object to `get_ai_metrics_from_response`.** The extractor is defined on a single message; aggregating across `result.messages` is the correct pattern.
+- **Do not assume there is a separate LangGraph provider package.** There is not. `@launchdarkly/server-sdk-ai-langchain` and `ldai_langchain` cover both.
+- **Do not import `LaunchDarklyCallbackHandler` from `ldai.langchain`.** Neither the class nor the dotted module path exists in the Python package. Use the helpers above.

skills/ai-configs/aiconfig-ai-metrics/references/strands-tracking.md

@@ -0,0 +1,92 @@
+# Strands Agents Metrics Tracking
+
+**There is no LaunchDarkly provider package for Strands.** Strands is a provider-agnostic agent SDK — the same `Agent` class runs against Anthropic, OpenAI, and Bedrock by swapping the `model` argument — so the tracking pattern plugs in at the agent layer, not the provider layer. Tier 3 (custom extractor + `trackMetricsOf`) is the canonical path.
+
+The Strands `AgentResult` object exposes a `metrics.accumulated_usage` dict (Python) / `metrics.accumulatedUsage` object (Node) that already aggregates token counts across every provider call the agent made in a single `invoke_async` turn — including any tool-calling round trips. That means one extractor call covers the whole turn, unlike the per-response shape from Anthropic or OpenAI direct.
+
+The key names inside `accumulated_usage` are camelCase even in Python: `inputTokens`, `outputTokens`, `totalTokens`.
+
+## Tier 1 is not available
+
+`ManagedModel` / `TrackedChat` do not currently ship a Strands runner. Strands owns its own agent loop and short-term memory (`SlidingWindowConversationManager`), so wrapping it in a LaunchDarkly managed runner would fight against the framework. Stay on Tier 3.
+
+## Tier 3 — Explicit `track_duration_of` + manual `track_tokens` (primary)
+
+This is the shape in the LaunchDarkly Strands integration guide. Use it when the call site is already async and you want token extraction split out from duration tracking.
+
+```python
+from ldai.tracker import TokenUsage
+
+
+def track_strands_metrics(tracker, result):
+    """Record token usage from a Strands AgentResult on the LD tracker."""
+    usage = getattr(result.metrics, "accumulated_usage", {}) or {}
+    input_tokens = usage.get("inputTokens", 0)
+    output_tokens = usage.get("outputTokens", 0)
+    total = usage.get("totalTokens", 0) or (input_tokens + output_tokens)
+    if total > 0:
+        tracker.track_tokens(
+            TokenUsage(input=input_tokens, output=output_tokens, total=total)
+        )
+
+
+async def run_turn(agent, tracker, user_input):
+    try:
+        result = await tracker.track_duration_of(lambda: agent.invoke_async(user_input))
+        tracker.track_success()
+        track_strands_metrics(tracker, result)
+        return result.message["content"][0]["text"]
+    except Exception:
+        tracker.track_error()
+        raise
+```
+
+**What this tracks:**
+- Duration — from the `track_duration_of` wrapper around `invoke_async`.
+- Tokens — from `accumulated_usage`, including any tool-calling round trips inside the turn.
+- Success / error — explicit, in the try/except.
+
+## Tier 3 — Single-call `track_metrics_of_async` variant
+
+If you prefer the single-call form that matches the rest of the provider-tracking references, fold the extractor into an `LDAIMetrics` return and use `track_metrics_of_async`:
+
+```python
+from ldai.providers.types import LDAIMetrics, TokenUsage
+
+
+def strands_extractor(result) -> LDAIMetrics:
+    usage = getattr(result.metrics, "accumulated_usage", {}) or {}
+    input_tokens = usage.get("inputTokens", 0)
+    output_tokens = usage.get("outputTokens", 0)
+    total = usage.get("totalTokens", 0) or (input_tokens + output_tokens)
+    return LDAIMetrics(
+        success=True,
+        usage=TokenUsage(input=input_tokens, output=output_tokens, total=total),
+    )
+
+
+async def run_turn(agent, tracker, user_input):
+    try:
+        result = await tracker.track_metrics_of_async(
+            lambda: agent.invoke_async(user_input),
+            strands_extractor,
+        )
+        return result.message["content"][0]["text"]
+    except Exception:
+        tracker.track_error()
+        raise
+```
+
+Pick the style that matches the rest of the codebase — the two variants record the same metrics.
+
+## Provider dispatch stays in your code
+
+Strands model classes are provider-specific (`AnthropicModel`, `OpenAIModel`, `BedrockModel`). To serve more than one provider from a single AI Config key, dispatch on `agent_config.provider.name` before constructing the `Agent`. See [agent-mode-frameworks.md § Strands Agent](../../aiconfig-migrate/references/agent-mode-frameworks.md) for the `create_strands_model` dispatcher, including the rule that `parameters.tools` must be dropped before being passed into the Strands model class (tools flow through the `Agent` constructor, not through model params).
+
+## Always flush before exit
+
+Strands examples are commonly short-lived scripts (`python run_agent.py ...`). Trailing analytics events can be lost if the client closes before flushing. Always call `ldclient.get().flush()` (and `ldclient.get().close()` on exit) after the last turn.
+
+## Node / TypeScript caveat
+
+The Strands TypeScript SDK ships `BedrockModel` and `OpenAIModel` only — no `AnthropicModel`. The same Tier-3 pattern applies (custom extractor over `result.metrics.accumulatedUsage`, then `tracker.trackMetricsOf` or explicit `trackDurationOf` + `trackTokens`), but multi-provider variations that include Anthropic require the Python SDK today.

skills/ai-configs/aiconfig-create/SKILL.md

@@ -38,7 +38,7 @@ When the user provides enough context (use case, model, mode), proceed through t
 
 Before creating, identify what you're building:
 
-- **What framework?** LangGraph, LangChain, CrewAI, OpenAI SDK, Anthropic SDK, custom
+- **What framework?** LangGraph, LangChain, CrewAI, Strands, OpenAI SDK, Anthropic SDK, custom
 - **What does the AI need?** Just text generation, or tools/function calling?
 - **Agent or completion?** See the decision matrix below
 
@@ -48,8 +48,8 @@ This choice is about **input schema and framework compatibility**, not execution
 
 | Your Need | Mode | Why |
 |-----------|------|-----|
-| LangGraph, CrewAI, AutoGen frameworks | **Agent** | Frameworks expect goal/instruction input |
-| Persistent instructions across interactions | **Agent** | Single instructions string, SDK method: `aiclient.agent()` |
+| LangGraph, CrewAI, Strands, AutoGen frameworks | **Agent** | Frameworks expect goal/instruction input |
+| Persistent instructions across interactions | **Agent** | Single instructions string, SDK method: `agent_config()` (Python) / `agentConfig()` (Node) |
 | Direct OpenAI/Anthropic API calls | **Completion** | Messages array maps directly to provider APIs |
 | Full control of message structure | **Completion** | System/user/assistant role-based messages |
 | One-off text generation | **Completion** | Standard chat format |
@@ -102,7 +102,7 @@ Example completion-mode call:
 ```
 
 **Optional:**
-- `parameters` -- model parameters like `{temperature: 0.7, maxTokens: 2000}`
+- `parameters` -- model parameters like `{temperature: 0.7, max_tokens: 2000}` (match the UI's snake_case keys)
 
 The tool returns the full verified config detail with the variation attached.
 

skills/ai-configs/aiconfig-custom-metrics/SKILL.md

@@ -480,7 +480,7 @@ class SessionMetricsTracker:
 1. **Create Before Track** - Metric must exist before tracking events
 2. **Use Numeric Metrics** - Set `isNumeric=True` for aggregation
 3. **Consistent Keys** - Use same key in `create_metric()` and `ld_client.track()`
-4. **Flush in Serverless** - Call `ld_client.flush()` before Lambda terminates
+4. **Always flush before close** - Call `ld_client.flush()` (await in Node) before `close()`. Trailing events are at risk of being lost otherwise, in short-lived scripts and long-running services alike. This is not a serverless-only rule; it applies to any process that exits.
 5. **Rate Limit** - Don't track on every keystroke
 
 ## Viewing Metrics

skills/ai-configs/aiconfig-migrate/README.md

@@ -22,7 +22,7 @@ Copy `skills/ai-configs/aiconfig-migrate/` into your agent client's skills path.
 
 - Remotely hosted LaunchDarkly MCP server
 - `LD_SDK_KEY` environment variable (server-side SDK key, starts with `sdk-`)
-- An application with hardcoded LLM calls (OpenAI, Anthropic, Bedrock, Gemini, LangChain, LangGraph, or CrewAI)
+- An application with hardcoded LLM calls (OpenAI, Anthropic, Bedrock, Gemini, LangChain, LangGraph, CrewAI, or Strands)
 
 ## Usage
 

skills/ai-configs/aiconfig-migrate/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: aiconfig-migrate
-description: "Migrate an application with hardcoded LLM prompts to a full LaunchDarkly AI Configs implementation in five stages: extract prompts, wrap in the AI SDK, add tools, add tracking, add evals/judges. Use when the user wants to externalize model/prompt configuration, move from direct provider calls (OpenAI, Anthropic, Bedrock, Gemini) to a managed AI Config, or stage a full hardcoded-to-LaunchDarkly migration."
+description: "Migrate an application with hardcoded LLM prompts to a full LaunchDarkly AI Configs implementation in five stages: extract prompts, wrap in the AI SDK, add tools, add tracking, add evals/judges. Use when the user wants to externalize model/prompt configuration, move from direct provider calls (OpenAI, Anthropic, Bedrock, Gemini, Strands) to a managed AI Config, or stage a full hardcoded-to-LaunchDarkly migration."
 license: Apache-2.0
 compatibility: Requires the remotely hosted LaunchDarkly MCP server
 metadata:
@@ -47,9 +47,9 @@ Run the phase-1 checklist and produce a structured summary. **This step writes n
 Use [phase-1-analysis-checklist.md](references/phase-1-analysis-checklist.md) to scan:
 
 1. **Language and package manager** — Python (pip/poetry/uv), TypeScript/JavaScript (npm/pnpm/yarn), Go, Ruby, .NET
-2. **LLM provider** — OpenAI, Anthropic, Bedrock, Gemini, LangChain, LangGraph, CrewAI
+2. **LLM provider** — OpenAI, Anthropic, Bedrock, Gemini, LangChain, LangGraph, CrewAI, Strands
 3. **Existing LaunchDarkly usage** — any pre-existing `LDClient` or `ldclient` initialization to reuse
-4. **Hardcoded model configs** — model name string literals, temperature / maxTokens / topP, system prompts, instruction strings
+4. **Hardcoded model configs** — model name string literals, temperature / max_tokens / top_p, system prompts, instruction strings
 5. **Mode decision** — completion mode (chat messages array) or agent mode (single instructions string). Completion mode is the default and the only mode that supports judges attached in the UI.
 
 **Phase 1 output** (return to user as a structured summary):
@@ -86,10 +86,16 @@ This manifest is the contract for the next four stages. Review it with the user.
 This is the first stage that writes code. It has six sub-steps.
 
 1. **Install the AI SDK.** Detect the package manager from Step 1, then install:
-   - Python: `launchdarkly-server-sdk` + `launchdarkly-server-sdk-ai`
+   - Python: `launchdarkly-server-sdk` + `launchdarkly-server-sdk-ai>=0.17.0`
    - Node.js/TypeScript: `@launchdarkly/node-server-sdk` + `@launchdarkly/server-sdk-ai`
    - Go: `github.com/launchdarkly/go-server-sdk/v7` + `github.com/launchdarkly/go-server-sdk/ldai`
 
+   Tier-2 provider packages (install in Stage 4, only if you're using the matching provider):
+   - OpenAI: `launchdarkly-server-sdk-ai-openai>=0.3.0` (Python) / `@launchdarkly/server-sdk-ai-openai` (Node)
+   - LangChain / LangGraph: `launchdarkly-server-sdk-ai-langchain>=0.4.1` (Python) / `@launchdarkly/server-sdk-ai-langchain` (Node)
+   - Vercel AI SDK (Node only): `@launchdarkly/server-sdk-ai-vercel`
+   - Anthropic, Gemini, Bedrock — no provider package published; use Tier-3 custom extractor (see `aiconfig-ai-metrics`)
+
 2. **Initialize `LDAIClient` once at startup.** Reuse any existing `LDClient` — do not create a second base client. Place the initialization in the same module that owns existing app config.
 
    **Python:**
@@ -124,7 +130,7 @@ This is the first stage that writes code. It has six sub-steps.
 
    fallback = AICompletionConfigDefault(
        enabled=True,
-       model=ModelConfig(name="gpt-4o", parameters={"temperature": 0.7, "maxTokens": 2000}),
+       model=ModelConfig(name="gpt-4o", parameters={"temperature": 0.7, "max_tokens": 2000}),
        provider=ProviderConfig(name="openai"),
        messages=[LDMessage(role="system", content="You are a helpful assistant...")],
    )
@@ -157,7 +163,7 @@ This is the first stage that writes code. It has six sub-steps.
    response = openai_client.chat.completions.create(
        model=config.model.name,
        temperature=params.get("temperature"),
-       max_tokens=params.get("maxTokens"),
+       max_tokens=params.get("max_tokens"),
        messages=[m.to_dict() for m in (config.messages or [])] + [
            {"role": "user", "content": user_input},
        ],
@@ -206,6 +212,7 @@ Skip this step if the audited app has no function calling / tools. Otherwise:
    - `anthropic.messages.create(tools=[...])` — Anthropic direct
    - `create_react_agent(tools=[...])` — LangGraph prebuilt ReAct
    - `Agent(tools=[...])` — CrewAI
+   - `Agent(tools=[...])` — Strands (Python `@tool`-decorated callables passed through the constructor; TS SDK uses Zod-schema tools)
    - **Custom `StateGraph`** — module-level `TOOLS = [...]` list referenced in **both** `model.bind_tools(TOOLS)` and `ToolNode(TOOLS)`. This is the `langchain-ai/react-agent` template shape; the list is usually in a `tools.py` module. Grep for `bind_tools(` and `ToolNode(` together — they will point at the same list.
 
    Record each tool's name, description, and JSON schema.
@@ -285,7 +292,7 @@ Hand off: print the AI Config key, variation key, provider, and whether the call
    }
    ```
 
-   For Anthropic direct, Bedrock (no provider package), Gemini, and custom HTTP, write a small extractor returning `LDAIMetrics` — see the delegate skill's [anthropic-tracking.md](../aiconfig-ai-metrics/references/anthropic-tracking.md) and [bedrock-tracking.md](../aiconfig-ai-metrics/references/bedrock-tracking.md). LangChain single-node and LangGraph go through the `launchdarkly-server-sdk-ai-langchain` / `@launchdarkly/server-sdk-ai-langchain` provider package with `LangChainProvider.get_ai_metrics_from_response`.
+   For Anthropic direct, Bedrock (no provider package), Gemini, and custom HTTP, write a small extractor returning `LDAIMetrics` — see the delegate skill's [anthropic-tracking.md](../aiconfig-ai-metrics/references/anthropic-tracking.md), [bedrock-tracking.md](../aiconfig-ai-metrics/references/bedrock-tracking.md), and [gemini-tracking.md](../aiconfig-ai-metrics/references/gemini-tracking.md). LangChain single-node and LangGraph go through the `launchdarkly-server-sdk-ai-langchain` / `@launchdarkly/server-sdk-ai-langchain` provider package. Build the model with `create_langchain_model(config)` / `LangChainProvider.createLangChainModel(config)` (forwards all variation parameters) and track with `get_ai_metrics_from_response` / `LangChainProvider.getAIMetricsFromResponse`. See [langchain-tracking.md](../aiconfig-ai-metrics/references/langchain-tracking.md).
 
 4. **Wire feedback tracking if the app has thumbs-up/down UI.** Both SDKs expose `trackFeedback` with a `{kind}` argument.
 
@@ -368,8 +375,10 @@ Delegate: **`aiconfig-online-evals`** (sub-step 3, optional — only for UI-atta
 | App uses LangChain `ChatOpenAI(model=...)` | Read `config.model.name` and pass it as `ChatOpenAI(model=config.model.name)`; keep LangChain for the call itself |
 | Retry wrapper around the provider call | Move tracker inside the retry — failures in the same request should share one `track_error`, and success tracking should fire only after the final attempt |
 | App has no tools — Stage 3 skipped | Move directly from Stage 2 verification to Stage 4 (tracking) |
-| Mode mismatch: user said agent, audit shows one-shot chat | Choose completion mode unless the app uses LangGraph `create_react_agent`, CrewAI `Agent`, or a similar goal-driven framework |
-| TypeScript app using Anthropic SDK | No `trackAnthropicMetrics` helper exists — use manual `trackDuration` + `trackTokens` + `trackSuccess`/`trackError` (see the Step 5 manual block) |
+| Mode mismatch: user said agent, audit shows one-shot chat | Choose completion mode unless the app uses LangGraph `create_react_agent`, CrewAI `Agent`, Strands `Agent`, or a similar goal-driven framework |
+| App uses Strands Agents (Python) | Agent mode. Build a `create_strands_model` dispatcher keyed on `agent_config.provider.name` that returns `AnthropicModel(model_id=..., max_tokens=...)` or `OpenAIModel(model_id=..., params=...)`. Drop `parameters.tools` before passing params to the model class — Strands receives tools via `Agent(tools=[...])`. Tracking is Tier 3: wrap `invoke_async` with `tracker.track_duration_of(...)` and record tokens from `result.metrics.accumulated_usage`. See [agent-mode-frameworks.md § Strands Agent](references/agent-mode-frameworks.md) and [strands-tracking.md](../aiconfig-ai-metrics/references/strands-tracking.md) |
+| Strands app on TypeScript | TS SDK ships `BedrockModel` and `OpenAIModel` only — cannot serve Anthropic-backed variations. Use the Python SDK if multi-provider variations are required |
+| TypeScript app using Anthropic SDK | No `trackAnthropicMetrics` helper exists. Use Tier 3: `trackMetricsOf` with a small custom extractor that reads `response.usage.input_tokens` / `response.usage.output_tokens` and returns `LDAIMetrics`. See [anthropic-tracking.md](../aiconfig-ai-metrics/references/anthropic-tracking.md) in the `aiconfig-ai-metrics` skill for the exact extractor |
 | Fallback would silently crash because `LD_SDK_KEY` is missing | Log a startup warning; proceed with the fallback. Never raise at import time |
 | Multi-agent graph (supervisor + workers) | Stop after migrating a single agent. Agent graphs are currently **Python-only** (`launchdarkly-server-sdk-ai.agent_graph`). Read [agent-graph-reference.md](references/agent-graph-reference.md) for the graph-level migration path — it is deliberately out of this skill's main scope |
 | Single-agent (ReAct, tool loop) + agent mode | Default to offline eval via the LD Playground + Datasets for Stage 5. UI-attached judges are completion-only today, and programmatic direct-judge adds per-call cost that is usually not worth it until after the migration is live and stable. Point at `/tutorials/offline-evals` |
@@ -395,7 +404,7 @@ Delegate: **`aiconfig-online-evals`** (sub-step 3, optional — only for UI-atta
 - Don't attempt a multi-agent graph migration in one pass. Migrate a single agent first; use [agent-graph-reference.md](references/agent-graph-reference.md) as the next-step read.
 - Don't use `track_request()` in Python — it does not exist in `launchdarkly-server-sdk-ai`. Use `track_metrics_of` with a provider-package or custom extractor, or drop to explicit `track_duration` + `track_tokens` + `track_success` / `track_error` if you're on the streaming path.
 - Don't tuple-unpack the return of `completion_config` / `agent_config` / `completionConfig` / `agentConfig`. They return a **single** config object (e.g. `AIAgentConfig`, `AICompletionConfig`), not `(config, tracker)`. The tracker is at `config.tracker`. LLMs hallucinate the tuple shape because pre-v0.x SDKs used to return one — the current API does not.
-- Don't import `LaunchDarklyCallbackHandler` from `ldai.langchain` — neither the class nor the dotted module path exists. The real Python LangChain helper package is `ldai_langchain` (top-level module, underscore). For single-node LangChain calls, use `track_metrics_of(fn, LangChainProvider.get_ai_metrics_from_response)` — the provider package normalizes `AIMessage.usage_metadata` for you across OpenAI / Anthropic / Bedrock / Gemini. See [sdk-ai-tracker-patterns.md](references/sdk-ai-tracker-patterns.md) for the full matrix.
+- Don't import `LaunchDarklyCallbackHandler` from `ldai.langchain` — neither the class nor the dotted module path exists. The real Python LangChain helper package is `ldai_langchain` (top-level module, underscore). For single-node LangChain calls, build the model with `create_langchain_model(config)` (forwards all variation parameters and handles LaunchDarkly→LangChain provider-name mapping internally) and track with `track_metrics_of_async(lambda: llm.ainvoke(messages), get_ai_metrics_from_response)`. Do not reach for `init_chat_model` + a hand-rolled provider-name mapping — that path silently drops every variation parameter (temperature, max_tokens, top_p). See [langchain-tracking.md](../aiconfig-ai-metrics/references/langchain-tracking.md) for both single-model and LangGraph patterns.
 
 ## Related Skills
 

skills/ai-configs/aiconfig-migrate/references/agent-mode-frameworks.md

@@ -1,6 +1,6 @@
 # Agent-Mode Frameworks
 
-How to wire an AI Config in **agent mode** into the frameworks that take a goal/instructions string: LangGraph, CrewAI, and custom ReAct loops. Also covers the **dynamic tool loading** pattern from the devrel-agents-tutorial — how to extract tool names from `config.tools` at runtime and instantiate the actual tool implementations without hardcoding.
+How to wire an AI Config in **agent mode** into the frameworks that take a goal/instructions string: LangGraph, CrewAI, Strands, and custom ReAct loops. Also covers the **dynamic tool loading** pattern from the devrel-agents-tutorial — how to extract tool names from `config.tools` at runtime and instantiate the actual tool implementations without hardcoding.
 
 ## When to pick agent mode
 
@@ -12,11 +12,14 @@ Completion mode is the default and covers direct provider calls (OpenAI, Anthrop
 | Takes `role`, `goal`, `backstory` | CrewAI `Agent` | `Agent(role="researcher", goal="...", backstory="...")` |
 | Custom ReAct loop with a system instruction separated from messages | hand-rolled | `system = "You can use search..."; while not done: ...` |
 | Multi-step tool use with persistent instructions across turns | LangGraph / LangChain `AgentExecutor` | The system prompt stays stable across a long interaction |
+| Provider-agnostic agent with `@tool` decorators and `invoke_async` | Strands `Agent` | `Agent(model=OpenAIModel(...), system_prompt="You are...", tools=[search])` |
 
 Agent mode returns an `instructions` string. Completion mode returns a `messages` array. Both modes support tools, parameters, and the same tracker — the only difference is the input shape the SDK returns to you.
 
 **Caveat:** judges cannot be attached to agent-mode variations via the LaunchDarkly UI. Agent mode evaluations must go through the programmatic judge API (`create_judge(...).evaluate(input, output)`). See `aiconfig-online-evals` for the programmatic path.
 
+**Model construction for LangChain / LangGraph.** When the framework runs on top of LangChain (which includes LangGraph's `create_react_agent` and most custom graphs), build the chat model with `create_langchain_model(ai_config)` (Python) or `LangChainProvider.createLangChainModel(aiConfig)` (Node). These helpers forward every variation parameter (`temperature`, `max_tokens`, `top_p`, …) and handle LaunchDarkly→LangChain provider-name mapping internally. Do not hand-roll `init_chat_model(model=..., model_provider=...)` — it silently drops every variation parameter. See [langchain-tracking.md](../../aiconfig-ai-metrics/references/langchain-tracking.md) for the canonical single-model and LangGraph patterns, including the per-message token-aggregation extractor used with `track_metrics_of_async` / `trackMetricsOf`.
+
 ## Wiring `agent_config` into each framework
 
 ### LangGraph `create_react_agent`
@@ -88,6 +91,116 @@ def build_crew_agent(ai_client, user_id: str):
 
 Prompt variables are cleaner and keep the AI Config human-readable in the UI.
 
+### Strands `Agent`
+
+Strands is a provider-agnostic, async-first agent SDK. The same `Agent` class runs against Anthropic, OpenAI, and Bedrock by swapping the `model` argument; tools are plain `@tool`-decorated Python functions passed through the constructor; and `SlidingWindowConversationManager` keeps short-term memory across `invoke_async` turns without external state. Agent-mode `instructions` maps directly to `Agent(system_prompt=...)`.
+
+Strands does not ship a first-party LaunchDarkly provider package. To serve multiple providers from a single AI Config key, dispatch on `agent_config.provider.name` and construct the matching Strands model class.
+
+**Provider dispatcher.** Drop `parameters.tools` before passing params into the Strands model class — LaunchDarkly surfaces attached tools via a flat `parameters.tools` shape in the variation payload, but Strands receives tools via the `Agent` constructor. Passing `tools` through a second time via model `params` is an error.
+
+```python
+from strands.models.anthropic import AnthropicModel
+from strands.models.openai import OpenAIModel
+
+
+def create_strands_model(agent_config):
+    """Map an LDAIAgentConfig to the matching Strands model class by provider."""
+    provider = (agent_config.provider.name if agent_config.provider else "").lower()
+    model_id = agent_config.model.name
+    params = dict(agent_config.model.to_dict().get("parameters") or {})
+    # LD surfaces attached tools via parameters.tools; Strands takes tools via
+    # Agent(tools=[...]). Drop the key before passing params to the model class.
+    params.pop("tools", None)
+
+    if provider == "anthropic":
+        # AnthropicModel requires max_tokens as a kwarg, not inside params.
+        max_tokens = int(
+            params.pop("max_tokens", None) or params.pop("maxTokens", None) or 1024
+        )
+        return AnthropicModel(model_id=model_id, max_tokens=max_tokens, params=params or None)
+    if provider == "openai":
+        # Pass parameters through as-is — gpt-5 wants max_completion_tokens,
+        # gpt-4o wants max_tokens. Keep that choice in the LD variation.
+        return OpenAIModel(model_id=model_id, params=params)
+    raise ValueError(f"Unsupported provider for Strands: {provider!r}")
+```
+
+**Call site.** Build the agent once per request, pull the tracker off the config, and wrap `invoke_async` with `track_duration_of` — Strands is Tier 3 (custom extractor) because there is no provider package.
+
+```python
+from strands import Agent, tool
+from strands.agent.conversation_manager.sliding_window_conversation_manager import (
+    SlidingWindowConversationManager,
+)
+from ldai.client import AIAgentConfigDefault, ModelConfig, ProviderConfig
+from ldai.tracker import TokenUsage
+from ldclient import Context
+
+
+@tool
+def get_order_status(order_id: str) -> str:
+    """Look up the status of a customer order by order ID."""
+    ...
+
+
+FALLBACK = AIAgentConfigDefault(
+    enabled=True,
+    model=ModelConfig(name="gpt-5", parameters={"max_completion_tokens": 2000}),
+    provider=ProviderConfig(name="openai"),
+    instructions="You are a helpful assistant.",
+)
+
+
+def track_strands_metrics(tracker, result):
+    """Record token usage from a Strands AgentResult on the LD tracker.
+
+    accumulated_usage aggregates tokens across every provider call in the turn,
+    including tool-calling round trips — unlike the single-response shape from
+    Anthropic or OpenAI direct.
+    """
+    usage = getattr(result.metrics, "accumulated_usage", {}) or {}
+    input_tokens = usage.get("inputTokens", 0)
+    output_tokens = usage.get("outputTokens", 0)
+    total = usage.get("totalTokens", 0) or (input_tokens + output_tokens)
+    if total > 0:
+        tracker.track_tokens(TokenUsage(input=input_tokens, output=output_tokens, total=total))
+
+
+async def run_turn(ai_client, user_id: str, user_input: str):
+    context = Context.builder(user_id).kind("user").build()
+    agent_config = ai_client.agent_config("strands-agent", context, FALLBACK)
+
+    if not agent_config.enabled:
+        return disabled_response()
+
+    agent = Agent(
+        name="order-assistant",
+        model=create_strands_model(agent_config),
+        system_prompt=agent_config.instructions,
+        tools=[get_order_status],
+        conversation_manager=SlidingWindowConversationManager(window_size=40),
+    )
+    tracker = agent_config.tracker
+
+    try:
+        result = await tracker.track_duration_of(lambda: agent.invoke_async(user_input))
+        tracker.track_success()
+        track_strands_metrics(tracker, result)
+        return result.message["content"][0]["text"]
+    except Exception:
+        tracker.track_error()
+        raise
+```
+
+**Key points:**
+- `system_prompt=agent_config.instructions` — the instructions string replaces the hardcoded system prompt.
+- `create_strands_model(agent_config)` is the provider-dispatch seam. Add a branch per provider the variation can serve.
+- The tracker is Tier 3: `tracker.track_duration_of(...)` + an explicit `track_tokens` call fed by `track_strands_metrics`. See [strands-tracking.md](../../aiconfig-ai-metrics/references/strands-tracking.md) for the single-call `track_metrics_of_async` variant and the per-field breakdown of `accumulated_usage`.
+- Always `ldclient.get().flush()` before process exit in short-lived scripts — trailing events can otherwise be lost.
+
+**TypeScript caveat.** The Strands TypeScript SDK ships `BedrockModel` and `OpenAIModel` only — it cannot run Anthropic-backed variations. If the app needs to serve both OpenAI and Anthropic from a single AI Config, use the Python SDK.
+
 ### Custom `StateGraph` (bind_tools + ToolNode)
 
 The most common LangGraph pattern in the wild is not `create_react_agent` — it's a custom `StateGraph` with a `call_model` node that does `model.bind_tools(TOOLS)`, a separate `"tools"` node that runs `ToolNode(TOOLS)`, and a conditional edge between them. This is the shape of the `langchain-ai/react-agent` template.
@@ -228,7 +341,7 @@ graph = builder.compile()
 - `TOOLS` (a static list) → `TOOL_IMPLEMENTATIONS` (a name-to-callable dict) + `build_tools_from_config(ai_config)` (a per-request builder). This is the dynamic tool factory pattern from the devrel-agents-tutorial, adapted to plain callables.
 - `call_model` fetches an `AIAgentConfig` per invocation, builds tools from `ai_config.tools`, binds them, and injects `ai_config.instructions` as the system message (replacing `runtime.context.system_prompt`).
 - The `"tools"` node is replaced with a factory that reads the active tool list from state — so both `bind_tools` and `ToolNode` always run against the same list. If you skip this step and leave `ToolNode(TOOLS)` hardcoded, the LLM and the executor will disagree on what's available and the graph will misbehave.
-- Tracker calls wrap the provider call inside `call_model`. On Node, use `trackDuration` / `trackSuccess` / `trackTokens` or `trackOpenAIMetrics` via the helper package.
+- Tracker calls wrap the provider call inside `call_model`. The snippet above uses explicit `track_duration` + `track_tokens` + `track_success` because the sample model is hand-constructed via `load_chat_model(name).bind_tools(...)` without passing variation parameters. If you switch to `create_langchain_model(ai_config).bind_tools(tools)`, variation parameters flow through and you can collapse the block to `track_metrics_of_async(lambda: model.ainvoke(...), get_ai_metrics_from_response)`. Same story on Node: `LangChainProvider.createLangChainModel(aiConfig)` + `tracker.trackMetricsOf(LangChainProvider.getAIMetricsFromResponse, ...)`.
 - The existing `Context` dataclass is kept as the fallback shape — its defaults become the `AIAgentConfigDefault` values, so the app still runs exactly as before when LaunchDarkly is unreachable.
 
 **Gotcha:** the `"tools"` node above reads `_active_tools` from state. That means your `State` TypedDict has to include it. If it doesn't, either add the field, or take the simpler route of fetching the AI Config **once at graph-compile time** (at module load) and accepting that tool changes require a restart. The per-invocation pattern above is strictly better but adds one state field.

skills/ai-configs/aiconfig-migrate/references/before-after-examples.md

@@ -47,7 +47,7 @@ FALLBACK = AICompletionConfigDefault(
     enabled=True,
     model=ModelConfig(
         name="gpt-4o",
-        parameters={"temperature": 0.7, "maxTokens": 2000},
+        parameters={"temperature": 0.7, "max_tokens": 2000},
     ),
     provider=ProviderConfig(name="openai"),
     messages=[LDMessage(role="system", content="You are a helpful assistant. Answer concisely.")],
@@ -64,7 +64,7 @@ def answer(user_id: str, user_question: str) -> str:
     response = openai_client.chat.completions.create(
         model=config.model.name,
         temperature=params.get("temperature"),
-        max_tokens=params.get("maxTokens"),
+        max_tokens=params.get("max_tokens"),
         messages=[m.to_dict() for m in (config.messages or [])] + [
             {"role": "user", "content": user_question},
         ],
@@ -122,7 +122,7 @@ const FALLBACK: LDAICompletionConfigDefault = {
   enabled: true,
   model: {
     name: 'claude-sonnet-4-5',
-    parameters: { maxTokens: 1024 },
+    parameters: { max_tokens: 1024 },
   },
   provider: { name: 'anthropic' },
   messages: [
@@ -154,7 +154,7 @@ export async function answer(userId: string, userQuestion: string): Promise<stri
 
   const response = await anthropic.messages.create({
     model: aiConfig.model?.name ?? 'claude-sonnet-4-5',
-    max_tokens: (aiConfig.model?.parameters?.maxTokens as number) ?? 1024,
+    max_tokens: (aiConfig.model?.parameters?.max_tokens as number) ?? 1024,
     system: systemMessage,
     messages,
   });

skills/ai-configs/aiconfig-migrate/references/fallback-defaults-pattern.md

@@ -22,7 +22,7 @@ CHAT_FALLBACK = AICompletionConfigDefault(
     enabled=True,
     model=ModelConfig(
         name="gpt-4o",
-        parameters={"temperature": 0.7, "maxTokens": 2000},
+        parameters={"temperature": 0.7, "max_tokens": 2000},
     ),
     provider=ProviderConfig(name="openai"),
     messages=[
@@ -69,7 +69,7 @@ const CHAT_FALLBACK: LDAICompletionConfigDefault = {
   enabled: true,
   model: {
     name: 'gpt-4o',
-    parameters: { temperature: 0.7, maxTokens: 2000 },
+    parameters: { temperature: 0.7, max_tokens: 2000 },
   },
   provider: { name: 'openai' },
   messages: [
@@ -123,7 +123,7 @@ A JSON/YAML file holds every config's fallback; a loader at startup builds the d
       "enabled": true,
       "model": {
         "name": "gpt-4o",
-        "parameters": { "temperature": 0.7, "maxTokens": 2000 }
+        "parameters": { "temperature": 0.7, "max_tokens": 2000 }
       },
       "provider": { "name": "openai" },
       "messages": [
@@ -266,6 +266,6 @@ Fallback drift is a feature, not a bug. If you regenerate on every deploy, a sta
 
 1. **Fallback mirrors pre-migration behavior.** If the hardcoded model was `gpt-4o`, the fallback model is `gpt-4o`. If the hardcoded temperature was `0.7`, the fallback temperature is `0.7`. The fallback is the contract that says "app behavior doesn't change if LaunchDarkly is unreachable."
 2. **Always set `enabled=True` in the fallback.** If the fallback has `enabled=False`, the disabled path runs every time LaunchDarkly is unreachable — an outage escalates into a full service outage. Make the fallback a real, working config unless the feature is explicitly off-by-default.
-3. **Fallback must be a fully-specified `AICompletionConfigDefault` / `AIAgentConfigDefault`.** Do not pass `AICompletionConfigDefault(enabled=False)` as a "placeholder" — the SDK will not synthesize missing fields. You must supply `model`, `provider`, and `messages`/`instructions` if `enabled=True`.
+3. **Fallback is optional — but if you pass one, specify it fully.** Omitting the fallback argument is valid; the SDK returns a disabled config when LaunchDarkly is unreachable, and your `if not config.enabled:` branch handles the disabled path. Pass a fallback when you want the app to keep serving traffic on LaunchDarkly unreachable — in that case it must be a fully-specified `AICompletionConfigDefault` / `AIAgentConfigDefault` with `model`, `provider`, and `messages`/`instructions`. Do not pass `AICompletionConfigDefault(enabled=False)` as a "placeholder" — it collapses into the disabled path and gives you nothing the omitted-fallback case wouldn't.
 4. **Do not delete the fallback after migration.** It is required for the `enabled=False` path and for SDK-unreachable scenarios. Treat it as load-bearing production code, not scaffolding.
 5. **Keep fallback type imports stable.** `AICompletionConfigDefault` is for completion mode; `AIAgentConfigDefault` is for agent mode. Using the wrong one will fail at runtime when the SDK tries to coerce the fallback.

skills/ai-configs/aiconfig-migrate/references/phase-1-analysis-checklist.md

@@ -93,11 +93,11 @@ Feeds into Stage 2 (install + wrap). Quoted from the `ai-configs-relaunch-guides
 
 | Provider | Package | Helper |
 |----------|---------|--------|
-| OpenAI | `@launchdarkly/server-sdk-ai-openai` | `OpenAIProvider.createAIMetrics` + `trackMetricsOf` |
-| LangChain | `@launchdarkly/server-sdk-ai-langchain` | **Manual today** — no single-call auto-helper. Use `tracker.trackDuration` + `trackTokens` + `trackSuccess`/`trackError` around `ainvoke`. |
-| Vercel AI SDK | `@launchdarkly/server-sdk-ai-vercel` | `trackVercelAISDKGenerateTextMetrics` |
+| OpenAI | `@launchdarkly/server-sdk-ai-openai` | `OpenAIProvider.getAIMetricsFromResponse` + `trackMetricsOf` |
+| LangChain / LangGraph | `@launchdarkly/server-sdk-ai-langchain` | `LangChainProvider.createLangChainModel(config)` (forwards all variation parameters and handles provider-name mapping) + `LangChainProvider.getAIMetricsFromResponse` with `trackMetricsOf` |
+| Vercel AI SDK | `@launchdarkly/server-sdk-ai-vercel` | `VercelAISDKProvider.getAIMetricsFromGenerateText` + `trackMetricsOf` |
 
-Python currently ships helper packages for OpenAI (`ldai_openai`) and LangChain (`ldai_langchain`). The LangChain Python package exposes runner classes (`LangChainModelRunner`, `LangChainAgentRunner`, `LangGraphAgentGraphRunner`) and helper functions (`get_ai_metrics_from_response`, `get_ai_usage_from_response`) — **not** a single-node callback handler. For single-node LangChain calls, use manual tracker wiring with `response.usage_metadata`. See [sdk-ai-tracker-patterns.md](sdk-ai-tracker-patterns.md) for the full matrix and the manual snippet.
+Python currently ships helper packages for OpenAI (`ldai_openai`) and LangChain (`ldai_langchain`). The LangChain Python package exposes `create_langchain_model(config)` (builds a LangChain chat model from the AI Config, forwarding every variation parameter and mapping LD provider names to LangChain equivalents), `convert_messages_to_langchain`, and `get_ai_metrics_from_response` — the same package covers LangGraph. Use `create_langchain_model(config)` + `track_metrics_of_async(lambda: llm.ainvoke(messages), get_ai_metrics_from_response)` as the canonical single-call pattern. See [langchain-tracking.md](../../aiconfig-ai-metrics/references/langchain-tracking.md) for both LangChain and LangGraph patterns and [sdk-ai-tracker-patterns.md](sdk-ai-tracker-patterns.md) for the full tracker-method matrix.
 
 ## Phase 1 output format
 

skills/ai-configs/aiconfig-online-evals/SKILL.md

@@ -330,6 +330,9 @@ async def async_main():
         print("Judge results:")
         print(json.dumps(results_to_display, indent=2, default=str))
 
+    # Always flush events before closing — trailing events are at risk of being
+    # lost otherwise, in short-lived scripts and long-running services alike.
+    ldclient.get().flush()
     ldclient.get().close()
 ```
 
@@ -385,6 +388,9 @@ async def async_main():
     print("Judge Response:")
     print(json.dumps(judge_response.to_dict(), indent=2, default=str))
 
+    # Always flush events before closing — trailing events are at risk of being
+    # lost otherwise, in short-lived scripts and long-running services alike.
+    ldclient.get().flush()
     ldclient.get().close()
 ```
 

skills/ai-configs/aiconfig-projects/SKILL.md

@@ -138,9 +138,16 @@ After creating the project, verify it works:
 2. **Test SDK integration:**
    Run a quick verification to ensure the SDK key works:
    ```python
-   from ldclient import set_config, Config
-   set_config(Config("{sdk_key}"))
+   import ldclient
+   from ldclient.config import Config
+
+   ldclient.set_config(Config("{sdk_key}"))
    # SDK initializes successfully
+
+   # Always flush events before closing — trailing events are at risk of being
+   # lost otherwise, in short-lived scripts and long-running services alike.
+   ldclient.get().flush()
+   ldclient.get().close()
    ```
 
 3. **Report results:**

skills/ai-configs/aiconfig-tools/SKILL.md

@@ -95,6 +95,130 @@ If you observe a UI-clear bug where attaching tools wipes other fields, **do not
 - Tool attached to variation
 - Flag any issues
 
+## Per-provider schema at the call site
+
+LaunchDarkly stores the tool schema once — the flat `{type, name, description, parameters}` shape you passed to `create-ai-tool`. Your application reads it back via `config.model.parameters.tools` (completion mode) or `agent_config.model.parameters.tools` (agent mode), then converts to the shape the provider SDK expects. LaunchDarkly never makes the provider call; your code does. The handlers that implement each tool also stay in application code — LaunchDarkly stores the schema, your application owns the behavior.
+
+| Provider / framework | Target shape | Where it goes on the call |
+|---|---|---|
+| OpenAI Chat Completions (direct SDK) | `{type: "function", function: {name, description, parameters}}` | top-level `tools=[...]` |
+| Anthropic direct SDK | `{name, description, input_schema}` — rename `parameters` → `input_schema` | top-level `tools=[...]` |
+| Bedrock Converse | `{toolSpec: {name, description, inputSchema: {json: parameters}}}` | inside `toolConfig.tools=[...]` |
+| Gemini (`google-genai`) | `{function_declarations: [{name, description, parameters}]}` (Python) / `{functionDeclarations: [...]}` (Node) | `GenerateContentConfig.tools=[...]` |
+| OpenAI Responses API | LaunchDarkly's flat shape passes through unchanged | top-level `tools=[...]` |
+| LangChain / LangGraph | `LangChainProvider.createLangChainModel(config)` and pass `ai_config.tools` (or your own `StructuredTool` list) into `bind_tools(...)` / `create_react_agent(tools=[...])` | framework-native; no per-call conversion |
+| Strands Agents | LaunchDarkly's flat shape; drop `parameters.tools` before passing params to the Strands model class (`AnthropicModel`, `OpenAIModel`) — Python `@tool`-decorated callables stay in code | `Agent(tools=[...])` constructor; no per-call conversion |
+
+Minimal conversion snippets (Python):
+
+```python
+ld_tools = (ai_config.model.to_dict().get("parameters") or {}).get("tools", []) or []
+
+# OpenAI Chat Completions
+openai_tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": t["name"],
+            "description": t.get("description", ""),
+            "parameters": t.get("parameters", {"type": "object", "properties": {}}),
+        },
+    }
+    for t in ld_tools
+]
+
+# Anthropic
+anthropic_tools = [
+    {
+        "name": t["name"],
+        "description": t.get("description", ""),
+        "input_schema": t.get("parameters", {"type": "object", "properties": {}}),
+    }
+    for t in ld_tools
+]
+
+# Bedrock Converse
+bedrock_tool_config = {
+    "tools": [
+        {
+            "toolSpec": {
+                "name": t["name"],
+                "description": t.get("description", ""),
+                "inputSchema": {"json": t.get("parameters", {"type": "object", "properties": {}})},
+            }
+        }
+        for t in ld_tools
+    ]
+}
+
+# Gemini
+gemini_tools = [
+    {
+        "function_declarations": [
+            {
+                "name": t["name"],
+                "description": t.get("description", ""),
+                "parameters": t.get("parameters", {"type": "object", "properties": {}}),
+            }
+            for t in ld_tools
+        ]
+    }
+] if ld_tools else []
+```
+
+## Agent loop with tool calls
+
+An agent that uses tools runs a short loop: call the provider, dispatch any tool calls, loop again, stop when the provider returns a final answer. Three rules apply regardless of provider:
+
+1. **Bound the loop.** `MAX_STEPS = 5` is a safe default. A runaway tool loop is almost always a prompt or schema bug, not a case that needs 50 iterations.
+2. **Track every tool invocation.** Call `tracker.track_tool_call(tool_name)` / `tracker.trackToolCall(toolName)` for each tool the agent actually executes. This is what the Monitoring tab counts as tool usage.
+3. **Break on the provider's "no more tool calls" signal.** The exact signal differs per provider: OpenAI Chat Completions → `choice.finish_reason != "tool_calls"`; Anthropic → `response.stop_reason != "tool_use"`; Bedrock Converse → `response["stopReason"] != "tool_use"`; Gemini → `response.function_calls` empty; OpenAI Responses API → no `function_call` items in `response.output`.
+
+Skeleton (Python, Anthropic — the other providers follow the same shape with their own stop-reason check and tool-result formatting):
+
+```python
+messages = [{"role": "user", "content": initial_input}]
+MAX_STEPS = 5
+for _ in range(MAX_STEPS):
+    response = tracker.track_metrics_of(
+        lambda: anthropic_client.messages.create(
+            model=agent.model.name,
+            system=agent.instructions,
+            messages=messages,
+            tools=anthropic_tools,
+            **params,
+        ),
+        anthropic_metrics,
+    )
+    if response.stop_reason != "tool_use":
+        break
+
+    messages.append({"role": "assistant", "content": response.content})
+
+    tool_results = []
+    for block in response.content:
+        if block.type != "tool_use":
+            continue
+        if block.name not in tool_handlers:
+            raise ValueError(f"Unknown tool: {block.name}")
+        result = tool_handlers[block.name](**block.input)
+        tracker.track_tool_call(block.name)
+        tool_results.append({
+            "type": "tool_result",
+            "tool_use_id": block.id,
+            "content": result,
+        })
+    messages.append({"role": "user", "content": tool_results})
+```
+
+Per-provider tool-call payload shapes live in the `aiconfig-ai-metrics` references:
+
+- [openai-tracking.md](../aiconfig-ai-metrics/references/openai-tracking.md) — Chat Completions + Responses API
+- [anthropic-tracking.md](../aiconfig-ai-metrics/references/anthropic-tracking.md) — `tool_use` blocks and `tool_result` payloads
+- [bedrock-tracking.md](../aiconfig-ai-metrics/references/bedrock-tracking.md) — `toolUse` / `toolResult` Converse format
+- [gemini-tracking.md](../aiconfig-ai-metrics/references/gemini-tracking.md) — `functionCalls` / `functionResponse` parts
+- [langchain-tracking.md](../aiconfig-ai-metrics/references/langchain-tracking.md) — LangGraph tool loop inherits from `create_react_agent`
+
 ## Orchestrator Note
 
 LangGraph, CrewAI, and AutoGen often generate schemas from function definitions. You still need to create tools in LaunchDarkly and attach keys to variations so the SDK knows what's available.

skills/ai-configs/aiconfig-update/SKILL.md

@@ -60,7 +60,7 @@ Then use `get-ai-config` to review the full detail:
 **Update a variation** -- Use `update-ai-config-variation`:
 - Switch model (provide new `modelConfigKey` and `modelName`)
 - Change instructions or messages
-- Tune parameters (temperature, maxTokens, etc.)
+- Tune parameters (temperature, max_tokens, etc.)
 - Attach or detach tools via the parameters object
 
 **Archive a config** -- Use `update-ai-config` with `archived: true`. Archiving is the **preferred** way to retire a config:

skills/ai-configs/aiconfig-variations/SKILL.md

@@ -46,7 +46,7 @@ What's the problem? Cost, quality, speed, accuracy? How will you measure success
 |------|--------------|
 | Reduce cost | Cheaper model (e.g., `gpt-4o-mini`) |
 | Improve quality | Better model or more detailed prompt |
-| Reduce latency | Faster model, lower `maxTokens` |
+| Reduce latency | Faster model, lower `max_tokens` |
 | Increase accuracy | Different model family (Claude vs GPT-4) |
 
 ### Step 3: Create Variations (Recommended: Clone with Overrides)