2026-02-03 07:43:05 -08:00
# LaunchDarkly Agent Skills
2026-05-21 17:04:50 -07:00
LaunchDarkly's public collection of agent skills and playbooks. These skills encode repeatable workflows for working with LaunchDarkly, so coding agents can execute common tasks safely and consistently.
2026-02-03 07:43:05 -08:00
## What Is This Repo?
Agent Skills are modular, text-based playbooks that teach an agent how to perform a workflow. This repo is designed to be a public, open-source home for LaunchDarkly skills and to align with the emerging Agent Skills Open Standard.
## Available Skills
2026-02-06 14:08:30 -08:00
### Feature Flags
2026-02-03 07:43:05 -08:00
| Skill | Description |
|-------|-------------|
2026-06-04 11:37:29 -04:00
| `feature-flags/launchdarkly-flag-command` | Resolve `/flag` style requests into fast flag lookup, detail, and disambiguation |
Add should-flag-change advisory skill + eval suite (#101)
* Add should-flag-change advisory skill + eval suite
New read-only skill that decides whether a code change should ship behind a
LaunchDarkly feature flag. Invoked ad hoc or in CI on a PR: it reads the diff
and surrounding code, weighs a decision framework (favoring flags for
user-facing and risky changes, weighting false negatives over false positives),
and ends with a structured recommend-flag verdict. It never creates or
modifies flags.
- skills/feature-flags/should-flag-change: SKILL.md (read-only scope boundary,
confusion-matrix decision framework, recommend-flag verdict contract) plus
README and marketplace.json
- evals: recommend-flag tool definition + mock, git_diff provider var injected
as a <git_diff> tag, and a two-tier suite (judgment vs agentic providers,
targeted per fixture) with 5 labeled fixtures
- evals/scripts/build-pr-fixture.js: build a fixture from a real PR via
gh pr diff + refs/pull/N/head, bootstrapping the label from LD SDK call sites
- wiring: package.json scripts, _manifest.js, README skill list, skills.json
* Add already-gated fixtures, PR-fixture tooling, and templating fix
Extends the should-flag-change eval suite and its fixture tooling based on
testing against real pull requests.
- promptfooconfig.yaml: add three synthetic fixtures for the "already gated"
case the suite didn't cover — a change that ships behind a flag should get
recommend: false ("already handled"), and its ungated mirror should get
recommend: true. Uses public SDK idioms; no proprietary source.
- provider: fix a Nunjucks var-render crash on diffs containing `{{ ... }}`
(JSX props, Go templates). Fixtures may wrap such content in a `{% raw %}`
block; stripRawWrapper removes it before the agent sees the diff.
- build-pr-fixture.js: read changed files via the GitHub contents API instead
of fetching refs/pull/N/head (avoids cloning a large external repo); add
--flag-pattern / --flag-token / --flag-file-pattern to teach it a codebase's
flag conventions at call time, and --counterfactual to build positive-recall
fixtures by stripping the flag gate. Kept repo-agnostic — no baked-in
conventions.
- tests: unit-test stripRawWrapper and the builder's pure functions
(addedLines, detectLdUsage, splitDiffByFile, dropFilesFromDiff,
extractFlagTokens, stripFlagGate).
* Combine should-flag-change and flag-and-release-change onto one branch
Bring the flag-and-release-change skill (originally PR #102) onto the
should-flag-change branch (#101) so the advisory "decide" step and the
"apply" step ship together.
Only the content of #102's two flag-and-release commits is included --
not that branch's unrelated observability skills (which sit on an
unmerged #99 commit). README skill list updated and skills.json
regenerated from the catalog script.
* Improve both flag skills with ideas from auto-factory and gonfalon
should-flag-change:
- Ancestor-gate analysis: detect an enclosing flag the change already
sits behind, and judge on its rollout state.
- Prerequisite/dependency signal as a reason to flag.
- Optional risk/blast-radius field on the verdict, orthogonal to
confidence (added to the eval tool schema, additive/non-breaking).
- Refactor-that-changes-a-contract case; net-new-vs-incremental
measurability nuance; unverified-claim confidence discipline.
flag-and-release-change:
- Capture and honor human release intent (release / hold / notBefore /
segment / prerequisite), fail-closed; three-layer precedence
(intent -> override -> policy -> default).
- "The deploy is not the release" framing.
- Fail-closed on non-409 flag-creation errors.
- Paired flag-on/flag-off tests run to green before push.
- Metric-adequacy check for guarded policies; prerequisite coupling.
- Off-path-invokes-no-new-code invariant; edge cases for duplicate
config, registering before the PR exists, and double-toggling.
Sources: launchdarkly-labs/launchdarkly-auto-factory (AI-config prompts,
ADRs) and launchdarkly/gonfalon .agents/skills. Portable-safe only --
repo-specific specifics (env matrix, release tags, context kinds) left
out as the per-repo customization surface.
* Extract flag-release skill; slim flag-and-release-change to an orchestrator
Decompose the "apply" half into composable, harness-callable units,
matching gonfalon's planning/add-flag/rollout shape while staying generic:
- New skill feature-flags/flag-release: records a flag's automated rollout
for a PR (match-release-policies preview, release intent + fail-closed
hold, precedence, prerequisites, metric-adequacy). Operates on an
existing flag; creates no flags and edits no code. auto-release.md moves
here as its core reference.
- flag-and-release-change becomes a thin portable orchestrator that
composes should-flag-change -> launchdarkly-flag-create -> flag-release,
owning only the PR workflow (clone/diff/push) and plan->implement
sequencing. Notes that an automation harness can bypass it and invoke
the three atoms directly.
Repo-specific values (env matrix, release tags, targeting context) remain
out of these generic skills -- that is the AgentControl customization seam.
* Add eval coverage for the new flag decision/release logic
should-flag-change:
- New fixtures: ancestor-gate (agentic; explores to an enclosing flag not
in the diff) and prerequisite-dependency (judgment).
- risk-calibration asserts folded into the auth (expect high) and
docs-only (expect low/absent) fixtures. All pass.
New flag-release suite (adds match-release-policies / list-release-policies
/ create-automated-rollout-config tool defs + mocks; environmentKey mock
replacement):
- Release-on-merge happy path: previews the policy, records both envs. PASS.
- Hold intent: KNOWN-RED tracked signal. sonnet-4-6 records the held env as
`policy` (wrongly believing policy holds); opus omits it correctly. The
assertion is correct (a held env must be omitted); do not weaken it.
New flag-and-release-change suite (both PASS):
- Plan phase is side-effect-free (no mutating MCP calls during planning).
- Fails closed: a create-flag 403 (via the restricted-project mock hook)
stops the run before any release is recorded.
Wiring: manifest + package.json scripts for both new suites; READMEs for
flag-release and flag-and-release-change. Sharpened flag-release's
hold-honoring wording (helps stronger models; sonnet gap tracked above).
90/90 unit tests pass.
* Address review feedback on the should-flag-change PR
1. Catalog leaked third-party skills from evals/node_modules (dotenv,
dotenvx, playwright) into the public skills.json, and CI's
generate_catalog.py --check failed on the mismatch. Add node_modules to
the validator's EXCLUDED_DIRS (generate_catalog reuses it) and regenerate
skills.json — back to 44 skills, --check clean.
2. Enforce the recommend-flag contract. A new suite-wide verdict_contract
assertion requires the tool to be called EXACTLY once and as the final
tool call; a run that calls it early, twice, or keeps working afterward
now fails instead of passing on the first call's boolean.
3. Guard the read-only safety contract. A new read_only_guard assertion bans
Edit/Write and mutating shell commands (redirects, rm/mv/cp, git
commit/push, package installs) across the suite, so the agentic tier can't
modify code or state and still pass. Previously only flag-mutating MCP
tools were checked.
Both new assertions pass on all 10 should-flag-change fixtures.
* Declare js-yaml as a direct dependency of the eval tooling
build-pr-fixture.js required js-yaml via an explicit ../node_modules path,
but js-yaml was only present transitively (via promptfoo). A promptfoo bump
that dropped or relocated it would break the fixture builder and its unit
test. Add js-yaml (^4.1.1, already the resolved version) to evals
devDependencies, update the lockfile, and use a plain require. Also drop the
now-unused node:path import.
* Enrich should-flag-change + add targeting context-availability reference (#108)
* feat(feature-flags): enrich should-flag-change and add context-availability reference
Layer portable, de-LaunchDarkly-internal lessons from our flag-planning
skills onto the public feature-flag skills.
should-flag-change:
- Add an explicit, named user-observability test as a gate before any
`recommend: false` verdict.
- Add a `verdict` field (suggested | already-flagged | not-suited) to the
recommend-flag output, keeping already-flagged (protected by an existing or
ancestor gate) distinct from not-suited (nothing to flag). `recommend` stays
the boolean a CI check keys on.
- Generalize a decision-posture tie-breaker (conservative vs. low-overhead)
for genuinely balanced calls, without internal "dogfood" wording.
Add a new SDK-agnostic targeting context-availability reference: match the
context kind to the surface where the flag is read (server/client/anonymous),
key vs attribute, and rollout bucketing. Wire it into flag-targeting and
flag-create, and reference it from should-flag-change.
Bump versions and regenerate skills.json.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(feature-flags): cover verdict taxonomy and targeting-context reasoning
- should-flag-change suite: assert the verdict field across the already-flagged,
not-suited, and suggested fixtures, and validate the enum plus recommend/verdict
agreement in the ambiguous fixture.
- flag-create suite: add a fixture where a client-side flag is asked to target a
server-only signal, asserting the agent flags that browser context can't carry
it and suggests an available approach.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Let the eval gate tolerate documented known-red fixtures
The flag-release suite carries an intentionally-red fixture (Sonnet mishandles
a held production environment) that must stay red until the skill or model
closes the gap. Previously any suite below 75% failed the CI job, so this
tracked signal blocked merge.
Declare such fixtures via a knownRed allowlist in _manifest.js (matched by the
fixture description, which promptfoo stores under testCase.description).
aggregate.js now keeps the honest score/badge red while gating on a gateScore
that excludes known-red fixtures, so only unexpected failures fail CI. A
known-red fixture that starts passing is surfaced so its entry can be removed.
* Revert "Let the eval gate tolerate documented known-red fixtures"
This reverts commit 5e870937e482f67dafba600b00dbce83f427d305.
* Restore known-red eval gate tolerance for flag-release hold fixture
Sonnet still records held production as `policy` (auto-releases on merge),
so the intentionally-red hold-intent fixture keeps failing the 75% suite
gate. Re-apply the gateScore allowlist so the tracked signal stays visible
without blocking CI.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(flag-release): make held-environment handling a mechanical step
The hold-intent fixture failed because the model maps "hold production" to
`releaseType: policy` — the tool describes `policy` as "defer to the
environment's release policy on merge", and "defer" reads like "hold". The
skill fought this with prose caveats, which the authors noted hadn't closed
the gap.
Restructure the Implement phase around a forced RELEASE-vs-HOLD bucket sort:
the environments array is built only from the RELEASE bucket, HOLD means
absence from the array, and the "policy defers to the policy, not to you"
misreading is called out at the point of the call. Remove the known-red gate
allowlist so the fixture counts again and the skill fix carries the suite.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(flag-release): de-trap the rollout tool description; name the hold-field hallucination
Root-cause dig on the hold-intent fixture: the model doesn't fail from a bare
reasoning gap — the eval's mock tool description diverged from the real hosted
tool and fed it a trap. The real create-automated-rollout-config says policy
"waits until merge, then performs the release"; the mock said policy "defers to
the release policy", and "defer" reads as "hold", steering the model to keep a
held env as policy. With louder omit-prose the model instead fabricated a
holdUntil field — which the mock silently accepts (.passthrough), so it never
learns the field is invalid.
- Make the mock tool description faithful to the real tool: drop "defer", say
"waits then performs", and state explicitly there is NO hold/notBefore/date
field (omit the env to hold it; unknown fields are rejected).
- Skill: name the exact tell — wanting holdUntil/notBefore/hold (or keeping an
env as policy "so it waits") means the env is HOLD; drop the entry, don't
invent a field. Date/reason go in the report.
- Keep the known-red gate as a backstop since the 2-test suite is a binary gate
that can flake even when the fix lands; the honest score still shows red if it
regresses.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(evals): drop known-red backstop now flag-release hold fixture passes
The de-trapped tool description + skill hallucination-naming flips Sonnet to the
correct behavior: it omits the held environment (recorded [{staging,simple}],
production absent) and scores 0.83 > 0.75. With a real fix in place the known-red
gate tolerance would only mask a future regression, so remove it and let the
fixture gate on its own.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(flag-release): broaden hold coverage and lock in the no-fabricated-field fix
Expand the flag-release suite from 2 to 5 fixtures, all exercising the
release-vs-hold sort the earlier fix addressed:
- add a no_fabricated_fields assertion (rejects holdUntil/notBefore/hold on any
recorded env) to the hold fixtures, locking in the regression fix
- multi-env mixed intent (dev+staging release, production hold) — the sort must
scale past two envs
- single-target date hold (the only env is held, so nothing releases)
- fail closed on ambiguous intent (release staging, hold unconfirmed production)
Retire the stale KNOWN-RED comment on the original hold fixture (now a passing
regression guard). More fixtures also stabilize the 75% gate: a single stochastic
miss on a 5-test suite stays green, where on 2 tests it went red.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(flag-and-release-change): cover the orchestrator's decide gate and create step
The orchestrator suite only had plan-side-effect-free and fail-closed fixtures.
Add two more targeting its own distinct, sandbox-reachable decisions (the record
step needs a git push the MCP-only provider can't do, and is covered directly by
the flag-release suite):
- decide gate: a docs-only change is judged not flag-worthy and creates/records
nothing, even when told to "flag and release if warranted"
- create step: an approved change produces exactly one boolean kill-switch,
created OFF, never toggled on by hand (over-flag + created-OFF guards)
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Ramon Niebla <rniebla@launchdarkly.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-21 11:50:05 -07:00
| `feature-flags/should-flag-change` | Advisory, read-only call on whether a code change (diff/PR) should ship behind a feature flag |
2026-02-13 09:53:10 -08:00
| `feature-flags/launchdarkly-flag-discovery` | Audit flags, find stale/launched flags, and assess removal readiness |
| `feature-flags/launchdarkly-flag-create` | Create new feature flags in a way that fits existing codebase patterns |
| `feature-flags/launchdarkly-flag-targeting` | Control targeting, rollouts, rules, and cross-environment config |
2026-02-03 07:43:05 -08:00
| `feature-flags/launchdarkly-flag-cleanup` | Safely remove flags from code using LaunchDarkly as the source of truth |
feat: add launchdarkly-flag-drift skill (#119)
* feat: add launchdarkly-flag-drift skill
Detect and reconcile drift between a feature flag's in-code SDK fallback
default and its LaunchDarkly default rule (fallthrough), updating only the
default argument without removing the flag or changing its evaluation.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(evals): add eval suite for launchdarkly-flag-drift
Add a promptfoo suite covering drift reconciliation, the no-drift
(no code change, no PR) case, and a registry-declared default, asserting
the agent resolves the fallthrough via get-flag and never mutates the flag.
Register the suite in the manifest and add npm scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(flag-drift): check fallthrough across all critical environments
Address review feedback that the skill only reconciled the in-code
default against a single environment. The in-code fallback default is a
single value that must stand in for every environment the build serves,
so the fallthrough is now resolved in each critical environment.
When critical environments agree, that shared value is the expected
default and reconciliation proceeds as before. When they disagree (e.g.
EU serves true but Federal serves false), the skill surfaces the
per-environment divergence and confirms which environment is
authoritative instead of silently reconciling to one. Updates the
prerequisites, workflow, edge cases, summary fields, and PR template
accordingly.
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
* test(evals): cover cross-environment fallthrough divergence for flag-drift
Add a divergence eval where one build serves both production and federal
environments that disagree on the fallthrough. The mock now returns the
opposite default (variation 0) for any /federal/i environment key, so the
skill must query each critical environment and surface the divergence
rather than blindly reconciling. Adds mock unit tests for the hook.
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
2026-07-28 12:30:54 -07:00
| `feature-flags/launchdarkly-flag-drift` | Detect and reconcile drift between an in-code SDK fallback default and the LaunchDarkly default rule |
2026-05-07 11:44:44 -07:00
| `feature-flags/launchdarkly-guarded-rollout` | Configure guarded rollouts with progressive traffic, metric monitoring, and rollback |
Add should-flag-change advisory skill + eval suite (#101)
* Add should-flag-change advisory skill + eval suite
New read-only skill that decides whether a code change should ship behind a
LaunchDarkly feature flag. Invoked ad hoc or in CI on a PR: it reads the diff
and surrounding code, weighs a decision framework (favoring flags for
user-facing and risky changes, weighting false negatives over false positives),
and ends with a structured recommend-flag verdict. It never creates or
modifies flags.
- skills/feature-flags/should-flag-change: SKILL.md (read-only scope boundary,
confusion-matrix decision framework, recommend-flag verdict contract) plus
README and marketplace.json
- evals: recommend-flag tool definition + mock, git_diff provider var injected
as a <git_diff> tag, and a two-tier suite (judgment vs agentic providers,
targeted per fixture) with 5 labeled fixtures
- evals/scripts/build-pr-fixture.js: build a fixture from a real PR via
gh pr diff + refs/pull/N/head, bootstrapping the label from LD SDK call sites
- wiring: package.json scripts, _manifest.js, README skill list, skills.json
* Add already-gated fixtures, PR-fixture tooling, and templating fix
Extends the should-flag-change eval suite and its fixture tooling based on
testing against real pull requests.
- promptfooconfig.yaml: add three synthetic fixtures for the "already gated"
case the suite didn't cover — a change that ships behind a flag should get
recommend: false ("already handled"), and its ungated mirror should get
recommend: true. Uses public SDK idioms; no proprietary source.
- provider: fix a Nunjucks var-render crash on diffs containing `{{ ... }}`
(JSX props, Go templates). Fixtures may wrap such content in a `{% raw %}`
block; stripRawWrapper removes it before the agent sees the diff.
- build-pr-fixture.js: read changed files via the GitHub contents API instead
of fetching refs/pull/N/head (avoids cloning a large external repo); add
--flag-pattern / --flag-token / --flag-file-pattern to teach it a codebase's
flag conventions at call time, and --counterfactual to build positive-recall
fixtures by stripping the flag gate. Kept repo-agnostic — no baked-in
conventions.
- tests: unit-test stripRawWrapper and the builder's pure functions
(addedLines, detectLdUsage, splitDiffByFile, dropFilesFromDiff,
extractFlagTokens, stripFlagGate).
* Combine should-flag-change and flag-and-release-change onto one branch
Bring the flag-and-release-change skill (originally PR #102) onto the
should-flag-change branch (#101) so the advisory "decide" step and the
"apply" step ship together.
Only the content of #102's two flag-and-release commits is included --
not that branch's unrelated observability skills (which sit on an
unmerged #99 commit). README skill list updated and skills.json
regenerated from the catalog script.
* Improve both flag skills with ideas from auto-factory and gonfalon
should-flag-change:
- Ancestor-gate analysis: detect an enclosing flag the change already
sits behind, and judge on its rollout state.
- Prerequisite/dependency signal as a reason to flag.
- Optional risk/blast-radius field on the verdict, orthogonal to
confidence (added to the eval tool schema, additive/non-breaking).
- Refactor-that-changes-a-contract case; net-new-vs-incremental
measurability nuance; unverified-claim confidence discipline.
flag-and-release-change:
- Capture and honor human release intent (release / hold / notBefore /
segment / prerequisite), fail-closed; three-layer precedence
(intent -> override -> policy -> default).
- "The deploy is not the release" framing.
- Fail-closed on non-409 flag-creation errors.
- Paired flag-on/flag-off tests run to green before push.
- Metric-adequacy check for guarded policies; prerequisite coupling.
- Off-path-invokes-no-new-code invariant; edge cases for duplicate
config, registering before the PR exists, and double-toggling.
Sources: launchdarkly-labs/launchdarkly-auto-factory (AI-config prompts,
ADRs) and launchdarkly/gonfalon .agents/skills. Portable-safe only --
repo-specific specifics (env matrix, release tags, context kinds) left
out as the per-repo customization surface.
* Extract flag-release skill; slim flag-and-release-change to an orchestrator
Decompose the "apply" half into composable, harness-callable units,
matching gonfalon's planning/add-flag/rollout shape while staying generic:
- New skill feature-flags/flag-release: records a flag's automated rollout
for a PR (match-release-policies preview, release intent + fail-closed
hold, precedence, prerequisites, metric-adequacy). Operates on an
existing flag; creates no flags and edits no code. auto-release.md moves
here as its core reference.
- flag-and-release-change becomes a thin portable orchestrator that
composes should-flag-change -> launchdarkly-flag-create -> flag-release,
owning only the PR workflow (clone/diff/push) and plan->implement
sequencing. Notes that an automation harness can bypass it and invoke
the three atoms directly.
Repo-specific values (env matrix, release tags, targeting context) remain
out of these generic skills -- that is the AgentControl customization seam.
* Add eval coverage for the new flag decision/release logic
should-flag-change:
- New fixtures: ancestor-gate (agentic; explores to an enclosing flag not
in the diff) and prerequisite-dependency (judgment).
- risk-calibration asserts folded into the auth (expect high) and
docs-only (expect low/absent) fixtures. All pass.
New flag-release suite (adds match-release-policies / list-release-policies
/ create-automated-rollout-config tool defs + mocks; environmentKey mock
replacement):
- Release-on-merge happy path: previews the policy, records both envs. PASS.
- Hold intent: KNOWN-RED tracked signal. sonnet-4-6 records the held env as
`policy` (wrongly believing policy holds); opus omits it correctly. The
assertion is correct (a held env must be omitted); do not weaken it.
New flag-and-release-change suite (both PASS):
- Plan phase is side-effect-free (no mutating MCP calls during planning).
- Fails closed: a create-flag 403 (via the restricted-project mock hook)
stops the run before any release is recorded.
Wiring: manifest + package.json scripts for both new suites; READMEs for
flag-release and flag-and-release-change. Sharpened flag-release's
hold-honoring wording (helps stronger models; sonnet gap tracked above).
90/90 unit tests pass.
* Address review feedback on the should-flag-change PR
1. Catalog leaked third-party skills from evals/node_modules (dotenv,
dotenvx, playwright) into the public skills.json, and CI's
generate_catalog.py --check failed on the mismatch. Add node_modules to
the validator's EXCLUDED_DIRS (generate_catalog reuses it) and regenerate
skills.json — back to 44 skills, --check clean.
2. Enforce the recommend-flag contract. A new suite-wide verdict_contract
assertion requires the tool to be called EXACTLY once and as the final
tool call; a run that calls it early, twice, or keeps working afterward
now fails instead of passing on the first call's boolean.
3. Guard the read-only safety contract. A new read_only_guard assertion bans
Edit/Write and mutating shell commands (redirects, rm/mv/cp, git
commit/push, package installs) across the suite, so the agentic tier can't
modify code or state and still pass. Previously only flag-mutating MCP
tools were checked.
Both new assertions pass on all 10 should-flag-change fixtures.
* Declare js-yaml as a direct dependency of the eval tooling
build-pr-fixture.js required js-yaml via an explicit ../node_modules path,
but js-yaml was only present transitively (via promptfoo). A promptfoo bump
that dropped or relocated it would break the fixture builder and its unit
test. Add js-yaml (^4.1.1, already the resolved version) to evals
devDependencies, update the lockfile, and use a plain require. Also drop the
now-unused node:path import.
* Enrich should-flag-change + add targeting context-availability reference (#108)
* feat(feature-flags): enrich should-flag-change and add context-availability reference
Layer portable, de-LaunchDarkly-internal lessons from our flag-planning
skills onto the public feature-flag skills.
should-flag-change:
- Add an explicit, named user-observability test as a gate before any
`recommend: false` verdict.
- Add a `verdict` field (suggested | already-flagged | not-suited) to the
recommend-flag output, keeping already-flagged (protected by an existing or
ancestor gate) distinct from not-suited (nothing to flag). `recommend` stays
the boolean a CI check keys on.
- Generalize a decision-posture tie-breaker (conservative vs. low-overhead)
for genuinely balanced calls, without internal "dogfood" wording.
Add a new SDK-agnostic targeting context-availability reference: match the
context kind to the surface where the flag is read (server/client/anonymous),
key vs attribute, and rollout bucketing. Wire it into flag-targeting and
flag-create, and reference it from should-flag-change.
Bump versions and regenerate skills.json.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(feature-flags): cover verdict taxonomy and targeting-context reasoning
- should-flag-change suite: assert the verdict field across the already-flagged,
not-suited, and suggested fixtures, and validate the enum plus recommend/verdict
agreement in the ambiguous fixture.
- flag-create suite: add a fixture where a client-side flag is asked to target a
server-only signal, asserting the agent flags that browser context can't carry
it and suggests an available approach.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Let the eval gate tolerate documented known-red fixtures
The flag-release suite carries an intentionally-red fixture (Sonnet mishandles
a held production environment) that must stay red until the skill or model
closes the gap. Previously any suite below 75% failed the CI job, so this
tracked signal blocked merge.
Declare such fixtures via a knownRed allowlist in _manifest.js (matched by the
fixture description, which promptfoo stores under testCase.description).
aggregate.js now keeps the honest score/badge red while gating on a gateScore
that excludes known-red fixtures, so only unexpected failures fail CI. A
known-red fixture that starts passing is surfaced so its entry can be removed.
* Revert "Let the eval gate tolerate documented known-red fixtures"
This reverts commit 5e870937e482f67dafba600b00dbce83f427d305.
* Restore known-red eval gate tolerance for flag-release hold fixture
Sonnet still records held production as `policy` (auto-releases on merge),
so the intentionally-red hold-intent fixture keeps failing the 75% suite
gate. Re-apply the gateScore allowlist so the tracked signal stays visible
without blocking CI.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(flag-release): make held-environment handling a mechanical step
The hold-intent fixture failed because the model maps "hold production" to
`releaseType: policy` — the tool describes `policy` as "defer to the
environment's release policy on merge", and "defer" reads like "hold". The
skill fought this with prose caveats, which the authors noted hadn't closed
the gap.
Restructure the Implement phase around a forced RELEASE-vs-HOLD bucket sort:
the environments array is built only from the RELEASE bucket, HOLD means
absence from the array, and the "policy defers to the policy, not to you"
misreading is called out at the point of the call. Remove the known-red gate
allowlist so the fixture counts again and the skill fix carries the suite.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(flag-release): de-trap the rollout tool description; name the hold-field hallucination
Root-cause dig on the hold-intent fixture: the model doesn't fail from a bare
reasoning gap — the eval's mock tool description diverged from the real hosted
tool and fed it a trap. The real create-automated-rollout-config says policy
"waits until merge, then performs the release"; the mock said policy "defers to
the release policy", and "defer" reads as "hold", steering the model to keep a
held env as policy. With louder omit-prose the model instead fabricated a
holdUntil field — which the mock silently accepts (.passthrough), so it never
learns the field is invalid.
- Make the mock tool description faithful to the real tool: drop "defer", say
"waits then performs", and state explicitly there is NO hold/notBefore/date
field (omit the env to hold it; unknown fields are rejected).
- Skill: name the exact tell — wanting holdUntil/notBefore/hold (or keeping an
env as policy "so it waits") means the env is HOLD; drop the entry, don't
invent a field. Date/reason go in the report.
- Keep the known-red gate as a backstop since the 2-test suite is a binary gate
that can flake even when the fix lands; the honest score still shows red if it
regresses.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(evals): drop known-red backstop now flag-release hold fixture passes
The de-trapped tool description + skill hallucination-naming flips Sonnet to the
correct behavior: it omits the held environment (recorded [{staging,simple}],
production absent) and scores 0.83 > 0.75. With a real fix in place the known-red
gate tolerance would only mask a future regression, so remove it and let the
fixture gate on its own.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(flag-release): broaden hold coverage and lock in the no-fabricated-field fix
Expand the flag-release suite from 2 to 5 fixtures, all exercising the
release-vs-hold sort the earlier fix addressed:
- add a no_fabricated_fields assertion (rejects holdUntil/notBefore/hold on any
recorded env) to the hold fixtures, locking in the regression fix
- multi-env mixed intent (dev+staging release, production hold) — the sort must
scale past two envs
- single-target date hold (the only env is held, so nothing releases)
- fail closed on ambiguous intent (release staging, hold unconfirmed production)
Retire the stale KNOWN-RED comment on the original hold fixture (now a passing
regression guard). More fixtures also stabilize the 75% gate: a single stochastic
miss on a 5-test suite stays green, where on 2 tests it went red.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(flag-and-release-change): cover the orchestrator's decide gate and create step
The orchestrator suite only had plan-side-effect-free and fail-closed fixtures.
Add two more targeting its own distinct, sandbox-reachable decisions (the record
step needs a git push the MCP-only provider can't do, and is covered directly by
the flag-release suite):
- decide gate: a docs-only change is judged not flag-worthy and creates/records
nothing, even when told to "flag and release if warranted"
- create step: an approved change produces exactly one boolean kill-switch,
created OFF, never toggled on by hand (over-flag + created-OFF guards)
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Ramon Niebla <rniebla@launchdarkly.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-21 11:50:05 -07:00
| `feature-flags/flag-release` | Record a flag's automated release for a PR, honoring release intent and per-environment release policies |
| `feature-flags/flag-and-release-change` | End-to-end PR orchestrator: decide → create + wire the flag → record its release (composes the skills above) |
2026-08-03 16:35:07 -07:00
| `feature-flags/launchdarkly-flag-qualitative-feedback-setup` | Add a qualitative user feedback widget tied to a flag, adapting to the project's framework and design system |
2026-02-06 14:08:30 -08:00
2026-05-21 17:04:50 -07:00
### AgentControl
2026-02-06 14:08:30 -08:00
| Skill | Description |
|-------|-------------|
2026-05-21 17:04:50 -07:00
| `agentcontrol/configs-create` | Create configs with variations for agent or completion mode |
| `agentcontrol/migrate` | Migrate an app with hardcoded LLM prompts to AgentControl in five stages (extract, wrap, tools, tracking, evals) |
| `agentcontrol/configs-update` | Update and delete configs, manage lifecycle |
| `agentcontrol/configs-variations` | Manage config variations for A/B testing |
2026-05-21 15:47:41 -07:00
| `agentcontrol/tools` | Create and attach tools for function calling |
2026-05-21 17:04:50 -07:00
| `agentcontrol/projects` | Create and manage projects to organize configs |
| `agentcontrol/online-evals` | Attach LLM-as-a-judge evaluators to configs |
| `agentcontrol/configs-targeting` | Configure targeting rules for config rollouts |
| `agentcontrol/snippets` | Create and manage reusable prompt snippets across configs |
2026-05-21 15:47:41 -07:00
| `agentcontrol/agent-graphs` | Create and manage multi-agent graphs with routing and handoffs |
2026-05-07 11:44:44 -07:00
### Experiments
| Skill | Description |
|-------|-------------|
| `experiments/launchdarkly-experiment-setup` | Set up experiments with metrics, treatments, and data collection |
2026-04-15 15:13:48 -05:00
### Metrics
| Skill | Description |
|-------|-------------|
| `metrics/launchdarkly-metric-choose` | Select the right metric type for an experiment |
| `metrics/launchdarkly-metric-create` | Create metrics and instrument tracking events |
| `metrics/launchdarkly-metric-instrument` | Add tracking calls to code for existing metrics |
## Install as a Claude Code Plugin
This repo is a [Claude Code plugin ](https://code.claude.com/docs/en/create-plugins ). Installing it gives you all the skills above plus the LaunchDarkly MCP server.
2026-06-15 16:32:57 -07:00
1. Add this repo as a plugin marketplace in Claude Code:
2026-04-15 15:13:48 -05:00
```
2026-06-15 16:32:57 -07:00
/plugin marketplace add launchdarkly/ai-tooling
` ``
2. Install the plugin:
` ``
/plugin install launchdarkly@launchdarkly-ai-tooling
2026-04-15 15:13:48 -05:00
` ``
3. Authenticate the LaunchDarkly MCP server when prompted with your [API access token](https://docs.launchdarkly.com/home/account/api).
Once installed, skills are available as ` /launchdarkly:<skill-name>` across all your projects, and the MCP server can read and modify your flags directly.
2026-02-06 14:08:30 -08:00
2026-04-15 12:40:36 -07:00
### Onboarding
| Skill | Description |
|-------|-------------|
2026-09-03 06:27:19 -07:00
| ` onboarding` | End-to-end LaunchDarkly setup: quiet scripted flow, SDK install, first flag with a live reveal, MCP offered afterwards |
2026-04-15 12:40:36 -07:00
| ` onboarding/mcp-configure` | Configure the LaunchDarkly hosted MCP server (OAuth, no API keys needed) |
| ` onboarding/sdk-install` | Install and initialize the correct SDK via detect, plan, and apply sub-steps |
2026-02-19 06:31:44 -08:00
## Install as a Cursor Plugin
This repo is a [Cursor plugin](https://cursor.com/docs/plugins/building). Installing it gives you all the skills above plus the LaunchDarkly MCP server, so the agent can read and modify your flags directly.
1. Open Cursor and go to **Settings > Plugins**.
2. Search for **LaunchDarkly** in the marketplace, or install from the repo URL:
` ``
2026-04-15 15:13:48 -05:00
https://github.com/launchdarkly/ai-tooling
2026-02-19 06:31:44 -08:00
` ``
Once installed, the skills and MCP server are available across all your projects.
2026-02-03 07:43:05 -08:00
## Quick Start (Local)
` ``bash
# Clone the repo
2026-04-15 15:13:48 -05:00
git clone https://github.com/launchdarkly/ai-tooling.git
cd ai-tooling
2026-02-03 07:43:05 -08:00
# If your agent supports skills.sh installs:
2026-04-15 15:13:48 -05:00
npx skills add launchdarkly/ai-tooling
2026-02-03 07:43:05 -08:00
# Or manually copy a skill into your agent's skills path:
cp -r skills/feature-flags/launchdarkly-flag-cleanup <your-agent-skills-dir>/
2026-04-16 12:09:10 -04:00
2026-02-03 07:43:05 -08:00
` ``
Then ask your agent something like:
2026-02-13 09:53:10 -08:00
` ``
Which feature flags are stale and should be cleaned up?
` ``
` ``
Create a feature flag for the new checkout flow
` ``
` ``
Roll out dark-mode to 25% of users in production
` ``
2026-02-03 07:43:05 -08:00
` ``
Remove the ` new-checkout-flow` feature flag from this codebase
` ``
## Install via skills.sh CLI
` ``bash
npx skills add <owner/repo>
` ``
## Contributing
See ` CONTRIBUTING.md` for how to add new skills and the conventions we follow.
## License
Apache-2.0
