mirror of
https://github.com/langfuse/skills.git
synced 2026-09-14 20:00:05 +08:00
fix(langfuse): verify attributes during SDK upgrades
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "langfuse",
|
||||
"description": "Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
|
||||
"version": "1.5.3",
|
||||
"version": "1.6.0",
|
||||
"author": {
|
||||
"name": "Langfuse",
|
||||
"email": "support@langfuse.com"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "langfuse",
|
||||
"version": "1.5.0",
|
||||
"version": "1.6.0",
|
||||
"description": "Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
|
||||
"author": {
|
||||
"name": "Langfuse",
|
||||
@@ -33,7 +33,7 @@
|
||||
"Set up Langfuse tracing in this app.",
|
||||
"Inspect Langfuse traces for recent failures.",
|
||||
"Migrate prompts into Langfuse.",
|
||||
"Prepare this application and its Langfuse project for the v4 platform migration."
|
||||
"Upgrade this application's Langfuse SDKs and instrumentation, verify session and other correlating attributes on every applicable child observation, and prepare its Langfuse project for the v4 platform migration."
|
||||
],
|
||||
"websiteURL": "https://langfuse.com",
|
||||
"privacyPolicyURL": "https://langfuse.com/privacy",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "langfuse",
|
||||
"displayName": "Langfuse",
|
||||
"version": "1.5.3",
|
||||
"version": "1.6.0",
|
||||
"description": "Skills for working with Langfuse — the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
|
||||
"author": {
|
||||
"name": "Langfuse",
|
||||
|
||||
@@ -36,8 +36,9 @@ Follow these principles for ALL Langfuse work:
|
||||
- creating a prompt or changing any part of an existing prompt, including small edits and debugging/tuning: references/prompt-engineering.md
|
||||
- capturing user feedback (thumbs, ratings, implicit signals) as scores on traces: references/user-feedback.md
|
||||
- further tips on using the Langfuse CLI: references/cli.md
|
||||
- upgrading or migrating Langfuse SDKs and preserving application instrumentation attributes: references/sdk-upgrade.md
|
||||
- upgrading legacy trace-level or dataset-item evaluators to observation-level or experiment evaluators: references/trace-evaluator-upgrade.md. Use the [evaluator migration guide](https://langfuse.com/faq/all/llm-as-a-judge-migration) as the primary reference.
|
||||
- preparing an application and Langfuse project for the v4 platform migration: references/v4-project-migration.md
|
||||
- preparing a Langfuse project for the v4 platform migration: references/v4-project-migration.md
|
||||
- judge calibration (LLM-as-a-Judge reliability, simple accuracy checks, advanced split-based validation, confusion matrices, and metric ingestion): references/judge-calibration.md
|
||||
- systematic error analysis — reading traces, building failure taxonomy, deciding what to fix: references/error-analysis.md
|
||||
- setting up CI/CD experiment gates with `langfuse/experiment-action`: references/ci-cd.md
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: langfuse-sdk-upgrade
|
||||
description: Upgrade Langfuse SDKs and application instrumentation while preserving trace attributes across observations. Use for Python or JS/TS SDK migrations, including the application side of a v4 platform migration.
|
||||
metadata:
|
||||
required_access:
|
||||
- CODEBASE
|
||||
- LANGFUSE_PROJECT_SCRIPT
|
||||
---
|
||||
|
||||
# Langfuse SDK upgrade
|
||||
|
||||
## Sources of truth
|
||||
|
||||
Determine every installed Langfuse SDK major version, then fetch each applicable leaf guide in full before editing:
|
||||
|
||||
- [SDK upgrade paths](https://langfuse.com/docs/observability/sdk/upgrade-path)
|
||||
- [Python v3 to v4](https://langfuse.com/docs/observability/sdk/upgrade-path/python-v3-to-v4)
|
||||
- [JS/TS v4 to v5](https://langfuse.com/docs/observability/sdk/upgrade-path/js-v4-to-v5)
|
||||
- [Instrumentation and attribute propagation](https://langfuse.com/docs/observability/sdk/instrumentation#add-attributes)
|
||||
- [Sessions and session-level metrics](https://langfuse.com/docs/observability/features/sessions)
|
||||
- [Direct OpenTelemetry ingestion](https://langfuse.com/integrations/native/opentelemetry)
|
||||
|
||||
Follow every intermediate guide when the installed version is more than one major behind. Use the current docs for implementation details; do not copy examples from this file.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Inventory every SDK, integration package, direct OpenTelemetry exporter, initialization site, instrumentation wrapper, lockfile, worker, script, and test that can emit Langfuse data.
|
||||
2. Find every source of correlating attributes, including `session_id`/`sessionId`, `user_id`/`userId`, tags, metadata, version, environment, and trace name. Search for the values and surrounding application concepts, not only removed SDK method names.
|
||||
3. Apply every relevant item from the exact version-specific guides. Preserve each correlating attribute by establishing its documented propagation scope before any observation-producing call that must inherit it.
|
||||
4. For sessions, propagate the session ID early enough that the root and every applicable child observation—including cost-bearing generations—receive the same value. For work crossing service boundaries, follow the current distributed-tracing and baggage guidance rather than assuming in-process context crosses the boundary.
|
||||
5. Run focused format, type, lint, and test checks, then exercise each changed ingestion path with representative application behavior.
|
||||
6. Fetch the emitted trace and verify the expected correlating attributes on the root and every applicable child observation. For a session path, also confirm the observations are grouped into the intended session and its cost includes the cost-bearing children.
|
||||
|
||||
Do not mark the upgrade ready when only dependency or compile-time checks passed. If runtime execution or trace inspection is unavailable, report the exact verification as blocked.
|
||||
|
||||
## Completion report
|
||||
|
||||
Report the versions before and after, changed instrumentation paths, attribute sources and propagation scopes, validation performed, inspected trace or session, and any remaining blocked verification.
|
||||
@@ -35,7 +35,8 @@ Discover unstable schemas before use.
|
||||
|
||||
- Inventory every Langfuse SDK, integration, OTEL exporter, initialization site, lockfile, raw request, generated client, script, notebook, and CI call.
|
||||
- Upgrade to the latest stable SDK major required by the current docs and apply every applicable breaking change. Record both declared and resolved versions; update an existing lockfile.
|
||||
- Put overall input/output on the root observation and propagate every trace attribute needed to filter or aggregate observations.
|
||||
- Find every source of correlating attributes, including session and user IDs, tags, metadata, version, environment, and trace name; do not search only for removed SDK methods.
|
||||
- Put overall input/output on the root observation. Establish the documented propagation scope before observation-producing calls so every applicable child receives the attributes needed for filtering and aggregation, including the session ID on cost-bearing generations.
|
||||
- For raw `/api/public/ingestion`, use the current Langfuse SDK in Python or JS/TS. For other languages, use the language's native OpenTelemetry API and follow the custom-ingestion guide.
|
||||
- When replacing synchronous ingestion, assess buffering, retries, flushing, shutdown, and error propagation. Do not claim identical delivery semantics without verification.
|
||||
- For other deprecated APIs, migrate the path, parameters, pagination, filters, field groups, response parsing, and downstream consumer together using the deprecated-API guide.
|
||||
@@ -68,6 +69,7 @@ Discover unstable schemas before use.
|
||||
|
||||
- Test applicable hierarchy, root input/output, propagated attributes, public/release/environment behavior, API pagination and parsing, delivery semantics, and absence of deprecated calls.
|
||||
- Before production cutover, send representative traces from the migrated instrumentation to a non-production Langfuse project and inspect the resulting observations there. Mocked tests do not verify backend ingestion or project behavior.
|
||||
- On a session path, confirm the root and every applicable child observation carry the intended session ID and that session cost includes the cost-bearing children. Keep SDK/instrumentation readiness blocked when emitted-trace inspection is unavailable.
|
||||
- Re-read rules and integrations after writes. Preserve disabled legacy rules for rollback; never claim completion without checking the Evaluators UI on the target host for legacy rows.
|
||||
- The readiness report must contain exactly these seven rows, each marked `ready`, `changed`, `manual action`, or `blocked`: project access; SDK/instrumentation; trace evaluators; dataset evaluators; direct APIs; exports; verification/rollback.
|
||||
- In evaluator rows, separate contracts verified against project data from targets suggested only by code inspection. For every row not marked `ready`, include the blocker, next action, and a direct UI link when applicable.
|
||||
|
||||
Reference in New Issue
Block a user