fix(langfuse): verify attributes during SDK upgrades

This commit is contained in:
Hassieb Pakzad
2026-08-20 18:10:11 +02:00
parent 0f9a20a874
commit c527ffae3f
6 changed files with 47 additions and 6 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "langfuse",
"description": "Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
"version": "1.5.3",
"version": "1.6.0",
"author": {
"name": "Langfuse",
"email": "support@langfuse.com"
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "langfuse",
"version": "1.5.0",
"version": "1.6.0",
"description": "Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
"author": {
"name": "Langfuse",
@@ -33,7 +33,7 @@
"Set up Langfuse tracing in this app.",
"Inspect Langfuse traces for recent failures.",
"Migrate prompts into Langfuse.",
"Prepare this application and its Langfuse project for the v4 platform migration."
"Upgrade this application's Langfuse SDKs and instrumentation, verify session and other correlating attributes on every applicable child observation, and prepare its Langfuse project for the v4 platform migration."
],
"websiteURL": "https://langfuse.com",
"privacyPolicyURL": "https://langfuse.com/privacy",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "langfuse",
"displayName": "Langfuse",
"version": "1.5.3",
"version": "1.6.0",
"description": "Skills for working with Langfuse — the open-source LLM engineering platform for tracing, prompt management, and evaluation.",
"author": {
"name": "Langfuse",
+2 -1
View File
@@ -36,8 +36,9 @@ Follow these principles for ALL Langfuse work:
- creating a prompt or changing any part of an existing prompt, including small edits and debugging/tuning: references/prompt-engineering.md
- capturing user feedback (thumbs, ratings, implicit signals) as scores on traces: references/user-feedback.md
- further tips on using the Langfuse CLI: references/cli.md
- upgrading or migrating Langfuse SDKs and preserving application instrumentation attributes: references/sdk-upgrade.md
- upgrading legacy trace-level or dataset-item evaluators to observation-level or experiment evaluators: references/trace-evaluator-upgrade.md. Use the [evaluator migration guide](https://langfuse.com/faq/all/llm-as-a-judge-migration) as the primary reference.
- preparing an application and Langfuse project for the v4 platform migration: references/v4-project-migration.md
- preparing a Langfuse project for the v4 platform migration: references/v4-project-migration.md
- judge calibration (LLM-as-a-Judge reliability, simple accuracy checks, advanced split-based validation, confusion matrices, and metric ingestion): references/judge-calibration.md
- systematic error analysis — reading traces, building failure taxonomy, deciding what to fix: references/error-analysis.md
- setting up CI/CD experiment gates with `langfuse/experiment-action`: references/ci-cd.md
+38
View File
@@ -0,0 +1,38 @@
---
name: langfuse-sdk-upgrade
description: Upgrade Langfuse SDKs and application instrumentation while preserving trace attributes across observations. Use for Python or JS/TS SDK migrations, including the application side of a v4 platform migration.
metadata:
required_access:
- CODEBASE
- LANGFUSE_PROJECT_SCRIPT
---
# Langfuse SDK upgrade
## Sources of truth
Determine every installed Langfuse SDK major version, then fetch each applicable leaf guide in full before editing:
- [SDK upgrade paths](https://langfuse.com/docs/observability/sdk/upgrade-path)
- [Python v3 to v4](https://langfuse.com/docs/observability/sdk/upgrade-path/python-v3-to-v4)
- [JS/TS v4 to v5](https://langfuse.com/docs/observability/sdk/upgrade-path/js-v4-to-v5)
- [Instrumentation and attribute propagation](https://langfuse.com/docs/observability/sdk/instrumentation#add-attributes)
- [Sessions and session-level metrics](https://langfuse.com/docs/observability/features/sessions)
- [Direct OpenTelemetry ingestion](https://langfuse.com/integrations/native/opentelemetry)
Follow every intermediate guide when the installed version is more than one major behind. Use the current docs for implementation details; do not copy examples from this file.
## Workflow
1. Inventory every SDK, integration package, direct OpenTelemetry exporter, initialization site, instrumentation wrapper, lockfile, worker, script, and test that can emit Langfuse data.
2. Find every source of correlating attributes, including `session_id`/`sessionId`, `user_id`/`userId`, tags, metadata, version, environment, and trace name. Search for the values and surrounding application concepts, not only removed SDK method names.
3. Apply every relevant item from the exact version-specific guides. Preserve each correlating attribute by establishing its documented propagation scope before any observation-producing call that must inherit it.
4. For sessions, propagate the session ID early enough that the root and every applicable child observation—including cost-bearing generations—receive the same value. For work crossing service boundaries, follow the current distributed-tracing and baggage guidance rather than assuming in-process context crosses the boundary.
5. Run focused format, type, lint, and test checks, then exercise each changed ingestion path with representative application behavior.
6. Fetch the emitted trace and verify the expected correlating attributes on the root and every applicable child observation. For a session path, also confirm the observations are grouped into the intended session and its cost includes the cost-bearing children.
Do not mark the upgrade ready when only dependency or compile-time checks passed. If runtime execution or trace inspection is unavailable, report the exact verification as blocked.
## Completion report
Report the versions before and after, changed instrumentation paths, attribute sources and propagation scopes, validation performed, inspected trace or session, and any remaining blocked verification.
@@ -35,7 +35,8 @@ Discover unstable schemas before use.
- Inventory every Langfuse SDK, integration, OTEL exporter, initialization site, lockfile, raw request, generated client, script, notebook, and CI call.
- Upgrade to the latest stable SDK major required by the current docs and apply every applicable breaking change. Record both declared and resolved versions; update an existing lockfile.
- Put overall input/output on the root observation and propagate every trace attribute needed to filter or aggregate observations.
- Find every source of correlating attributes, including session and user IDs, tags, metadata, version, environment, and trace name; do not search only for removed SDK methods.
- Put overall input/output on the root observation. Establish the documented propagation scope before observation-producing calls so every applicable child receives the attributes needed for filtering and aggregation, including the session ID on cost-bearing generations.
- For raw `/api/public/ingestion`, use the current Langfuse SDK in Python or JS/TS. For other languages, use the language's native OpenTelemetry API and follow the custom-ingestion guide.
- When replacing synchronous ingestion, assess buffering, retries, flushing, shutdown, and error propagation. Do not claim identical delivery semantics without verification.
- For other deprecated APIs, migrate the path, parameters, pagination, filters, field groups, response parsing, and downstream consumer together using the deprecated-API guide.
@@ -68,6 +69,7 @@ Discover unstable schemas before use.
- Test applicable hierarchy, root input/output, propagated attributes, public/release/environment behavior, API pagination and parsing, delivery semantics, and absence of deprecated calls.
- Before production cutover, send representative traces from the migrated instrumentation to a non-production Langfuse project and inspect the resulting observations there. Mocked tests do not verify backend ingestion or project behavior.
- On a session path, confirm the root and every applicable child observation carry the intended session ID and that session cost includes the cost-bearing children. Keep SDK/instrumentation readiness blocked when emitted-trace inspection is unavailable.
- Re-read rules and integrations after writes. Preserve disabled legacy rules for rollback; never claim completion without checking the Evaluators UI on the target host for legacy rows.
- The readiness report must contain exactly these seven rows, each marked `ready`, `changed`, `manual action`, or `blocked`: project access; SDK/instrumentation; trace evaluators; dataset evaluators; direct APIs; exports; verification/rollback.
- In evaluator rows, separate contracts verified against project data from targets suggested only by code inspection. For every row not marked `ready`, include the blocker, next action, and a direct UI link when applicable.