Files
Ben Younes c805410878 fix(pi): bridge MCP tools into pi.registerTool() so the LLM can call them (#426) (#472)
* fix(pi): bridge MCP tools into pi.registerTool() so the LLM can call them (#426)

Pi 0.73.x has no native MCP support — its README is explicit:

> No MCP. Build CLI tools with READMEs (see Skills), or build an
> extension that adds MCP support.

Without a bridge inside the context-mode Pi extension, the routing
block tells the LLM to call `ctx_execute` / `ctx_search` / etc. but
those tools never enter Pi's tool list and the LLM cannot reach them.
The reporter measured 18 sessions over 2 days: ~2,500 tokens of
system-prompt overhead per window, 0 actual ctx_* calls, 447 events
recorded but never retrieved. Net ROI on Pi was negative.

This adds a stdio JSON-RPC client (`MCPStdioClient`) plus a thin
bootstrap (`bootstrapMCPTools`) that:

  - spawns `server.bundle.mjs` as a long-lived MCP child,
  - performs the standard MCP handshake (initialize →
    notifications/initialized),
  - lists tools once via `tools/list`, and
  - registers each tool through `pi.registerTool({ name, label,
    description, parameters, execute })` so the LLM sees the canonical
    bare names (matching what hooks/core/tool-naming.mjs emits for
    Pi).

Each Pi `execute()` callback forwards into the MCP child via
`tools/call`. Errors are translated to `throw` (Pi's contract for
"tool failed") so the LLM sees the MCP server's diagnostic text.

Lifecycle:

  - Bridge bootstrap is fire-and-forget at extension load — the rest
    of the extension (session capture, hooks, slash commands) is not
    blocked by spawn / handshake latency.
  - `session_shutdown` terminates the child via SIGTERM.
  - A missing `server.bundle.mjs` or any spawn / handshake error is
    surfaced once on stderr, then the extension keeps running with
    only the existing hooks + commands. Defense-in-depth so the bridge
    can never break Pi sessions for users with broken installs.

## Why a JSON Schema parameters object instead of TypeBox

MCP `tools/list` returns JSON Schema. Pi's parameter validator accepts
JSON Schema directly (TypeBox just produces JSON Schema with extra
Symbol metadata for type inference). Passing the schema through
unchanged avoids a runtime translation pass and keeps the bridge a
true thin layer over the MCP protocol — what works in Claude Code,
Gemini CLI, and the other adapters now also works in Pi.

## No new runtime dependencies

Pure `node:child_process` + `node:path`. The `@earendil-works/pi-*`
packages are NOT pulled in as build deps — `pi` is typed structurally
as `any` (matching the existing src/pi-extension.ts style) and the
bridge only touches the documented `pi.registerTool()` shape.

## Tests

Added two new `describe` blocks in `tests/pi-extension.test.ts`:

  1. `MCPStdioClient` (5 tests) — wire-protocol contract pinned with
     fake stdio servers: id-matched responses, concurrent in-flight
     requests with out-of-order delivery, child-exit cancellation,
     timeout, non-JSON noise tolerance.
  2. `bootstrapMCPTools` (2 integration tests) — spawn the real
     `start.mjs` MCP server, assert that the canonical ctx_* set
     (`ctx_execute`, `ctx_execute_file`, `ctx_search`, `ctx_index`,
     `ctx_batch_execute`, `ctx_fetch_and_index`, `ctx_doctor`,
     `ctx_stats`, `ctx_purge`) is registered, and round-trip
     `ctx_index` through `tools/call` to confirm execute() forwards
     args and returns text.

## Test plan

  - [x] `npm run build`
  - [x] `npm run typecheck` clean
  - [x] `npm test` — 73 files, 2406 pass / 25 skipped / 0 fail
  - [x] `npx vitest run tests/pi-extension.test.ts` — 44 pass
        (37 pre-existing + 7 new for the bridge)
  - [x] see-real-bug repro: `pi.registerTool` count = 0 in installed
        binary (`/home/$USER/.nvm/.../context-mode/build/pi-extension.js`)
        on `next` @ 1f70bee, plus Pi README's "No MCP" stance, plus
        the issue reporter's 18-session measurements — all three
        agree.
  - [-] Live LLM tool-call probe in Pi: blocked — free-tier Gemini
        quota was exhausted on every available key during the fix
        session. The integration test exercises the same code path
        (real MCP server + the Pi-facing registerTool surface), so
        the regression contract is enforced from CI.

## Out of scope

  - Removing the MCP server stanza from the Pi install README. Once
    this lands, the `~/.pi/agent/mcp.json` step is still harmless but
    no longer load-bearing. Cleanup left to a docs-only follow-up.
  - In-process refactor of server.ts handlers. The subprocess bridge
    is the same model used by every other adapter; a refactor that
    inlines the handlers is its own scope.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* fix(pi): address self-review findings on the MCP bridge (#426)

Three follow-up changes from the empirical self-review on PR #472:

1. **C1 HIGH — wiring not test-covered.**
   Phase A of the empirical review only failed because removing
   `src/pi-mcp-bridge.ts` produced an import error, not because the
   bug reproduced behaviorally. If a future refactor dropped the
   `bootstrapMCPTools(pi, …)` call from `src/pi-extension.ts` while
   keeping the bridge module intact, every existing bridge test
   stayed green and the bug silently re-entered.

   Fix: export `_mcpBridgeReady: Promise<void>` from
   `src/pi-extension.ts`. Bootstrap is still fire-and-forget (so
   spawn / handshake latency does not block session_start), but the
   promise gives tests a deterministic await point. Reset to a fresh
   promise on every `piExtension(pi)` call so multiple registrations
   in one process do not see a stale resolution.

   New test in `tests/pi-extension.test.ts` ("pi-extension.ts wiring
   (#426 regression guard)"): calls `registerPiExtension(api)`,
   awaits `_mcpBridgeReady`, asserts `api.registerTool.mock.calls`
   includes at least the canonical `ctx_execute` / `ctx_search` /
   `ctx_index` / `ctx_batch_execute` / `ctx_fetch_and_index` set.
   Verified red-on-revert: with the bridge module intact but the
   `bootstrapMCPTools(...)` call reverted to next, this test fails
   with `registeredNames: []`. Pre-fix it would have stayed green.

2. **C2 LOW — duplicated path resolution in the integration tests.**
   `tests/pi-extension.test.ts` had `path.dirname(...) +
   path.resolve(here, "..", "start.mjs")` recomputed in each `it()`.
   Lifted to a single `mcpEntry` const at the top of the
   `bootstrapMCPTools — registers every ctx_* tool with Pi` describe
   block, plus a shared `mcpEnv` for the `CONTEXT_MODE_DISABLE_VERSION_CHECK`
   override. One place to update if `start.mjs` ever moves.

3. **C3 LOW — dead `running` getter on MCPStdioClient.**
   Exported in the original commit but had zero callers anywhere in
   `src/` or `tests/`. Dropped — five lines, no behavioral impact.

## Test plan

- [x] `npm run build`
- [x] `npm run typecheck` — clean
- [x] `npm test` — 73/73 files, 2407 pass / 25 skipped / 0 fail
- [x] `npx vitest run tests/pi-extension.test.ts -t "MCP bridge|wiring"` — 8 pass
- [x] Phase A re-validation: revert ONLY the wiring in
      `src/pi-extension.ts` (keep `src/pi-mcp-bridge.ts` intact),
      run the wiring test → fails with `registeredNames: []`. Restore
      and the test goes green again.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor(openclaw): consolidate src/openclaw/* into src/adapters/openclaw/

Pre-fix layout split OpenClaw across two locations:
  - src/adapters/openclaw/ — config, hooks, index, session-db (standard
    adapter pattern matching every other platform)
  - src/openclaw/         — mcp-tools, workspace-router (rogue location)

The split predates the adapter pattern: workspace-router.ts was added
first by Pedro Almeida (#aa8d93c), then mcp-tools.ts by the maintainer
(#ff0a9a2 v1.0.107), while the adapter dir was bootstrapped later by
the copilot-swe-agent (#5fd6a9e). Source-of-truth for every platform
should live under src/adapters/<platform>/, so we move the two
stragglers in.

## Changes

  - git mv src/openclaw/mcp-tools.ts        → src/adapters/openclaw/mcp-tools.ts
  - git mv src/openclaw/workspace-router.ts → src/adapters/openclaw/workspace-router.ts
  - rmdir src/openclaw

  - src/openclaw-plugin.ts: 3 import-path updates
  - tests/plugins/openclaw.test.ts: 1 import-path update
  - tests/core/cli.test.ts: 1 readFileSync source-grep path update
    (the existing PR #183 path-traversal regression test reads the
    workspace-router source file directly to grep for safe-regex
    patterns; pin updated to the new location)

## Test plan

  - [x] npm run typecheck — clean
  - [x] npx vitest run tests/plugins/openclaw.test.ts tests/core/cli.test.ts
        → 225/225 pass
  - [x] npm test — 73 files, 2405+ pass / 25 skipped / 0 fail

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor: flatten src/concurrency/runPool.ts → src/runPool.ts

The src/concurrency/ directory held a single file. A whole directory
for one module is structural noise — flatten it to src/runPool.ts.

## Changes

  - git mv src/concurrency/runPool.ts → src/runPool.ts
  - rmdir src/concurrency
  - src/server.ts: import path updated
  - tests/core/server.test.ts: import path updated

## Test plan

  - [x] npm run typecheck — clean
  - [x] npx vitest run tests/core/server.test.ts -t "runPool" — pass

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* refactor: relocate plugin entry files into src/adapters/<platform>/

Pre-fix layout had three platform plugin entry files at the src/ root:

  src/pi-extension.ts        — Pi Coding Agent extension
  src/pi-mcp-bridge.ts       — Pi MCP bridge (added in #426)
  src/openclaw-plugin.ts     — OpenClaw gateway plugin
  src/opencode-plugin.ts     — OpenCode plugin

Every other platform follows the src/adapters/<name>/ pattern (config,
hooks, index, …). The four root-level files were the last hold-outs:
inconsistent layout, plus they made adapter discovery harder for new
contributors.

## Changes (file moves)

  - git mv src/pi-extension.ts        → src/adapters/pi/extension.ts
  - git mv src/pi-mcp-bridge.ts       → src/adapters/pi/mcp-bridge.ts
  - git mv src/openclaw-plugin.ts     → src/adapters/openclaw/plugin.ts
  - git mv src/opencode-plugin.ts     → src/adapters/opencode/plugin.ts

## Internal import-path updates inside the moved files

  - ./session/db.js    → ../../session/db.js   (depth +2)
  - ./types.js         → ../../types.js
  - ./adapters/X/Y.js  → ./Y.js                (now sibling)
  - ./adapters/types.js → ../types.js          (now parent)
  - ./pi-mcp-bridge.js → ./mcp-bridge.js       (renamed + sibling)

## Runtime path-resolution updates

The plugins read sibling resources (hooks/, package.json, etc.) via
`resolve(buildDir, "..")`. After the move buildDir lives 2 dirs
deeper, so every `..` is now `../../..`:

  - resolve(buildDir, "..")                                 → resolve(buildDir, "..", "..", "..")
  - resolve(buildDir, "..", "hooks", "core", "routing.mjs")    → resolve(buildDir, "..", "..", "..", "hooks", "core", "routing.mjs")
  - (and similar for routing-block / tool-naming / auto-injection)

For opencode/plugin.ts the version-from-package.json walker prepends
`../../../package.json` to its search list (keeps the legacy
`../package.json` and `./package.json` entries as fall-backs so
unbundled or old-layout dev environments still resolve).

## Build-output paths in package.json

tsc preserves src/ structure under build/, so:

  ./build/pi-extension.js     → ./build/adapters/pi/extension.js
  ./build/openclaw-plugin.js  → ./build/adapters/openclaw/plugin.js
  ./build/opencode-plugin.js  → ./build/adapters/opencode/plugin.js

Updated:

  - package.json: pi.extensions[0], openclaw.extensions[0], main,
    exports["."], exports["./plugin"], exports["./openclaw"]
  - .pi/extensions/context-mode/index.ts: re-export delegate path
  - .openclaw-plugin/index.ts: re-export delegate path + JSDoc

## Test-side updates

  - tests/pi-extension.test.ts: dynamic-import paths updated
  - tests/opencode-plugin.test.ts: dynamic-import paths updated
  - tests/plugins/openclaw.test.ts: dynamic-import paths updated
  - tests/core/cli.test.ts: 4 dynamic-import paths + 1
    `readFileSync(src/openclaw-plugin.ts)` source-grep updated to the
    new location
  - src/adapters/detect.ts: comment-line ref updated
  - tests/adapters/detect.test.ts: comment-line ref updated

## Test plan

  - [x] npm run build                                        clean
  - [x] npm run typecheck                                    clean
  - [x] npm test                                             73 files,
        2407 pass / 25 skipped / 0 fail
  - [x] npx vitest run tests/opencode-plugin.test.ts         33/33 pass
        (regression: marker test that needed package.json walker fix)
  - [x] npx vitest run tests/plugins/openclaw.test.ts        225/225 pass
  - [x] npx vitest run tests/pi-extension.test.ts            45/45 pass
        (incl. the wiring guard added in the previous commit)
  - [x] npx vitest run tests/core/cli.test.ts -t "openclaw-plugin.ts doctor/upgrade"
        passes against the new src/adapters/openclaw/plugin.ts location
  - [x] Manual sanity: every old root-level path (build/pi-extension.js,
        src/opencode-plugin.ts, etc.) is gone from the repo — grep
        confirms zero stale refs in src/ + tests/ + package.json + the
        .pi/.openclaw-plugin/ thin wrappers.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* fix(ci): update E2E + install scripts for relocated openclaw plugin path

The structural refactor in 4911c07 (src/openclaw-plugin.ts → src/adapters/
openclaw/plugin.ts) moved the build output from build/openclaw-plugin.js
to build/adapters/openclaw/plugin.js. Three scripts still pointed at the
legacy path and broke on next-CI.

## OpenClaw E2E (failing on ubuntu-latest + macos-latest)

  scripts/test-openclaw-e2e.sh:34
    join(process.cwd(), "build", "openclaw-plugin.js")

The Phase 1 plugin-load check failed at " build/openclaw-plugin.js
exists" → exit 1. Updated to look for the new path first, fall back to
the legacy one for transition safety:

  build/adapters/openclaw/plugin.js → fall back → build/openclaw-plugin.js

Loaded-tag tracks which path actually resolved.

## OpenClaw global install (would have broken at user-install time)

  scripts/install-openclaw-plugin.sh:49 (auto-generated index.ts stub)

Updated the absolute re-export path written into the generated stub
plus the jiti cache-clear glob (now matches both
`build-adapters-openclaw-plugin.*.cjs` and the legacy
`build-openclaw-plugin.*.cjs` filenames).

## Bonus: security.js path was wrong post-refactor

The opencode + openclaw plugins called `routing.initSecurity(buildDir)`
where buildDir = build/adapters/<platform>/. That made initSecurity look
for build/adapters/<platform>/security.js — which never exists. The
security module lives at build/security.js (top-level). The fix-open
fallback meant tests still passed but every plugin load emitted a
spurious WARNING about deny-policy enforcement being off.

  - opencode/plugin.ts: pass `resolve(buildDir, "..", "..")` (= build/)
  - openclaw/plugin.ts: same

Verified locally: `bash scripts/test-openclaw-e2e.sh` → 39/39 pass, no
security warning.

## Test plan

  - [x] npm run build              clean
  - [x] npm run typecheck          clean
  - [x] npm test                   73 files, 2405+ pass / 25 skipped /
        0 fail (2 pre-existing flake worker-pool timeouts on
        kiro-hooks + insight-cors; both pass when run in isolation)
  - [x] bash scripts/test-openclaw-e2e.sh
        → "Results: 39 passed  0 warned  0 failed"  +  " E2E test PASSED"

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

* docs(openclaw): update Key Files paths after src/adapters/<platform>/ refactor

Independent PR review on #472 caught 3 stale path strings in
`docs/adapters/openclaw.md` that the structural refactor (4911c07)
missed:

  - src/openclaw-plugin.ts          → src/adapters/openclaw/plugin.ts
  - src/openclaw/workspace-router.ts → src/adapters/openclaw/workspace-router.ts (×2)

Doc-only — no code paths reference these strings.

Co-Authored-By: Ora Studio <noreply@oratelecom.net>

---------

Co-authored-by: Ora Studio <noreply@oratelecom.net>
2026-05-08 02:30:46 +03:00

6.9 KiB

OpenClaw Adapter

context-mode plugin for the OpenClaw gateway, targeting Pi Agent sessions.

Overview

OpenClaw is the gateway/platform that manages agent sessions, extensions, and tool routing. Pi Agent is OpenClaw's coding agent — it runs within OpenClaw and provides Read, Write, Edit, and Bash tools for software development tasks.

The context-mode adapter hooks into Pi Agent sessions specifically, intercepting tool calls to route data-heavy operations through the sandbox and tracking session events for compaction recovery.

Supported Configurations

  • Pi Agent sessions with coding tools (Read/Write/Edit/Bash) — fully supported.
  • Custom agents with coding tools — may work but are untested. The adapter relies on tool names matching Pi Agent's conventions.

Installation

Quick install

npm run install:openclaw

This runs scripts/install-openclaw-plugin.sh, which handles building, extension setup, runtime registration, and gateway restart.

Prerequisites

  • Node.js must be in PATH (required for the build and registration steps)
  • OpenClaw must have been started once — the script needs openclaw.json, which OpenClaw creates on first launch
  • OPENCLAW_STATE_DIR must point to your OpenClaw state directory (default: /openclaw). Pass it as an argument: npm run install:openclaw -- /path/to/state

Manual install

For advanced users or custom setups:

bash scripts/install-openclaw-plugin.sh [OPENCLAW_STATE_DIR]

See scripts/install-openclaw-plugin.sh for details.

Troubleshooting

"openclaw.json not found" OpenClaw creates this file on first launch. Start OpenClaw once (openclaw gateway start), then re-run the install script. This is the most common issue for users who install context-mode before ever starting OpenClaw.

"OPENCLAW_STATE_DIR (/path) does not exist. Is OpenClaw installed?" The state directory doesn't exist at the expected path. If you installed OpenClaw via npm (not git clone), check where it stores state — common locations are ~/.openclaw or /openclaw. Pass the correct path: npm run install:openclaw -- /path/to/state.

Plugin installed but not loading Clear the jiti cache (rm -f /tmp/jiti/context-mode-*.cjs) and restart the gateway. If the issue persists, verify the plugin appears in openclaw plugins list.

Plugin loads but ctx_* tools are missing from the agent's tool list The plugin registers its hooks via api.on(...) / api.registerCommand(...), but the agent-callable ctx_* tools live in the MCP server (server.bundle.mjs). OpenClaw surfaces them by spawning the server as an MCP sidecar declared in mcp.servers.context-mode. The install script writes this entry automatically (step 5); if you configured OpenClaw manually, verify with openclaw mcp list and add it if missing:

openclaw mcp set context-mode \
  "{\"command\":\"node\",\"args\":[\"/absolute/path/to/context-mode/server.bundle.mjs\"]}"
openclaw gateway restart

After the restart, the agent's tool inventory should include context-mode__ctx_execute, context-mode__ctx_search, context-mode__ctx_fetch_and_index, and the rest of the ctx_* surface (OpenClaw prefixes MCP-sourced tools with the server name).

Hook Registration

The adapter uses two different registration APIs, matching OpenClaw's internal architecture:

  • api.on() for lifecycle and tool hooks: session_start, before_tool_call, after_tool_call, before_compaction, after_compaction, before_prompt_build, before_model_resolve. These are typed event emitters with structured payloads.
  • api.registerHook() for command hooks: command:new, command:reset, command:stop. These use colon-delimited names and the generic hook registration system.

Using the wrong API (e.g., api.registerHook("before_tool_call", ...)) registers silently but the hook never fires. This distinction is critical.

Synchronous register()

OpenClaw silently discards the return value of register(). If register() is async, all hooks registered inside it are lost. The adapter uses the initPromise pattern:

register(api): void {
  const initPromise = (async () => { /* async setup */ })();
  api.on("after_tool_call", async (e) => {
    await initPromise;
    // handle event
  });
}

Session Continuity

Hook Method Status
after_tool_call api.on() Working
before_compaction api.on() Working
session_start api.on() Working
command:new api.registerHook() Working
command:reset api.registerHook() Working
command:stop api.registerHook() Working

Graceful Degradation

If compaction hooks fail to fire (e.g., on older OpenClaw versions), the adapter falls back to DB snapshot reconstruction — rebuilding session state from the events already persisted in SQLite by after_tool_call. This produces a less precise snapshot than the PreCompact path but preserves critical state (active files, tasks, errors).

Previously Known Upstream Issues

Both issues below have been resolved in upstream OpenClaw:

  • #4967 — Compaction hooks not firing. Closed as duplicate of #3728; fix merged.
  • #5513api.on() hooks not invoked for tool lifecycle events. Fixed in PR #9761.

Minimum Version

Required: OpenClaw >2026.1.29

This is the first release that includes the api.on() fix from PR #9761, which shipped on 2026-01-29.

What breaks on older versions: Lifecycle hooks registered via api.on() — including before_compaction, after_compaction, session_start, and tool interception hooks — may silently fail to fire.

Graceful degradation: If compaction hooks don't fire, the adapter falls back to DB snapshot reconstruction, rebuilding session state from events already persisted by after_tool_call. This produces a less precise snapshot than the PreCompact path but preserves critical state (active files, tasks, errors). The adapter will not crash on older versions, but compaction recovery quality will be reduced.

Workspace Routing

The adapter includes a workspace router (src/adapters/openclaw/workspace-router.ts) that resolves project paths from Pi Agent session metadata, ensuring session databases and routing instructions are scoped per-workspace.

Key Files

File Purpose
src/adapters/openclaw/plugin.ts Main plugin entry (sync register, initPromise pattern)
src/adapters/openclaw/workspace-router.ts Workspace path resolution for session scoping
.openclaw-plugin/ Plugin manifest (index.ts, openclaw.plugin.json, package.json)
scripts/install-openclaw-plugin.sh One-shot installer