mirror of
https://github.com/thedotmack/claude-mem.git
synced 2026-09-20 04:23:02 +08:00
dc36ee7d45b98884c58aba1860054c5bfb1145d0
200 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alex Newman
|
57d865619f |
fix: upgrade shell-quote to 1.9.0 (CVE-2026-13311) (#4018)
Pin shell-quote to 1.9.0 in the plugin manifest, lockfile, root devDependency, and build-hooks generator so a normal build cannot regenerate the CVE-affected ^1.8.3 range. Rebase of #3646. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> |
||
|
Alex Newman
|
a61b86ddde |
fix(packaging): ship bug-report CLI (#4009)
Co-authored-by: dajiaohuang <mikewushuwen@outlook.com> |
||
|
Alex Newman
|
c166e30d09 |
feat(grok-bot): Phase 1 hack-on-Memory JIT — rich timeline INDEX (#3959)
* feat(grok-bot): Phase 1 Memory JIT — rich timeline index via #3953 transport Compile an editable CCS L1 markdown bucket into a slide-off timeline INDEX (one observation per line, ID kept) and land it through zz-claude-mem-inject.md → host Memory mid-attach. mtime-stable rewrite when the fact block is unchanged. Pilot allowlist remains the default; AGENT_IDS=* stays optional. Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> * fix(grok-bot): stamp CCS L1 bucket at ccs/seats/<id>/TIMELINE.md Concrete default is <agentDataRoot>/ccs/seats/<agentId>/TIMELINE.md so bots can edit it. PRIVATE.md is a reserved sibling (never written). house/ and groups/ stay reserved for later inherit. L1 pilot remains the Orifice/Grok Memory allowlist. Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> |
||
|
Claude
|
1d6eb84aaf |
fix(grok-bot): drop the inject fetch clock from the staged fact line
The worker's inject header carries the time it was fetched. Keeping it in the fact block made every poll a content change, so the host saw a "memory changed" delta and re-announced the section on the next turn for a timestamp and nothing else. Facts already carry their own date; strip the clock and the refresh pass goes to `unchanged` until observations actually move. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0181Cyg5g9ee7QvPynfh7iV7 |
||
|
Claude
|
8511f87410 |
feat(grok-bot): silent session inject into the Grok Bot prompt path
GrokBotAwarenessPusher gets needles into an agent's memory folder, but it
writes them as `- <date> [awareness] ...`, which the sand host's fact grammar
(`/^-\s+\((\d{4}-\d{2}-\d{2})\)\s+(.+?)\s*$/`) does not match — those lines
never reach the prompt. Nothing else pulls session context in automatically,
so a Grok seat only sees claude-mem when the model remembers to call a tool.
This adds a standalone watcher that closes that gap using only APIs that
already exist:
worker GET /api/context/inject (projects / platformSource only)
-> host fact grammar, in a file the shim owns:
agents/<id>/memory/log/zz-claude-mem-inject.md
-> the host's own WatchedDirectory + frozen-section machinery
(getFrozenSectionUpdatesForTurn -> promptWithInstructionsUpdate)
attaches the delta to the agent's next cold / non-resume turn.
No host patch, no new memory API, no listening socket (outbound HTTP to the
local worker only), and no full frozen-head rebuild.
Two details the host forced:
- Tier. The host ranks recalled log facts by
`log2(importance) + createdAt / 30d`, and importance comes only from a line
prefix (`[episode] ` 1.5, `[note] ` 0.5, else 1). log2(1.5) is worth ~17.5
days of recency, so on a seat whose log is already full of episode summaries
a plain-tier line loses every slot in the 4000-char recall budget and never
renders at all. Episode tier is the default; `CLAUDE_MEM_GROK_BOT_INJECT_TIER`
can drop it to plain/note on a quieter seat.
- Line budget. Every emitted line spends its own length out of that same
4000-char budget, evicting the agent's own memory, so the default is 2 lines:
project, stats and the freshest observation IDs, plus how to fetch the rest.
Gated off by default and allowlisted per agent id. Writes are atomic, refuse
any path outside the agent's memory/log, and refuse any filename but the one
the shim owns — the host owns YYYY-MM.md and profile.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181Cyg5g9ee7QvPynfh7iV7
|
||
|
Cursor Agent
|
913a67fa40 |
docs: add OpenRouter list-price history for expense reports
Daily published $/MTok input+output for CMEM-relevant models (2026-07-01 through 2026-09-08), plus work-day popularity guesses and a five-model July–September sample. Join observation tokens to the CSV; do not use OpenRouter generation or activity APIs. Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> |
||
|
Alex Newman
|
ad3dcaa2fc |
feat(worker): read-only Observation TV broadcast behind CLAUDE_MEM_TV_TOKEN
* feat(ui): observation TV — fullscreen fading titles off the existing SSE stream Adds a standalone, dependency-free page that consumes the same /stream the React viewer does and plays each observation's title as a fullscreen fading card. Live arrivals play first; a seeded backlog from /api/observations cycles while the worker is idle, so the screen is never blank. Picture-in-picture without a broadcast library: Document PiP (Chromium) moves the real DOM into the floating window so the CSS fades keep running, and everywhere else — including iOS Safari, the phone case — the card is painted to a canvas whose captureStream() feeds a muted video into native PiP. Served two ways: express.static already exposes plugin/ui, so /tv.html works with no route change, and a /tv alias is cached at boot the same way viewer.html is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6QPdnPducVehMwCM2HYNC * docs(plans): observation TV read-only broadcast + shared-secret token Phased plan for the locked 2026-09-05 decision: expose Observation TV to a second device on the LAN without exposing the rest of the worker. The worker has no request authentication anywhere; its only defence is the loopback bind, and the codebase says so out loud (ServerService.ts:129-131). So CLAUDE_MEM_WORKER_HOST=0.0.0.0 today does not put the TV on the LAN, it puts GET /api/settings — which returns the user's Gemini and OpenRouter API keys in plaintext — on the LAN, alongside the settings writer, the row deletes, bulk import, and better-auth's key issuance. The design is one guard middleware mounted at position zero in the Server constructor, the only spot that covers /api/auth/*, /api/admin/*, the static mount, and every route registered later. It is a no-op for loopback and, for non-loopback requests, default-deny with a four-path exact-match allowlist behind a new CLAUDE_MEM_TV_TOKEN. An empty token means the guard is never mounted, so every existing install — including the documented Docker 0.0.0.0 setup — is byte-identical to today. Phase 0 is written out rather than delegated: ~45 routes inventoried with file:line, the copy-ready patterns named (requireLocalhost, parseBearerToken, safeEqualHex, the securityHeaders opt-in precedent), and five traps recorded, including that SettingsDefaultsManager.get() cannot see settings.json and that the worker never calls finalizeRoutes() so the guard must write its own responses. Appendix B lists every rejected option with its reason — cloudflared first among them. Plan only. Nothing implemented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMh2GZST1UgKDSML17qCmh * feat(worker): read-only Observation TV broadcast behind CLAUDE_MEM_TV_TOKEN The worker's HTTP surface (45+ routes) has no request authentication; the loopback bind is its only defence. So setting CLAUDE_MEM_WORKER_HOST=0.0.0.0 — which the Docker docs tell people to do — puts GET /api/settings (provider API keys in plaintext), POST /api/admin/restart, DELETE /api/observation/:id, POST /api/import and better-auth on the LAN. Add one guard middleware, mounted at position zero in the Server constructor — the only spot that covers /api/auth/*, /api/admin/*, the static mount and every route registered later, including routes that do not exist yet. It is a no-op for loopback and, for non-loopback requests, default-deny with an exact-match four-path allowlist behind a shared secret: /tv, /tv.html, /stream, GET /api/observations A GET/HEAD method gate kills every mutation; non-allowlisted paths get 404 so a scanner is not told which routes exist; the token is compared constant-time and accepted as Authorization: Bearer, X-Api-Key, or ?token= (the query form exists only because EventSource cannot set headers). The token is never logged. Empty token means the guard is never mounted, so every existing install behaves exactly as before and CLAUDE_MEM_WORKER_HOST keeps its 127.0.0.1 default. A boot-time SECURITY warning fires when the host is non-loopback with no token — warn, not refuse, so the documented Docker deployment keeps working. Also fixes createCorsMiddleware forwarding next(new Error('CORS not allowed')): the worker never calls finalizeRoutes(), so that reached Express's default handler and returned a 500 HTML stack trace with absolute filesystem paths — newly reachable from the LAN. It now writes its own 403 JSON. tv.html carries the token through to both of its calls, and cards now show platform_source with a per-source accent colour in both the DOM and canvas render paths. No new dependencies. 38 tests in tests/server/tv-remote-guard.test.ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xcn8Gf6ACkfDqLYaULAj2k --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
0b723c3069 |
fix: stop sync-plugin-manifests clobbering Cursor/Grok plugin copy
syncCursorPlugin() copied `description` and `homepage` straight from the
root package.json into claude-mem-cursor/.cursor-plugin/plugin.json and
claude-mem-grok-bot/.cursor-plugin/plugin.json. Those are independent
marketplace plugins for products that are not Claude Code, and they carry
their own curated copy from
|
||
|
Copilot
|
1c7aaff38d |
Add independent Cursor/Grok Bot marketplace plugins and host-observer install flow
Ship independent Cursor/Grok Bot marketplace plugins, host-observer probe, and Grok transcript discovery. Fixes Greptile P1s: source-specific manual sessions, probe-before-persist host observer, occupied-port rejection, and Grok Bot agent-data root discovery. |
||
|
Alex Newman
|
fe9180c297 |
fix(codex): make startup context injection fast and bounded (#3789)
* fix(codex): simplify startup context injection * fix(codex): bound cold worker startup * fix(codex): cover supported cold starts |
||
|
Alex Newman
|
6682ea6e3e |
fix(windows): address PR #3661 review and tester findings
Four fixes from the megafix review cycle: - mirror-dir.cjs: refuse identical or ancestor/descendant source and destination roots before any filesystem mutation. The --delete pass enumerated the destination first, so a source nested inside the destination was seen as stale and recursively deleted (Greptile P1). - worker-logs.cjs: track file identity (dev/ino) in the --follow loop. A rename-and-recreate rotation whose replacement matched the old offset's byte size was skipped forever by the size-only check (Greptile P2). - windows-git-bash-preflight: derive bin\bash.exe from every `git` PATH hit instead of only the first. Git for Windows puts both mingw64\bin\git.exe and cmd\git.exe on PATH; when mingw64 sorts first the derivation missed the real bash and doctor reported a false negative (tester report, D:\JavaTool\GIT layout). - doctor: a missing .install-version marker with node_modules present is now a warn, not a required failure. The marker is written only by the npx installer — marketplace-flow and dev build-and-sync installs never have one, so the check always failed for them and its repair suggestion would overwrite the checkout under test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JT1VTKoaTf7VfePb7nVfwz |
||
|
Alex Newman
|
b544c43401 |
fix(windows): reconcile rsync -a metadata on the quick-check path, bound the log tail
Addresses two P1s from review. 1. The size+mtime quick check returned before synchronizing mode bits, so a source that changed permissions without changing size or mtime left the destination stale. The direction that bites: a file whose execute bit was revoked stayed executable in the marketplace. The quick check exists to skip copying *content*, not to skip reconciling *metadata*. Re-auditing the full `-a` (-rlptgoD) set rather than only the reported bit found a second, larger gap: directory permissions and directory mtimes were never applied at all. The A/B against real rsync missed it because the snapshot recorded directories as the literal string 'dir'. On the repo tree that was 205 diverging entries on every sync. Now reconciled: permissions (-p, full 0o7777 so setuid/setgid/sticky are included) and mtimes (-t) on files, directories and symlinks. Directories are reconciled after their children, since writing children bumps the destination mtime and tightening permissions first would lock the mirror out of its own target. Deliberately not reconciled: owner/group (-o/-g, which rsync itself can only apply as root) and devices/specials (-D, root-only and absent from a source checkout). -a implies none of -H/-A/-X, so hardlinks, ACLs and xattrs are out of scope for both tools. 2. worker-logs read, decoded and split the entire daily log before keeping its last 50 lines. Against a 64 MiB log under a 16 MiB old-space limit that aborted with exit 134 and printed nothing - failing exactly when someone is debugging a busy worker. It now walks backwards in 64 KiB chunks until it has one newline more than it needs, which also guarantees the chunk boundary is discarded along with the partial line in front of it, so a multi-byte character split across chunks cannot reach the output. --follow is unchanged and now seeks from the size it already stat'd. Tests: the A/B snapshot now covers directory mode+mtime and symlink mtime, and runs metadata-only passes (chmod with size and mtime held constant, mtime-only, directory-only). Against the pre-fix mirror it reports PARITY BROKEN on all five passes; against the fix, all five are identical to rsync. The log tail gains a 64 MiB case run under --max-old-space-size=16, which spawns node rather than process.execPath - under `bun test` execPath is bun, which ignores the flag and would have made the case prove nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
bb1db97dd4 |
fix(windows): make build-and-sync run without rsync or a POSIX shell
Windows users cannot build and test claude-mem from source. `npm install` and `npm run build` already work on native Windows (windows.yml proves it on every PR), but every step after them is bash-only: - sync-marketplace.cjs shelled out to the rsync binary, which does not exist on Windows and is not installed by anything in this repo. Git for Windows does not bundle it. - build-and-sync used `cd ~/.claude/plugins/...`, and the sync script's own execSync calls passed `~` paths to the shell. `~` is bash-only home expansion. - worker:logs / worker:tail used `tail` and `date +%Y-%m-%d`. Replaces rsync with scripts/mirror-dir.cjs, a stdlib-only mirror that reproduces `rsync -a --delete --exclude=...`: the same pattern syntax (anchored `/x`, dir-only `x/`, basename-at-any-depth, `*` / `**` / `?`), the same size+mtime quick check, and crucially the same receiver-side protection, so `--delete` still refuses to remove excluded paths like the marketplace's .git and node_modules. Verified byte-identical against real rsync over the whole repo tree (1125 entries: paths, types, sizes, modes, mtimes, content hashes, symlink targets) and again after planting stale and protected files in both destinations. Home-relative paths now go through os.homedir() and cwd instead of the shell. worker:logs becomes scripts/worker-logs.cjs, which also fixes the pre-existing `tail -f 50` typo in worker:tail. Windows CI now runs the real sync script against a scratch USERPROFILE and asserts the --delete semantics, runs the mirror tests, and exercises the log tail - the sync path was previously untested on Windows, which is how this shipped in the first place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
9326fae06a | feat: add custom mode creator skill (#3487) | ||
|
Alex Newman
|
1839006979 | Merge branch 'pr3282' into mainline | ||
|
Alex Newman
|
7d3f1879eb |
fix(worker): single version oracle — never rank plugin cache dirs by mtime
The 2026-07-22 restart storm: claude-mem had four resolvers that ranked plugin cache dirs by directory mtime (worker-utils cacheWorkerScriptCandidates, the inline node L() and Codex Windows resolvers in hook-shell-template, and the POSIX `ls -dt` hook resolver), while the staleness detector compared the live worker against the marketplace package.json. When Claude Code stamped the superseded 13.11.0 cache dir with .orphaned_at, its mtime became newest, so every recycle respawned 13.11.0 while hooks running 13.12.0 kept demanding a restart — 485 respawns in ~2 hours, each spawning SDK and Chroma children, until the host exhausted its process table. The fix is an invariant, not a mitigation: one deterministic version oracle shared by detection and respawn. - resolveWorkerScript() ranks candidates by version descending (numeric major.minor.patch, release ahead of prerelease at the same base, reverse-lexical tiebreak) and skips .orphaned_at-stamped dirs; stable sort keeps cache → marketplace → cwd precedence on ties. - ensureWorkerRunning() resolves once and feeds the same result to both checkVersionMatch(port, expectedVersion) and the lazy-spawn script, so a mismatch can only ever resolve toward the version the respawn will actually produce — the loop is structurally impossible. - checkVersionMatch now takes the caller's expected version; getInstalledPluginVersion (the second oracle) is deleted. - All three generated bootstrap resolvers (mcp node launcher, Codex Windows launcher, POSIX hook prelude) embed the identical ordering; the POSIX variant uses a zero-padded sort key and balanced (pattern) case forms (unmatched parens are a parser error inside $(...)). - scripts/build-hooks.js gains --write-shell-templates, the regeneration mode its hand-edit tripwire error message points at. - Regression coverage: comparator/selection unit tests plus a functional shell-matrix test that rebuilds the exact storm layout (orphaned old dir with newest mtime) and asserts the new version wins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
b3eafc82b9 |
feat(sync): complete SyncHub-only client cutover
Retire legacy client adoption and per-kind sync state, establish the one-time launch boundary, and expose authenticated Hub health. Add bounded device metadata and a loopback-only two-client protocol-v2 matrix E2E. |
||
|
Alex Newman
|
57f8b9a110 | fix(sync-hub): harden verifier authentication | ||
|
Alex Newman
|
fe9caa5c88 |
docs(sync) + chore: final verification, docs, and stale-bundle fix (Phase 6)
Full-matrix e2e harness (scripts/sync-matrix-e2e.ts, 39/39): fresh-device bootstrap, >500-op offline catch-up, concurrent writes, all four mutation types in both arrival orders, epoch reset with native-corpus re-push, and kill-switch degradation — plus a two-real-daemon run proving the session-start context-inject pull end-to-end. docs/public/cloud-sync.mdx rewritten for the two-lane architecture (durable HTTP lanes, advisory WebSocket, poll-mode fallback, mutation convergence, settings reference) with availability framed honestly — no live hub URLs. Canary header comment scoped to what it measures. plugin/scripts/worker-service.cjs regenerated — the Phase 5 commit had shipped it stale (missing pollModeOnly/onSyncModeHint/X-Sync-Mode). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
528f133ad1 |
feat(sync): guardrails + monitoring (Phase 5)
Hourly watchdog as a scheduled handler on the stateless Worker: queries the DO GraphQL Analytics datasets (duration GB-s as the hibernation- defeat detector, rows read/written as runaway detectors, request count), thresholds derived from the validated workload model and env-overridable with min(alert,kill) breach gating, Discord alerts in the release-notify embed shape, and severe duration/rows breaches auto-tripping the kill switch — KV trip strictly before Discord so a dead webhook can never disable the brake. GraphQL failure alerts nothing and trips nothing. Kill switch: KV flag (AUTH_CACHE, control:kill-switch), per-isolate 30s cache, fail-open, checked before auth so every response — including auth failures — carries X-Sync-Mode: poll while tripped; WS upgrades refused before the DO is woken. Clients honor header presence at any status, treat absence as cleared only on OK, close the socket, suppress reconnects, and keep polling — product stays complete in poll mode. Suspended clients now tear the socket down (idle needs no speed layer), making the documented propagation bound structural. Canary: two-device trickle script with bounded --flood for alert-chain rehearsal. DEPLOY.md runbook covers provisioning, secrets, cron, canary deployment, and the trip/degrade/recover verification procedure. The DO itself is untouched — zero diff under src/do/. Hub 68+15 tests, client sync suites 113, e2e trip/recover PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
8fe1ba8658 |
feat(sync-hub): per-user Durable Object sync hub (Phase 1 — HTTP lanes)
Greenfield workers/sync-hub wrangler project implementing the two-lane
sync plan's server core: SyncHub DO (per-user ordered op log, idempotent
pushOps, paginated getChanges, daily compaction alarm) plus a stateless
front Worker that owns token verification (canonical-userId binding,
KV-cached verdicts, fail-closed) and batch caps. ESLint anti-pattern
guards scoped to src/do; 35 vitest tests via vitest-pool-workers.
Marketplace rsync now excludes /workers. Plan doc included.
Deploy prerequisites (flagged in wrangler.jsonc): confirm cmem.ai verify
endpoint path + {userId} response contract; create AUTH_CACHE KV namespace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
posthog[bot]
|
b19849205f |
fix(hooks): stop SessionStart start hook emitting two JSON objects
The SessionStart `start` hook ran `worker-service.cjs start` and then
appended `; echo '{"continue":true,"suppressOutput":true}'`. But `start`
already prints its own status JSON via exitWithStatus/buildStatusOutput
({"continue":true,"status":"ready","suppressOutput":true}), so stdout
ended up with two JSON objects concatenated — invalid JSON. Claude Code
could not parse it, ignored suppressOutput, and rendered the raw text at
the top of every session.
Drop the redundant trailingJson echo from the SessionStart `start`
config so the hook emits exactly one valid JSON object. Updated the
generator (scripts/build-hooks.js) and the distribution parity test to
match.
Generated-By: PostHog Code
Task-Id: 6dc27a3d-873f-4b6e-b869-95d7b33d3fb7
|
||
|
Alex Newman
|
817eb9ae4a |
Merge remote-tracking branch 'origin/main' into freckle-nail
# Conflicts: # README.md # docs/public/branches.mdx # src/services/worker/README.md |
||
|
Alex Newman
|
accd986760 | docs: refresh release docs and translations | ||
|
Alex Newman
|
2282220d1b |
fix(codex): add Windows hook overrides
(cherry picked from commit
|
||
|
Alex Newman
|
38fc189b5c |
fix(build): ship sqlite runtime modules
(cherry picked from commit
|
||
|
Alex Newman
|
41113d3454 |
fix: use health oracle and configured worker endpoint
(cherry picked from commit
|
||
|
Alex Newman
|
f64c878a8b |
chore: remove unused dependencies (ponytail audit wave 2, P17)
Drops 11 devDependencies with zero remaining imports after the wave-1 code swaps landed: handlebars/yaml/zod-to-json-schema/ts-prune (dead weight, no callers), picocolors (replaced by node:util styleText), glob (replaced by Bun.Glob), cors/@types/cors (replaced by an inline allowlist middleware), jimp (its sole caller, the banner-frame generator script, was deleted), tsx (bug-report script now runs via bun, matching every sibling script), and tree-sitter-elixir (the grammar can never produce a symbol; removed from build-hooks.js's plugin manifest generator and check-postinstall-allowlist.js too). Also drops the tree-kill optionalDependency — every remaining "tree-kill" reference in src/ is prose describing the self-implemented taskkill/pkill fallback, not an import of the package. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
33fc8a0ce5 |
chore: remove unused cmem-sdk client surface (ponytail audit wave 2, P15)
createCmemClient and its in-process client graph (Postgres pool, provider
selection, capture/search/generate/close) had zero product callers — only
its own tests exercised it. Deletes the client, its tsup build config and
bundle guard, the sdk-node example, and the docs pages, and drops the
"." and "./sdk" package exports. Moves pg/zod/@modelcontextprotocol/sdk
from dependencies to devDependencies since nothing in the shipped bundles
resolves them at runtime anymore (esbuild inlines them; plugin/ carries
its own zod).
src/sdk/{parser,prompts,output-classifier,hardened-options}.ts are kept —
the worker's provider/response pipeline uses them directly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Alex Newman
|
2a0a407340 |
chore: remove over-engineering findings from repo-wide ponytail audit (wave 1)
Cuts verified dead code, duplication, and speculative abstractions across 14 disjoint areas of the tree: sqlite layer, worker HTTP routes, search pipeline, server, chroma sync, viewer UI, npx-cli, MCP server, shared/utils, telemetry/infra, integrations, scripts, tests, and stale plans/evals docs. Also unifies the Chroma search pipeline onto SearchOrchestrator/strategies and ports dual-project (merged_into_project) scoping plus dateRange filtering into ChromaSearchStrategy, which corpus builds need but had silently lost. Full audit trail: 65-agent verification workflow wf_160a7862-1a6, 14-agent execution wf_c7e48c0f-164, 6-agent fixup wf_865f86a4-c6b. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
108c5c65a1 |
Merge remote-tracking branch 'origin/main' into mutual-aardvark
# Conflicts: # plugin/scripts/context-generator.cjs # plugin/scripts/mcp-server.cjs # plugin/scripts/server-service.cjs # plugin/scripts/transcript-watcher.cjs # plugin/scripts/worker-service.cjs # plugin/ui/viewer-bundle.js # src/server/routes/v1/ServerV1PostgresRoutes.ts # src/server/runtime/ServerService.ts # src/server/runtime/create-server-service.ts # src/server/runtime/types.ts # tests/server/server-service.test.ts |
||
|
Alex Newman
|
4557920565 |
Merge remote-tracking branch 'origin/main' into mutual-aardvark
# Conflicts: # package.json # plugin/scripts/context-generator.cjs # plugin/scripts/mcp-server.cjs # plugin/scripts/server-service.cjs # plugin/scripts/worker-service.cjs # plugin/ui/viewer-bundle.js # src/server/runtime/create-server-service.ts # tests/sdk/parse-summary.test.ts # tests/sdk/parser.test.ts |
||
|
Alex Newman
|
16b2c72d57 | fix(codex): repair startup hooks and stale mcp config | ||
|
Alex Newman
|
edc5cf7d59 |
Ponytail audit (−10.4k lines) + worker-restart hardening + deterministic dependency closure (#3021)
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure Approach A — commit & ship plugin/bun.lock so the plugin's runtime node_modules install is deterministic, fixing the recurring `Cannot find module 'zod/v3'` (#2730). - align generated plugin zod range to root (^4.4.3) in build-hooks.js - new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a build artifact after build-hooks.js writes plugin/package.json - track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist) - install with `bun install --frozen-lockfile --ignore-scripts` at runtime Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 2: fail loud at install time on a broken dependency closure Strengthen verifyCriticalModules to assert each dependency is actually importable via require.resolve (not merely a directory), and assert the worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini. A partial/stale install now fails `npx claude-mem install` immediately instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`. Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry point) fall back to resolving <dep>/package.json so a healthy install isn't falsely rejected. Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3 subpath (throws), a complete zod (passes), and a bin-only dep (passes). Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop) Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script. Against fresh temp dirs (never the repo's node_modules) it: - copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`, asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled worker asserting no `Cannot find module` — the direct #2730 regression guard; - `npm pack`s, installs the tarball into a second temp dir, and load-tests the published bin entrypoint, warning loudly on any declared main/exports target missing from the tarball (latent #2537 gap). Exits non-zero naming the missing module on any failure; cleans up all temp dirs and the tarball in a finally. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 4: gate CI and publish on the clean-room dependency closure - ci.yml: new `clean-room-deps` job (between build and the docker e2e job) runs a frozen-lockfile drift check on the committed plugin lockfile, then `npm run build` + `npm run smoke:clean-room`. The drift step catches a contributor who changed plugin deps without regenerating plugin/bun.lock. - npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between build and `npm publish`, so a broken runtime closure cannot be published on a tag push (ci.yml does not run on tags). Secrets block untouched. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: doc recluster note + Phase 0 execution slice for #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: gen-plugin-lockfile degrades gracefully when bun is absent The Windows build CI job has no bun on PATH; regenerating the lockfile there threw and failed the build. The committed plugin/bun.lock is already the deterministic closure, so skip regeneration (non-fatal) when bun is missing and a lockfile exists; fail loud only when neither is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main (v13.5.1) + plan-10 work Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main v13.5.5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): daily upgrade pass — agent SDK 0.3.172, better-auth 1.6.16, posthog-node 5.36.15, dompurify 3.4.9 - Bump @anthropic-ai/claude-agent-sdk 0.2.141 -> 0.3.172 (tsc + full test suite green) - Remove deprecated @types/dompurify stub (dompurify ships its own types) - Add overrides.tmp ^0.2.7 to clear GHSA-52f5-9888-hmc6 / GHSA-ph9p-34f9-6g65 via np -> listr-input -> inquirer -> external-editor -> tmp chain - npm audit: 0 vulnerabilities; npm outdated: clean - package-lock.json is gitignored in this repo, so not committed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * plan: worker-restart single-source-of-truth — 7-phase fix for restart races Phased plan from the adversarially-verified diagnosis (wf_f07f3541-b05): kill the cache mirror, single verified restart initiator, self-replacing restart endpoint, unified spawn gate with lockfile, PID-file demotion, test data-dir isolation, soak verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): delete sync-script cache-mirror and HTTP restart trigger Phase 1 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The installed-version cache mirror wrote version-N code into the version-(N-1) cache dir, manufacturing permanent version disagreement; the HTTP POST to /api/admin/restart raced the CLI restart that follows it in build-and-sync. Both are deleted; the CLI worker:restart in the marketplace copy is now the single restart initiator, and the sleep 1 between the two mechanisms is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): restart proves itself or exits 1 Phase 2 of plans/2026-06-10-worker-restart-single-source-of-truth.md. worker-service restart now captures the old worker pid, waits for the port with the same platform-scaled 15s budget as stop, spawns the marketplace copy of worker-service.cjs when present, then polls /api/health until the pid changes and the version matches this build's baked __DEFAULT_PACKAGE_VERSION__ — success is printed to stdout, deadline (platform-scaled 30s) exits 1 with the last observed health payload and the spawned script path. The --daemon generic start-failure path now exits 1 instead of masquerading as success; the three duplicate-suppression exits remain 0. New helper src/services/restart-verify.ts (worker-service.ts bootstraps on import, so the helper lives in an import-safe module) with 8 tests covering pid-flip success, stale pid, wrong version, unreachable timeout, 503-degraded acceptance, and null-oldPid version-only verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): self-replacing worker — old worker spawns its successor Phase 3 of plans/2026-06-10-worker-restart-single-source-of-truth.md. /api/admin/restart was kill-only: hooks that POSTed it then raced the dying worker with their own lazy-spawn (the observed recycle ping-pong). Now the dying worker spawns its successor itself — after a re-entrancy- guarded, deadline-bounded (platform-scaled 10s) graceful shutdown, and only once its port is confirmed free; stop and signal shutdowns stay kill-only. The hook recycle path waits for that successor via /api/health polling (HOOK_READINESS_TIMEOUT_MS budget) and lazy-spawns only as a fallback, with a warn-only version re-check so a hook never recycles more than once per invocation. Shutdown sequence lives in import-safe src/services/worker-shutdown.ts (worker-service.ts bootstraps on import); registerSignalHandlers no longer pre-sets isShuttingDown — the supervisor's shutdownInitiated guard owns signal dedupe, and pre-setting would no-op the new entry guard. 13 new tests cover re-entrancy, deadline expiry/rejection, handoff ordering, kill-only reasons, successor-wait vs lazy-spawn fallback, and pre-graceful bookkeeping failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): one spawn gate; CLI restart defers to the self-replacing worker Phase 4 of plans/2026-06-10-worker-restart-single-source-of-truth.md. Three uncoordinated spawn paths (hook lazy-spawn, MCP worker-spawner, CLI) with two different bun resolvers produced 3-launcher collisions within a single second. Now a wx-flag lockfile (<DATA_DIR>/spawn.lock, 60s mtime staleness with re-stat-before-unlink, owner-checked release) gates every external spawn: lock losers never fail — they skip the spawn and wait for the winner's worker. resolveBunRuntime is deleted in favor of ProcessManager's resolveWorkerRuntimePath (adds BUN_PATH, ~/.bun/bin, brew, which fallbacks), closing the kill-then-can't-respawn path; mcp-server prefers the marketplace worker script so stale cache dirs stop spawning stale workers. Integration fix surfaced by live verification: the CLI restart raced the Phase 3 self-replacement handoff (the successor re-binds the port in ~200ms, so waitForPortFree always timed out and restart exited 1 while the restart had actually succeeded). The CLI now verifies the worker's self-spawned successor directly, and only spawns — gate- wrapped, after the port frees — as the fallback when no worker was running, the shutdown POST was rejected, or no successor appeared. The dying worker's handoff is intentionally ungated: it spawns only after its own port closes, and hooks wait on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): demote the PID file — health and port are the liveness oracle Phase 5 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The dying worker's shutdown cascade deleted the PID file unconditionally as its final act, clobbering the successor's freshly-written file; status then required portInUse AND pidInfo, so a healthy worker reported as "not running". Now every PID-file deletion is owner-guarded: the supervisor cascade deletes only its own pid (removeOwnedPidFile), and the CLI stop/restart-fallback, the restart handoff, and the daemon start-failure cleanup go through removePidFileIfOwner (owner-or-dead — a live successor's file always survives; corrupt files are left for the next boot's validator). status sources from GET /api/health alone (pid, version, uptime, workerPath; 503-degraded counts as running and now surfaces its queue detail), with port-in-use-but-unreachable and not-running fallbacks — all exit 0 as before. The --daemon duplicate gate checks the port first (ground truth) and the PID file second (advisory, for the freed-port-but-undeleted-file window); duplicate suppression stays exit 0. writePidFile/touchPidFile remain — the file is diagnostics, and the worker stays its only writer. Also fixes combined-run test pollution: spawn-gate and worker-utils timeout tests now eagerly import paths.js before setting a temp CLAUDE_MEM_DATA_DIR, so the import-time DATA_DIR const can't freeze on a deleted temp dir for suites loaded later in the same bun process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: no test ever touches the real ~/.claude-mem again Phase 6 of plans/2026-06-10-worker-restart-single-source-of-truth.md. process-manager and graceful-shutdown tests wrote corrupt JSON and sentinel PIDs (2147483647) into the real ~/.claude-mem/worker.pid and drove the real supervisor.json cascade under a snapshot-restore that a killed run would skip — that pollution contaminated production logs and a prior diagnosis. Both files now set a temp CLAUDE_MEM_DATA_DIR at the top of the file before dynamically importing the code under test (ESM hoisting makes beforeEach too late), assert their paths landed outside the real dir, and derive PID_FILE from the same frozen paths module the code uses so test and code can never diverge under bun's shared module cache. The snapshot-restore scaffolding is deleted; zero assertions changed. tests/preload.ts gains a tripwire: when CLAUDE_MEM_DATA_DIR is unset it fills a per-run temp dir, so no test in any file can fall through to the real data dir. Fallout made explicit: worker-spawn child processes get an explicit temp dir; install-error-matrix restores rather than deletes the env var; settings-defaults-manager pins the unset-env default it was implicitly relying on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): bootstrap notices go to stderr, never stdout CI on PR #2894 caught the latent bug: on the first boot in a fresh data dir, SettingsDefaultsManager printed '[SETTINGS] Created settings file with defaults: ...' to stdout before the start command's JSON hook payload, corrupting the machine-readable contract every fresh install's first hook invocation relies on. The Phase 6 per-run temp data dir made the cold-dir case deterministic in CI, exposing it. Both informational notices (creation, nested-schema migration) now use console.warn — stderr — matching the function's existing failure-path idiom; two regression tests pin stdout silence on both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): address PR #2894 review — dedupe script resolver, skip futile port wait Both inline copies of the marketplace-first script-candidate list in worker-service.ts (restart fallback + successor handoff injection) now call the exported resolveWorkerScriptPath() ?? __filename, so the candidate list lives in one place. verifyRestartedWorker's failure result gains lastPollSawHealth; when the self-replacement handoff verification timed out while a live (but unverifiable) worker was still serving on the port, the CLI fallback now skips its port-free wait — the port cannot free while that worker lives, so the wait only burned its full platform-scaled budget before the same final verification ran anyway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): stale Claude CLI can no longer silently kill every observation findClaudeExecutable validated candidates with --version only, so an abandoned old install shadowing a current CLI in PATH (e.g. npm-global 2.0.42 next to the auto-updating native installer) passed validation, then died at every Observer spawn: the SDK passes --permission-mode dontAsk (hardened-options) which old CLIs reject with exit 1. Result: healthy worker, zero observations, no visible error (#2782 family; previously #1857/#2049/#1866/#2142 in the same class). - Probe every candidate (which -a + known install paths) with `--permission-mode dontAsk --version`: one spawn proves both flag compatibility and version, no API call (~150ms) - Prefer the newest capable version; PATH order only breaks ties - Explicit CLAUDE_CODE_PATH still wins but fails loud with version and remedy when too old, instead of dying silently at spawn - All-too-old throws an error naming each candidate, its version, and how to fix; resolution success logs at INFO with the chosen version - Cache successful resolution 15 min (resolver runs per SDK query); never cache failure so a CLI update is picked up without restart - SDK child keeps a 2KB stderr tail and includes it in the exit WARN, so "unknown option" deaths are diagnosable at default log level Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(audit): add ponytail audit master plan + SQLite removal companion Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-0): remove commit-hash fabrication verification Removes the commit-verification system whose second-system effects (greedy hex regex false-positives + compensating scaffolding) cost more than the edge case it caught. Deletes src/sdk/commit-verification.ts + its 2 tests, the verify-before-persist block + stripFabricatedHashesFromSummary in ResponseProcessor, and the now-dead fabricated_count/fabrication_* telemetry in buffer.ts/scrub.ts/npx-cli telemetry.ts. Consequence: PostHog Fabrication Rate tile goes dark (no other producer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-1): remove duplicate SQLite stack, test the shipping path Two parallel SQLite systems lived in src/services/sqlite/. The worker only ever used SessionStore (which runs its own inline migrations); the second stack — ClaudeMemDatabase/MigrationRunner + a free-function CRUD API — was reachable only from tests. This removes the dead duplicate and rewrites its tests against the real SessionStore path. Deleted: Database.ts, migrations/runner.ts, index.ts, SchemaRepair.ts, all 6 CRUD barrels, transactions.ts, and the sessions/summaries/prompts/timeline/import submodules (18 files). Trimmed the 5 survivor leaf functions to their single live export. Removed dead SessionStore methods getSessionSummaryById (broken) and storeObservationsAndMarkComplete (0 callers). Replaced 9 old test files (coupled to the dead stack) with 7 focused session-store-*.test.ts suites + a rewritten cleanup-v12_4_3 test, all exercising the worker's real path. Net ~-3,262 src lines. tsc clean; full suite 2253 pass / 0 fail; dead-reference sweep zero. No production behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-6): drop dead storage methods, dedupe EnvManager, stdlib swaps Dead-code deletes (all zero-caller verified): sqlite ProjectsRepository.upsert + getByRootPath, ServerSessionsRepository.getByMemorySessionId, MemoryItemsRepository.getByLegacyObservationId, pg AgentEventsRepository.createMany + TeamsRepository.getMember; shared getWorkerSocketPath, createBackupFilename, and the deprecated ENV_FILE_PATH const. Refactors (behavior-preserving): EnvManager's three open-coded 5-key credential blocks collapse to one CREDENTIAL_KEYS loop (whitelist semantics kept, no Object.assign); parseEnvFile line-parser → util.parseEnv; fetchWithTimeout's manual setTimeout race → AbortSignal.timeout (caller timeout-message contract preserved). Net ~-106 lines. tsc clean; full suite 2253 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-4): sweep dead/redundant code from src/server Deletes (zero-caller verified): jobs/outbox.ts (canonical path is IngestEventsService/EndSessionService) + its test; the src/server/mcp/ surface (live MCP server is src/servers/mcp-server.ts) + its test; the always-inert ServerBetaProviderRegistry/EventBroadcaster boundaries (no Active variant exists) + their /v1/info payload keys; the unused isServerClassified guard; dead option sessionDebounceWindowMs (route + IngestEventsService). Dedupe: hoist the 5 byte-identical auth helpers shared by auth.ts/postgres-auth.ts into request-auth-helpers.ts; collapse resolveSummaryQueue/resolveEventQueue into one resolveQueue(lane). Kept (plan premise didn't verify): the ServerV1Routes runtime? option — a live smoke test sets and asserts it. Disabled* boundary subclasses kept (real Active counterparts chosen at runtime). Net ~-570 source lines. tsc clean; full suite 2242 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-7): dead-code sweep + dedup across services + npx-cli Deletes (zero-caller verified, tests removed in lockstep): ProcessManager getChildProcesses + parseElapsedTime (ps-reaper leftovers), ModeManager getObservationConcepts/validateType/getTypeLabel, AgentFormatter renderAgentFileHeader/renderAgentColumnKey/renderAgentContextIndex (return-[] inert, HeaderRenderer arms collapsed to Human-only), asyncHandler export, PriorMessages.userMessage (always ''), and 3 dead barrels (services/context, services/server, services/server/Middleware). Dedup: shared toError() helper (src/utils/to-error.ts) across 8 in-scope sites; asMs hoisted to telemetry/common; spawnPlugin() folds 4 spawnHidden blocks; countObserverSessionRows extracts the CleanupV12_4_3 count trio; IS_WINDOWS imported from paths.ts (was redeclared 3x); detectOsVersion inlined to os.release(). Kept (plan premise didn't verify): bun-resolver's spawnSync (npx-cli runs under Node — Bun.which would ReferenceError); context-generator.ts (a build entrypoint + test mock seam) — barrel still removed; Middleware.ts consumer Server.ts repointed to the real middleware. Net ~-279 lines. tsc clean; full suite 2232 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-5): delete orphans + dead UI/util code across cli/supervisor/ui Orphan deletes (zero-caller verified): src/bin/ (import-xml-observations, cleanup-duplicates — not wired to any entrypoint), src/adapters/ (abandoned parallel impl; real adapters are src/cli/adapters/), core/schemas/context-pack.ts + core/schemas/index.ts barrel. Companion tests removed in lockstep. Dead functions: 7 cursor-utils helpers (kept the 6 live registry/context writers), logger correlationId/sessionId/timing. YAGNI trims: gemini-cli metadata block (never read), useStats collapsed to fire-and-forget (+ dead Stats/WorkerStats/DatabaseStats types), useTheme resolvedTheme, useContextPreview.refresh from the public result; env-sanitizer ENV_PROXY_VARS folded into ENV_EXACT_MATCHES. Dedup: useGitHubStars + formatNumber inlined into their single consumer GitHubStarsButton (kept lowercase-k display — matches GitHub, Intl would uppercase); waitForExit extracted in process-registry and shared with shutdown's reapSession (placed low to avoid a circular import). Net ~-1085 lines (incl. test deletions). tsc + viewer typecheck clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-2a): integrations/sync dead exports + scoped Chroma cleanup Dead-export deletes (zero-caller verified): syncContextToAgentsMd, updateWindsurfContextForProject, handleWindsurfCommand, detectClaudeCode (+ orphaned exec/promisify imports), getScriptExtension/detectPlatform, getVersionCheckAbsolutePath (+ companion plugin-distribution test edit). ChromaSyncState flush/resetCache/dirty (bump/replace persist eagerly, so dirty was always false). Scoped to the Chroma write path only (SQLite/search/telemetry copies stay live): dropped the created_at ISO field (kept created_at_epoch, the field actually read back) and the discovery_tokens param/field from ChromaSync's local Stored interfaces + the 3 live call sites. Stdlib/shrink: parseSemver/compareSemver → localeCompare numeric (ordering verified); OpenClaw roots hoisted to one const; bootstrapWatermarks max-loop → Math.max; Goose YAML builders collapsed to one withHeader flag. Defers the 5-installer consolidation to a separate pass. Net ~-187 lines. tsc clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-2b): delete dead fetchAndInjectOpenCodeContext Orphaned when slice-2a removed syncContextToAgentsMd (its only caller chain). Confirmed zero callers. Kept fetchRealContextFromWorker (live at :323). Per feasibility analysis, the plan's 5-installer table-driven consolidation (headline -2400) was descoped: the 5 installers write 5 different config schemas and 4 different context formats with thin test coverage, so a 5->1 collapse would relocate divergence into callbacks and risk untested IDE-config write paths for a line-count-only win. Plan gate ('only if behavior genuinely identical') not met. tsc clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-3): worker dead-code sweep + provider/search consolidation Search pipeline: delete the unreachable half built speculatively (TimelineBuilder, ResultFormatter instance methods [kept the one live static], search/filters/, SearchStrategy.canHandle/name + BaseSearchStrategy, HybridSearchStrategy no-op search(), dead SearchOrchestrator wrappers, ChromaQueryResult) + companion tests. Dedup the 5 genuinely-identical chroma->recency->hydrate->FTS paths into one hybridSemanticHydrate helper and the 3 timeline renderers into renderTimeline() (search()/decisions()/changes()/howItWorks() left as-is — different shape). Providers: extract OpenAICompatibleProvider base from the Gemini/OpenRouter twins, preserving every per-provider divergence via flags/abstracts (truncation guard, token estimation, empty-response handling, endpointClass, the Gemini-only RPM throttle). Dedup parseRetryAfterMs into retry.ts. http/agents/session: delete the IngestEventBus machinery + ingestPrompt, the read-only POST /api/processing dup, FallbackErrorHandler.shouldFallbackToClaude, RateLimitStore.getAll/clear, SessionCompletionHandler.completeByDbId, SessionCleanupHelper, 3 dead agent types, and no-op empty-passthrough validateBody; collapse the 6 CorpusRoutes 404 blocks into one helper. Kept (false-dead, caller verified): SSEEventPayload (live via WorkerRef), RateLimitStore.get (live via shouldAbortForQuota). Both adversarial reviews confirmed behavior preserved. Net ~-2,640 lines. tsc clean; full suite 2080 pass / 0 fail; dead-ref sweep zero. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(plugin): regenerate distribution artifacts after ponytail audit Rebuilds the worker/mcp/server-beta/transcript-watcher/context-generator bundles and the viewer UI from the post-audit source (Slices 0-7). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(env): require Node >=20.12.0 to match util.parseEnv usage EnvManager adopted stdlib util.parseEnv (added in Node 20.12.0) during the ponytail audit, but engines.node still advertised >=20.0.0. On Node 20.0-20.11 parseEnv is undefined: loadClaudeMemEnv() silently returns {} (credentials never load) and saveClaudeMemEnv() throws. Declare the real floor in both the npm package and the generated plugin manifest (via its generator in build-hooks.js so rebuilds stay consistent). Greptile PR #3021 P1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
a521dd97f6 |
build: demote bundle-size budgets from hard failures to advisory warnings
The thresholds were eyeballed headroom numbers, and a drifted local node_modules could fail a release build on source that CI had already verified. The sizes still print and warn so a sudden jump (a server-only dep leaking into a bundle, #2584/#1645) stays visible without blocking. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
c0b96288a7 |
fix(restart): worker-restart single source of truth — self-replacing worker, spawn gate, verified restarts (#2894)
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure Approach A — commit & ship plugin/bun.lock so the plugin's runtime node_modules install is deterministic, fixing the recurring `Cannot find module 'zod/v3'` (#2730). - align generated plugin zod range to root (^4.4.3) in build-hooks.js - new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a build artifact after build-hooks.js writes plugin/package.json - track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist) - install with `bun install --frozen-lockfile --ignore-scripts` at runtime Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 2: fail loud at install time on a broken dependency closure Strengthen verifyCriticalModules to assert each dependency is actually importable via require.resolve (not merely a directory), and assert the worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini. A partial/stale install now fails `npx claude-mem install` immediately instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`. Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry point) fall back to resolving <dep>/package.json so a healthy install isn't falsely rejected. Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3 subpath (throws), a complete zod (passes), and a bin-only dep (passes). Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop) Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script. Against fresh temp dirs (never the repo's node_modules) it: - copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`, asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled worker asserting no `Cannot find module` — the direct #2730 regression guard; - `npm pack`s, installs the tarball into a second temp dir, and load-tests the published bin entrypoint, warning loudly on any declared main/exports target missing from the tarball (latent #2537 gap). Exits non-zero naming the missing module on any failure; cleans up all temp dirs and the tarball in a finally. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 4: gate CI and publish on the clean-room dependency closure - ci.yml: new `clean-room-deps` job (between build and the docker e2e job) runs a frozen-lockfile drift check on the committed plugin lockfile, then `npm run build` + `npm run smoke:clean-room`. The drift step catches a contributor who changed plugin deps without regenerating plugin/bun.lock. - npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between build and `npm publish`, so a broken runtime closure cannot be published on a tag push (ci.yml does not run on tags). Secrets block untouched. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: doc recluster note + Phase 0 execution slice for #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: gen-plugin-lockfile degrades gracefully when bun is absent The Windows build CI job has no bun on PATH; regenerating the lockfile there threw and failed the build. The committed plugin/bun.lock is already the deterministic closure, so skip regeneration (non-fatal) when bun is missing and a lockfile exists; fail loud only when neither is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main (v13.5.1) + plan-10 work Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main v13.5.5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): daily upgrade pass — agent SDK 0.3.172, better-auth 1.6.16, posthog-node 5.36.15, dompurify 3.4.9 - Bump @anthropic-ai/claude-agent-sdk 0.2.141 -> 0.3.172 (tsc + full test suite green) - Remove deprecated @types/dompurify stub (dompurify ships its own types) - Add overrides.tmp ^0.2.7 to clear GHSA-52f5-9888-hmc6 / GHSA-ph9p-34f9-6g65 via np -> listr-input -> inquirer -> external-editor -> tmp chain - npm audit: 0 vulnerabilities; npm outdated: clean - package-lock.json is gitignored in this repo, so not committed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * plan: worker-restart single-source-of-truth — 7-phase fix for restart races Phased plan from the adversarially-verified diagnosis (wf_f07f3541-b05): kill the cache mirror, single verified restart initiator, self-replacing restart endpoint, unified spawn gate with lockfile, PID-file demotion, test data-dir isolation, soak verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): delete sync-script cache-mirror and HTTP restart trigger Phase 1 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The installed-version cache mirror wrote version-N code into the version-(N-1) cache dir, manufacturing permanent version disagreement; the HTTP POST to /api/admin/restart raced the CLI restart that follows it in build-and-sync. Both are deleted; the CLI worker:restart in the marketplace copy is now the single restart initiator, and the sleep 1 between the two mechanisms is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): restart proves itself or exits 1 Phase 2 of plans/2026-06-10-worker-restart-single-source-of-truth.md. worker-service restart now captures the old worker pid, waits for the port with the same platform-scaled 15s budget as stop, spawns the marketplace copy of worker-service.cjs when present, then polls /api/health until the pid changes and the version matches this build's baked __DEFAULT_PACKAGE_VERSION__ — success is printed to stdout, deadline (platform-scaled 30s) exits 1 with the last observed health payload and the spawned script path. The --daemon generic start-failure path now exits 1 instead of masquerading as success; the three duplicate-suppression exits remain 0. New helper src/services/restart-verify.ts (worker-service.ts bootstraps on import, so the helper lives in an import-safe module) with 8 tests covering pid-flip success, stale pid, wrong version, unreachable timeout, 503-degraded acceptance, and null-oldPid version-only verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): self-replacing worker — old worker spawns its successor Phase 3 of plans/2026-06-10-worker-restart-single-source-of-truth.md. /api/admin/restart was kill-only: hooks that POSTed it then raced the dying worker with their own lazy-spawn (the observed recycle ping-pong). Now the dying worker spawns its successor itself — after a re-entrancy- guarded, deadline-bounded (platform-scaled 10s) graceful shutdown, and only once its port is confirmed free; stop and signal shutdowns stay kill-only. The hook recycle path waits for that successor via /api/health polling (HOOK_READINESS_TIMEOUT_MS budget) and lazy-spawns only as a fallback, with a warn-only version re-check so a hook never recycles more than once per invocation. Shutdown sequence lives in import-safe src/services/worker-shutdown.ts (worker-service.ts bootstraps on import); registerSignalHandlers no longer pre-sets isShuttingDown — the supervisor's shutdownInitiated guard owns signal dedupe, and pre-setting would no-op the new entry guard. 13 new tests cover re-entrancy, deadline expiry/rejection, handoff ordering, kill-only reasons, successor-wait vs lazy-spawn fallback, and pre-graceful bookkeeping failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): one spawn gate; CLI restart defers to the self-replacing worker Phase 4 of plans/2026-06-10-worker-restart-single-source-of-truth.md. Three uncoordinated spawn paths (hook lazy-spawn, MCP worker-spawner, CLI) with two different bun resolvers produced 3-launcher collisions within a single second. Now a wx-flag lockfile (<DATA_DIR>/spawn.lock, 60s mtime staleness with re-stat-before-unlink, owner-checked release) gates every external spawn: lock losers never fail — they skip the spawn and wait for the winner's worker. resolveBunRuntime is deleted in favor of ProcessManager's resolveWorkerRuntimePath (adds BUN_PATH, ~/.bun/bin, brew, which fallbacks), closing the kill-then-can't-respawn path; mcp-server prefers the marketplace worker script so stale cache dirs stop spawning stale workers. Integration fix surfaced by live verification: the CLI restart raced the Phase 3 self-replacement handoff (the successor re-binds the port in ~200ms, so waitForPortFree always timed out and restart exited 1 while the restart had actually succeeded). The CLI now verifies the worker's self-spawned successor directly, and only spawns — gate- wrapped, after the port frees — as the fallback when no worker was running, the shutdown POST was rejected, or no successor appeared. The dying worker's handoff is intentionally ungated: it spawns only after its own port closes, and hooks wait on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): demote the PID file — health and port are the liveness oracle Phase 5 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The dying worker's shutdown cascade deleted the PID file unconditionally as its final act, clobbering the successor's freshly-written file; status then required portInUse AND pidInfo, so a healthy worker reported as "not running". Now every PID-file deletion is owner-guarded: the supervisor cascade deletes only its own pid (removeOwnedPidFile), and the CLI stop/restart-fallback, the restart handoff, and the daemon start-failure cleanup go through removePidFileIfOwner (owner-or-dead — a live successor's file always survives; corrupt files are left for the next boot's validator). status sources from GET /api/health alone (pid, version, uptime, workerPath; 503-degraded counts as running and now surfaces its queue detail), with port-in-use-but-unreachable and not-running fallbacks — all exit 0 as before. The --daemon duplicate gate checks the port first (ground truth) and the PID file second (advisory, for the freed-port-but-undeleted-file window); duplicate suppression stays exit 0. writePidFile/touchPidFile remain — the file is diagnostics, and the worker stays its only writer. Also fixes combined-run test pollution: spawn-gate and worker-utils timeout tests now eagerly import paths.js before setting a temp CLAUDE_MEM_DATA_DIR, so the import-time DATA_DIR const can't freeze on a deleted temp dir for suites loaded later in the same bun process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: no test ever touches the real ~/.claude-mem again Phase 6 of plans/2026-06-10-worker-restart-single-source-of-truth.md. process-manager and graceful-shutdown tests wrote corrupt JSON and sentinel PIDs (2147483647) into the real ~/.claude-mem/worker.pid and drove the real supervisor.json cascade under a snapshot-restore that a killed run would skip — that pollution contaminated production logs and a prior diagnosis. Both files now set a temp CLAUDE_MEM_DATA_DIR at the top of the file before dynamically importing the code under test (ESM hoisting makes beforeEach too late), assert their paths landed outside the real dir, and derive PID_FILE from the same frozen paths module the code uses so test and code can never diverge under bun's shared module cache. The snapshot-restore scaffolding is deleted; zero assertions changed. tests/preload.ts gains a tripwire: when CLAUDE_MEM_DATA_DIR is unset it fills a per-run temp dir, so no test in any file can fall through to the real data dir. Fallout made explicit: worker-spawn child processes get an explicit temp dir; install-error-matrix restores rather than deletes the env var; settings-defaults-manager pins the unset-env default it was implicitly relying on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): bootstrap notices go to stderr, never stdout CI on PR #2894 caught the latent bug: on the first boot in a fresh data dir, SettingsDefaultsManager printed '[SETTINGS] Created settings file with defaults: ...' to stdout before the start command's JSON hook payload, corrupting the machine-readable contract every fresh install's first hook invocation relies on. The Phase 6 per-run temp data dir made the cold-dir case deterministic in CI, exposing it. Both informational notices (creation, nested-schema migration) now use console.warn — stderr — matching the function's existing failure-path idiom; two regression tests pin stdout silence on both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): address PR #2894 review — dedupe script resolver, skip futile port wait Both inline copies of the marketplace-first script-candidate list in worker-service.ts (restart fallback + successor handoff injection) now call the exported resolveWorkerScriptPath() ?? __filename, so the candidate list lives in one place. verifyRestartedWorker's failure result gains lastPollSawHealth; when the self-replacement handoff verification timed out while a live (but unverifiable) worker was still serving on the port, the CLI fallback now skips its port-free wait — the port cannot free while that worker lives, so the wait only burned its full platform-scaled budget before the same final verification ran anyway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
YOMXXX
|
04832dcfa9 |
fix(build): add transcript-watcher.cjs build target (closes #2450) (#2592)
* fix(build): add transcript-watcher.cjs build target (closes #2450) src/npx-cli/commands/runtime.ts:230 hard-codes a reference to plugin/scripts/transcript-watcher.cjs, but scripts/build-hooks.js never compiled this entry. The fallback `spawnBunWorkerCommand` path silently no-ops on systems where bun isn't reachable, so `claude-mem transcript watch` is effectively dead in the published npm bundle. This adds: - A thin entry (src/services/transcripts/transcript-watcher-entry.ts) that dispatches process.argv to the existing runTranscriptCommand function in cli.ts (which until now had no caller in the build). - A new esbuild target in build-hooks.js modeled on context-generator (Node 18 CJS, minified, bun:sqlite external) plus the standard stripHardcodedDirname pass and chmod 0755. - Updated build summary so the new artifact shows up in the "compiled successfully" footer. The compiled bundle is 137 KB. `node plugin/scripts/transcript-watcher.cjs` prints the usage line and exits 1 as expected; `... validate`, `... init`, and `... watch` all route correctly. No behavior change on platforms that previously had a working bun fallback - the explicit path now just wins the existsSync check. * fix(build): externalize zod + add bundle-size guard for transcript-watcher Addresses Greptile review on #2592: 1. Add `zod` to the external list. worker-service and server-beta-service both externalize zod and rely on plugin/package.json to provide it at runtime; the watcher build was inconsistent. The current processor.ts chain does not actually pull in zod (bundle size unchanged at 137.64 KB), but the external entry is defensive — any future import in src/services/transcripts/ that touches zod will now resolve against the plugin's declared dependency rather than getting silently inlined (and potentially creating a duplicate-instance hazard). 2. Add a 200 KB size guard mirroring the 600 KB ceiling on mcp-server.cjs. The watcher is meant to be a thin file-tail loop; if a transitive import ever drags a heavy module (worker-service, an SDK runtime) into the chain, this catches it at build time with an informative error pointing at processor.ts / watcher.ts. No functional change to the runtime artifact (size identical, behavior identical). Both follow-ups recommended by Greptile. |
||
|
Alex Newman
|
07ac3a54d5 |
fix(mcp): cross-platform Node launcher for plugin .mcp.json (closes #2792) (#2807)
* fix(mcp): cross-platform Node launcher for plugin .mcp.json (closes #2792) plugin/.mcp.json used `command: "sh"` with an `sh -c` discovery prelude, which Claude Code cannot spawn on Windows when Git's usr/bin is not on PATH — so the mcp-search tools never registered (#2792/#2790/#2714/#2461). Replace it with a pure-Node `command: "node"` + `node -e <payload>` launcher emitted by buildMcpNodeLauncher() in hook-shell-template.ts. The payload mirrors the POSIX prelude's plugin-root discovery order ($CLAUDE_PLUGIN_ROOT/$PLUGIN_ROOT, extra candidates, mtime-sorted cache roots, then the marketplace dir), spawns the resolved mcp-server.cjs with inherited stdio, forwards SIGTERM/SIGINT/SIGHUP, and propagates the child's exit code/signal. No shell dependency. The generator remains the single source of truth: build-hooks.js verifies plugin/.mcp.json args[1] matches buildShellCommand({host:'mcp'}), and the plugin-distribution tests are updated to assert the Node payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): address review — exit 1 on signal re-raise failure, drop unused trailingCommand Greptile review of #2807: - Signal-kill fallback now exits 1 (not 0) when re-raising the child's terminating signal throws, so an unexpected kill is no longer reported to the host as clean success. Regenerated plugin/.mcp.json to match. - trailingCommand is now optional and intentionally ignored by the mcp Node launcher (spawn target derives from requireFile). Documented in buildMcpNodeLauncher, dropped the meaningless shell tokens from the mcp call sites, and made shell hosts fail loud if trailingCommand is missing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
9efb04da84 |
fix(hooks): bun-runner.js parses on pre-ES2020 Node (closes #2791) (#2801)
The Stop hook crashes every session with 'SyntaxError: Unexpected token .' when a host invokes bun-runner.js under a bundled pre-ES2020 Node whose ESM loader rejects optional chaining (?.). Replace the only ?. site with explicit guards and add a build-time guard forbidding ?. / ?? in this launcher so it can't regress. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
cf450cec00 |
fix(build): enforce shipped dependency-closure boundary (plan-10, closes #2783) (#2800)
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure Approach A — commit & ship plugin/bun.lock so the plugin's runtime node_modules install is deterministic, fixing the recurring `Cannot find module 'zod/v3'` (#2730). - align generated plugin zod range to root (^4.4.3) in build-hooks.js - new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a build artifact after build-hooks.js writes plugin/package.json - track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist) - install with `bun install --frozen-lockfile --ignore-scripts` at runtime Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 2: fail loud at install time on a broken dependency closure Strengthen verifyCriticalModules to assert each dependency is actually importable via require.resolve (not merely a directory), and assert the worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini. A partial/stale install now fails `npx claude-mem install` immediately instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`. Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry point) fall back to resolving <dep>/package.json so a healthy install isn't falsely rejected. Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3 subpath (throws), a complete zod (passes), and a bin-only dep (passes). Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop) Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script. Against fresh temp dirs (never the repo's node_modules) it: - copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`, asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled worker asserting no `Cannot find module` — the direct #2730 regression guard; - `npm pack`s, installs the tarball into a second temp dir, and load-tests the published bin entrypoint, warning loudly on any declared main/exports target missing from the tarball (latent #2537 gap). Exits non-zero naming the missing module on any failure; cleans up all temp dirs and the tarball in a finally. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 4: gate CI and publish on the clean-room dependency closure - ci.yml: new `clean-room-deps` job (between build and the docker e2e job) runs a frozen-lockfile drift check on the committed plugin lockfile, then `npm run build` + `npm run smoke:clean-room`. The drift step catches a contributor who changed plugin deps without regenerating plugin/bun.lock. - npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between build and `npm publish`, so a broken runtime closure cannot be published on a tag push (ci.yml does not run on tags). Secrets block untouched. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: doc recluster note + Phase 0 execution slice for #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: gen-plugin-lockfile degrades gracefully when bun is absent The Windows build CI job has no bun on PATH; regenerating the lockfile there threw and failed the build. The committed plugin/bun.lock is already the deterministic closure, so skip regeneration (non-fatal) when bun is missing and a lockfile exists; fail loud only when neither is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
fb869ffd7c |
build(sdk): cmem-sdk package skeleton + tsup build + import guard (phase 2)
Wires up the deferred SDK export slot from commit
|
||
|
Alex Newman
|
166b956e88 |
Merge remote-tracking branch 'origin/main' into mutual-aardvark
# Conflicts: # package.json # plugin/scripts/context-generator.cjs # plugin/scripts/mcp-server.cjs # plugin/scripts/server-service.cjs # plugin/scripts/worker-service.cjs # plugin/ui/viewer-bundle.js # src/npx-cli/commands/install.ts # src/server/runtime/ServerService.ts # src/server/runtime/create-server-service.ts # src/services/worker-service.ts # tests/hooks/server-client.test.ts |
||
|
Alex Newman
|
34c734a1a4 |
refactor(server): rename server-beta → server runtime (plan phase 1)
Foundational rename: the server runtime is now `server`, not `server-beta`.
Removes the literal-string regression in `runtime-selector.ts` where only
`CLAUDE_MEM_RUNTIME='server-beta'` was accepted (anything else silently
fell back to the worker runtime).
What changed (1a–1d per plans/2026-05-25-cmem-sdk-and-server-rename.md):
- 1a Regression fix: `selectRuntime()` accepts both `'server'` and
`'server-beta'` (canonicalizing to `'server'`); new settings keys
`CLAUDE_MEM_SERVER_{URL,API_KEY,PROJECT_ID}` read first, legacy
`*_BETA_*` keys fall back via a `pickFirstNonEmpty` helper.
- 1b Code identifiers: ~80 `ServerBeta*`/`serverBeta*`/`SERVER_BETA_*`
symbols (classes, types, functions, vars, non-persisted constants) →
`Server*`/`server*`/`SERVER_*`.
- 1c File renames: 14 files moved via `git mv` (tracked as renames);
build target `server-beta-service` → `server-service` (emits
`plugin/scripts/server-service.cjs`); dispatch sites in
`worker-service.ts`, `runtime.ts`, `ServerService.ts` keep a documented
`existsSync` fallback to the legacy `.cjs` for installs running from a
pre-rename plugin cache (plan §1c line 149).
- 1d Persisted-value back-compat (zero-risk path):
- DB table name `server_beta_schema_migrations` preserved
- Job/source enum strings (`server_beta_generate_event`, …) preserved
- `lockedBy: 'server-beta-worker'` literal preserved
- Installer writes new canonical settings keys + `'server'` runtime
value going forward; reads dual-accept old + new.
Verification:
- Typecheck: 24/24 baseline errors unchanged (no new errors introduced).
- Tests: 1810 pass / 54 fail / 19 skip — failure count matches the
pre-Phase-1 baseline; no new failures introduced.
- Build: `npm run build` succeeds; `plugin/scripts/server-service.cjs`
emitted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Alex Newman
|
8151cd5a8f |
fix(spawn): plan-02 canonical ${CLAUDE_PLUGIN_ROOT} resolution + Windows spawn fixes
- Centralize Rule B absolute-path bake (src/services/integrations/install-paths.ts) used by Cursor/Gemini/Windsurf/Mcp installers; centralize Rule A shell template (src/build/hook-shell-template.ts). - Build-time guardrail asserts hooks.json/codex-hooks.json/.mcp.json match the canonical generator (prevents drift); validation matrix test executes the resolution pipeline. - #2696: quote cmd.exe uvx args so 'protobuf<7'/'onnxruntime>=1.20' aren't parsed as redirection on Windows (ChromaMcpManager). - #2695: Codex CLI spawn uses shell:true + quoting on Windows so PATHEXT resolves codex.cmd (no more spawnSync ENOENT). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
9c56dda79f |
fix(env): plan-06 worker env isolation — block effort vars, spawn-env discipline, $TIER
- Add CLAUDE_CODE_EFFORT_LEVEL / CLAUDE_CODE_ALWAYS_ENABLE_EFFORT to BLOCKED_ENV_VARS (defense-in-depth for #2357, alongside the CLAUDE_CODE_* prefix filter). - Document the OAuth-skip predicate contract (#2375 leak path already closed upstream). - lint:spawn-env CI check forbids spawning children with raw process.env; fixed 6 real violations (worker-service, ServerBetaService, process-registry, runtime x3) to use sanitizeEnv. - $TIER alias (#2289): resolveTierAlias + CLAUDE_MEM_TIER_FAST/SMART_MODEL settings, wired into ClaudeProvider + KnowledgeAgent model resolution. - Anchor bugs #2375/#2357 verified already-fixed with tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
7a33685c46 |
fix(install): plan-04 installer failure transparency + diagnostic CLI
- Explicit failure taxonomy + single installerError() decision point replacing ad-hoc swallow sites; 'Installation Complete' only when all ABORT-level deps satisfied, else 'Installation Partial' with remediation. - npm install strict-first; --legacy-peer-deps only on confirmed ERESOLVE, announced. - Missing uv/bun = ABORT with platform-specific remediation as primary message. - Postinstall allowlist guard (scripts/check-postinstall-allowlist.js) prevents a re-run of the tree-sitter-swift install hang. - 48-cell IDE x failure-mode test matrix. - #2548: 'npx claude-mem doctor' read-only diagnostic probe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
6821d85245 |
refactor(hooks): plan-01 hook IO discipline — typed stdout/stderr/exit channels
- src/shared/hook-io.ts: single module routing intent (DIAGNOSTIC/MODEL_CONTEXT/ USER_HINT/BLOCKING_FEEDBACK/EXIT_SIGNAL) -> correct channel; only place that calls console.log/process.stderr.write/process.exit for the hook path. - Handlers are pure (return HookResult); hookCommand orchestrates hook-io. - Fix #2292: recordWorkerUnreachable now surfaces via emitBlockingError instead of being swallowed by the stderr no-op; exit-code strategy preserved verbatim. - Quiet-on-success: buffered library stderr dropped on graceful exit (#2694). - lint:hook-io CI check forbids direct stream writes in handlers/** and adapters/**. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
7a6eaca565 |
fix(build): plan-10 artifact hygiene — green typecheck, drop better-auth from worker, CI gate
- Fix 24 TS errors (logger Component union, Express 5 query params, clack spinner.stop arity, React 19 ref typing, dead SdkSessionRecord import). #2538 - Externalize better-auth from worker bundle (3.0MB -> 2.29MB). #2584 - Add worker-bundle size guard in build-hooks.js + .github/workflows/ci.yml running typecheck/build/test as required checks. #2570 #2538 - Tighten npm files allowlist / .npmignore so maintainer CLAUDE.md never ships. #2537 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
e53d1530ff |
fix(mcp): drop root .mcp.json so plugin's mcp-search isn't duplicated (#2411)
The repo shipped both a root-level .mcp.json and plugin/.mcp.json with
identical mcp-search launchers — kept in sync by a build-time guard and
a test. The root file was a holdover from when devs working inside the
repo could load mem-search without installing the plugin. With the
plugin universally installed, every plugin user now sees `/doctor` warn:
Plugin (claude-mem @ plugin:claude-mem:mcp-search): MCP server
"mcp-search" skipped — same command/URL as already-configured
"mcp-search"
…because Claude Code dedupes by command and skips the plugin's
namespaced registration. The duplicate is functionally harmless but
suppresses the canonical `plugin:claude-mem:mcp-search` entry.
This removes the root .mcp.json entirely and re-points everything that
referenced it at the bundled plugin copy:
- .mcp.json: deleted
- .codex-plugin/plugin.json: mcpServers → ./plugin/.mcp.json
- package.json: drop .mcp.json from files
- scripts/build-hooks.js: drop root-file requirement + sync check
- scripts/sync-marketplace.cjs: drop syncManagedFiles entry
- src/npx-cli/commands/install.ts: drop from allowedTopLevelEntries
- tests/infrastructure/plugin-distribution.test.ts: drop two tests
enforcing the now-removed root file
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Alex Newman
|
e7bbb2a9aa |
server-beta: Phases 4–13 — event pipeline, generation, MCP, compat, Docker, team audit, observability (#2383)
* feat(server-beta): Phase 4 — Postgres event-to-generation-job pipeline Adds POST /v1/events, /v1/events/batch, GET /v1/jobs/:id, GET /v1/events/:id, and POST /v1/memories on the server-beta runtime, backed by Postgres. - Event row + outbox generation-job row insert in one withPostgresTransaction. - BullMQ enqueue happens after commit; enqueue failure leaves the row queued for Phase 3 startup reconciliation. - ?generate=false skips the outbox; ?wait=true returns queue status only, never observation IDs (provider generation is Phase 5). - Batch pre-validates all event projectIds against api-key scope before any write; mixed-project batches reject 403 with zero side effects. - /v1/memories is a direct insert alias — no generator, no outbox. - Cross-tenant /v1/jobs/:id returns 404 to avoid leaking row existence. - New PostgresAuthMiddleware reads api_keys by SHA-256 hash; populates req.authContext.teamId/projectId; legacy ServerV1Routes (SQLite, used by worker runtime) is left untouched. - Tests: unit suite hardened with stubbed pool.query so route registration is safe; integration tests skip cleanly without CLAUDE_MEM_TEST_POSTGRES_URL. Verification: 87 pass / 1 skip / 0 fail. No new typecheck errors. Required greps for WorkerService and MemoryItemsRepository in src/server/routes/v1 and src/server/runtime return no hits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 5 — provider observation generator Adds independent provider generation under src/server/generation/ with no worker coupling. Server beta can now generate observations end-to-end: event -> outbox -> BullMQ -> provider -> parser -> persisted observation. - ProviderObservationGenerator orchestrates: lock outbox (queued -> processing), reload agent_event from Postgres (BullMQ payload is advisory only), call provider, hand raw text to processGeneratedResponse, route errors via markGenerationFailed with retryable flag from ServerClassifiedProviderError. - processGeneratedResponse parses with parseAgentXml, persists via PostgresObservationRepository with deterministic generation_key = generation:v1:{job_id}:{index}:{fingerprint}, links via PostgresObservationSourcesRepository, advances outbox status, appends observation_generation_job_events, audits — all in one withPostgresTransaction. Idempotent on retry via UNIQUE constraints. - Three provider adapters under src/server/generation/providers/: Claude, Gemini, OpenRouter. Self-contained — no imports from src/services/worker/*. Worker providers unchanged. - Shared error classification + prompt builder under providers/shared/. Prompt builder strips <private> at the edge; fully-private batches emit <skip_summary /> without billing the provider. - ActiveServerBetaGenerationWorkerManager wires BullMQ Worker via ServerJobQueue.start(...) with concurrency 1 + autorun:false + worker.on('error') per BullMQ docs. - New GET /v1/events/:id/observations on ServerV1PostgresRoutes returns observations linked via observation_sources, team/project scoped. Verification: 104 pass / 4 skip / 0 fail. No typecheck regressions. Anti-pattern greps clean for services/worker imports under src/server, WorkerRef/ActiveSession/SessionStore in src/server/generation. Deferred: ModeManager loading uses a stable fallback observation type list; summary and reindex queue lanes are not yet wired. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 6 — independent server session semantics server_sessions is now the canonical Server beta session model. Sessions are independent of legacy worker ActiveSession state. - PostgresServerSessionRepository extended: findByExternalIdForScope, endSession (idempotent via COALESCE(ended_at, now())), markGenerationStarted/Completed/Failed, listUnprocessedEvents (filters agent_events with completed agent_event jobs). - ServerSessionRuntimeRepository wraps the repo; every method requires explicit team_id + project_id and validates scope via assertProjectOwnership. - SessionGenerationPolicy supports per-event (default), debounce (BullMQ delayed-job replace via getJob+remove+add), and end-of-session. Configured via CLAUDE_MEM_SERVER_SESSION_POLICY and CLAUDE_MEM_SERVER_SESSION_DEBOUNCE_MS env vars; per-team override hooks are exposed on ServerV1PostgresRoutesOptions for future settings layer. - POST /v1/sessions/start (find-or-create on (project_id, external_session_id), GET /v1/sessions/:id (scoped 404), POST /v1/sessions/:id/end (transactional: end + create summary outbox via UNIQUE collapse + enqueue post-commit). Re-ending is fully idempotent. - processSessionSummaryResponse persists summary as kind='summary' observation with the same idempotency model (generation_key + observation_sources UNIQUE). - ProviderObservationGenerator dispatches on source_type: agent_event -> processGeneratedResponse, session_summary -> processSessionSummaryResponse; loadEvents handles session-summary by loading unprocessed events. - ActiveServerBetaGenerationWorkerManager wires summary BullMQ lane alongside event lane (concurrency=1, autorun=false, error listener attached per BullMQ docs). Verification: 110 pass / 6 skip / 0 fail. Net typecheck error count unchanged at 24 (pre-existing, none in Phase 6 files). Anti-pattern greps clean for ActiveSession/SessionStore in src/server/runtime, no worker imports anywhere in src/server. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 7 — hook routing without worker dependency Hooks can now talk directly to server-beta when CLAUDE_MEM_RUNTIME=server-beta is selected, with a clean worker fallback when server-beta is unhealthy. - src/services/hooks/server-beta-client.ts — typed HTTP client for /v1/sessions/start, /v1/events, /v1/sessions/:id/end. Throws ServerBetaClientError with kind classification (missing_api_key, transport, timeout, http_error, invalid_response) and isFallbackEligible helper. Zero imports from services/worker/. - src/services/hooks/runtime-selector.ts — reads CLAUDE_MEM_RUNTIME from settings, returns worker or server-beta context, logs [server-beta-fallback] reason=<code> on every config-time fallback. - src/services/hooks/server-beta-bootstrap.ts — Postgres-backed API key bootstrap. Find-or-creates local-hook-team + local-hook-project, generates cmem_<random> key (SHA-256 hashed), inserts into api_keys with scopes events:write/sessions:write/observations:read/jobs:read. Settings file written with chmod 0600. rotateServerBetaApiKey() wired to a new `claude-mem server keys rotate` command. - src/cli/handlers/{observation,session-init,summarize}.ts — every hook handler tries server-beta first when configured, falls through to the existing worker path on transport/5xx/429/missing-key. One WARN line per fallback. Hook JSON output shape unchanged. - src/shared/SettingsDefaultsManager.ts — three new keys with defaults: CLAUDE_MEM_SERVER_BETA_URL, CLAUDE_MEM_SERVER_BETA_API_KEY, CLAUDE_MEM_SERVER_BETA_PROJECT_ID. - src/npx-cli/commands/install.ts — when installer selects server-beta runtime and CLAUDE_MEM_SERVER_DATABASE_URL is set, bootstraps a local API key automatically. Warns and continues if the DB URL is missing. plugin/scripts/*.cjs bundles rebuilt via npm run build to pick up the new hook handler code path. No plaintext keys in the bundle (verified). Verification: 16 hook unit tests pass; 275 server/storage/services tests pass with 7 pre-existing failures (verified independent of this change via git stash --include-untracked). Build clean. No new typecheck errors in Phase 7 files. Anti-pattern guards verified: - /api/sessions/observations only reached via explicit fallback path - server-beta runtime never starts the worker process - API keys live only in ~/.claude-mem/settings.json (chmod 0600), never in the bundle (grep confirmed) - Worker fallback preserved, observable via single WARN line per call Deferred: semantic context injection (UserPromptSubmit hook) stays worker-only; server-beta does not yet expose /v1/context/semantic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 8 — MCP backed by server-beta core MCP tools now route through server-beta in server-beta mode while keeping worker-mode search/timeline/get_observations tools fully working. - src/servers/mcp-server.ts — five new observation_* tools registered: observation_add, observation_record_event, observation_search, observation_context, observation_generation_status. Three memory_* compatibility aliases delegate to the canonical handlers. Worker auto-start is gated when selectRuntime() === 'server-beta' so MCP in server-beta mode never spawns the worker. - src/services/hooks/server-beta-client.ts — addObservation, searchObservations, contextObservations, getJobStatus added so MCP shares one transport with hooks (Phase 7). - src/server/routes/v1/ServerV1PostgresRoutes.ts — POST /v1/search and POST /v1/context REST cores backed by PostgresObservationRepository full-text search (GIN tsvector from Phase 1). - Existing memory_search/timeline/get_observations tools call callWorkerAPI unchanged in worker mode; worker tests unaffected. Verification: 39 pass / 4 skip / 0 fail on targeted suite. Pre-existing 7 baseline failures verified independent (git stash). No new typecheck errors. WorkerService grep clean across src/servers/mcp-server.ts and src/server/. Anti-pattern guards verified: - No duplicate generation logic in MCP — observation_record_event hits /v1/events which owns event+outbox+enqueue inside one tx - WorkerService not imported anywhere under MCP server-beta path - No hardcoded worker URLs — all transport via Phase 7 ServerBetaClient - memory_* aliases retained, single handler per pair Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 9 — compatibility adapters without coupling Legacy /api/sessions/observations and /api/sessions/summarize endpoints keep working on server-beta runtime by translating to AgentEvent and session-end calls — no worker code, no route duplication. - src/server/services/IngestEventsService.ts — shared event-ingest path used by both /v1/events and the compat adapter. Owns transactional event row + outbox row + lifecycle log + post-commit BullMQ enqueue, honors Phase 6 SessionGenerationPolicy. - src/server/services/EndSessionService.ts — shared session-end path used by both /v1/sessions/:id/end and the compat adapter. Idempotent ended_at + summary outbox + deterministic summary job id. - src/server/compat/SessionsObservationsAdapter.ts — translates legacy POST /api/sessions/observations payload (Claude Code transcript shape) -> AgentEvent (source_adapter='claude-code-compat', event_type='tool_use') -> IngestEventsService.ingestOne. Resolves contentSessionId to server_sessions via find-or-create. - src/server/compat/SessionsSummarizeAdapter.ts — translates legacy POST /api/sessions/summarize -> EndSessionService.end. Preserves the legacy agentId -> {status:'skipped', reason:'subagent_context'} behavior so existing clients see the same response shape. - src/server/routes/v1/ServerV1PostgresRoutes.ts — refactored to delegate to the new shared services (-203 LoC net) so /v1 and /api compat both call the SAME canonical code path. - src/server/runtime/ServerBetaService.ts — registers both compat adapters alongside ServerV1PostgresRoutes, sharing service instances. - docs/server-beta-parity-map.md — full enumeration of legacy /api/* routes labeled native, adapter, or unsupported (with reasons). Viewer read-path adapters explicitly listed as unsupported pending a future viewer-rewrite phase. Verification: 7 compat tests pass, 6 v1-routes tests still pass (refactor preserved behavior), 4 session-routes tests pass. Pre- existing 16 baseline failures verified independent via git stash. Zero new typecheck errors. Anti-pattern guards verified: - No services/worker/http/routes or WorkerService imports under src/server/compat or src/server/runtime - Compat adapters are thin translators with names ending in *Adapter and a top-of-file comment noting they are legacy compatibility - /v1/* remains the canonical Server beta API; compat adapters call shared services rather than acting as a parallel API Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 10 — Docker stack and deployable runtime Server beta now ships as a Docker stack with no worker process anywhere and a separate horizontal generation worker for scaling. - src/server/runtime/create-server-beta-service.ts — validateServerBetaEnv() fails fast on missing CLAUDE_MEM_SERVER_DATABASE_URL, requires CLAUDE_MEM_QUEUE_ENGINE=bullmq in Docker, rejects CLAUDE_MEM_AUTH_MODE=local-dev and CLAUDE_MEM_ALLOW_LOCAL_DEV_BYPASS inside containers (detected via /.dockerenv or CLAUDE_MEM_DOCKER=1). Adds CLAUDE_MEM_GENERATION_DISABLED so the HTTP service can run generator-free. - src/server/runtime/ServerBetaService.ts — runServerBetaGenerationWorker for the dedicated consumer process; runServerBetaApiKeyCli is a new Postgres-backed `server api-key` command (the legacy worker CLI wrote to SQLite and was invisible to the Postgres runtime); getQueueHealth shim feeds /api/health a consistent ObservationQueueHealth shape. - src/npx-cli/commands/{runtime,server}.ts — `claude-mem server worker start` subcommand that boots only the BullMQ consumer. - docker/claude-mem/{Dockerfile,entrypoint.sh} — entrypoint forces CLAUDE_MEM_DOCKER=1 + CLAUDE_MEM_RUNTIME=server-beta and exposes three modes: server (HTTP only, generation disabled), worker (BullMQ consumer), shell. Worker bundle is no longer the default CMD. - docker-compose.yml — full stack: postgres + valkey + claude-mem-server (HTTP-only) + claude-mem-worker (generation consumer). Wires service-to-service env vars. - scripts/e2e-server-beta-docker.sh + docker/e2e/server-beta-e2e.mjs — E2E now hits /v1/sessions/start, /v1/events?wait=true, /v1/jobs/:id; asserts no worker-service.cjs process anywhere in the stack; one-shot docker compose run --rm verifies local-dev auth is rejected with the expected stderr; restart-and-verify confirms Postgres durability and BullMQ retry idempotency. - docs/server.md — full Phase 10 doc: stack diagram, env table, worker mode, auth-in-Docker policy. - docs/api.md — event generation semantics (wait=true, generationJob). Verification: full Docker E2E PASSED on live daemon (phase1 + phase2 + restart-and-verify + revoked-key + no-worker- process + local-dev-rejected). Unit tests 292 pass / 9 skip / 7 fail (7 fails pre-existing baseline). Zero new typecheck errors. Anti-pattern guards verified: - entrypoint never execs worker-service.cjs; E2E greps prove no worker process anywhere in the stack - validateServerBetaEnv refuses local-dev auth in Docker with explicit remediation message; ALLOW_LOCAL_DEV_BYPASS rejected the same way - Docker requires CLAUDE_MEM_QUEUE_ENGINE=bullmq; in-process queue rejected at startup - claude-mem worker / worker-service / WorkerService greps clean in docker/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 11 — team-aware generation with audit chain Generation jobs now carry team_id/project_id/api_key_id/actor_id/ source_adapter from enqueue through execution; the outbox is reloaded from Postgres before any side effect so BullMQ payload can never act as auth authority. - src/server/jobs/types.ts — ServerGenerationJobPayloadSchema (Zod discriminated union) requires team_id, project_id, generation_job_id, source_adapter, api_key_id, actor_id (nullable), source_type, source_id, plus event_id / server_session_id per kind. assertServerGenerationJobPayload is called at enqueue (outbox.ts) and again at execution boundary. - src/server/services/{IngestEventsService,EndSessionService}.ts + SessionGenerationPolicy.ts — thread identity context (apiKeyId, actorId, sourceAdapter) into both event and summary BullMQ payloads. - src/server/generation/ProviderObservationGenerator.ts — loadCanonicalOutbox loads the outbox row WITHOUT scope filter, then compares candidate.team_id/project_id to payload.team_id/project_id; mismatch -> ServerGenerationScopeViolationError (non-retryable), failed status, generation_job.scope_violation audit. isApiKeyRevoked checks api_keys (revoked_at, expires_at, row missing) before any provider call; revoked -> generation_job.revoked_key audit + non- retryable failure. generation_job.processing audit emitted on lock. - src/server/generation/processGeneratedResponse.ts — generated observations carry team_id/project_id/server_session_id from the reloaded source row (not job payload). observation_sources.metadata records source_adapter, actor_id, api_key_id for traceability. observation.created audit per observation; generation_job.completed audit per terminal transition. All audit rows reference the same generation_job_id in details. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/teams/:id/jobs and GET /v1/projects/:id/jobs with SQL-layer scoping (WHERE team_id=$1 [AND project_id=$2] [AND status=$3]); cross-tenant returns 404 to avoid leaking row existence. Pagination via status/limit/offset. audit_log rows for event.received, event.batch_received, observation.read. - src/server/compat/{SessionsObservationsAdapter,SessionsSummarizeAdapter}.ts — propagate apiKeyId and sourceAdapter='claude-code-compat'. Verification: 162 pass / 10 skip / 0 fail. Pre-existing failures in tests/services/queue and tests/services/worker confirmed independent via git stash. Zero new typecheck errors in server-beta files. Required greps: rg "team_id.*req\.body|project_id.*req\.body" src/server -> 0 matches Audit chain integration test passes — generation_job.processing, observation.created, and generation_job.completed audit rows all share the same generation_job_id reference. Anti-pattern guards verified: - BullMQ payload never acts as auth authority — Postgres outbox reload with mismatch check happens before every side effect - team_id / project_id never derived from request body for scope decisions; always req.authContext.teamId / projectId - Application-layer team/project filtering forbidden — listJobsForScope pushes scope into the SQL WHERE clause - Project-scoped key on cross-project /v1/teams/:id/jobs returns 404 - Revoked api keys cause non-retryable failure with audit before any provider call Deferred: a redundant generation_job.queued audit_log row (already covered by observation_generation_job_events lifecycle log per Phase 1 schema split). Compat adapters set actor_id=null but propagate api_key_id which is the canonical reference downstream. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 12 — observability and operations Operators can now inspect, retry, and cancel generation jobs from the CLI; queue lane metrics flow into /api/health and /v1/info; every request gets a stable request_id that flows through HTTP -> audit -> outbox -> generator -> completion log. - src/server/middleware/request-id.ts — honors safe inbound X-Request-Id, mints uuid v4 otherwise. Set on req.requestId and echoed via response header so external traces can correlate. - src/server/jobs/ServerJobQueue.ts — QueueEvents wired with completed, failed, progress, stalled, error listeners; lifecycle counters exposed via observe() API. Logs emitted as [generation] job=<id> source_type=<...> duration=<ms> attempts=<N> reason=<message>. Stalled and error counters survive worker restart. - src/server/jobs/types.ts — ServerGenerationJob payload schema extended with optional request_id; flows through from HTTP into every BullMQ job. - src/server/queue/ObservationQueueEngine.ts — health snapshot now carries per-lane (event, summary) counts via ObservationQueueHealthLaneSnapshot. - src/server/runtime/{ActiveServerBetaQueueManager, ActiveServerBetaGenerationWorkerManager,ServerBetaService}.ts — per-lane getJobCounts feed /api/health and /v1/info; stalled events audit through audit_log with action generation_job.stalled. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/jobs (status/source_type/since/limit/offset, scope from api-key, payload stripped unless ?include=payload AND admin scope), POST /v1/jobs/:id/retry (idempotent; queued -> no-op; audit generation_job.retried_by_operator), POST /v1/jobs/:id/cancel (terminal -> no-op; audit generation_job.cancelled_by_operator; generator reload-before-side-effects already prevents double work). - src/server/services/IngestEventsService.ts + SessionGenerationPolicy.ts + ProviderObservationGenerator.ts — request_id propagated end to end. Generator extracts request_id from BullMQ payload and includes it in lock/processing/completion logs and audit details. - src/npx-cli/commands/server-jobs.ts + src/npx-cli/commands/server.ts — `claude-mem server jobs status|failed|retry|cancel`. status compares Postgres outbox counts to BullMQ queue counts and surfaces divergence. failed prints attempts + last_error message. --team and --project filters. Verification: 350 pass / 12 skip / 7 fail (pre-existing baseline, verified independent via git stash). 18 new tests added (request-id middleware, server-jobs CLI seams, jobs list/retry/cancel routes Postgres-gated). Zero new typecheck errors. Anti-pattern guards verified: - agent_events.payload only emitted in /v1/jobs response inside the admin-gated branch (?include=payload + admin scope) — returns 403 otherwise - jobs retry on a queued row is a no-op (no double BullMQ enqueue, no double UPDATE) - Every operator action writes to audit_log with the *_by_operator action and request_id correlation in details - Stalled events audit through generation_job.stalled Sample correlated trace (one request_id end to end): HTTP middleware: req.requestId = 'req-abc' audit event.received: details.requestId = 'req-abc' BullMQ payload: { request_id: 'req-abc', generation_job_id: 'gj_x' } generator lock log: [generation] job locked { jobId, requestId } audit generation_job.processing: details.requestId = 'req-abc' completion log: [generation] job=evt_... duration=1230ms Deferred: live /api/health round-trip integration test (needs Redis); stalled event live integration test (needs Redis); storing request_id on the observations row itself (spec did not require). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(server-beta): add Phase 13 release readiness report Captures the final verification gate: tests (1749 pass, 45 fail all pre-existing baseline, zero regressions), required greps clean, Docker E2E green end-to-end, all 7 exit criteria met, build clean, typecheck unchanged from main. Documents deferred items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild server-beta-service bundle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile review on PR #2383 - ProviderObservationGenerator.lockOutbox: skip duplicate worker run when another lock is active instead of returning the row, which previously let two BullMQ workers issue the (paid, rate-limited) external provider call before the persistence-layer terminal-status guard collapsed the duplicate. Reconciliation still recovers from a stale lock on startup or next retry. - docker-compose.yml: require POSTGRES_USER/PASSWORD/DB env vars (no defaults). Stack refuses to start without explicit secrets. Added a header warning that the file must not be deployed unmodified. - e2e-server-beta-docker.sh: export ephemeral test creds for the new required env vars so the Docker E2E driver still runs unattended. - ServerBetaService api-key list: bound query with LIMIT/OFFSET (default 100, max 500) and add optional --team filter to prevent unintentional cross-tenant key metadata disclosure on shared admin hosts. - SessionGenerationPolicy: fix dead `??` fallback for NaN parseInt result; use `||` so DEFAULT_DEBOUNCE_MS actually applies. - ServerV1PostgresRoutes: `?wait=true` now actually waits — polls the outbox row until terminal status (timeout 30s, 100ms interval) on both /v1/events and /v1/events/batch. Returns `waitTimedOut: true` if the cap is hit so callers can re-poll the status endpoints. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address CodeRabbit + Greptile second review on PR #2383 P1 fixes - Operator retry endpoint was re-publishing the Postgres outbox metadata column as the BullMQ payload; the worker's assertServerGenerationJobPayload always rejected it, leaving the row stuck in queued until startup reconciliation. Persist the BullMQ payload on the outbox row at create-time inside IngestEventsService and EndSessionService, then re-enqueue that canonical payload on retry. Major fixes - prompt-builder: escape server_session_id when interpolating into the XML prompt; previously a session id containing `<`, `&`, or quotes could inject XML into the provider input. - ServerJobQueue: route both worker.on('stalled') and the QueueEvents 'stalled' subscriber through a single notifyStalled helper that dedupes by jobId for 30s, so counters.stalled increments once per stall. QueueEvents 'error' now routes through notifyQueueError so it increments counters.errored and runs onError listeners — keeping observability symmetric across both sources. - ServerV1PostgresRoutes: convert PostgresObservationRepository from three dynamic imports to a single static import for consistency. - mcp-server / ServerBetaClient: actually forward the observation_record_event tool's `generate` flag through to the /v1/events endpoint as `?generate=false` instead of voiding it. - server-sessions.markGenerationFailed: guard jsonb_set against a null error payload so the failure path can't null out metadata before the generation_status='failed' write commits. Minor fixes - server-sessions.endSession: keep updated_at stable on repeated calls so the documented idempotency contract holds. - SettingsDefaultsManager + ServerBetaService.getServerBetaPort: derive the server-beta default port from UID (37877 + uid%100), matching the worker port pattern, so two users on the same host don't collide. Docker stacks always pass CLAUDE_MEM_SERVER_PORT explicitly so the containerized deployment is unaffected. - server-session-runtime test: close the pg.Pool in afterAll. - server-beta-release-readiness.md: escape pipes inside table inline code, add `text` language tag to the fenced log block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile + CodeRabbit third review on PR #2383 P1 fixes - SessionsObservationsAdapter.resolveServerSession: catch unique-violation (23505) on concurrent compat inserts and re-fetch instead of returning 500. Two compat callers carrying the same contentSessionId can both observe `existing===null` and race on the (project_id, external_session_id) unique constraint; the second now resolves to the raced row instead of dropping the event. - /v1/events/batch: pass `sourceAdapter: null` to ingestBatch so each event's BullMQ payload (and persisted outbox payload column) reflects its own event.sourceAdapter via buildEventBullmqPayload's fallback, rather than stamping the whole batch with the first event's adapter. Minor - server-session-runtime test afterEach: wrap DROP SCHEMA in try/finally so client.release() always runs even if the drop throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): drop `pool as never` cast — pg.Pool already matches PostgresPool Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): retry of completed job now 409s instead of duplicating retryGenerationJob previously fell through to the reset+re-enqueue path when called on a job in `completed` status. The observations index dedupes on (generation_job_id, parsed_observation_index, content) but LLM output is non-deterministic, so a second provider run almost always produced a different content string and bypassed the index, persisting a parallel set of observation rows attributed to the same generation job. Match cancelGenerationJob's 409 guard for completed jobs. failed and cancelled remain valid retry targets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild bundles after rebase onto main Regenerates the three plugin bundles so they reflect the rebased source state. Mechanical rebuild output only — no source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): wrap resolveServerSession in try/catch for structured error response Greptile P1 on PR #2383: resolveServerSession was called before the try/catch in both compat adapters, so Postgres errors during session lookup (timeout, pool exhaustion, etc.) escaped to Express's default error handler and returned HTML/text 500s. Legacy clients calling response.json() would get a parse failure instead of the documented { stored: false, reason: 'internal_error' } (or { status: 'error', reason: 'internal_error' } for the summarize adapter) shape. Move the resolveServerSession call inside the existing try block in both adapters so any failure flows through the structured catch handler. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): catch 23505 unique violation in POST /v1/sessions/start Greptile P1 on PR #2383: concurrent requests with the same externalSessionId can both pass the findByExternalIdForScope check, both call repo.create, and the loser hits the (project_id, external_session_id) unique constraint. The handler treated that as an unknown error and returned a 500. Apply the same pattern resolveServerSession already uses: catch error.code '23505' when externalSessionId is set, refetch the row inserted by the winning request, and return 200 with that session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |