mirror of
https://github.com/thedotmack/claude-mem.git
synced 2026-09-20 04:23:02 +08:00
feat(sync): complete SyncHub-only client cutover
Retire legacy client adoption and per-kind sync state, establish the one-time launch boundary, and expose authenticated Hub health. Add bounded device metadata and a loopback-only two-client protocol-v2 matrix E2E.
This commit is contained in:
Alex Newman
+35
-36
@@ -44,14 +44,16 @@ that content must stay on your machine.
|
||||
**The database is the queue.** Every synced table carries a `synced_at` column
|
||||
(`NULL` = not in the log yet). After each write, the worker nudges a debounced
|
||||
flusher that drains `WHERE synced_at IS NULL` and stamps rows on success. That
|
||||
one mechanism **is** live sync, backfill, offline catch-up, and retry — a
|
||||
never-synced install simply has everything `NULL`, and anything that fails to
|
||||
upload stays `NULL` until a later flush picks it up.
|
||||
one mechanism handles live sync, offline catch-up, and retry. Rows written
|
||||
after the SyncHub launch boundary start as `NULL`; anything that fails to
|
||||
upload stays `NULL` until a later flush picks it up. Pre-launch local rows are
|
||||
not treated as a cloud migration corpus.
|
||||
|
||||
- **Debounced:** write bursts coalesce; the flusher runs ~1.5 s after the
|
||||
last write (250 ms while the speed layer is connected), and only one
|
||||
flush runs at a time.
|
||||
- **Batched:** ops drain in requests of up to 500 ops / 2 MB.
|
||||
- **Batched:** ops drain in requests of up to 500 ops / 4,000,000 encoded
|
||||
bytes. Each individual canonical body is capped at 256,000 encoded bytes.
|
||||
- **Timeboxed:** every request has a 30 s timeout — a dead network can
|
||||
never hang the worker.
|
||||
- **Retrying:** a failed upload leaves rows `NULL` and retries on the next
|
||||
@@ -109,10 +111,8 @@ turn sync off.
|
||||
|
||||
Run the `/cloud-sync` skill in Claude Code to check status and walk through
|
||||
setup. It writes the settings into `~/.claude-mem/settings.json` (mode
|
||||
`0600`) without ever echoing the token, restarts the worker, and polls status
|
||||
until the pending counts fall. If you previously used the legacy standalone
|
||||
sync client, the skill retires it and carries your device identity over — see
|
||||
[Migrating from the standalone client](#migrating-from-the-standalone-client).
|
||||
`0600`) without ever echoing the token, restarts the worker, and verifies the
|
||||
installed client can reach SyncHub.
|
||||
|
||||
## Settings
|
||||
|
||||
@@ -122,9 +122,13 @@ sync client, the skill retires it and carries your device identity over — see
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_USER_ID` | `''` | Your cmem.ai user id (from **cmem.ai → Connect**). All your devices share it — it names your hub log. |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_HUB_URL` | `''` | Sync hub base URL. **Empty = sync off.** |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_WS` | `'true'` | Advisory WebSocket speed layer. `'false'` = HTTP polling only; sync stays fully correct at poll latency. |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID` | `''` | Stable identity of this machine. Resolved at first start — adopted from the legacy client's state file if present, otherwise a fresh UUID — then persisted back here. Don't edit it: every row is attributed to its origin device, and a changed id forks every previously synced row into a duplicate. |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID` | `''` | Stable identity of this machine. A fresh UUID is minted and persisted on first start. Don't edit it: every row is attributed to its origin device. |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME` | machine hostname | Human-readable label for this device. |
|
||||
| `CLAUDE_MEM_CLOUD_SYNC_URL` | (legacy) | The pre-hub per-kind endpoint base. Unused since the hub cutover; kept only so existing settings files round-trip. |
|
||||
|
||||
A Hub accepts at most 64 distinct device ids per account. Existing devices
|
||||
continue to sync normally at the limit; a new device receives
|
||||
`409 device_limit_exceeded`. Status/metadata reads and renaming an unknown
|
||||
device do not create phantom devices.
|
||||
|
||||
## Status endpoint
|
||||
|
||||
@@ -143,44 +147,39 @@ Configured:
|
||||
{
|
||||
"configured": true,
|
||||
"deviceId": "2f6b1c9e-7d41-4c1a-9b0e-3d5f8a2c6e10",
|
||||
"pending": { "observations": 0, "summaries": 0, "prompts": 2, "mutations": 0 },
|
||||
"pending": { "observations": 0, "summaries": 0, "prompts": 2, "mutations": 0, "tombstones": 0 },
|
||||
"quarantine": { "count": 0, "latestReason": null },
|
||||
"lastFlushAt": 1783981042731,
|
||||
"lastError": null
|
||||
"lastError": null,
|
||||
"hub": {
|
||||
"checkedAt": 1783981042800,
|
||||
"reachable": true,
|
||||
"epoch": "1783981042000",
|
||||
"headSeq": "42",
|
||||
"projectedSeq": "42",
|
||||
"error": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Every configured status request makes an authenticated, read-only
|
||||
`GET /v1/sync/status` directly to SyncHub, including when all pending counts
|
||||
are zero. The probe does not append an operation or advance a pull cursor.
|
||||
`hub.reachable: false` and `hub.error` therefore expose a bad token, wrong Hub
|
||||
URL, malformed response, timeout, or network failure
|
||||
that an empty queue would otherwise hide.
|
||||
|
||||
- `pending` — rows (and queued mutation ops) still waiting to upload. Counts
|
||||
near 0 mean the hub's log has everything this device wrote.
|
||||
- `lastFlushAt` — epoch ms of the last successful flush, `null` before the first.
|
||||
- `lastError` — message from the most recent failed flush, `null` when healthy.
|
||||
It never contains the token.
|
||||
- `hub` — the most recent authenticated SyncHub probe. Treat
|
||||
`hub.reachable: true` as the connectivity check; `lastError: null` alone is
|
||||
not enough. Sequence and epoch fields remain decimal strings.
|
||||
|
||||
Not configured:
|
||||
|
||||
```json
|
||||
{ "configured": false }
|
||||
```
|
||||
|
||||
## Migrating from the standalone client
|
||||
|
||||
Before sync moved into the worker, cloud sync ran as a separate standalone
|
||||
client (`cloud-sync.mjs` plus a `.cloud-sync.env` credentials file and a
|
||||
`cloud-sync.pid` file). The worker supersedes it entirely, so the
|
||||
`/cloud-sync` skill retires it during setup:
|
||||
|
||||
- **Stopped:** if `cloud-sync.pid` points at a live daemon, it is killed.
|
||||
- **Retired:** `cloud-sync.mjs`, `.cloud-sync.env`, and `cloud-sync.pid` are
|
||||
renamed with a `.retired` suffix (archived, not deleted). Your token is copied
|
||||
from `.cloud-sync.env` into `settings.json` first, so setup needs no re-pasting.
|
||||
- **Preserved:** `~/.claude-mem/cloud-sync-state.json` is deliberately **left
|
||||
in place** — the worker adopts its `deviceId` into
|
||||
`CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID` so this machine keeps one identity across
|
||||
the migration. Renaming or deleting that file before the first worker start
|
||||
would mint a new device identity and fork every previously synced row into a
|
||||
duplicate.
|
||||
|
||||
<Note>
|
||||
On first contact with the hub, every device re-pushes its own corpus once
|
||||
(the hub's dedupe makes this safe) — that is how the shared log is built, and
|
||||
it doubles as the initial backfill of your existing database.
|
||||
</Note>
|
||||
|
||||
@@ -103,6 +103,7 @@
|
||||
"test:infra": "bun test tests/infrastructure/",
|
||||
"test:server": "bun test tests/server/",
|
||||
"e2e:server:docker": "bash scripts/e2e-server-docker.sh",
|
||||
"e2e:sync-matrix": "bun scripts/sync-matrix-e2e.ts",
|
||||
"check:postinstall-allowlist": "node scripts/check-postinstall-allowlist.js",
|
||||
"smoke:clean-room": "node scripts/smoke-clean-room.cjs",
|
||||
"prepublishOnly": "npm run build && node scripts/check-postinstall-allowlist.js",
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+163
-171
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+493
-484
File diff suppressed because one or more lines are too long
@@ -9,155 +9,99 @@ allowed-tools:
|
||||
|
||||
# Cloud Sync (cmem.ai Pro)
|
||||
|
||||
The worker syncs memories itself: every write nudges a background flusher that
|
||||
drains unsynced rows to cmem.ai. There is no daemon to install or babysit. This
|
||||
skill is a thin front-end — check status, and on first run collect credentials,
|
||||
retire the old standalone client, and restart the worker so it picks them up.
|
||||
The installed worker syncs through SyncHub. There is one client, one durable
|
||||
operation log, and no separate sync daemon. This skill checks status or writes
|
||||
the three connection values issued by **cmem.ai → Connect**.
|
||||
|
||||
**Security rule for this entire skill:** NEVER print the sync token, never put
|
||||
it in a command-line argument (argv is visible to other processes), and never
|
||||
log it. It travels only inside heredoc-fed stdin scripts or files that already
|
||||
hold it. When confirming, report its length — not its value.
|
||||
**Security rule:** never print the sync token, put it in argv, or log it.
|
||||
Confirm only its length. Preserve every unrelated setting and keep
|
||||
`~/.claude-mem/settings.json` mode `0600`.
|
||||
|
||||
## 1. Check status
|
||||
|
||||
Resolve the worker port (env → `~/.claude-mem/settings.json` → per-UID default
|
||||
`37700 + (uid % 100)`, matching how the worker picks its own port; re-run this
|
||||
in any fresh shell before the curls in step 5):
|
||||
Resolve the worker port and query the always-registered status route:
|
||||
|
||||
```bash
|
||||
PORT="${CLAUDE_MEM_WORKER_PORT:-$(node -e "const fs=require('fs'),p=require('path'),os=require('os');const uid=(typeof process.getuid==='function'?process.getuid():77);const fallback=String(37700+(uid%100));try{const s=JSON.parse(fs.readFileSync(p.join(os.homedir(),'.claude-mem','settings.json'),'utf-8'));process.stdout.write(String(s.CLAUDE_MEM_WORKER_PORT||fallback));}catch{process.stdout.write(fallback);}" 2>/dev/null)}"
|
||||
curl -s "http://127.0.0.1:${PORT}/api/sync/status"
|
||||
```
|
||||
|
||||
Responses:
|
||||
- `configured: true` and `hub.reachable: true` → the worker completed an
|
||||
authenticated `GET /v1/sync/status` against SyncHub. Report `deviceId`,
|
||||
pending counts, `lastFlushAt`, `lastError`, and the Hub head/checkpoint;
|
||||
stop unless the user asked to replace the connection.
|
||||
- `configured: true` and `hub.reachable: false` → report `hub.error` and say
|
||||
the SyncHub connection is not verified. A zero pending count or
|
||||
`lastError: null` is not success because an empty queue performs no push.
|
||||
- `configured: false` → continue.
|
||||
- Connection refused, 404, or 503 immediately after restart → retry every
|
||||
three seconds for about 30 seconds before diagnosing the worker.
|
||||
|
||||
- `{"configured": true, "deviceId": ..., "pending": {"observations": N, "summaries": N, "prompts": N}, "lastFlushAt": ..., "lastError": ...}` → go to step 2.
|
||||
- `{"configured": false}` → go to step 3.
|
||||
- **404 / 503 / connection refused** → the route registers late during worker
|
||||
startup, so a request right after a restart can miss it. Retry every 3s for
|
||||
~15s before concluding anything. If 404 persists, the running worker predates
|
||||
cloud sync — restart it with the command in step 5, wait, and retry.
|
||||
## 2. Obtain the connection
|
||||
|
||||
## 2. Already configured → report and stop
|
||||
Ask for all three values shown by **cmem.ai → Connect**:
|
||||
|
||||
Report the three pending counts, `lastFlushAt`, and `lastError` (null means
|
||||
healthy). Pending counts near 0 mean the cloud copy is current. Done — do not
|
||||
run the setup steps below.
|
||||
1. sync token;
|
||||
2. user id;
|
||||
3. SyncHub URL.
|
||||
|
||||
## 3. Not configured → obtain credentials
|
||||
The Hub URL must be an absolute `https://` URL. Do not substitute the cmem.ai
|
||||
application API URL; the installed client talks only to SyncHub.
|
||||
|
||||
Priority order:
|
||||
## 3. Write installed-client settings
|
||||
|
||||
**(a) Legacy standalone client present.** If `~/.claude-mem/.cloud-sync.env`
|
||||
exists, the user already set up the old standalone sync client. Tell them:
|
||||
"Found your existing standalone cloud-sync setup — migrating it into the
|
||||
worker. Your token and device identity carry over; nothing re-uploads." Do NOT
|
||||
`cat` the file or print its values. Migrate it with this script (it reads the
|
||||
file itself, so the token never enters the conversation):
|
||||
|
||||
```bash
|
||||
node - <<'EOF'
|
||||
const fs = require('fs'), os = require('os'), path = require('path');
|
||||
const dir = path.join(os.homedir(), '.claude-mem');
|
||||
const env = fs.readFileSync(path.join(dir, '.cloud-sync.env'), 'utf8');
|
||||
const get = (k) => (env.match(new RegExp('^' + k + '=(.*)$', 'm')) || [])[1]?.trim().replace(/^["']|["']$/g, '') || '';
|
||||
const token = get('CMEM_SYNC_TOKEN'), userId = get('CMEM_USER_ID');
|
||||
if (!token || !userId) { console.error('legacy env file is missing CMEM_SYNC_TOKEN or CMEM_USER_ID'); process.exit(1); }
|
||||
const file = path.join(dir, 'settings.json');
|
||||
const settings = fs.existsSync(file) ? JSON.parse(fs.readFileSync(file, 'utf8')) : {};
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_TOKEN = token;
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_USER_ID = userId;
|
||||
fs.writeFileSync(file, JSON.stringify(settings, null, 2) + '\n');
|
||||
fs.chmodSync(file, 0o600);
|
||||
console.log(`migrated: token length ${token.length}, user id length ${userId.length}`);
|
||||
EOF
|
||||
```
|
||||
|
||||
**(b) No legacy file.** Use AskUserQuestion to ask the user to paste two
|
||||
values from **cmem.ai → Connect**: their sync token and their user id. Then
|
||||
write them with the same merge script, embedding the two values as string
|
||||
literals inside the quoted heredoc (heredoc body is stdin, not argv — the
|
||||
token stays off the command line; do not echo it back afterward):
|
||||
Substitute the collected values inside this quoted stdin script. Do not echo
|
||||
them before or after running it:
|
||||
|
||||
```bash
|
||||
node - <<'EOF'
|
||||
const fs = require('fs'), os = require('os'), path = require('path');
|
||||
const token = 'PASTE_TOKEN_HERE';
|
||||
const userId = 'PASTE_USER_ID_HERE';
|
||||
const file = path.join(os.homedir(), '.claude-mem', 'settings.json');
|
||||
const hubUrl = 'PASTE_HUB_URL_HERE';
|
||||
if (!token || !userId || !/^https:\/\/[^\s]+$/.test(hubUrl)) {
|
||||
console.error('token, user id, and an https SyncHub URL are required');
|
||||
process.exit(1);
|
||||
}
|
||||
const dir = path.join(os.homedir(), '.claude-mem');
|
||||
const file = path.join(dir, 'settings.json');
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const settings = fs.existsSync(file) ? JSON.parse(fs.readFileSync(file, 'utf8')) : {};
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_TOKEN = token;
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_USER_ID = userId;
|
||||
fs.writeFileSync(file, JSON.stringify(settings, null, 2) + '\n');
|
||||
const target = settings.env && typeof settings.env === 'object' ? settings.env : settings;
|
||||
target.CLAUDE_MEM_CLOUD_SYNC_TOKEN = token;
|
||||
target.CLAUDE_MEM_CLOUD_SYNC_USER_ID = userId;
|
||||
target.CLAUDE_MEM_CLOUD_SYNC_HUB_URL = hubUrl.replace(/\/+$/, '');
|
||||
fs.writeFileSync(file, JSON.stringify(settings, null, 2) + '\n', { mode: 0o600 });
|
||||
fs.chmodSync(file, 0o600);
|
||||
console.log(`saved: token length ${token.length}, user id length ${userId.length}`);
|
||||
console.log(`saved cloud connection: token length ${token.length}, user id length ${userId.length}`);
|
||||
EOF
|
||||
```
|
||||
|
||||
Merge rules (both paths, non-negotiable): parse the existing JSON and merge
|
||||
the two keys in — never clobber other settings, never rewrite the file from a
|
||||
template, and restore file mode 0600 after writing (both scripts above do
|
||||
this).
|
||||
These are the only required connection keys. The worker mints and persists a
|
||||
device id on first start and defaults the device name to the hostname.
|
||||
|
||||
## 4. Retire the legacy daemon
|
||||
|
||||
The standalone client is superseded by worker-native sync. If it is still
|
||||
running it would double-upload, so shut it down and archive its artifacts:
|
||||
|
||||
```bash
|
||||
D="$HOME/.claude-mem"
|
||||
if [ -f "$D/cloud-sync.pid" ]; then
|
||||
LEGACY_PID=$(cat "$D/cloud-sync.pid")
|
||||
if [ -n "$LEGACY_PID" ] && ps -p "$LEGACY_PID" -o command= 2>/dev/null | grep -q cloud-sync; then
|
||||
kill "$LEGACY_PID"
|
||||
fi
|
||||
fi
|
||||
for f in cloud-sync.mjs .cloud-sync.env cloud-sync.pid; do
|
||||
[ -f "$D/$f" ] && mv "$D/$f" "$D/$f.retired"
|
||||
done
|
||||
ls "$D" | grep retired
|
||||
```
|
||||
|
||||
**Leave `~/.claude-mem/cloud-sync-state.json` exactly where it is.** The
|
||||
worker's migration stamps already-synced rows from its cursors and adopts its
|
||||
device id — renaming or deleting it forks every cloud row into a duplicate.
|
||||
|
||||
## 5. Restart the worker and watch it drain
|
||||
|
||||
The worker reads credentials at startup, so restart it:
|
||||
## 4. Restart and verify
|
||||
|
||||
```bash
|
||||
curl -s -X POST "http://127.0.0.1:${PORT}/api/admin/restart"
|
||||
```
|
||||
|
||||
The old worker spawns its own successor once its port closes — do not spawn or
|
||||
kill anything yourself; the POST is the entire restart. Then poll:
|
||||
Poll the status route every five seconds for up to 30 seconds while the
|
||||
successor starts. Success means `configured: true`, `hub.reachable: true`, and
|
||||
`lastError: null`. The local route always makes an authenticated, read-only
|
||||
SyncHub status probe, even when every pending count is zero; it never uses a
|
||||
legacy cmem.ai Pro status route and never appends or advances sync state.
|
||||
Pending counts describe only writes made after the SyncHub launch baseline;
|
||||
setup does not migrate a pre-launch local corpus.
|
||||
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:${PORT}/api/sync/status"
|
||||
```
|
||||
If `hub.reachable` is false, report `hub.error`. If `lastError` is non-null,
|
||||
report it too. Ask the user to verify the three values in **cmem.ai →
|
||||
Connect**. Never include the token.
|
||||
|
||||
every ~5s. Tolerate connection-refused/404 for the first ~30s (successor
|
||||
booting, route registering late). Expect `configured: true` with the pending
|
||||
counts falling as the flusher drains. Stop polling when either:
|
||||
## 5. Report
|
||||
|
||||
- all three pending counts reach 0, or
|
||||
- the counts stop changing across 3 consecutive polls with `lastError` null —
|
||||
a large first backfill flushes in batches and can take minutes; report the
|
||||
current counts and note the worker keeps draining in the background.
|
||||
|
||||
If `lastError` is non-null and pending is not moving, report the error text
|
||||
verbatim (it never contains the token) and suggest re-checking the token and
|
||||
user id against cmem.ai → Connect.
|
||||
|
||||
## 6. Report
|
||||
|
||||
- **Status check (already configured):** pending counts, last flush time,
|
||||
last error.
|
||||
- **First-time setup:** device id from the status response, what the counts
|
||||
drained to, whether a legacy client was migrated/retired, and this one-line
|
||||
privacy note:
|
||||
Report device id, pending counts, last successful flush, Hub reachability and
|
||||
checkpoint, and any Hub/flush error. End with this privacy note:
|
||||
|
||||
> Cloud sync uploads your observation narratives and full prompt text to your
|
||||
> cmem.ai account.
|
||||
|
||||
+118
-101
File diff suppressed because one or more lines are too long
@@ -1,9 +1,9 @@
|
||||
var __IMPORT_META_URL__ = require("node:url").pathToFileURL(__filename).href;
|
||||
"use strict";var h=Object.defineProperty;var b=Object.getOwnPropertyDescriptor;var w=Object.getOwnPropertyNames;var F=Object.prototype.hasOwnProperty;var P=(o,r)=>{for(var e in r)h(o,e,{get:r[e],enumerable:!0})},N=(o,r,e,t)=>{if(r&&typeof r=="object"||typeof r=="function")for(let n of w(r))!F.call(o,n)&&n!==e&&h(o,n,{get:()=>r[n],enumerable:!(t=b(r,n))||t.enumerable});return o};var U=o=>N(h({},"__esModule",{value:!0}),o);var K={};P(K,{parseFileList:()=>z});module.exports=U(K);var l=require("fs"),k=require("path");var i=require("path"),v=require("os"),f=require("fs"),_=require("url");var M=null;function B(o){return(M??process.stderr.write.bind(process.stderr))(o)}function u(o){B(o)}var rr=process.platform==="win32";function G(o){return o.replace(/^\uFEFF/,"")}function g(o){return JSON.parse(G(o))}function j(){return typeof __dirname<"u"?__dirname:(0,i.dirname)((0,_.fileURLToPath)(__IMPORT_META_URL__))}var ir=j();function W(){if(process.env.CLAUDE_MEM_DATA_DIR)return process.env.CLAUDE_MEM_DATA_DIR;let o=(0,i.join)((0,v.homedir)(),".claude-mem"),r=(0,i.join)(o,"settings.json");try{if((0,f.existsSync)(r)){let e=g((0,f.readFileSync)(r,"utf-8")),t=e.env??e;if(t.CLAUDE_MEM_DATA_DIR)return t.CLAUDE_MEM_DATA_DIR}}catch{}return o}var c=W(),H=process.env.CLAUDE_CONFIG_DIR||(0,i.join)((0,v.homedir)(),".claude"),or=(0,i.join)(H,"plugins","marketplaces","thedotmack"),J=(0,i.join)(c,"logs"),sr=(0,i.join)(c,"settings.json"),cr=(0,i.join)(c,"claude-mem.db"),Y=(0,i.join)(c,"observer-sessions"),lr=(0,i.basename)(Y);var D={dataDir:()=>c,workerPid:()=>(0,i.join)(c,"worker.pid"),serverPid:()=>(0,i.join)(c,".server-beta.pid"),serverPort:()=>(0,i.join)(c,".server-beta.port"),serverRuntime:()=>(0,i.join)(c,".server-beta.runtime.json"),settings:()=>(0,i.join)(c,"settings.json"),database:()=>(0,i.join)(c,"claude-mem.db"),chroma:()=>(0,i.join)(c,"chroma"),combinedCerts:()=>(0,i.join)(c,"combined_certs.pem"),transcriptsConfig:()=>(0,i.join)(c,"transcript-watch.json"),transcriptsState:()=>(0,i.join)(c,"transcript-watch-state.json"),cloudSyncState:()=>(0,i.join)(c,"cloud-sync-state.json"),corpora:()=>(0,i.join)(c,"corpora"),supervisorRegistry:()=>(0,i.join)(c,"supervisor.json"),envFile:()=>(0,i.join)(c,".env"),logsDir:()=>J};var O=(s=>(s[s.DEBUG=0]="DEBUG",s[s.INFO=1]="INFO",s[s.WARN=2]="WARN",s[s.ERROR=3]="ERROR",s[s.SILENT=4]="SILENT",s))(O||{}),$=null,C=class{level=null;useColor;logFilePath=null;logFileInitialized=!1;constructor(){this.useColor=process.stdout.isTTY??!1}ensureLogFileInitialized(){if(!this.logFileInitialized){this.logFileInitialized=!0;try{let r=D.logsDir();(0,l.existsSync)(r)||(0,l.mkdirSync)(r,{recursive:!0});let e=new Date().toISOString().split("T")[0];this.logFilePath=(0,k.join)(r,`claude-mem-${e}.log`)}catch(r){console.error("[LOGGER] Failed to initialize log file:",r instanceof Error?r.message:String(r)),this.logFilePath=null}}}getLevel(){if(this.level===null)try{let r=D.settings();if((0,l.existsSync)(r)){let e=(0,l.readFileSync)(r,"utf-8"),n=(g(e).CLAUDE_MEM_LOG_LEVEL||"INFO").toUpperCase();this.level=O[n]??1}else this.level=1}catch(r){console.error("[LOGGER] Failed to load log level from settings:",r instanceof Error?r.message:String(r)),this.level=1}return this.level}formatData(r){if(r==null)return"";if(typeof r=="string")return r;if(typeof r=="number"||typeof r=="boolean")return r.toString();if(typeof r=="object"){if(r instanceof Error)return this.getLevel()===0?`${r.message}
|
||||
${r.stack}`:r.message;if(Array.isArray(r))return`[${r.length} items]`;let e=Object.keys(r);return e.length===0?"{}":e.length<=3?JSON.stringify(r):`{${e.length} keys: ${e.slice(0,3).join(", ")}...}`}return String(r)}formatTool(r,e){if(!e)return r;let t=e;if(typeof e=="string")try{t=JSON.parse(e)}catch{t=e}if(r==="Bash"&&t.command)return`${r}(${t.command})`;if(t.file_path)return`${r}(${t.file_path})`;if(t.notebook_path)return`${r}(${t.notebook_path})`;if(r==="Glob"&&t.pattern)return`${r}(${t.pattern})`;if(r==="Grep"&&t.pattern)return`${r}(${t.pattern})`;if(t.url)return`${r}(${t.url})`;if(t.query)return`${r}(${t.query})`;if(r==="Task"){if(t.subagent_type)return`${r}(${t.subagent_type})`;if(t.description)return`${r}(${t.description})`}return r==="Skill"&&t.skill?`${r}(${t.skill})`:r==="LSP"&&t.operation?`${r}(${t.operation})`:r}formatTimestamp(r){let e=r.getFullYear(),t=String(r.getMonth()+1).padStart(2,"0"),n=String(r.getDate()).padStart(2,"0"),s=String(r.getHours()).padStart(2,"0"),d=String(r.getMinutes()).padStart(2,"0"),S=String(r.getSeconds()).padStart(2,"0"),m=String(r.getMilliseconds()).padStart(3,"0");return`${e}-${t}-${n} ${s}:${d}:${S}.${m}`}log(r,e,t,n,s){if(r<this.getLevel())return;this.ensureLogFileInitialized();let d=this.formatTimestamp(new Date),S=O[r].padEnd(5),m=e.padEnd(6),y="";n?.correlationId?y=`[${n.correlationId}] `:n?.sessionId&&(y=`[session-${n.sessionId}] `);let a="";if(s!=null)if(s instanceof Error)a=this.getLevel()===0?`
|
||||
"use strict";var h=Object.defineProperty;var b=Object.getOwnPropertyDescriptor;var w=Object.getOwnPropertyNames;var F=Object.prototype.hasOwnProperty;var P=(o,r)=>{for(var e in r)h(o,e,{get:r[e],enumerable:!0})},N=(o,r,e,t)=>{if(r&&typeof r=="object"||typeof r=="function")for(let n of w(r))!F.call(o,n)&&n!==e&&h(o,n,{get:()=>r[n],enumerable:!(t=b(r,n))||t.enumerable});return o};var U=o=>N(h({},"__esModule",{value:!0}),o);var K={};P(K,{parseFileList:()=>z});module.exports=U(K);var l=require("fs"),k=require("path");var i=require("path"),v=require("os"),f=require("fs"),_=require("url");var M=null;function B(o){return(M??process.stderr.write.bind(process.stderr))(o)}function u(o){B(o)}var rr=process.platform==="win32";function G(o){return o.replace(/^\uFEFF/,"")}function g(o){return JSON.parse(G(o))}function W(){return typeof __dirname<"u"?__dirname:(0,i.dirname)((0,_.fileURLToPath)(__IMPORT_META_URL__))}var ir=W();function j(){if(process.env.CLAUDE_MEM_DATA_DIR)return process.env.CLAUDE_MEM_DATA_DIR;let o=(0,i.join)((0,v.homedir)(),".claude-mem"),r=(0,i.join)(o,"settings.json");try{if((0,f.existsSync)(r)){let e=g((0,f.readFileSync)(r,"utf-8")),t=e.env??e;if(t.CLAUDE_MEM_DATA_DIR)return t.CLAUDE_MEM_DATA_DIR}}catch{}return o}var c=j(),H=process.env.CLAUDE_CONFIG_DIR||(0,i.join)((0,v.homedir)(),".claude"),or=(0,i.join)(H,"plugins","marketplaces","thedotmack"),J=(0,i.join)(c,"logs"),sr=(0,i.join)(c,"settings.json"),cr=(0,i.join)(c,"claude-mem.db"),Y=(0,i.join)(c,"observer-sessions"),lr=(0,i.basename)(Y);var D={dataDir:()=>c,workerPid:()=>(0,i.join)(c,"worker.pid"),serverPid:()=>(0,i.join)(c,".server-beta.pid"),serverPort:()=>(0,i.join)(c,".server-beta.port"),serverRuntime:()=>(0,i.join)(c,".server-beta.runtime.json"),settings:()=>(0,i.join)(c,"settings.json"),database:()=>(0,i.join)(c,"claude-mem.db"),chroma:()=>(0,i.join)(c,"chroma"),combinedCerts:()=>(0,i.join)(c,"combined_certs.pem"),transcriptsConfig:()=>(0,i.join)(c,"transcript-watch.json"),transcriptsState:()=>(0,i.join)(c,"transcript-watch-state.json"),corpora:()=>(0,i.join)(c,"corpora"),supervisorRegistry:()=>(0,i.join)(c,"supervisor.json"),envFile:()=>(0,i.join)(c,".env"),logsDir:()=>J};var O=(s=>(s[s.DEBUG=0]="DEBUG",s[s.INFO=1]="INFO",s[s.WARN=2]="WARN",s[s.ERROR=3]="ERROR",s[s.SILENT=4]="SILENT",s))(O||{}),$=null,C=class{level=null;useColor;logFilePath=null;logFileInitialized=!1;constructor(){this.useColor=process.stdout.isTTY??!1}ensureLogFileInitialized(){if(!this.logFileInitialized){this.logFileInitialized=!0;try{let r=D.logsDir();(0,l.existsSync)(r)||(0,l.mkdirSync)(r,{recursive:!0});let e=new Date().toISOString().split("T")[0];this.logFilePath=(0,k.join)(r,`claude-mem-${e}.log`)}catch(r){console.error("[LOGGER] Failed to initialize log file:",r instanceof Error?r.message:String(r)),this.logFilePath=null}}}getLevel(){if(this.level===null)try{let r=D.settings();if((0,l.existsSync)(r)){let e=(0,l.readFileSync)(r,"utf-8"),n=(g(e).CLAUDE_MEM_LOG_LEVEL||"INFO").toUpperCase();this.level=O[n]??1}else this.level=1}catch(r){console.error("[LOGGER] Failed to load log level from settings:",r instanceof Error?r.message:String(r)),this.level=1}return this.level}formatData(r){if(r==null)return"";if(typeof r=="string")return r;if(typeof r=="number"||typeof r=="boolean")return r.toString();if(typeof r=="object"){if(r instanceof Error)return this.getLevel()===0?`${r.message}
|
||||
${r.stack}`:r.message;if(Array.isArray(r))return`[${r.length} items]`;let e=Object.keys(r);return e.length===0?"{}":e.length<=3?JSON.stringify(r):`{${e.length} keys: ${e.slice(0,3).join(", ")}...}`}return String(r)}formatTool(r,e){if(!e)return r;let t=e;if(typeof e=="string")try{t=JSON.parse(e)}catch{t=e}if(r==="Bash"&&t.command)return`${r}(${t.command})`;if(t.file_path)return`${r}(${t.file_path})`;if(t.notebook_path)return`${r}(${t.notebook_path})`;if(r==="Glob"&&t.pattern)return`${r}(${t.pattern})`;if(r==="Grep"&&t.pattern)return`${r}(${t.pattern})`;if(t.url)return`${r}(${t.url})`;if(t.query)return`${r}(${t.query})`;if(r==="Task"){if(t.subagent_type)return`${r}(${t.subagent_type})`;if(t.description)return`${r}(${t.description})`}return r==="Skill"&&t.skill?`${r}(${t.skill})`:r==="LSP"&&t.operation?`${r}(${t.operation})`:r}formatTimestamp(r){let e=r.getFullYear(),t=String(r.getMonth()+1).padStart(2,"0"),n=String(r.getDate()).padStart(2,"0"),s=String(r.getHours()).padStart(2,"0"),d=String(r.getMinutes()).padStart(2,"0"),m=String(r.getSeconds()).padStart(2,"0"),S=String(r.getMilliseconds()).padStart(3,"0");return`${e}-${t}-${n} ${s}:${d}:${m}.${S}`}log(r,e,t,n,s){if(r<this.getLevel())return;this.ensureLogFileInitialized();let d=this.formatTimestamp(new Date),m=O[r].padEnd(5),S=e.padEnd(6),y="";n?.correlationId?y=`[${n.correlationId}] `:n?.sessionId&&(y=`[session-${n.sessionId}] `);let a="";if(s!=null)if(s instanceof Error)a=this.getLevel()===0?`
|
||||
${s.message}
|
||||
${s.stack}`:` ${s.message}`;else if(this.getLevel()===0&&typeof s=="object")try{a=`
|
||||
`+JSON.stringify(s,null,2)}catch{a=" "+this.formatData(s)}else a=" "+this.formatData(s);let R="";if(n){let{sessionId:p,memorySessionId:E,correlationId:V,...L}=n;Object.keys(L).length>0&&(R=` {${Object.entries(L).map(([T,A])=>`${T}=${A}`).join(", ")}}`)}let x=`[${d}] [${S}] [${m}] ${y}${t}${R}${a}`;if(this.logFilePath)try{(0,l.appendFileSync)(this.logFilePath,x+`
|
||||
`+JSON.stringify(s,null,2)}catch{a=" "+this.formatData(s)}else a=" "+this.formatData(s);let R="";if(n){let{sessionId:p,memorySessionId:E,correlationId:V,...L}=n;Object.keys(L).length>0&&(R=` {${Object.entries(L).map(([T,A])=>`${T}=${A}`).join(", ")}}`)}let x=`[${d}] [${m}] [${S}] ${y}${t}${R}${a}`;if(this.logFilePath)try{(0,l.appendFileSync)(this.logFilePath,x+`
|
||||
`,"utf8")}catch(p){let E=p instanceof Error?p:new Error(String(p));u(`[LOGGER] Failed to write to log file: ${E.message}
|
||||
${E.stack??""}
|
||||
`)}else u(x+`
|
||||
|
||||
@@ -103,8 +103,8 @@ async function main(): Promise<void> {
|
||||
const tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-kill-e2e-'));
|
||||
const dbA = new Database(':memory:');
|
||||
const dbB = new Database(':memory:');
|
||||
new SessionStore(dbA, { cloudSyncStatePath: join(tempDir, 'no-legacy-a.json') });
|
||||
new SessionStore(dbB, { cloudSyncStatePath: join(tempDir, 'no-legacy-b.json') });
|
||||
new SessionStore(dbA);
|
||||
new SessionStore(dbB);
|
||||
seedSession(dbA);
|
||||
|
||||
// Device A: the push drain.
|
||||
@@ -116,7 +116,6 @@ async function main(): Promise<void> {
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME: 'e2e-a',
|
||||
}, {
|
||||
settingsPath: join(tempDir, 'settings-a.json'),
|
||||
legacyStatePath: join(tempDir, 'no-legacy-a.json'),
|
||||
debounceMs: 100,
|
||||
});
|
||||
|
||||
@@ -127,6 +126,7 @@ async function main(): Promise<void> {
|
||||
token: TOKEN,
|
||||
userId: USER,
|
||||
deviceId: DEV_B,
|
||||
deviceName: 'e2e-b',
|
||||
activePollMs: 1_000,
|
||||
idlePollMs: 1_000, // fast re-probe while the socket is live
|
||||
minPullGapMs: 0,
|
||||
|
||||
+692
-416
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,6 @@
|
||||
import { Database, type SQLQueryBindings } from 'bun:sqlite';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { randomUUID } from 'crypto';
|
||||
import { DATA_DIR, DB_PATH, ensureDir, OBSERVER_SESSIONS_PROJECT, paths } from '../../shared/paths.js';
|
||||
import { DATA_DIR, DB_PATH, ensureDir, OBSERVER_SESSIONS_PROJECT } from '../../shared/paths.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import {
|
||||
TableColumnInfo,
|
||||
@@ -75,7 +74,7 @@ interface SdkSessionDetailRow {
|
||||
export class SessionStore {
|
||||
public db: Database;
|
||||
|
||||
constructor(dbPathOrDb: string | Database = DB_PATH, options: { cloudSyncStatePath?: string; cloudSyncHubUrl?: string } = {}) {
|
||||
constructor(dbPathOrDb: string | Database = DB_PATH) {
|
||||
if (dbPathOrDb instanceof Database) {
|
||||
this.db = dbPathOrDb;
|
||||
} else {
|
||||
@@ -114,13 +113,12 @@ export class SessionStore {
|
||||
this.ensureSDKSessionsPlatformContentIdentity();
|
||||
this.ensureUserPromptsSessionDbId();
|
||||
this.ensurePendingMessagesSessionToolUniqueIndex();
|
||||
this.ensureSyncedAtColumns(options.cloudSyncStatePath ?? paths.cloudSyncState());
|
||||
this.requeuePromptCloudSyncAfterMapperFix();
|
||||
this.ensureSyncedAtColumns();
|
||||
this.ensureSyncOriginColumns();
|
||||
this.ensureSyncOutbox();
|
||||
this.ensureSyncEntityLedger();
|
||||
this.ensureSyncRevisionTextAffinity();
|
||||
this.requeueAllForHubCutover(options.cloudSyncHubUrl);
|
||||
this.initializeSyncHubLaunchBaseline();
|
||||
}
|
||||
|
||||
private getIndexColumns(indexName: string): string[] {
|
||||
@@ -456,13 +454,11 @@ export class SessionStore {
|
||||
}
|
||||
}
|
||||
|
||||
private ensureSyncedAtColumns(cloudSyncStatePath: string): void {
|
||||
private ensureSyncedAtColumns(): void {
|
||||
// Not gated on a schema_versions row: the community-edge line already
|
||||
// consumed versions 36-38 without adding synced_at, so affected DBs have
|
||||
// those version rows but not the columns. The PRAGMA checks are the real
|
||||
// guard; version 39 is recorded for bookkeeping only.
|
||||
let columnsAdded = false;
|
||||
|
||||
for (const table of ['observations', 'session_summaries', 'user_prompts']) {
|
||||
const tableInfo = this.db.query(`PRAGMA table_info(${table})`).all() as TableColumnInfo[];
|
||||
const hasSyncedAt = tableInfo.some(col => col.name === 'synced_at');
|
||||
@@ -470,47 +466,14 @@ export class SessionStore {
|
||||
if (!hasSyncedAt) {
|
||||
this.db.run(`ALTER TABLE ${table} ADD COLUMN synced_at INTEGER`);
|
||||
logger.debug('DB', `Added synced_at column to ${table} table`);
|
||||
columnsAdded = true;
|
||||
}
|
||||
|
||||
this.db.run(`CREATE INDEX IF NOT EXISTS idx_${table}_unsynced ON ${table}(id) WHERE synced_at IS NULL`);
|
||||
}
|
||||
|
||||
// Legacy cursor adoption is once-only: it runs only in the call that
|
||||
// created the columns.
|
||||
if (columnsAdded) {
|
||||
this.stampRowsSyncedByLegacyClient(cloudSyncStatePath);
|
||||
}
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(39, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* One-time cloud repair (version 40): every prompt synced before the
|
||||
* CloudSync mapper fix went to the cloud with memory_session_id =
|
||||
* content_session_id and project = 'unknown', so the cloud viewer could
|
||||
* never attach a prompt to its session. Re-nulling synced_at makes the
|
||||
* next flush re-push the full prompt history through the fixed mapper
|
||||
* (sdk_sessions join); the server upserts on (user_id, device_id,
|
||||
* local_id) with a change guard, so corrected rows overwrite in place and
|
||||
* still-identical rows (no local mapping) cost nothing. Runs after
|
||||
* ensureSyncedAtColumns — the column must exist. Harmless when cloud sync
|
||||
* is unconfigured: rows simply sit unsynced, which is their natural state.
|
||||
*/
|
||||
private requeuePromptCloudSyncAfterMapperFix(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(40) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
const res = this.db.prepare(`
|
||||
UPDATE user_prompts SET synced_at = NULL WHERE synced_at IS NOT NULL
|
||||
`).run();
|
||||
logger.info('DB', 'Requeued prompt cloud sync after mapper fix (v40)', {
|
||||
requeued: res.changes
|
||||
});
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(40, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Two-lane sync origins (version 41): every synced table learns where a row
|
||||
* came from. Native rows keep the origin columns NULL (NULL = this device);
|
||||
@@ -768,97 +731,101 @@ export class SessionStore {
|
||||
.run(45, new Date().toISOString());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Hub cutover one-shot (plan Phase 3 task 5): when settings point at a
|
||||
* sync hub (CLAUDE_MEM_CLOUD_SYNC_HUB_URL non-empty) that this DB has not
|
||||
* cut over to yet, re-null `synced_at` on every NATIVE row once so this
|
||||
* device re-pushes its whole corpus into that hub's log; the hub's
|
||||
* (origin_device, kind, origin_id, rev) unique index dedupes replays.
|
||||
* Replica rows (origin_device_id NOT NULL) are excluded — they are another
|
||||
* device's corpus and must never be pushed under this device's identity.
|
||||
*
|
||||
* GATING — keyed on HUB IDENTITY, not a version number: the hub URL the
|
||||
* cutover last ran against is stored in sync_state ('cutover_hub_url'),
|
||||
* written in the SAME transaction as the requeue. The one-shot fires
|
||||
* whenever a non-empty hub URL differs from the stored value — so it is
|
||||
* exactly-once per (DB, hub URL): the first configuration fires it, every
|
||||
* later boot with the same URL is a no-op, and pointing at a DIFFERENT hub
|
||||
* later fires it again (a burned version row would leave the corpus
|
||||
* permanently un-pushed into the new hub's empty log — silent fleet-wide
|
||||
* data loss). A schema_versions row 43 is still recorded as a legacy
|
||||
* bookkeeping marker but is NOT consulted, so pre-fix DBs that burned v43
|
||||
* (and have no stored cutover_hub_url) self-heal with one extra re-push —
|
||||
* hub dedupe makes over-firing safe. Callers that don't know the settings
|
||||
* (tests, CLI utilities) pass no URL and stay inert — the worker, which
|
||||
* owns the push drain, passes the URL via DatabaseManager.
|
||||
*
|
||||
* The other leg of hub-identity change — the SAME URL whose DO log was
|
||||
* lost/rebuilt (new epoch) — is handled by SyncApply.handleEpoch, which
|
||||
* re-nulls native rows on an epoch MISMATCH.
|
||||
* One-time launch boundary (v47) plus its durable revision exclusions
|
||||
* (v48). This product line has no released cloud corpus to migrate, so the
|
||||
* exact native revisions present at launch are a local-only baseline. The
|
||||
* exclusion ledger survives Hub epoch changes; if one of those rows is
|
||||
* edited later, its higher revision is eligible for ordinary sync/rebuild.
|
||||
* Fresh databases run this while empty.
|
||||
*/
|
||||
private requeueAllForHubCutover(hubUrl: string | undefined): void {
|
||||
// Normalize like CloudSync does, so "https://hub" and "https://hub/"
|
||||
// are one hub identity, not a spurious re-fire.
|
||||
const normalized = (hubUrl ?? '').trim().replace(/\/+$/, '');
|
||||
if (normalized === '') return;
|
||||
private initializeSyncHubLaunchBaseline(): void {
|
||||
const tables = [
|
||||
{ table: 'observations', kind: 'observation' },
|
||||
{ table: 'session_summaries', kind: 'summary' },
|
||||
{ table: 'user_prompts', kind: 'prompt' },
|
||||
] as const;
|
||||
const exclusionTableExisted = this.db.prepare(`
|
||||
SELECT 1 AS present FROM sqlite_master
|
||||
WHERE type = 'table' AND name = 'sync_launch_exclusions'
|
||||
`).get() !== undefined;
|
||||
this.db.run(`
|
||||
CREATE TABLE IF NOT EXISTS sync_launch_exclusions (
|
||||
kind TEXT NOT NULL CHECK (kind IN ('observation', 'summary', 'prompt')),
|
||||
origin_local_id TEXT NOT NULL,
|
||||
through_rev TEXT NOT NULL,
|
||||
PRIMARY KEY (kind, origin_local_id)
|
||||
)
|
||||
`);
|
||||
|
||||
const stored = this.db.prepare(`SELECT v FROM sync_state WHERE k = 'cutover_hub_url'`).get() as { v: string } | undefined;
|
||||
if (stored?.v === normalized) return;
|
||||
const applied = this.db.prepare(
|
||||
'SELECT version, applied_at FROM schema_versions WHERE version = ?'
|
||||
).get(47) as { version: number; applied_at: string } | undefined;
|
||||
|
||||
let requeued = 0;
|
||||
const tx = this.db.transaction(() => {
|
||||
for (const table of ['observations', 'session_summaries', 'user_prompts']) {
|
||||
const res = this.db.prepare(`
|
||||
UPDATE ${table} SET synced_at = NULL
|
||||
WHERE synced_at IS NOT NULL AND origin_device_id IS NULL
|
||||
`).run();
|
||||
requeued += res.changes;
|
||||
if (!applied) {
|
||||
const now = Date.now();
|
||||
const tx = this.db.transaction(() => {
|
||||
// Recompute if a migration fixture deliberately removes v47. In a
|
||||
// real pre-v47 database this table is newly created and already empty.
|
||||
this.db.run('DELETE FROM sync_launch_exclusions');
|
||||
for (const { table, kind } of tables) {
|
||||
this.db.prepare(`
|
||||
INSERT INTO sync_launch_exclusions (kind, origin_local_id, through_rev)
|
||||
SELECT ?, CAST(id AS TEXT), CAST(sync_rev AS TEXT)
|
||||
FROM ${table}
|
||||
WHERE origin_device_id IS NULL
|
||||
`).run(kind);
|
||||
this.db.prepare(`
|
||||
UPDATE ${table} SET synced_at = ?
|
||||
WHERE synced_at IS NULL AND origin_device_id IS NULL
|
||||
`).run(now);
|
||||
}
|
||||
this.db.run('DELETE FROM sync_outbox');
|
||||
this.db.run('DELETE FROM sync_content_outbox');
|
||||
this.db.run('DELETE FROM sync_dead_letter');
|
||||
// Adopt the launch Hub as a genuinely first epoch. Retaining a cursor
|
||||
// or epoch from a pre-launch test Hub would make SyncApply interpret
|
||||
// the first connection as a rebuild. Parked mutations belong to that
|
||||
// discarded test log, so pre-launch sync_state is stale control-plane
|
||||
// state; the exclusion ledger above is the only boundary state kept.
|
||||
this.db.run('DELETE FROM sync_state');
|
||||
const appliedAt = new Date(now).toISOString();
|
||||
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(47, appliedAt);
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(48, appliedAt);
|
||||
});
|
||||
tx();
|
||||
return;
|
||||
}
|
||||
|
||||
// Repair databases that ran the earlier v47 implementation before the
|
||||
// explicit exclusion ledger existed. v47 stamped the launch baseline at
|
||||
// its applied_at millisecond. Rows still stamped at/before that boundary
|
||||
// are the excluded launch revisions; NULL or later stamps are post-launch
|
||||
// writes/acks and must remain eligible for an epoch rebuild.
|
||||
const exclusionsApplied = this.db.prepare(
|
||||
'SELECT version FROM schema_versions WHERE version = ?'
|
||||
).get(48) as SchemaVersion | undefined;
|
||||
if (exclusionsApplied && exclusionTableExisted) return;
|
||||
const boundaryMs = Date.parse(applied.applied_at);
|
||||
if (!Number.isSafeInteger(boundaryMs) || boundaryMs < 0) {
|
||||
throw new Error(`schema v48: invalid v47 applied_at ${applied.applied_at}`);
|
||||
}
|
||||
const repair = this.db.transaction(() => {
|
||||
for (const { table, kind } of tables) {
|
||||
this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sync_launch_exclusions (kind, origin_local_id, through_rev)
|
||||
SELECT ?, CAST(id AS TEXT), CAST(sync_rev AS TEXT)
|
||||
FROM ${table}
|
||||
WHERE origin_device_id IS NULL
|
||||
AND synced_at > 0
|
||||
AND synced_at <= ?
|
||||
`).run(kind, boundaryMs);
|
||||
}
|
||||
this.db.prepare(`
|
||||
INSERT INTO sync_state (k, v) VALUES ('cutover_hub_url', ?)
|
||||
ON CONFLICT(k) DO UPDATE SET v = excluded.v
|
||||
`).run(normalized);
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(43, new Date().toISOString());
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)')
|
||||
.run(48, new Date().toISOString());
|
||||
});
|
||||
tx();
|
||||
|
||||
logger.info('DB', 'Requeued full corpus for sync hub cutover', {
|
||||
hubUrl: normalized,
|
||||
previousHubUrl: stored?.v ?? null,
|
||||
requeued,
|
||||
});
|
||||
}
|
||||
|
||||
// Rows the standalone cloud-sync client already uploaded (its cursors live in
|
||||
// cloud-sync-state.json) are stamped so they are not re-uploaded. The state
|
||||
// file is left in place — device-id adoption still reads it.
|
||||
private stampRowsSyncedByLegacyClient(statePath: string): void {
|
||||
if (!existsSync(statePath)) return;
|
||||
|
||||
let state: { lastId?: number; lastSummaryId?: number; lastPromptId?: number };
|
||||
try {
|
||||
state = JSON.parse(readFileSync(statePath, 'utf-8'));
|
||||
} catch (error) {
|
||||
logger.warn('DB', 'Failed to read legacy cloud-sync state, skipping synced_at adoption', { statePath }, error instanceof Error ? error : new Error(String(error)));
|
||||
return;
|
||||
}
|
||||
if (state === null || typeof state !== 'object') {
|
||||
logger.warn('DB', 'Legacy cloud-sync state is not an object, skipping synced_at adoption', { statePath });
|
||||
return;
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cursors: Array<[table: string, lastSyncedId: unknown]> = [
|
||||
['observations', state.lastId],
|
||||
['session_summaries', state.lastSummaryId],
|
||||
['user_prompts', state.lastPromptId],
|
||||
];
|
||||
|
||||
for (const [table, lastSyncedId] of cursors) {
|
||||
if (!(typeof lastSyncedId === 'number' && lastSyncedId > 0)) continue;
|
||||
this.db.prepare(`UPDATE ${table} SET synced_at = ? WHERE id <= ? AND synced_at IS NULL`).run(now, lastSyncedId);
|
||||
logger.debug('DB', `Stamped synced_at on ${table} rows already uploaded by the legacy cloud-sync client`, { lastSyncedId });
|
||||
}
|
||||
repair();
|
||||
}
|
||||
|
||||
private dropDeadPendingMessagesColumns(): void {
|
||||
|
||||
+135
-57
@@ -5,8 +5,9 @@
|
||||
// trailing debounce coalesces bursts into one `flush()`, which drains
|
||||
// `WHERE synced_at IS NULL AND origin_device_id IS NULL` in batches, POSTs to
|
||||
// the per-user sync hub (workers/sync-hub), and stamps rows on ack. That
|
||||
// single mechanism IS live sync, backfill, offline catch-up, and retry — no
|
||||
// second process, no cursor files. Mutation ops (custom title, prompt→session
|
||||
// single mechanism handles post-launch live sync, offline catch-up, and retry
|
||||
// — no historical/pre-launch backfill, second process, or cursor files.
|
||||
// Mutation ops (custom title, prompt→session
|
||||
// repair, project remaps) ride the same flush from the `sync_outbox` table
|
||||
// (migration v42). Already-frozen content (especially tombstones) drains
|
||||
// first; mutations drain before newly materialized row snapshots so title
|
||||
@@ -30,7 +31,7 @@
|
||||
// Acked mutation ops are DELETEd from sync_outbox (queue entries, not data).
|
||||
//
|
||||
// SIZE CONTRACT: each canonical body is at most 256,000 UTF-8 bytes and each
|
||||
// request is packed below 4,000,000 bytes / 500 ops. Invalid historical rows
|
||||
// request is packed below 4,000,000 bytes / 500 ops. Invalid queued rows
|
||||
// are moved to a durable dead-letter with their exact rejection reason so a
|
||||
// poison row cannot wedge later work. Mutation semantics are never clamped or
|
||||
// rewritten: all bounded fields are validated in UTF-8 bytes before append.
|
||||
@@ -46,7 +47,7 @@ import { randomUUID } from 'crypto';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { parseJsonWithBom, writeJsonFileAtomic } from '../../shared/atomic-json.js';
|
||||
import { SettingsDefaultsManager, type SettingsDefaults } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH, paths } from '../../shared/paths.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
import {
|
||||
assertCanonicalDecimal,
|
||||
buildContentOperation,
|
||||
@@ -65,6 +66,15 @@ const BATCH = 200;
|
||||
const MAX_BODY_BYTES = 4_000_000;
|
||||
// Hub cap: ≤500 ops per POST /v1/sync/ops request.
|
||||
const MAX_OPS_PER_PUSH = 500;
|
||||
const EMPTY_PUSH_REQUEST_BYTES = Buffer.byteLength(
|
||||
JSON.stringify({ protocol_version: 2, ops: [] }),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
/** Exact encoded request bytes from the sum of serialized op wrapper bytes. */
|
||||
function pushRequestBytes(opBytes: number, opCount: number): number {
|
||||
return EMPTY_PUSH_REQUEST_BYTES + opBytes + Math.max(0, opCount - 1);
|
||||
}
|
||||
type LocalRow = Record<string, unknown> & { id: string; sync_rev: string };
|
||||
type OpBody = Record<string, unknown>;
|
||||
|
||||
@@ -306,8 +316,6 @@ export interface CloudSyncOptions {
|
||||
fetchImpl?: typeof fetch;
|
||||
/** settings.json path where a newly resolved device id is persisted. */
|
||||
settingsPath?: string;
|
||||
/** Legacy standalone-client state file (~/.claude-mem/cloud-sync-state.json). */
|
||||
legacyStatePath?: string;
|
||||
/** Trailing debounce for notify() bursts. */
|
||||
debounceMs?: number;
|
||||
/**
|
||||
@@ -331,6 +339,14 @@ export interface CloudSyncStatus {
|
||||
quarantine: { count: number; latestReason: string | null };
|
||||
lastFlushAt: number | null;
|
||||
lastError: string | null;
|
||||
hub: {
|
||||
checkedAt: number | null;
|
||||
reachable: boolean | null;
|
||||
epoch: string | null;
|
||||
headSeq: string | null;
|
||||
projectedSeq: string | null;
|
||||
error: string | null;
|
||||
};
|
||||
}
|
||||
|
||||
export class CloudSync {
|
||||
@@ -341,7 +357,6 @@ export class CloudSync {
|
||||
private readonly deviceName: string;
|
||||
private readonly fetchImpl: typeof fetch;
|
||||
private readonly settingsPath: string;
|
||||
private readonly legacyStatePath: string;
|
||||
private readonly debounceMs: number;
|
||||
private readonly fastDebounceMs: number;
|
||||
private readonly backoffInitialMs: number;
|
||||
@@ -358,6 +373,14 @@ export class CloudSync {
|
||||
private stopped = false;
|
||||
private lastFlushAt: number | null = null;
|
||||
private lastError: string | null = null;
|
||||
private hubStatus: CloudSyncStatus['hub'] = {
|
||||
checkedAt: null,
|
||||
reachable: null,
|
||||
epoch: null,
|
||||
headSeq: null,
|
||||
projectedSeq: null,
|
||||
error: null,
|
||||
};
|
||||
/** True while SyncClient's advisory socket is live (setFastDebounce). */
|
||||
private fastDebounce = false;
|
||||
/**
|
||||
@@ -380,15 +403,13 @@ export class CloudSync {
|
||||
this.db = db;
|
||||
this.token = settings.CLAUDE_MEM_CLOUD_SYNC_TOKEN ?? '';
|
||||
this.userId = settings.CLAUDE_MEM_CLOUD_SYNC_USER_ID ?? '';
|
||||
// Hard cutover (plan Phase 3 task 5, open decision 2): the hub URL has NO
|
||||
// default. Empty ⇒ sync is OFF entirely — there is no legacy per-kind
|
||||
// fallback lane.
|
||||
// Launch contract: the Hub URL has no default. Empty means sync is off;
|
||||
// there is no application-API or per-kind fallback lane.
|
||||
this.hubUrl = (settings.CLAUDE_MEM_CLOUD_SYNC_HUB_URL ?? '').trim().replace(/\/+$/, '');
|
||||
// Human-readable device label for the dashboard's Devices panel.
|
||||
this.deviceName = (settings.CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME || hostname() || '').slice(0, 80);
|
||||
this.fetchImpl = options.fetchImpl ?? globalThis.fetch;
|
||||
this.settingsPath = options.settingsPath ?? USER_SETTINGS_PATH;
|
||||
this.legacyStatePath = options.legacyStatePath ?? paths.cloudSyncState();
|
||||
this.debounceMs = options.debounceMs ?? 1_500;
|
||||
this.fastDebounceMs = options.fastDebounceMs ?? 250;
|
||||
this.backoffInitialMs = options.backoffInitialMs ?? 30_000;
|
||||
@@ -439,8 +460,9 @@ export class CloudSync {
|
||||
}
|
||||
|
||||
/**
|
||||
* Kick one flush (non-blocking). This IS backfill: a never-synced install
|
||||
* simply has everything `synced_at IS NULL`.
|
||||
* Kick one non-blocking catch-up flush for eligible post-launch writes.
|
||||
* The v47 launch baseline is deliberately stamped/excluded and is not a
|
||||
* historical corpus to upload.
|
||||
*/
|
||||
start(): void {
|
||||
if (!this.isActive()) {
|
||||
@@ -450,7 +472,7 @@ export class CloudSync {
|
||||
});
|
||||
return;
|
||||
}
|
||||
logger.info('CLOUD_SYNC', 'Cloud sync active — kicking startup drain', {
|
||||
logger.info('CLOUD_SYNC', 'Cloud sync active — kicking post-launch catch-up drain', {
|
||||
hubUrl: this.hubUrl,
|
||||
deviceId: this.deviceId,
|
||||
deviceName: this.deviceName,
|
||||
@@ -536,9 +558,92 @@ export class CloudSync {
|
||||
quarantine: this.quarantineStatus(),
|
||||
lastFlushAt: this.lastFlushAt,
|
||||
lastError: this.lastError,
|
||||
hub: { ...this.hubStatus },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Status for the local `/api/sync/status` route. Even with an empty queue,
|
||||
* authenticate directly against SyncHub so "nothing to upload" cannot be
|
||||
* mistaken for a working connection. This is a read-only Hub status GET:
|
||||
* it never appends an operation or advances a client cursor.
|
||||
*/
|
||||
async statusWithHubProbe(): Promise<CloudSyncStatus> {
|
||||
if (!this.stopped && this.isActive()) {
|
||||
await this.probeHubStatus();
|
||||
}
|
||||
return this.status();
|
||||
}
|
||||
|
||||
private async probeHubStatus(): Promise<void> {
|
||||
let checkedAt = Date.now();
|
||||
try {
|
||||
const response = await this.fetchImpl(`${this.hubUrl}/v1/sync/status`, {
|
||||
method: 'GET',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${this.token}`,
|
||||
'X-User-Id': this.userId,
|
||||
'X-Device-Id': this.deviceId,
|
||||
...(this.deviceName ? { 'X-Device-Name': this.deviceName } : {}),
|
||||
},
|
||||
signal: AbortSignal.timeout(this.requestTimeoutMs),
|
||||
});
|
||||
checkedAt = Date.now();
|
||||
const syncMode = response.headers.get('X-Sync-Mode');
|
||||
if (syncMode !== null || response.ok) this.emitSyncMode(syncMode);
|
||||
if (!response.ok) {
|
||||
const body = (await response.text().catch(() => '')).slice(0, 200);
|
||||
throw new Error(`sync hub status ${response.status}: ${body}`);
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = await response.json();
|
||||
} catch {
|
||||
throw new Error('sync hub status: response is not JSON');
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
||||
throw new Error('sync hub status: response must be an object');
|
||||
}
|
||||
const record = parsed as Record<string, unknown>;
|
||||
if (record.protocol_version !== 2) {
|
||||
throw new Error('sync hub status: response requires protocol_version 2');
|
||||
}
|
||||
if (
|
||||
typeof record.epoch !== 'string'
|
||||
|| typeof record.head_seq !== 'string'
|
||||
|| typeof record.projected_seq !== 'string'
|
||||
) {
|
||||
throw new Error('sync hub status: response requires decimal-string epoch/head_seq/projected_seq');
|
||||
}
|
||||
const epoch = assertCanonicalDecimal(record.epoch, { positive: true });
|
||||
const headSeq = assertCanonicalDecimal(record.head_seq);
|
||||
const projectedSeq = assertCanonicalDecimal(record.projected_seq);
|
||||
if (compareCanonicalDecimals(projectedSeq, headSeq) > 0) {
|
||||
throw new Error('sync hub status: projected_seq exceeds head_seq');
|
||||
}
|
||||
this.hubStatus = {
|
||||
checkedAt,
|
||||
reachable: true,
|
||||
epoch,
|
||||
headSeq,
|
||||
projectedSeq,
|
||||
error: null,
|
||||
};
|
||||
} catch (error) {
|
||||
const raw = error instanceof Error ? error.message : String(error);
|
||||
const safe = this.token === '' ? raw : raw.split(this.token).join('[REDACTED]');
|
||||
this.hubStatus = {
|
||||
checkedAt,
|
||||
reachable: false,
|
||||
epoch: null,
|
||||
headSeq: null,
|
||||
projectedSeq: null,
|
||||
error: safe,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Queue and apply a local content deletion atomically. The tombstone stays
|
||||
* durable in sync_content_outbox until Hub projection acknowledges it.
|
||||
@@ -630,7 +735,10 @@ export class CloudSync {
|
||||
let bytes = 0;
|
||||
for (const row of rows) {
|
||||
const size = Buffer.byteLength(JSON.stringify(row), 'utf8');
|
||||
if (batch.length > 0 && (batch.length >= MAX_OPS_PER_PUSH || bytes + size > MAX_BODY_BYTES)) {
|
||||
if (batch.length > 0 && (
|
||||
batch.length >= MAX_OPS_PER_PUSH
|
||||
|| pushRequestBytes(bytes + size, batch.length + 1) > MAX_BODY_BYTES
|
||||
)) {
|
||||
await this.sendOps(batch);
|
||||
if (this.stopped) return;
|
||||
batch = [];
|
||||
@@ -708,7 +816,10 @@ export class CloudSync {
|
||||
`).run(op.body, op.operation_sha256, row.id);
|
||||
}
|
||||
const size = Buffer.byteLength(JSON.stringify(op), 'utf8');
|
||||
if (buf.length > 0 && (bufBytes + size > MAX_BODY_BYTES || buf.length >= MAX_OPS_PER_PUSH)) {
|
||||
if (buf.length > 0 && (
|
||||
pushRequestBytes(bufBytes + size, buf.length + 1) > MAX_BODY_BYTES
|
||||
|| buf.length >= MAX_OPS_PER_PUSH
|
||||
)) {
|
||||
await send();
|
||||
if (this.stopped) return;
|
||||
}
|
||||
@@ -825,6 +936,10 @@ export class CloudSync {
|
||||
}
|
||||
|
||||
private async pushOps(ops: WireOp[]): Promise<PushResponse> {
|
||||
const requestBody = JSON.stringify({ protocol_version: 2, ops });
|
||||
if (Buffer.byteLength(requestBody, 'utf8') > MAX_BODY_BYTES) {
|
||||
throw new Error(`sync hub push invariant: request exceeds ${MAX_BODY_BYTES} encoded bytes`);
|
||||
}
|
||||
const res = await this.fetchImpl(`${this.hubUrl}/v1/sync/ops`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
@@ -834,7 +949,7 @@ export class CloudSync {
|
||||
'X-Device-Id': this.deviceId,
|
||||
...(this.deviceName ? { 'X-Device-Name': this.deviceName } : {}),
|
||||
},
|
||||
body: JSON.stringify({ protocol_version: 2, ops }),
|
||||
body: requestBody,
|
||||
signal: AbortSignal.timeout(this.requestTimeoutMs),
|
||||
});
|
||||
// Mode hint BEFORE the ok-check: the kill-switch header rides error
|
||||
@@ -1301,50 +1416,13 @@ export class CloudSync {
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Resolve this install's stable device id, in priority order:
|
||||
* 1. CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID from settings (already resolved once);
|
||||
* 2. the legacy standalone client's cloud-sync-state.json deviceId;
|
||||
* 3. a freshly minted randomUUID().
|
||||
*
|
||||
* CRITICAL: never mint a new id while a legacy state file exists — the hub
|
||||
* keys ops on (origin_device, kind, origin_id, rev), so a new id forks
|
||||
* every previously pushed row into a duplicate entity. If the legacy file
|
||||
* is unreadable, fail closed (sync disabled) rather than guess.
|
||||
* Resolve this launch client's stable device id from settings, or mint and
|
||||
* immediately persist one. There is no standalone-client state to adopt.
|
||||
*/
|
||||
private resolveDeviceId(configuredId: string): string {
|
||||
if (configuredId) return configuredId;
|
||||
|
||||
if (existsSync(this.legacyStatePath)) {
|
||||
try {
|
||||
const parsed = parseJsonWithBom<{ deviceId?: unknown }>(readFileSync(this.legacyStatePath, 'utf-8'));
|
||||
const legacyId = parsed && typeof parsed === 'object' ? parsed.deviceId : undefined;
|
||||
if (typeof legacyId !== 'string' || legacyId === '') {
|
||||
throw new Error('legacy cloud-sync state has no valid deviceId');
|
||||
}
|
||||
try {
|
||||
this.persistDeviceId(legacyId);
|
||||
} catch (persistError) {
|
||||
// Adoption survives a failed persist: the legacy file still holds
|
||||
// the id, so the next start re-adopts the SAME id — no fork risk.
|
||||
logger.warn('CLOUD_SYNC', 'Adopted legacy device id but failed to persist it to settings; will re-adopt on next start', {
|
||||
settingsPath: this.settingsPath,
|
||||
}, persistError instanceof Error ? persistError : new Error(String(persistError)));
|
||||
}
|
||||
logger.info('CLOUD_SYNC', 'Adopted device id from legacy cloud-sync state', {
|
||||
deviceId: legacyId,
|
||||
statePath: this.legacyStatePath,
|
||||
});
|
||||
return legacyId;
|
||||
} catch (error) {
|
||||
this.lastError = 'legacy cloud-sync state unreadable — sync disabled to avoid forking device identity';
|
||||
logger.error('CLOUD_SYNC', 'Legacy cloud-sync state exists but is unusable; refusing to mint a new device id (fix or delete the file)', {
|
||||
statePath: this.legacyStatePath,
|
||||
}, error instanceof Error ? error : new Error(String(error)));
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
// First run on a fresh install: mint and persist immediately, so a later
|
||||
// First run: mint and persist immediately, so a later
|
||||
// transient failure can't mint a different one and fork device identity.
|
||||
const minted = randomUUID();
|
||||
try {
|
||||
|
||||
@@ -358,15 +358,12 @@ export class SyncApply {
|
||||
* and the cursor was reset to 0 — the caller must discard the current page
|
||||
* and re-pull from 0. First-ever epoch is adopted without a reset.
|
||||
*
|
||||
* An epoch MISMATCH means the hub's log was lost/rebuilt: everything this
|
||||
* device previously pushed is gone from the new log, so in the same
|
||||
* transaction every NATIVE row's synced_at is re-nulled — the push drain
|
||||
* re-uploads the corpus into the rebuilt log (hub dedupe makes over-firing
|
||||
* safe). Replica rows are untouched: they are another device's corpus and
|
||||
* must never be pushed under this identity; THEIR origin devices re-push
|
||||
* them the same way. Without this, the pull side would self-heal while
|
||||
* this device's history silently never re-entered the log — every counter
|
||||
* healthy, other devices converging on empty history.
|
||||
* An epoch MISMATCH means the hub's log was lost/rebuilt: eligible native
|
||||
* revisions this device previously pushed are re-nulled so the push drain
|
||||
* can repopulate the rebuilt log. The one-time v47 launch baseline is
|
||||
* deliberately excluded through the exact revisions recorded in
|
||||
* sync_launch_exclusions; a later edit has a higher revision and is
|
||||
* eligible. Replica rows and quarantined (-1) rows are never requeued.
|
||||
*/
|
||||
handleEpoch(epoch: string): boolean {
|
||||
const stored = this.getEpoch();
|
||||
@@ -376,17 +373,35 @@ export class SyncApply {
|
||||
this.setState('epoch', epoch);
|
||||
if (stored !== null) {
|
||||
this.setState('cursor', '0');
|
||||
for (const table of ['observations', 'session_summaries', 'user_prompts']) {
|
||||
for (const { table, kind } of [
|
||||
{ table: 'observations', kind: 'observation' },
|
||||
{ table: 'session_summaries', kind: 'summary' },
|
||||
{ table: 'user_prompts', kind: 'prompt' },
|
||||
]) {
|
||||
requeued += this.db.prepare(`
|
||||
UPDATE ${table} SET synced_at = NULL
|
||||
WHERE synced_at IS NOT NULL AND origin_device_id IS NULL
|
||||
`).run().changes;
|
||||
WHERE synced_at > 0
|
||||
AND origin_device_id IS NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM sync_launch_exclusions AS launch
|
||||
WHERE launch.kind = ?
|
||||
AND launch.origin_local_id = CAST(${table}.id AS TEXT)
|
||||
AND (
|
||||
LENGTH(launch.through_rev) > LENGTH(CAST(${table}.sync_rev AS TEXT))
|
||||
OR (
|
||||
LENGTH(launch.through_rev) = LENGTH(CAST(${table}.sync_rev AS TEXT))
|
||||
AND launch.through_rev >= CAST(${table}.sync_rev AS TEXT)
|
||||
)
|
||||
)
|
||||
)
|
||||
`).run(kind).changes;
|
||||
}
|
||||
}
|
||||
});
|
||||
tx();
|
||||
if (stored !== null) {
|
||||
logger.warn('SYNC_APPLY', 'Sync hub epoch changed — cursor reset, full re-pull required, native corpus requeued for re-push', {
|
||||
logger.warn('SYNC_APPLY', 'Sync hub epoch changed — cursor reset, full re-pull required, eligible native revisions requeued', {
|
||||
oldEpoch: stored,
|
||||
newEpoch: epoch,
|
||||
requeued,
|
||||
|
||||
@@ -138,6 +138,8 @@ export interface SyncClientOptions {
|
||||
userId: string;
|
||||
/** MUST be the CloudSync-resolved device id (single identity source). */
|
||||
deviceId: string;
|
||||
/** Human-readable Hub device label (CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME). */
|
||||
deviceName?: string;
|
||||
/** Injectable for tests; defaults to globalThis.fetch. */
|
||||
fetchImpl?: typeof fetch;
|
||||
/** Poll interval while a session is active. */
|
||||
@@ -203,6 +205,7 @@ export class SyncClient {
|
||||
private readonly token: string;
|
||||
private readonly userId: string;
|
||||
private readonly deviceId: string;
|
||||
private readonly deviceName: string;
|
||||
private readonly fetchImpl: typeof fetch;
|
||||
private readonly activePollMs: number;
|
||||
private readonly idlePollMs: number;
|
||||
@@ -262,6 +265,7 @@ export class SyncClient {
|
||||
this.token = options.token ?? '';
|
||||
this.userId = options.userId ?? '';
|
||||
this.deviceId = options.deviceId;
|
||||
this.deviceName = (options.deviceName ?? '').trim().slice(0, 80);
|
||||
this.fetchImpl = options.fetchImpl ?? globalThis.fetch;
|
||||
this.activePollMs = options.activePollMs ?? 30_000;
|
||||
this.idlePollMs = options.idlePollMs ?? 300_000;
|
||||
@@ -523,6 +527,7 @@ export class SyncClient {
|
||||
'Authorization': `Bearer ${this.token}`,
|
||||
'X-User-Id': this.userId,
|
||||
'X-Device-Id': this.deviceId,
|
||||
...(this.deviceName ? { 'X-Device-Name': this.deviceName } : {}),
|
||||
},
|
||||
// Plain short request — never a held connection (directive #4).
|
||||
signal: AbortSignal.timeout(Math.max(1, Math.min(this.requestTimeoutMs, remaining))),
|
||||
@@ -608,6 +613,7 @@ export class SyncClient {
|
||||
'Authorization': `Bearer ${this.token}`,
|
||||
'X-User-Id': this.userId,
|
||||
'X-Device-Id': this.deviceId,
|
||||
...(this.deviceName ? { 'X-Device-Name': this.deviceName } : {}),
|
||||
},
|
||||
});
|
||||
this.socket = ws;
|
||||
|
||||
@@ -525,6 +525,7 @@ export class WorkerService implements WorkerRef {
|
||||
token: settings.CLAUDE_MEM_CLOUD_SYNC_TOKEN,
|
||||
userId: settings.CLAUDE_MEM_CLOUD_SYNC_USER_ID,
|
||||
deviceId: pullDeviceId,
|
||||
deviceName: settings.CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME,
|
||||
isSessionActive: () => this.sessionManager.getActiveSessionCount() > 0,
|
||||
// Advisory WebSocket (plan Phase 4): enabled by default alongside
|
||||
// the hub URL; CLAUDE_MEM_CLOUD_SYNC_WS='false' pins HTTP-only.
|
||||
@@ -646,9 +647,10 @@ export class WorkerService implements WorkerRef {
|
||||
}
|
||||
|
||||
// Cloud sync startup drain (non-blocking). The database is the queue:
|
||||
// everything unsynced is simply `synced_at IS NULL`, so this one kick
|
||||
// IS backfill, offline catch-up, and retry. Null when no token/user
|
||||
// id/hub URL is configured (DatabaseManager gates construction).
|
||||
// eligible post-launch writes remain `synced_at IS NULL`, so this one
|
||||
// kick handles catch-up and retry without migrating the pre-launch
|
||||
// baseline. Null when no token/user id/hub URL is configured
|
||||
// (DatabaseManager gates construction).
|
||||
this.dbManager.getCloudSync()?.start();
|
||||
// Pull loop start (plan Phase 3 task 3): immediate catch-up pull, then
|
||||
// 30 s active / 5 min idle / suspended after 1 h without sessions.
|
||||
|
||||
@@ -22,13 +22,10 @@ export class DatabaseManager {
|
||||
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
|
||||
// cloudSyncHubUrl gates the one-shot v43 hub-cutover requeue inside the
|
||||
// migration chain (SessionStore.requeueAllForHubCutover) — it must run
|
||||
// BEFORE the first drain so the whole corpus is queued when the hub URL
|
||||
// first appears.
|
||||
this.sessionStore = new SessionStore(this.db, {
|
||||
cloudSyncHubUrl: settings.CLAUDE_MEM_CLOUD_SYNC_HUB_URL,
|
||||
});
|
||||
// The launch schema is SyncHub-native. SessionStore marks any pre-launch
|
||||
// local corpus as a nonqueued baseline once; only subsequent writes enter
|
||||
// the canonical v2 outbox.
|
||||
this.sessionStore = new SessionStore(this.db);
|
||||
this.sessionSearch = new SessionSearch(this.db);
|
||||
|
||||
const chromaEnabled = settings.CLAUDE_MEM_CHROMA_ENABLED !== 'false';
|
||||
@@ -38,10 +35,9 @@ export class DatabaseManager {
|
||||
logger.info('DB', 'Chroma disabled via CLAUDE_MEM_CHROMA_ENABLED=false, using SQLite-only search');
|
||||
}
|
||||
|
||||
// Cloud sync is active ⇔ token AND user id AND hub URL are all non-empty
|
||||
// (no separate enabled flag; empty hub URL = sync OFF — the hard cutover,
|
||||
// plan Phase 3 task 5). Inactive installs get null so the write-site
|
||||
// `getCloudSync()?.notify()` nudges are free no-ops.
|
||||
// Cloud sync is active iff token, user id, and Hub URL are all non-empty.
|
||||
// Inactive installs get null so the write-site `getCloudSync()?.notify()`
|
||||
// nudges are free no-ops.
|
||||
if (
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_TOKEN !== '' &&
|
||||
settings.CLAUDE_MEM_CLOUD_SYNC_USER_ID !== '' &&
|
||||
|
||||
@@ -21,12 +21,15 @@ export class CloudSyncRoutes extends BaseRouteHandler {
|
||||
app.get('/api/sync/status', this.handleGetStatus.bind(this));
|
||||
}
|
||||
|
||||
private handleGetStatus = this.wrapHandler((_req: Request, res: Response): void => {
|
||||
// CloudSync.status() carries counts and metadata only — never the token.
|
||||
const status = this.dbManager.getCloudSync()?.status();
|
||||
if (!status) {
|
||||
private handleGetStatus = this.wrapHandler(async (_req: Request, res: Response): Promise<void> => {
|
||||
const cloudSync = this.dbManager.getCloudSync();
|
||||
if (!cloudSync) {
|
||||
logger.debug('CLOUD_SYNC', 'Status requested but cloud sync is not configured');
|
||||
res.json({ configured: false });
|
||||
return;
|
||||
}
|
||||
res.json(status ?? { configured: false });
|
||||
// Always performs an authenticated, read-only SyncHub status GET. An
|
||||
// empty local queue alone is not evidence that the connection works.
|
||||
res.json(await cloudSync.statusWithHubProbe());
|
||||
});
|
||||
}
|
||||
|
||||
@@ -70,7 +70,6 @@ export interface SettingsDefaults {
|
||||
// entirely (the old per-kind cmem.ai lane was deleted in the hub cutover).
|
||||
CLAUDE_MEM_CLOUD_SYNC_TOKEN: string;
|
||||
CLAUDE_MEM_CLOUD_SYNC_USER_ID: string;
|
||||
CLAUDE_MEM_CLOUD_SYNC_URL: string; // legacy per-kind endpoint base — unused since the hub cutover, kept so existing settings files round-trip
|
||||
CLAUDE_MEM_CLOUD_SYNC_HUB_URL: string;
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: string;
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME: string;
|
||||
@@ -162,9 +161,8 @@ export class SettingsDefaultsManager {
|
||||
// Worker-native cloud sync: credentials come from cmem.ai → Connect.
|
||||
CLAUDE_MEM_CLOUD_SYNC_TOKEN: '',
|
||||
CLAUDE_MEM_CLOUD_SYNC_USER_ID: '',
|
||||
CLAUDE_MEM_CLOUD_SYNC_URL: 'https://cmem.ai/api/pro/sync', // legacy, unused since the hub cutover
|
||||
CLAUDE_MEM_CLOUD_SYNC_HUB_URL: '', // sync-hub base URL (e.g. https://sync.cmem.ai). Empty = sync OFF (hard cutover, plan Phase 3 task 5)
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '', // Resolved at first CloudSync start (legacy state file → adopt; else randomUUID), then persisted back here
|
||||
CLAUDE_MEM_CLOUD_SYNC_HUB_URL: '', // sync-hub base URL (e.g. https://sync.cmem.ai). Empty = sync OFF
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '', // Minted at first CloudSync start, then persisted back here
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME: hostname(), // Human-readable label for the cmem.ai Devices panel
|
||||
CLAUDE_MEM_CLOUD_SYNC_WS: 'true', // Advisory WebSocket speed layer (plan Phase 4). 'false' = HTTP polling only — sync stays fully correct, just poll-latency (prime directive #2)
|
||||
CLAUDE_MEM_TELEGRAM_ENABLED: 'true',
|
||||
|
||||
@@ -72,7 +72,6 @@ export const paths = {
|
||||
combinedCerts: () => join(DATA_DIR, 'combined_certs.pem'),
|
||||
transcriptsConfig: () => join(DATA_DIR, 'transcript-watch.json'),
|
||||
transcriptsState: () => join(DATA_DIR, 'transcript-watch-state.json'),
|
||||
cloudSyncState: () => join(DATA_DIR, 'cloud-sync-state.json'),
|
||||
corpora: () => join(DATA_DIR, 'corpora'),
|
||||
supervisorRegistry: () => join(DATA_DIR, 'supervisor.json'),
|
||||
envFile: () => join(DATA_DIR, '.env'),
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { describe, expect, it } from 'bun:test';
|
||||
import { readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
const root = join(import.meta.dir, '../..');
|
||||
const script = readFileSync(join(root, 'scripts/sync-matrix-e2e.ts'), 'utf8');
|
||||
|
||||
describe('sync matrix E2E safety contract', () => {
|
||||
it('uses only the local Miniflare runner and explicit loopback guards', () => {
|
||||
expect(script).toContain('test/run-miniflare-pro-e2e.mjs');
|
||||
expect(script).toContain("const HUB_DIR = resolve(import.meta.dir, '../workers/sync-hub')");
|
||||
expect(script).toContain("'--worker-root', HUB_DIR");
|
||||
expect(script).toContain("hostname: '127.0.0.1'");
|
||||
expect(script).toContain('refused non-loopback URL');
|
||||
expect(script).not.toContain('wrangler');
|
||||
expect(script).not.toContain('cmem.ai');
|
||||
expect(script).not.toContain('https://');
|
||||
});
|
||||
|
||||
it('spawns the Hub with an allowlisted environment instead of inherited secrets or code selectors', () => {
|
||||
expect(script).toContain("const CHILD_ENV_ALLOWLIST = ['PATH', 'TMPDIR', 'TMP', 'TEMP', 'LANG', 'LC_ALL']");
|
||||
expect(script).toContain('env: childEnvironment({');
|
||||
expect(script).not.toContain('...process.env');
|
||||
expect(script).not.toContain('CMEM_HUB_WORKER_ROOT: process.env');
|
||||
});
|
||||
|
||||
it('defines exactly two real client identities and canonical protocol-v2 pushes', () => {
|
||||
expect(script).toContain("const DEVICE_IDS = { a: 'matrix-device-a', b: 'matrix-device-b' }");
|
||||
expect(script).not.toContain('matrix-device-c');
|
||||
expect(script).not.toContain('rawPush');
|
||||
expect(script).toContain('SessionStore');
|
||||
expect(script).toContain('CloudSync');
|
||||
expect(script).toContain('SyncApply');
|
||||
expect(script).toContain('SyncClient');
|
||||
expect(script).toContain('protocol v2');
|
||||
});
|
||||
|
||||
it('is exposed as the package E2E command', () => {
|
||||
const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf8')) as { scripts?: Record<string, string> };
|
||||
expect(pkg.scripts?.['e2e:sync-matrix']).toBe('bun scripts/sync-matrix-e2e.ts');
|
||||
});
|
||||
});
|
||||
@@ -504,8 +504,7 @@ describe('SessionStore migrations', () => {
|
||||
// anyone reverts the predicate.
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
const missingStatePath = '/nonexistent/claude-mem-cloud-sync-state.json';
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
new SessionStore(db);
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
@@ -520,7 +519,7 @@ describe('SessionStore migrations', () => {
|
||||
|
||||
// A second construction over the fully migrated DB must be a no-op for
|
||||
// session_summaries.
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
new SessionStore(db);
|
||||
|
||||
const row = db.prepare(`
|
||||
SELECT synced_at, origin_device_id, origin_local_id, sync_rev
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { SessionStore } from '../../src/services/sqlite/SessionStore.js';
|
||||
import { CloudSync } from '../../src/services/sync/CloudSync.js';
|
||||
import { SyncApply } from '../../src/services/sync/SyncApply.js';
|
||||
|
||||
const SYNCED_TABLES = ['observations', 'session_summaries', 'user_prompts'] as const;
|
||||
|
||||
@@ -20,295 +19,44 @@ function stampedCount(db: Database, table: string): number {
|
||||
return (db.prepare(`SELECT COUNT(*) AS n FROM ${table} WHERE synced_at IS NOT NULL`).get() as { n: number }).n;
|
||||
}
|
||||
|
||||
function contentSnapshot(db: Database): Record<string, Array<Record<string, unknown>>> {
|
||||
return Object.fromEntries(SYNCED_TABLES.map(table => [
|
||||
table,
|
||||
(db.prepare(`SELECT * FROM ${table} ORDER BY id`).all() as Array<Record<string, unknown>>)
|
||||
.map(({ synced_at: _syncedAt, ...content }) => content),
|
||||
]));
|
||||
}
|
||||
|
||||
function rowCount(db: Database, table: string): number {
|
||||
return (db.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get() as { n: number }).n;
|
||||
}
|
||||
|
||||
function seedRows(db: Database): void {
|
||||
const now = new Date().toISOString();
|
||||
const epoch = Date.now();
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run('content-sync', 'memory-sync', 'sync-project', now, epoch);
|
||||
|
||||
const insertObs = db.prepare(`
|
||||
INSERT INTO observations (memory_session_id, project, type, content_hash, created_at, created_at_epoch)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', ?, ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 5; i++) insertObs.run(`hash-${i}`, now, epoch + i);
|
||||
|
||||
const insertSummary = db.prepare(`
|
||||
INSERT INTO session_summaries (memory_session_id, project, request, created_at, created_at_epoch)
|
||||
VALUES ('memory-sync', 'sync-project', 'request', ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 3; i++) insertSummary.run(now, epoch + i);
|
||||
|
||||
const insertPrompt = db.prepare(`
|
||||
INSERT INTO user_prompts (content_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES ('content-sync', ?, 'prompt', ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 4; i++) insertPrompt.run(i + 1, now, epoch + i);
|
||||
}
|
||||
|
||||
/**
|
||||
* A modern (v35-era) schema WITHOUT synced_at columns, seeded by hand so the
|
||||
* migration's column adoption and legacy stamping can be exercised against
|
||||
* pre-existing rows. `throughVersion: 38` reproduces the community-edge
|
||||
* collision: schema_versions rows 36-38 exist while synced_at does not.
|
||||
*/
|
||||
function seedPreSyncedAtDb(db: Database, throughVersion: number): void {
|
||||
const now = new Date().toISOString();
|
||||
const epoch = Date.now();
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
applied_at TEXT NOT NULL
|
||||
)
|
||||
`);
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
content_session_id TEXT NOT NULL,
|
||||
memory_session_id TEXT UNIQUE,
|
||||
project TEXT NOT NULL,
|
||||
platform_source TEXT NOT NULL DEFAULT 'claude',
|
||||
user_prompt TEXT,
|
||||
started_at TEXT NOT NULL,
|
||||
started_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
status TEXT NOT NULL DEFAULT 'active' CHECK(status IN ('active', 'completed', 'failed')),
|
||||
worker_port INTEGER,
|
||||
prompt_counter INTEGER DEFAULT 0,
|
||||
custom_title TEXT
|
||||
)
|
||||
`);
|
||||
db.run('CREATE UNIQUE INDEX ux_sdk_sessions_platform_content ON sdk_sessions(platform_source, content_session_id)');
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT,
|
||||
type TEXT NOT NULL,
|
||||
title TEXT,
|
||||
subtitle TEXT,
|
||||
facts TEXT,
|
||||
narrative TEXT,
|
||||
concepts TEXT,
|
||||
files_read TEXT,
|
||||
files_modified TEXT,
|
||||
prompt_number INTEGER,
|
||||
discovery_tokens INTEGER DEFAULT 0,
|
||||
content_hash TEXT,
|
||||
agent_type TEXT,
|
||||
agent_id TEXT,
|
||||
merged_into_project TEXT,
|
||||
generated_by_model TEXT,
|
||||
metadata TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
)
|
||||
`);
|
||||
db.run('CREATE UNIQUE INDEX ux_observations_session_hash ON observations(memory_session_id, content_hash)');
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
prompt_number INTEGER,
|
||||
discovery_tokens INTEGER DEFAULT 0,
|
||||
merged_into_project TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
)
|
||||
`);
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE user_prompts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_db_id INTEGER,
|
||||
content_session_id TEXT NOT NULL,
|
||||
prompt_number INTEGER NOT NULL,
|
||||
prompt_text TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(session_db_id) REFERENCES sdk_sessions(id) ON DELETE CASCADE
|
||||
)
|
||||
`);
|
||||
|
||||
db.run(`
|
||||
CREATE TABLE pending_messages (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_db_id INTEGER NOT NULL,
|
||||
content_session_id TEXT NOT NULL,
|
||||
tool_use_id TEXT,
|
||||
message_type TEXT NOT NULL CHECK(message_type IN ('observation', 'summarize')),
|
||||
tool_name TEXT,
|
||||
tool_input TEXT,
|
||||
tool_response TEXT,
|
||||
cwd TEXT,
|
||||
last_user_message TEXT,
|
||||
last_assistant_message TEXT,
|
||||
prompt_number INTEGER,
|
||||
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'processing')),
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
agent_type TEXT,
|
||||
agent_id TEXT,
|
||||
FOREIGN KEY (session_db_id) REFERENCES sdk_sessions(id) ON DELETE CASCADE
|
||||
)
|
||||
`);
|
||||
db.run(`
|
||||
CREATE UNIQUE INDEX ux_pending_session_tool
|
||||
ON pending_messages(session_db_id, tool_use_id)
|
||||
WHERE tool_use_id IS NOT NULL
|
||||
`);
|
||||
|
||||
const insertVersion = db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)');
|
||||
for (let version = 4; version <= throughVersion; version++) insertVersion.run(version, now);
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (id, content_session_id, memory_session_id, project, started_at, started_at_epoch)
|
||||
VALUES (1, 'content-sync', 'memory-sync', 'sync-project', ?, ?)
|
||||
`).run(now, epoch);
|
||||
|
||||
const insertObs = db.prepare(`
|
||||
INSERT INTO observations (memory_session_id, project, type, content_hash, created_at, created_at_epoch)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', ?, ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 5; i++) insertObs.run(`hash-${i}`, now, epoch + i);
|
||||
|
||||
const insertSummary = db.prepare(`
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', 'hash-1', ?, ?)
|
||||
`).run(now, epoch);
|
||||
db.prepare(`
|
||||
INSERT INTO session_summaries (memory_session_id, project, request, created_at, created_at_epoch)
|
||||
VALUES ('memory-sync', 'sync-project', 'request', ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 3; i++) insertSummary.run(now, epoch + i);
|
||||
|
||||
const insertPrompt = db.prepare(`
|
||||
INSERT INTO user_prompts (session_db_id, content_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (1, 'content-sync', ?, 'prompt', ?, ?)
|
||||
`);
|
||||
for (let i = 0; i < 4; i++) insertPrompt.run(i + 1, now, epoch + i);
|
||||
`).run(now, epoch);
|
||||
db.prepare(`
|
||||
INSERT INTO user_prompts (content_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES ('content-sync', 1, 'prompt', ?, ?)
|
||||
`).run(now, epoch);
|
||||
}
|
||||
|
||||
function expectStampedThroughCursors(db: Database, before: number): void {
|
||||
const observations = syncedAtById(db, 'observations');
|
||||
expect(observations.get(1)).toBeGreaterThanOrEqual(before);
|
||||
expect(observations.get(2)).toBeGreaterThanOrEqual(before);
|
||||
expect(observations.get(3)).toBeGreaterThanOrEqual(before);
|
||||
expect(observations.get(4)).toBeNull();
|
||||
expect(observations.get(5)).toBeNull();
|
||||
|
||||
const summaries = syncedAtById(db, 'session_summaries');
|
||||
expect(summaries.get(1)).toBeGreaterThanOrEqual(before);
|
||||
expect(summaries.get(2)).toBeGreaterThanOrEqual(before);
|
||||
expect(summaries.get(3)).toBeNull();
|
||||
|
||||
// Prompts end up NULL regardless of the legacy cursor: the v40 repair
|
||||
// migration re-nulls every prompt's synced_at right after v39 stamps them,
|
||||
// because the legacy client uploaded prompts with the broken
|
||||
// memory_session_id/project mapping and they must re-push through the
|
||||
// fixed mapper.
|
||||
const prompts = syncedAtById(db, 'user_prompts');
|
||||
expect(prompts.get(1)).toBeNull();
|
||||
expect(prompts.get(2)).toBeNull();
|
||||
expect(prompts.get(3)).toBeNull();
|
||||
expect(prompts.get(4)).toBeNull();
|
||||
}
|
||||
|
||||
describe('SessionStore synced_at migration (v39)', () => {
|
||||
let tempDir: string;
|
||||
let missingStatePath: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-synced-at-'));
|
||||
missingStatePath = join(tempDir, 'does-not-exist.json');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('adds synced_at columns and partial unsynced indexes to all three tables', () => {
|
||||
describe('SessionStore SyncHub launch baseline', () => {
|
||||
it('creates synced_at columns, unsynced indexes, and the durable launch boundary', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(columnNames(db, table).has('synced_at')).toBe(true);
|
||||
|
||||
const index = db.prepare(`
|
||||
SELECT sql FROM sqlite_master WHERE type = 'index' AND name = ?
|
||||
`).get(`idx_${table}_unsynced`) as { sql: string } | undefined;
|
||||
expect(index?.sql).toContain('synced_at IS NULL');
|
||||
}
|
||||
|
||||
const version = db.prepare('SELECT version FROM schema_versions WHERE version = 39').get() as { version: number } | undefined;
|
||||
expect(version?.version).toBe(39);
|
||||
|
||||
const plan = db.prepare('EXPLAIN QUERY PLAN SELECT id FROM observations WHERE synced_at IS NULL').all() as Array<{ detail: string }>;
|
||||
expect(plan.some(row => row.detail.includes('idx_observations_unsynced'))).toBe(true);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('is idempotent: repeat construction, even without the version-39 row, does not throw or duplicate columns', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
expect(() => new SessionStore(db, { cloudSyncStatePath: missingStatePath })).not.toThrow();
|
||||
|
||||
// The version row is bookkeeping only — losing it must not break re-runs.
|
||||
db.run('DELETE FROM schema_versions WHERE version = 39');
|
||||
expect(() => new SessionStore(db, { cloudSyncStatePath: missingStatePath })).not.toThrow();
|
||||
|
||||
for (const table of SYNCED_TABLES) {
|
||||
const syncedAtColumns = (db.query(`PRAGMA table_info(${table})`).all() as Array<{ name: string }>)
|
||||
.filter(col => col.name === 'synced_at');
|
||||
expect(syncedAtColumns.length).toBe(1);
|
||||
}
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('adds columns, indexes, and stamps legacy rows even when community-edge version rows 36-38 already exist', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
seedPreSyncedAtDb(db, 38);
|
||||
|
||||
// Collision preconditions: version rows 36-38 present, synced_at absent.
|
||||
const collidingVersions = db.prepare('SELECT COUNT(*) AS n FROM schema_versions WHERE version IN (36, 37, 38)').get() as { n: number };
|
||||
expect(collidingVersions.n).toBe(3);
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(columnNames(db, table).has('synced_at')).toBe(false);
|
||||
}
|
||||
|
||||
const statePath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(statePath, JSON.stringify({
|
||||
deviceId: 'ee1b7637-test',
|
||||
lastId: 3,
|
||||
lastSummaryId: 2,
|
||||
lastPromptId: 2,
|
||||
}));
|
||||
|
||||
const before = Date.now();
|
||||
new SessionStore(db, { cloudSyncStatePath: statePath });
|
||||
|
||||
new SessionStore(db);
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(columnNames(db, table).has('synced_at')).toBe(true);
|
||||
const index = db.prepare(`
|
||||
@@ -316,162 +64,355 @@ describe('SessionStore synced_at migration (v39)', () => {
|
||||
`).get(`idx_${table}_unsynced`) as { sql: string } | undefined;
|
||||
expect(index?.sql).toContain('synced_at IS NULL');
|
||||
}
|
||||
|
||||
expectStampedThroughCursors(db, before);
|
||||
|
||||
const version = db.prepare('SELECT version FROM schema_versions WHERE version = 39').get() as { version: number } | undefined;
|
||||
expect(version?.version).toBe(39);
|
||||
|
||||
// The state file is left in place — later phases still read it.
|
||||
expect(existsSync(statePath)).toBe(true);
|
||||
expect(db.prepare('SELECT version FROM schema_versions WHERE version = 47').get()).not.toBeNull();
|
||||
expect(db.prepare('SELECT version FROM schema_versions WHERE version = 48').get()).not.toBeNull();
|
||||
expect(db.prepare(`
|
||||
SELECT sql FROM sqlite_master WHERE type = 'table' AND name = 'sync_launch_exclusions'
|
||||
`).get()).not.toBeNull();
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('stamps rows at or below the legacy cursors on a v35-era DB when cloud-sync-state.json exists', () => {
|
||||
it('preserves all content while marking only the native pre-launch corpus and clearing stale sync state', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
seedPreSyncedAtDb(db, 35);
|
||||
new SessionStore(db);
|
||||
seedRows(db);
|
||||
db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, content_hash, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', 'already stamped', 'hash-2',
|
||||
'2026-07-20T00:00:00.000Z', 1752969600000, 777)
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, content_hash, created_at, created_at_epoch,
|
||||
synced_at, origin_device_id, origin_local_id)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', 'replica', 'hash-3',
|
||||
'2026-07-20T00:00:01.000Z', 1752969601000, NULL, 'device-other', '9')
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO sync_outbox (op_uuid, rev, body, created_at_epoch)
|
||||
VALUES ('old-mutation', '1', '{"op":"set_title"}', 1)
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO sync_content_outbox
|
||||
(entity_id, kind, origin_local_id, entity_rev, body, operation_sha256, deleted, created_at_epoch)
|
||||
VALUES ('old-doc', 'observation', '1', '1', '{}', 'hash', 1, 1)
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO sync_dead_letter
|
||||
(lane, queue_key, kind, origin_local_id, entity_rev, reason, raw_body, created_at_epoch)
|
||||
VALUES ('content', 'old-doc', 'observation', '1', '1', 'pre-launch fixture', '{}', 1)
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO sync_entity_heads
|
||||
(entity_id, kind, origin_device_id, origin_local_id, entity_rev,
|
||||
operation_sha256, deleted, updated_at_epoch)
|
||||
VALUES ('preserved-head', 'observation', 'device-self', '1', '4', 'head-hash', 0, 1)
|
||||
`).run();
|
||||
const insertState = db.prepare('INSERT INTO sync_state (k, v) VALUES (?, ?)');
|
||||
insertState.run('cursor', '42');
|
||||
insertState.run('epoch', 'pre-launch-epoch');
|
||||
insertState.run('cutover_hub_url', 'https://pre-launch-hub.test');
|
||||
insertState.run('parked_title:mem:old', 'stale title');
|
||||
|
||||
const statePath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(statePath, JSON.stringify({
|
||||
deviceId: 'ee1b7637-test',
|
||||
lastId: 3,
|
||||
lastSummaryId: 2,
|
||||
lastPromptId: 2,
|
||||
}));
|
||||
const contentBefore = contentSnapshot(db);
|
||||
|
||||
const before = Date.now();
|
||||
new SessionStore(db, { cloudSyncStatePath: statePath });
|
||||
// Reproduce a database created before the v47 launch boundary landed.
|
||||
db.run('DELETE FROM schema_versions WHERE version = 47');
|
||||
new SessionStore(db);
|
||||
|
||||
expectStampedThroughCursors(db, before);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('stamps nothing when no state file exists', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
seedPreSyncedAtDb(db, 35);
|
||||
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(columnNames(db, table).has('synced_at')).toBe(true);
|
||||
expect(stampedCount(db, table)).toBe(0);
|
||||
expect(contentSnapshot(db)).toEqual(contentBefore);
|
||||
for (const table of ['session_summaries', 'user_prompts']) {
|
||||
expect(stampedCount(db, table)).toBe(1);
|
||||
}
|
||||
expect(db.prepare(`
|
||||
SELECT title, synced_at, origin_device_id FROM observations ORDER BY id
|
||||
`).all()).toEqual([
|
||||
{ title: null, synced_at: expect.any(Number), origin_device_id: null },
|
||||
{ title: 'already stamped', synced_at: 777, origin_device_id: null },
|
||||
{ title: 'replica', synced_at: null, origin_device_id: 'device-other' },
|
||||
]);
|
||||
expect(rowCount(db, 'sync_outbox')).toBe(0);
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(0);
|
||||
expect(rowCount(db, 'sync_dead_letter')).toBe(0);
|
||||
expect(rowCount(db, 'sync_state')).toBe(0);
|
||||
expect(rowCount(db, 'sync_entity_heads')).toBe(1);
|
||||
expect(db.prepare(`
|
||||
SELECT kind, origin_local_id, through_rev
|
||||
FROM sync_launch_exclusions
|
||||
ORDER BY kind, origin_local_id
|
||||
`).all()).toEqual([
|
||||
{ kind: 'observation', origin_local_id: '1', through_rev: '1' },
|
||||
{ kind: 'observation', origin_local_id: '2', through_rev: '1' },
|
||||
{ kind: 'prompt', origin_local_id: '1', through_rev: '1' },
|
||||
{ kind: 'summary', origin_local_id: '1', through_rev: '1' },
|
||||
]);
|
||||
expect(db.prepare('SELECT entity_rev, operation_sha256 FROM sync_entity_heads').get())
|
||||
.toEqual({ entity_rev: '4', operation_sha256: 'head-hash' });
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not re-run stamping once the columns exist, even if a state file appears later', () => {
|
||||
it('preserves excluded pre-launch revisions across epoch changes while requeueing post-launch native rows', () => {
|
||||
const db = new Database(':memory:');
|
||||
let sync: CloudSync | null = null;
|
||||
try {
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
new SessionStore(db);
|
||||
seedRows(db);
|
||||
|
||||
const statePath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(statePath, JSON.stringify({ deviceId: 'late', lastId: 5, lastSummaryId: 3, lastPromptId: 4 }));
|
||||
// Reproduce a database whose content existed when the one-time launch
|
||||
// boundary was applied.
|
||||
db.run('DELETE FROM schema_versions WHERE version IN (47, 48)');
|
||||
db.run('DELETE FROM sync_launch_exclusions');
|
||||
new SessionStore(db);
|
||||
|
||||
new SessionStore(db, { cloudSyncStatePath: statePath });
|
||||
const baseline = Object.fromEntries(SYNCED_TABLES.map(table => [
|
||||
table,
|
||||
db.prepare(`SELECT id, synced_at FROM ${table} ORDER BY id`).all(),
|
||||
]));
|
||||
expect(rowCount(db, 'sync_launch_exclusions')).toBe(3);
|
||||
|
||||
const now = new Date().toISOString();
|
||||
const epoch = Date.now() + 10_000;
|
||||
db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, content_hash, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', 'post-launch observation',
|
||||
'post-launch-hash', ?, ?, ?)
|
||||
`).run(now, epoch, epoch);
|
||||
db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(memory_session_id, project, request, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('memory-sync', 'sync-project', 'post-launch summary', ?, ?, ?)
|
||||
`).run(now, epoch + 1, epoch + 1);
|
||||
db.prepare(`
|
||||
INSERT INTO user_prompts
|
||||
(content_session_id, prompt_number, prompt_text, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('content-sync', 2, 'post-launch prompt', ?, ?, ?)
|
||||
`).run(now, epoch + 2, epoch + 2);
|
||||
|
||||
const apply = new SyncApply(db, { deviceId: 'epoch-boundary-device' });
|
||||
expect(apply.handleEpoch('epoch-one')).toBe(false);
|
||||
expect(apply.handleEpoch('epoch-two')).toBe(true);
|
||||
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(stampedCount(db, table)).toBe(0);
|
||||
const rows = db.prepare(`SELECT id, synced_at FROM ${table} ORDER BY id`).all() as Array<{
|
||||
id: number;
|
||||
synced_at: number | null;
|
||||
}>;
|
||||
expect(rows[0]).toEqual((baseline[table] as Array<{ id: number; synced_at: number }>)[0]);
|
||||
expect(rows.at(-1)?.synced_at).toBeNull();
|
||||
expect(db.prepare(`
|
||||
SELECT id FROM ${table}
|
||||
WHERE synced_at IS NULL AND origin_device_id IS NULL
|
||||
`).all()).toEqual([{ id: rows.at(-1)?.id }]);
|
||||
}
|
||||
|
||||
sync = new CloudSync(db, {
|
||||
CLAUDE_MEM_CLOUD_SYNC_TOKEN: 'test-token',
|
||||
CLAUDE_MEM_CLOUD_SYNC_USER_ID: 'test-user',
|
||||
CLAUDE_MEM_CLOUD_SYNC_HUB_URL: 'https://hub.test',
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: 'epoch-boundary-device',
|
||||
});
|
||||
expect(sync.status().pending).toEqual({
|
||||
observations: 1,
|
||||
summaries: 1,
|
||||
prompts: 1,
|
||||
mutations: 0,
|
||||
tombstones: 0,
|
||||
});
|
||||
|
||||
// The exclusion is revision-bounded, not a permanent row-id ban. A
|
||||
// post-launch edit of a baseline row has a higher native revision and
|
||||
// must re-enter a later rebuilt Hub log.
|
||||
db.prepare(`
|
||||
UPDATE observations SET sync_rev = '2', synced_at = ? WHERE id = 1
|
||||
`).run(epoch + 3);
|
||||
expect(apply.handleEpoch('epoch-three')).toBe(true);
|
||||
expect(db.prepare('SELECT synced_at FROM observations WHERE id = 1').get())
|
||||
.toEqual({ synced_at: null });
|
||||
} finally {
|
||||
sync?.stop();
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('stamps nothing when the state file contains the JSON literal null', () => {
|
||||
it('repairs an earlier v47 database without excluding later writes', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
seedPreSyncedAtDb(db, 38);
|
||||
new SessionStore(db);
|
||||
const appliedAt = (db.prepare(`
|
||||
SELECT applied_at FROM schema_versions WHERE version = 47
|
||||
`).get() as { applied_at: string }).applied_at;
|
||||
const boundaryMs = Date.parse(appliedAt);
|
||||
|
||||
const statePath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(statePath, 'null');
|
||||
|
||||
expect(() => new SessionStore(db, { cloudSyncStatePath: statePath })).not.toThrow();
|
||||
|
||||
// The migration must complete: version recorded, columns added, no rows stamped.
|
||||
const version = db.prepare('SELECT version FROM schema_versions WHERE version = 39').get() as { version: number } | undefined;
|
||||
expect(version?.version).toBe(39);
|
||||
|
||||
for (const table of SYNCED_TABLES) {
|
||||
expect(columnNames(db, table).has('synced_at')).toBe(true);
|
||||
expect(stampedCount(db, table)).toBe(0);
|
||||
}
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('stamps nothing when the state file is unreadable JSON', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
seedPreSyncedAtDb(db, 35);
|
||||
|
||||
const statePath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(statePath, 'not json{');
|
||||
|
||||
expect(() => new SessionStore(db, { cloudSyncStatePath: statePath })).not.toThrow();
|
||||
|
||||
expect(stampedCount(db, 'observations')).toBe(0);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('SessionStore v40 prompt requeue (one-time cloud repair)', () => {
|
||||
let tempDir: string;
|
||||
let missingStatePath: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-v40-requeue-'));
|
||||
missingStatePath = join(tempDir, 'does-not-exist.json');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('records version 40 and never re-nulls prompts stamped after the repair ran', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
|
||||
const version = db.prepare('SELECT version FROM schema_versions WHERE version = 40').get() as { version: number } | undefined;
|
||||
expect(version?.version).toBe(40);
|
||||
|
||||
// Prompts synced through the FIXED mapper after the repair must keep
|
||||
// their stamps across restarts — a repeat requeue would re-push the
|
||||
// whole history on every worker boot.
|
||||
seedRows(db);
|
||||
db.run('UPDATE user_prompts SET synced_at = 1751234567890');
|
||||
new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
for (const table of SYNCED_TABLES) db.run(`UPDATE ${table} SET synced_at = ${boundaryMs}`);
|
||||
db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, content_hash, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('memory-sync', 'sync-project', 'discovery', 'later write', 'later-write-hash',
|
||||
?, ?, ?)
|
||||
`).run(new Date(boundaryMs + 1).toISOString(), boundaryMs + 1, boundaryMs + 1);
|
||||
|
||||
expect(stampedCount(db, 'user_prompts')).toBe(4);
|
||||
// Earlier v47 builds had only the applied_at boundary and stamps, not
|
||||
// the explicit revision ledger introduced by v48.
|
||||
db.run('DROP TABLE sync_launch_exclusions');
|
||||
db.run('DELETE FROM schema_versions WHERE version = 48');
|
||||
new SessionStore(db);
|
||||
|
||||
expect(rowCount(db, 'sync_launch_exclusions')).toBe(3);
|
||||
expect(db.prepare(`
|
||||
SELECT origin_local_id FROM sync_launch_exclusions
|
||||
WHERE kind = 'observation' ORDER BY origin_local_id
|
||||
`).all()).toEqual([{ origin_local_id: '1' }]);
|
||||
|
||||
const apply = new SyncApply(db, { deviceId: 'v48-repair-device' });
|
||||
expect(apply.handleEpoch('repair-one')).toBe(false);
|
||||
expect(apply.handleEpoch('repair-two')).toBe(true);
|
||||
expect(db.prepare(`
|
||||
SELECT id, synced_at FROM observations ORDER BY id
|
||||
`).all()).toEqual([
|
||||
{ id: 1, synced_at: boundaryMs },
|
||||
{ id: 2, synced_at: null },
|
||||
]);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('runs once and cannot clear post-boundary queues after restarts or lower-version migration repair', () => {
|
||||
const db = new Database(':memory:');
|
||||
try {
|
||||
const store = new SessionStore(db);
|
||||
seedRows(db);
|
||||
for (const table of SYNCED_TABLES) expect(stampedCount(db, table)).toBe(0);
|
||||
store.createSDKSession('content-edit', 'sync-project', 'prompt', 'Post-launch title', 'claude');
|
||||
db.prepare(`
|
||||
INSERT INTO sync_content_outbox
|
||||
(entity_id, kind, origin_local_id, entity_rev, body, operation_sha256, deleted, created_at_epoch)
|
||||
VALUES ('post-launch-doc', 'observation', '1', '1', '{}', 'post-launch-hash', 0, 2)
|
||||
`).run();
|
||||
db.prepare(`
|
||||
INSERT INTO sync_dead_letter
|
||||
(lane, queue_key, kind, origin_local_id, entity_rev, reason, raw_body, created_at_epoch)
|
||||
VALUES ('content', 'post-launch-bad', 'observation', '2', '1', 'post-launch fixture', '{}', 2)
|
||||
`).run();
|
||||
|
||||
// Simulate an older build repairing the lower v44-v46 bookkeeping rows.
|
||||
// Unknown v47 remains in schema_versions, as SQLite migrations must.
|
||||
db.run('DELETE FROM schema_versions WHERE version IN (44, 45, 46)');
|
||||
|
||||
new SessionStore(db);
|
||||
for (const table of SYNCED_TABLES) expect(stampedCount(db, table)).toBe(0);
|
||||
expect(rowCount(db, 'sync_outbox')).toBe(1);
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(1);
|
||||
expect(rowCount(db, 'sync_dead_letter')).toBe(1);
|
||||
expect((db.prepare('SELECT COUNT(*) AS n FROM schema_versions WHERE version = 47').get() as { n: number }).n).toBe(1);
|
||||
|
||||
new SessionStore(db);
|
||||
expect(rowCount(db, 'sync_outbox')).toBe(1);
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(1);
|
||||
expect(rowCount(db, 'sync_dead_letter')).toBe(1);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps later edits, deletes, and revives in the canonical queues across restart', async () => {
|
||||
const db = new Database(':memory:');
|
||||
let sync: CloudSync | null = null;
|
||||
try {
|
||||
const store = new SessionStore(db);
|
||||
seedRows(db);
|
||||
|
||||
store.createSDKSession('content-edit', 'sync-project', 'prompt', 'Post-launch title', 'claude');
|
||||
expect(rowCount(db, 'sync_outbox')).toBe(1);
|
||||
new SessionStore(db);
|
||||
expect(rowCount(db, 'sync_outbox')).toBe(1);
|
||||
|
||||
let seq = 0;
|
||||
let failNext = false;
|
||||
const fetchImpl = (async (_input: RequestInfo | URL, init?: RequestInit) => {
|
||||
if (failNext) {
|
||||
failNext = false;
|
||||
return new Response('offline', { status: 503 });
|
||||
}
|
||||
const request = JSON.parse(String(init?.body)) as {
|
||||
ops: Array<{ body: string; operation_sha256: string }>;
|
||||
};
|
||||
const acked = request.ops.map(op => {
|
||||
const body = JSON.parse(op.body) as {
|
||||
id: string;
|
||||
kind: string;
|
||||
origin_local_id: string | null;
|
||||
entity_rev: string;
|
||||
};
|
||||
seq += 1;
|
||||
return {
|
||||
id: body.id,
|
||||
kind: body.kind,
|
||||
origin_local_id: body.origin_local_id,
|
||||
entity_rev: body.entity_rev,
|
||||
operation_sha256: op.operation_sha256,
|
||||
seq: String(seq),
|
||||
};
|
||||
});
|
||||
return Response.json({ acked, head_seq: String(seq), projected_seq: String(seq) });
|
||||
}) as typeof fetch;
|
||||
sync = new CloudSync(db, {
|
||||
CLAUDE_MEM_CLOUD_SYNC_TOKEN: 'token',
|
||||
CLAUDE_MEM_CLOUD_SYNC_USER_ID: 'user',
|
||||
CLAUDE_MEM_CLOUD_SYNC_HUB_URL: 'https://hub.test',
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: 'device-launch-boundary',
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME: 'launch-boundary-test',
|
||||
}, {
|
||||
fetchImpl,
|
||||
});
|
||||
|
||||
expect(sync.queueDelete('observation', '1')).toBe('2');
|
||||
expect(db.prepare(`
|
||||
SELECT entity_rev, deleted FROM sync_content_outbox WHERE entity_id LIKE 'observation:%'
|
||||
`).get()).toEqual({ entity_rev: '2', deleted: 1 });
|
||||
new SessionStore(db);
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(1);
|
||||
await sync.flush();
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(0);
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO observations
|
||||
(id, memory_session_id, project, type, title, content_hash, created_at, created_at_epoch)
|
||||
VALUES (1, 'memory-sync', 'sync-project', 'discovery', 'revived after launch', 'hash-revived',
|
||||
'2026-07-20T00:00:02.000Z', 1752969602000)
|
||||
`).run();
|
||||
failNext = true;
|
||||
await sync.flush();
|
||||
expect(db.prepare(`
|
||||
SELECT entity_rev, deleted FROM sync_content_outbox
|
||||
WHERE entity_id LIKE 'observation:%' ORDER BY id DESC LIMIT 1
|
||||
`).get()).toEqual({ entity_rev: '3', deleted: 0 });
|
||||
|
||||
const queuedBeforeRestart = rowCount(db, 'sync_content_outbox');
|
||||
new SessionStore(db);
|
||||
expect(rowCount(db, 'sync_content_outbox')).toBe(queuedBeforeRestart);
|
||||
} finally {
|
||||
sync?.stop();
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('SessionStore prompt re-push hooks (memory id lands after first sync)', () => {
|
||||
let tempDir: string;
|
||||
let missingStatePath: string;
|
||||
let db: Database;
|
||||
let store: SessionStore;
|
||||
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-prompt-requeue-'));
|
||||
missingStatePath = join(tempDir, 'does-not-exist.json');
|
||||
db = new Database(':memory:');
|
||||
store = new SessionStore(db, { cloudSyncStatePath: missingStatePath });
|
||||
store = new SessionStore(db);
|
||||
|
||||
const now = new Date().toISOString();
|
||||
const epoch = Date.now();
|
||||
@@ -482,7 +423,6 @@ describe('SessionStore prompt re-push hooks (memory id lands after first sync)',
|
||||
insertSession.run('sess-1', 'mem-a', now, epoch);
|
||||
insertSession.run('sess-2', 'mem-b', now, epoch);
|
||||
|
||||
// All prompts start out synced (as if the pre-registration push happened).
|
||||
const insertPrompt = db.prepare(`
|
||||
INSERT INTO user_prompts (session_db_id, content_session_id, prompt_number, prompt_text, created_at, created_at_epoch, synced_at)
|
||||
VALUES (?, ?, ?, 'prompt', ?, ?, 1751234567890)
|
||||
@@ -494,22 +434,18 @@ describe('SessionStore prompt re-push hooks (memory id lands after first sync)',
|
||||
|
||||
afterEach(() => {
|
||||
db.close();
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('updateMemorySessionId requeues only that session\'s prompts', () => {
|
||||
store.updateMemorySessionId(1, 'mem-a2');
|
||||
|
||||
const prompts = syncedAtById(db, 'user_prompts');
|
||||
expect(prompts.get(1)).toBeNull();
|
||||
expect(prompts.get(2)).toBeNull();
|
||||
expect(prompts.get(3)).toBe(1751234567890); // other session untouched
|
||||
expect(prompts.get(3)).toBe(1751234567890);
|
||||
});
|
||||
|
||||
it('updateMemorySessionId(null) clears the mapping without requeueing', () => {
|
||||
store.updateMemorySessionId(1, null);
|
||||
|
||||
// Re-pushing now would only re-send the fallback shape — nothing to repair.
|
||||
expect(stampedCount(db, 'user_prompts')).toBe(3);
|
||||
});
|
||||
|
||||
|
||||
@@ -45,4 +45,27 @@ describe('skill docs placement (#1651)', () => {
|
||||
expect(content).not.toContain('tree-sitter');
|
||||
expect(content).not.toContain('Bundled Languages');
|
||||
});
|
||||
|
||||
it('cloud-sync/SKILL.md requires the authenticated Hub probe for success', () => {
|
||||
const path = join(SKILLS_DIR, 'cloud-sync/SKILL.md');
|
||||
expect(existsSync(path)).toBe(true);
|
||||
const content = readFileSync(path, 'utf-8');
|
||||
|
||||
expect(content).toContain('hub.reachable: true');
|
||||
expect(content).toContain('hub.reachable: false');
|
||||
expect(content).toContain('authenticated, read-only');
|
||||
expect(content).toContain('GET /v1/sync/status');
|
||||
expect(content).toContain('never appends or advances');
|
||||
expect(content).not.toContain('/api/pro/sync/status');
|
||||
});
|
||||
|
||||
it('cloud sync copy keeps the launch boundary and 4,000,000-byte request contract accurate', () => {
|
||||
const docs = readFileSync(join(import.meta.dir, '../../docs/public/cloud-sync.mdx'), 'utf-8');
|
||||
const source = readFileSync(join(import.meta.dir, '../../src/services/sync/CloudSync.ts'), 'utf-8');
|
||||
|
||||
expect(docs).toContain('up to 500 ops / 4,000,000 encoded');
|
||||
expect(docs).not.toContain('500 ops / 2 MB');
|
||||
expect(source).toContain('no historical/pre-launch backfill');
|
||||
expect(source).not.toContain('This IS backfill');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -33,6 +33,18 @@ function buildHandler(routes: CloudSyncRoutes): (req: Request, res: Response) =>
|
||||
return handler!;
|
||||
}
|
||||
|
||||
async function invoke(
|
||||
handler: (req: Request, res: Response) => void,
|
||||
req: Partial<Request>,
|
||||
res: Partial<Response>,
|
||||
jsonSpy: ReturnType<typeof mock>,
|
||||
): Promise<void> {
|
||||
handler(req as Request, res as Response);
|
||||
for (let index = 0; index < 100 && jsonSpy.mock.calls.length === 0; index++) {
|
||||
await new Promise(resolve => setTimeout(resolve, 1));
|
||||
}
|
||||
}
|
||||
|
||||
describe('CloudSyncRoutes — GET /api/sync/status', () => {
|
||||
beforeEach(() => {
|
||||
loggerSpies = [
|
||||
@@ -49,60 +61,80 @@ describe('CloudSyncRoutes — GET /api/sync/status', () => {
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
it('returns the service status with pending counts when cloud sync is configured', () => {
|
||||
it('performs the authenticated Hub probe even when pending counts are zero', async () => {
|
||||
const status: CloudSyncStatus = {
|
||||
configured: true,
|
||||
deviceId: 'device-fixture',
|
||||
pending: { observations: 3, summaries: 2, prompts: 1, mutations: 0, tombstones: 0 },
|
||||
pending: { observations: 0, summaries: 0, prompts: 0, mutations: 0, tombstones: 0 },
|
||||
quarantine: { count: 0, latestReason: null },
|
||||
lastFlushAt: 1751990400000,
|
||||
lastError: null,
|
||||
hub: {
|
||||
checkedAt: 1751990400100,
|
||||
reachable: true,
|
||||
epoch: '10',
|
||||
headSeq: '20',
|
||||
projectedSeq: '20',
|
||||
error: null,
|
||||
},
|
||||
};
|
||||
const probe = mock(async () => status);
|
||||
const mockDbManager = {
|
||||
getCloudSync: () => ({ status: () => status }),
|
||||
getCloudSync: () => ({ statusWithHubProbe: probe }),
|
||||
};
|
||||
const handler = buildHandler(new CloudSyncRoutes(mockDbManager as any));
|
||||
|
||||
const { req, res, jsonSpy, statusSpy } = createMockReqRes();
|
||||
handler(req as Request, res as Response);
|
||||
await invoke(handler, req, res, jsonSpy);
|
||||
|
||||
expect(probe).toHaveBeenCalledTimes(1);
|
||||
expect(jsonSpy).toHaveBeenCalledTimes(1);
|
||||
expect(jsonSpy).toHaveBeenCalledWith(status);
|
||||
expect(statusSpy).not.toHaveBeenCalled(); // implicit 200
|
||||
});
|
||||
|
||||
it('returns {configured: false} with 200 (not 500) when no service exists', () => {
|
||||
it('returns {configured: false} with 200 (not 500) when no service exists', async () => {
|
||||
const mockDbManager = {
|
||||
getCloudSync: () => null,
|
||||
};
|
||||
const handler = buildHandler(new CloudSyncRoutes(mockDbManager as any));
|
||||
|
||||
const { req, res, jsonSpy, statusSpy } = createMockReqRes();
|
||||
handler(req as Request, res as Response);
|
||||
await invoke(handler, req, res, jsonSpy);
|
||||
|
||||
expect(jsonSpy).toHaveBeenCalledTimes(1);
|
||||
expect(jsonSpy).toHaveBeenCalledWith({ configured: false });
|
||||
expect(statusSpy).not.toHaveBeenCalled(); // no error status set
|
||||
});
|
||||
|
||||
it('never leaks the sync token in the response payload', () => {
|
||||
it('surfaces Hub probe failure and never leaks the sync token', async () => {
|
||||
const mockDbManager = {
|
||||
getCloudSync: () => ({
|
||||
status: () => ({
|
||||
statusWithHubProbe: async () => ({
|
||||
configured: true,
|
||||
deviceId: 'device-fixture',
|
||||
pending: { observations: 0, summaries: 0, prompts: 0 },
|
||||
pending: { observations: 0, summaries: 0, prompts: 0, mutations: 0, tombstones: 0 },
|
||||
quarantine: { count: 0, latestReason: null },
|
||||
lastFlushAt: null,
|
||||
lastError: null,
|
||||
hub: {
|
||||
checkedAt: 1751990400100,
|
||||
reachable: false,
|
||||
epoch: null,
|
||||
headSeq: null,
|
||||
projectedSeq: null,
|
||||
error: 'sync hub status 401: denied',
|
||||
},
|
||||
}),
|
||||
}),
|
||||
};
|
||||
const handler = buildHandler(new CloudSyncRoutes(mockDbManager as any));
|
||||
|
||||
const { req, res, jsonSpy } = createMockReqRes();
|
||||
handler(req as Request, res as Response);
|
||||
await invoke(handler, req, res, jsonSpy);
|
||||
|
||||
const payload = (jsonSpy.mock.calls[0] as unknown[])[0] as Record<string, unknown>;
|
||||
expect(payload.hub).toMatchObject({ reachable: false, error: 'sync hub status 401: denied' });
|
||||
const keys = Object.keys(payload).map(k => k.toLowerCase());
|
||||
expect(keys.some(k => k.includes('token'))).toBe(false);
|
||||
expect(JSON.stringify(payload).toLowerCase()).not.toContain('token');
|
||||
|
||||
@@ -18,7 +18,7 @@ describe('DataRoutes synchronized delete APIs', () => {
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'cmem-delete-routes-'));
|
||||
db = new Database(':memory:');
|
||||
store = new SessionStore(db, { cloudSyncStatePath: join(tempDir, 'missing-state.json') });
|
||||
store = new SessionStore(db);
|
||||
sync = new CloudSync(db, {
|
||||
CLAUDE_MEM_CLOUD_SYNC_TOKEN: 'test-token',
|
||||
CLAUDE_MEM_CLOUD_SYNC_USER_ID: 'test-user',
|
||||
@@ -27,7 +27,6 @@ describe('DataRoutes synchronized delete APIs', () => {
|
||||
CLAUDE_MEM_CLOUD_SYNC_DEVICE_NAME: 'test',
|
||||
}, {
|
||||
settingsPath: join(tempDir, 'settings.json'),
|
||||
legacyStatePath: join(tempDir, 'missing-state.json'),
|
||||
fetchImpl: mock(async () => new Response('{}', { status: 500 })) as typeof fetch,
|
||||
});
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'fs';
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { SessionStore } from '../../../src/services/sqlite/SessionStore.js';
|
||||
@@ -127,7 +127,6 @@ describe('CloudSync', () => {
|
||||
let db: Database;
|
||||
let store: SessionStore;
|
||||
let settingsPath: string;
|
||||
let missingLegacyPath: string;
|
||||
|
||||
function makeSettings(overrides: Partial<CloudSyncSettingKeys> = {}): CloudSyncSettingKeys {
|
||||
return {
|
||||
@@ -148,7 +147,6 @@ describe('CloudSync', () => {
|
||||
return new CloudSync(db, makeSettings(settingsOverrides), {
|
||||
fetchImpl,
|
||||
settingsPath,
|
||||
legacyStatePath: missingLegacyPath,
|
||||
debounceMs: 25,
|
||||
backoffInitialMs: 20,
|
||||
backoffMaxMs: 200,
|
||||
@@ -305,9 +303,8 @@ describe('CloudSync', () => {
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-cloud-sync-'));
|
||||
settingsPath = join(tempDir, 'settings.json');
|
||||
missingLegacyPath = join(tempDir, 'no-such-cloud-sync-state.json');
|
||||
db = new Database(':memory:');
|
||||
store = new SessionStore(db, { cloudSyncStatePath: missingLegacyPath });
|
||||
store = new SessionStore(db);
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-abc', 'mem-1', 'proj-x', ?, 1751234567000, 'active')
|
||||
@@ -530,6 +527,85 @@ describe('CloudSync', () => {
|
||||
expect(status.lastError).toBeNull();
|
||||
});
|
||||
|
||||
it('authenticates a read-only Hub status probe even when the local queue is empty', async () => {
|
||||
const calls: Array<{ url: string; method: string; headers: Headers; body: unknown; hasSignal: boolean }> = [];
|
||||
const impl = (async (input: RequestInfo | URL, init?: RequestInit) => {
|
||||
calls.push({
|
||||
url: String(input),
|
||||
method: init?.method ?? 'GET',
|
||||
headers: new Headers(init?.headers),
|
||||
body: init?.body,
|
||||
hasSignal: init?.signal != null,
|
||||
});
|
||||
return Response.json({
|
||||
protocol_version: 2,
|
||||
epoch: '18446744073709551615',
|
||||
head_seq: '9007199254740993',
|
||||
projected_seq: '9007199254740993',
|
||||
op_count: 7,
|
||||
device_count: 2,
|
||||
});
|
||||
}) as typeof fetch;
|
||||
const sync = makeCloudSync(impl);
|
||||
|
||||
// An empty drain performs no write request and therefore proves nothing
|
||||
// about connectivity. The status route's probe must still hit the Hub.
|
||||
await sync.flush();
|
||||
expect(calls).toHaveLength(0);
|
||||
const status = await sync.statusWithHubProbe();
|
||||
|
||||
expect(calls).toHaveLength(1);
|
||||
expect(calls[0].url).toBe('https://hub.test/v1/sync/status');
|
||||
expect(calls[0].method).toBe('GET');
|
||||
expect(calls[0].body).toBeUndefined();
|
||||
expect(calls[0].hasSignal).toBe(true);
|
||||
expect(calls[0].headers.get('Authorization')).toBe('Bearer test-token-1234');
|
||||
expect(calls[0].headers.get('X-User-Id')).toBe('user-42');
|
||||
expect(calls[0].headers.get('X-Device-Id')).toBe('device-fixture');
|
||||
expect(calls[0].headers.get('X-Device-Name')).toBe('test-host');
|
||||
expect(status.pending).toEqual({ observations: 0, summaries: 0, prompts: 0, mutations: 0, tombstones: 0 });
|
||||
expect(status.hub).toMatchObject({
|
||||
reachable: true,
|
||||
epoch: '18446744073709551615',
|
||||
headSeq: '9007199254740993',
|
||||
projectedSeq: '9007199254740993',
|
||||
error: null,
|
||||
});
|
||||
expect(status.hub.checkedAt).toBeNumber();
|
||||
});
|
||||
|
||||
it('surfaces Hub authentication, network, and malformed-status failures without leaking the token', async () => {
|
||||
const scenarios: Array<{ response: Response | Error; error: RegExp }> = [
|
||||
{
|
||||
response: new Response('denied test-token-1234', { status: 401 }),
|
||||
error: /sync hub status 401: denied \[REDACTED\]/,
|
||||
},
|
||||
{ response: new Error('connect ECONNREFUSED'), error: /ECONNREFUSED/ },
|
||||
{
|
||||
response: Response.json({ protocol_version: 2, epoch: '1', head_seq: '2', projected_seq: '3' }),
|
||||
error: /projected_seq exceeds head_seq/,
|
||||
},
|
||||
];
|
||||
for (const scenario of scenarios) {
|
||||
const impl = (async () => {
|
||||
if (scenario.response instanceof Error) throw scenario.response;
|
||||
return scenario.response.clone();
|
||||
}) as typeof fetch;
|
||||
const sync = makeCloudSync(impl);
|
||||
const status = await sync.statusWithHubProbe();
|
||||
expect(status.hub).toMatchObject({
|
||||
reachable: false,
|
||||
epoch: null,
|
||||
headSeq: null,
|
||||
projectedSeq: null,
|
||||
});
|
||||
expect(status.hub.error).toMatch(scenario.error);
|
||||
expect(JSON.stringify(status)).not.toContain('test-token-1234');
|
||||
expect(status.lastError).toBeNull();
|
||||
sync.stop();
|
||||
}
|
||||
});
|
||||
|
||||
it('packs oversized pages into multiple bodies, each under the request cap', async () => {
|
||||
// Each operation stays under the 256KB canonical-body cap while the
|
||||
// combined page crosses the 4MB request packing budget.
|
||||
@@ -1323,40 +1399,14 @@ describe('CloudSync', () => {
|
||||
});
|
||||
|
||||
describe('device id resolution', () => {
|
||||
it('adopts the legacy cloud-sync-state.json deviceId and never mints a new one', () => {
|
||||
const legacyPath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(legacyPath, JSON.stringify({
|
||||
deviceId: 'legacy-dev-123',
|
||||
lastId: 10,
|
||||
lastSummaryId: 2,
|
||||
lastPromptId: 3,
|
||||
}));
|
||||
|
||||
it('uses the settings-configured device id without rewriting settings', () => {
|
||||
const { impl } = makeFetchMock();
|
||||
const sync = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '' }, { legacyStatePath: legacyPath });
|
||||
expect(sync.status().deviceId).toBe('legacy-dev-123');
|
||||
|
||||
// Persisted back to settings so future starts skip legacy resolution.
|
||||
const persisted = JSON.parse(readFileSync(settingsPath, 'utf-8'));
|
||||
expect(persisted.CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID).toBe('legacy-dev-123');
|
||||
|
||||
// A second instance resolving from scratch adopts the SAME id.
|
||||
const again = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '' }, { legacyStatePath: legacyPath });
|
||||
expect(again.status().deviceId).toBe('legacy-dev-123');
|
||||
});
|
||||
|
||||
it('prefers the settings-configured device id over the legacy file', () => {
|
||||
const legacyPath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(legacyPath, JSON.stringify({ deviceId: 'legacy-dev-123' }));
|
||||
|
||||
const { impl } = makeFetchMock();
|
||||
const sync = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: 'settings-dev-9' }, { legacyStatePath: legacyPath });
|
||||
const sync = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: 'settings-dev-9' });
|
||||
expect(sync.status().deviceId).toBe('settings-dev-9');
|
||||
// No resolution ran, so nothing was persisted.
|
||||
expect(existsSync(settingsPath)).toBe(false);
|
||||
});
|
||||
|
||||
it('mints a UUID and persists it when neither settings nor legacy state exist', () => {
|
||||
it('mints a UUID and persists it when settings have no device id', () => {
|
||||
const { impl } = makeFetchMock();
|
||||
const sync = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '' });
|
||||
|
||||
@@ -1366,26 +1416,6 @@ describe('CloudSync', () => {
|
||||
const persisted = JSON.parse(readFileSync(settingsPath, 'utf-8'));
|
||||
expect(persisted.CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID).toBe(deviceId);
|
||||
});
|
||||
|
||||
it('fails closed (no uploads, no minting) when the legacy state file is corrupt', async () => {
|
||||
seedObservation();
|
||||
const legacyPath = join(tempDir, 'cloud-sync-state.json');
|
||||
writeFileSync(legacyPath, 'not json{');
|
||||
|
||||
const { impl, calls } = makeFetchMock();
|
||||
const sync = makeCloudSync(impl, { CLAUDE_MEM_CLOUD_SYNC_DEVICE_ID: '' }, { legacyStatePath: legacyPath });
|
||||
|
||||
sync.start();
|
||||
sync.notify();
|
||||
await sleep(120);
|
||||
|
||||
expect(calls.length).toBe(0);
|
||||
expect(sync.status().deviceId).toBe('');
|
||||
expect(sync.status().lastError).toContain('legacy cloud-sync state unreadable');
|
||||
expect(pendingCount('observations')).toBe(1);
|
||||
// Nothing persisted — a new id here would fork every cloud row.
|
||||
expect(existsSync(settingsPath)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('sync-mode piggyback (kill switch, plan Phase 5 task 2)', () => {
|
||||
|
||||
@@ -1,145 +0,0 @@
|
||||
// Phase 3 verification (plan 2026-07-17, task 5): the settings-conditional
|
||||
// hub-cutover one-shot, keyed on HUB IDENTITY (sync_state 'cutover_hub_url'),
|
||||
// not a burnable version number. It must fire exactly once per (DB, hub URL):
|
||||
// first configuration fires it, same-URL reboots are no-ops, and a DIFFERENT
|
||||
// hub URL later fires it again — otherwise a device pointed at a new hub
|
||||
// would never push its corpus into the new (empty) log while every counter
|
||||
// read healthy. The requeue and the identity write commit in one
|
||||
// transaction; settings-less constructors stay inert.
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { SessionStore } from '../../../src/services/sqlite/SessionStore.js';
|
||||
|
||||
const ISO = '2026-07-09T00:00:00.000Z';
|
||||
|
||||
describe('hub cutover one-shot (hub-identity keyed)', () => {
|
||||
let tempDir: string;
|
||||
let dbPath: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-hub-cutover-'));
|
||||
dbPath = join(tempDir, 'claude-mem.db');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function open(hubUrl?: string): SessionStore {
|
||||
return new SessionStore(dbPath, {
|
||||
cloudSyncStatePath: join(tempDir, 'none.json'),
|
||||
...(hubUrl !== undefined ? { cloudSyncHubUrl: hubUrl } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
function seed(store: SessionStore): void {
|
||||
store.db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-1', 'mem-1', 'proj-x', ?, 1751234567000, 'active')
|
||||
`).run(ISO);
|
||||
// A native row already pushed through some earlier lane…
|
||||
store.db.prepare(`
|
||||
INSERT INTO observations (memory_session_id, project, type, title, created_at, created_at_epoch, synced_at)
|
||||
VALUES ('mem-1', 'proj-x', 'discovery', 'native', ?, 1751234567890, 111)
|
||||
`).run(ISO);
|
||||
// …and a replica applied from the hub (another device's corpus).
|
||||
store.db.prepare(`
|
||||
INSERT INTO observations (memory_session_id, project, type, title, created_at, created_at_epoch, synced_at, origin_device_id, origin_local_id)
|
||||
VALUES ('mem-1', 'proj-x', 'discovery', 'replica', ?, 1751234567891, 222, 'device-other', '9')
|
||||
`).run(ISO);
|
||||
}
|
||||
|
||||
function syncedAt(db: Database, title: string): number | null {
|
||||
return (db.prepare('SELECT synced_at FROM observations WHERE title = ?').get(title) as { synced_at: number | null }).synced_at;
|
||||
}
|
||||
|
||||
function cutoverHubUrl(db: Database): string | null {
|
||||
const row = db.prepare(`SELECT v FROM sync_state WHERE k = 'cutover_hub_url'`).get() as { v: string } | undefined;
|
||||
return row?.v ?? null;
|
||||
}
|
||||
|
||||
it('stays inert while the hub URL is absent (settings-less constructors included)', () => {
|
||||
const store = open();
|
||||
seed(store);
|
||||
expect(cutoverHubUrl(store.db)).toBeNull();
|
||||
store.db.close();
|
||||
|
||||
const again = open(); // e.g. a CLI-path constructor that knows no settings
|
||||
expect(cutoverHubUrl(again.db)).toBeNull();
|
||||
expect(syncedAt(again.db, 'native')).toBe(111);
|
||||
again.db.close();
|
||||
});
|
||||
|
||||
it('fires when the hub URL first appears: native rows requeued, replicas untouched, identity stored', () => {
|
||||
const before = open();
|
||||
seed(before);
|
||||
before.db.close();
|
||||
|
||||
const cutover = open('https://hub-one.test');
|
||||
expect(cutoverHubUrl(cutover.db)).toBe('https://hub-one.test');
|
||||
expect(syncedAt(cutover.db, 'native')).toBeNull(); // re-pushes its own corpus
|
||||
expect(syncedAt(cutover.db, 'replica')).toBe(222); // another device's corpus — never ours to push
|
||||
// Legacy bookkeeping marker still recorded (not consulted for gating).
|
||||
expect(cutover.db.prepare('SELECT version FROM schema_versions WHERE version = 43').get()).not.toBeNull();
|
||||
cutover.db.close();
|
||||
});
|
||||
|
||||
it('is a no-op on later boots with the same hub URL (trailing-slash variants included)', () => {
|
||||
const first = open('https://hub-one.test');
|
||||
seed(first);
|
||||
// Simulate the drain re-stamping after the cutover push.
|
||||
first.db.prepare(`UPDATE observations SET synced_at = 333 WHERE title = 'native'`).run();
|
||||
first.db.close();
|
||||
|
||||
const same = open('https://hub-one.test');
|
||||
expect(syncedAt(same.db, 'native')).toBe(333);
|
||||
same.db.close();
|
||||
|
||||
const slashed = open('https://hub-one.test///');
|
||||
expect(syncedAt(slashed.db, 'native')).toBe(333); // normalized — same identity
|
||||
slashed.db.close();
|
||||
});
|
||||
|
||||
it('re-fires exactly once when the hub URL CHANGES, and clearing the URL is inert', () => {
|
||||
const first = open('https://hub-one.test');
|
||||
seed(first);
|
||||
first.db.prepare(`UPDATE observations SET synced_at = 333 WHERE title = 'native'`).run();
|
||||
first.db.close();
|
||||
|
||||
// (a) URL cleared: sync is OFF; nothing fires, identity is retained.
|
||||
const cleared = open('');
|
||||
expect(syncedAt(cleared.db, 'native')).toBe(333);
|
||||
expect(cutoverHubUrl(cleared.db)).toBe('https://hub-one.test');
|
||||
cleared.db.close();
|
||||
|
||||
// A DIFFERENT hub appears: its log has none of our corpus — re-fire.
|
||||
const second = open('https://hub-two.test');
|
||||
expect(cutoverHubUrl(second.db)).toBe('https://hub-two.test');
|
||||
expect(syncedAt(second.db, 'native')).toBeNull();
|
||||
expect(syncedAt(second.db, 'replica')).toBe(222);
|
||||
second.db.prepare(`UPDATE observations SET synced_at = 444 WHERE title = 'native'`).run();
|
||||
second.db.close();
|
||||
|
||||
// Same second URL again: exactly-once per (DB, hub URL).
|
||||
const again = open('https://hub-two.test');
|
||||
expect(syncedAt(again.db, 'native')).toBe(444);
|
||||
again.db.close();
|
||||
});
|
||||
|
||||
it('self-heals pre-fix DBs that burned v43 without a stored hub identity', () => {
|
||||
const legacy = open();
|
||||
seed(legacy);
|
||||
// Simulate the pre-fix state: v43 burned, no cutover_hub_url row.
|
||||
legacy.db.prepare(`INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (43, ?)`).run(ISO);
|
||||
legacy.db.close();
|
||||
|
||||
const healed = open('https://hub-one.test');
|
||||
expect(syncedAt(healed.db, 'native')).toBeNull(); // one extra re-push; hub dedupe makes it safe
|
||||
expect(cutoverHubUrl(healed.db)).toBe('https://hub-one.test');
|
||||
healed.db.close();
|
||||
});
|
||||
});
|
||||
@@ -66,7 +66,7 @@ describe('mutation sites', () => {
|
||||
|
||||
beforeEach(() => {
|
||||
db = new Database(':memory:');
|
||||
store = new SessionStore(db, { cloudSyncStatePath: join(tempDir, 'none.json') });
|
||||
store = new SessionStore(db);
|
||||
});
|
||||
|
||||
afterEach(() => db.close());
|
||||
@@ -119,7 +119,7 @@ describe('mutation sites', () => {
|
||||
|
||||
beforeEach(() => {
|
||||
db = new Database(':memory:');
|
||||
store = new SessionStore(db, { cloudSyncStatePath: join(tempDir, 'none.json') });
|
||||
store = new SessionStore(db);
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, platform_source, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-1', NULL, 'proj-x', 'claude', ?, 1751234567000, 'active')
|
||||
@@ -211,7 +211,7 @@ describe('mutation sites', () => {
|
||||
|
||||
beforeEach(() => {
|
||||
dbPath = join(tempDir, 'claude-mem.db');
|
||||
const store = new SessionStore(dbPath, { cloudSyncStatePath: join(tempDir, 'none.json') });
|
||||
const store = new SessionStore(dbPath);
|
||||
store.db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-1', 'mem-1', 'parent/wt', ?, 1751234567000, 'active')
|
||||
@@ -332,7 +332,7 @@ describe('mutation sites', () => {
|
||||
const dataDir = join(tempDir, 'data');
|
||||
mkdirSync(dataDir);
|
||||
const dbPath = join(dataDir, 'claude-mem.db');
|
||||
const store = new SessionStore(dbPath, { cloudSyncStatePath: join(tempDir, 'none.json') });
|
||||
const store = new SessionStore(dbPath);
|
||||
store.db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-1', 'mem-1', 'mainrepo/wt', ?, 1751234567000, 'active')
|
||||
@@ -383,7 +383,7 @@ describe('mutation sites', () => {
|
||||
const dataDir = join(tempDir, 'data');
|
||||
mkdirSync(dataDir);
|
||||
const dbPath = join(dataDir, 'claude-mem.db');
|
||||
const store = new SessionStore(dbPath, { cloudSyncStatePath: join(tempDir, 'none.json') });
|
||||
const store = new SessionStore(dbPath);
|
||||
store.db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-1', 'mem-1', 'stale-name', ?, 1751234567000, 'active')
|
||||
|
||||
@@ -109,7 +109,6 @@ describe('SyncApply', () => {
|
||||
let tempDir: string;
|
||||
let db: Database;
|
||||
let settingsPath: string;
|
||||
let missingLegacyPath: string;
|
||||
|
||||
function makeApply(options: { deviceId?: string; chromaSync?: ChromaSyncLike | null } = {}): SyncApply {
|
||||
return new SyncApply(db, {
|
||||
@@ -159,9 +158,8 @@ describe('SyncApply', () => {
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-sync-apply-'));
|
||||
settingsPath = join(tempDir, 'settings.json');
|
||||
missingLegacyPath = join(tempDir, 'no-such-cloud-sync-state.json');
|
||||
db = new Database(':memory:');
|
||||
new SessionStore(db, { cloudSyncStatePath: missingLegacyPath });
|
||||
new SessionStore(db);
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES ('sess-abc', 'mem-1', 'proj-x', ?, 1751234567000, 'active')
|
||||
@@ -202,7 +200,7 @@ describe('SyncApply', () => {
|
||||
});
|
||||
|
||||
it('is idempotent — re-running the constructor changes nothing', () => {
|
||||
new SessionStore(db, { cloudSyncStatePath: missingLegacyPath });
|
||||
new SessionStore(db);
|
||||
const cols = db.query('PRAGMA table_info(observations)').all() as Array<{ name: string }>;
|
||||
expect(cols.filter(c => c.name === 'origin_device_id').length).toBe(1);
|
||||
});
|
||||
@@ -497,7 +495,6 @@ describe('SyncApply', () => {
|
||||
const sync = new CloudSync(db, makeSettings(), {
|
||||
fetchImpl,
|
||||
settingsPath,
|
||||
legacyStatePath: missingLegacyPath,
|
||||
debounceMs: 25,
|
||||
backoffInitialMs: 20,
|
||||
backoffMaxMs: 200,
|
||||
|
||||
@@ -147,6 +147,7 @@ describe('SyncClient advisory WebSocket', () => {
|
||||
token: 'test-token-1234',
|
||||
userId: 'user-42',
|
||||
deviceId: SELF,
|
||||
deviceName: 'test laptop',
|
||||
fetchImpl,
|
||||
webSocketImpl: ws,
|
||||
// Slow poll tiers by default: WS behavior must not hide behind polls.
|
||||
@@ -176,7 +177,7 @@ describe('SyncClient advisory WebSocket', () => {
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-sync-ws-'));
|
||||
db = new Database(':memory:');
|
||||
new SessionStore(db, { cloudSyncStatePath: join(tempDir, 'no-legacy.json') });
|
||||
new SessionStore(db);
|
||||
apply = new SyncApply(db, { deviceId: SELF });
|
||||
clients = [];
|
||||
});
|
||||
@@ -187,7 +188,7 @@ describe('SyncClient advisory WebSocket', () => {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('connects to the ws URL with the exact auth header trio', async () => {
|
||||
it('connects to the ws URL with auth and device metadata headers', async () => {
|
||||
const { impl } = makeHub({ epoch: '1' });
|
||||
const { ctor, sockets } = makeWsFactory();
|
||||
makeClient(impl, ctor).start();
|
||||
@@ -199,6 +200,7 @@ describe('SyncClient advisory WebSocket', () => {
|
||||
'Authorization': 'Bearer test-token-1234',
|
||||
'X-User-Id': 'user-42',
|
||||
'X-Device-Id': SELF,
|
||||
'X-Device-Name': 'test laptop',
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -79,6 +79,7 @@ describe('SyncClient', () => {
|
||||
token: 'test-token-1234',
|
||||
userId: 'user-42',
|
||||
deviceId: SELF,
|
||||
deviceName: 'test laptop',
|
||||
fetchImpl,
|
||||
// This suite covers the HTTP lanes exactly as they behave with the
|
||||
// advisory socket absent (prime directive #2: deleting the socket path
|
||||
@@ -108,7 +109,7 @@ describe('SyncClient', () => {
|
||||
beforeEach(() => {
|
||||
tempDir = mkdtempSync(join(tmpdir(), 'claude-mem-sync-client-'));
|
||||
db = new Database(':memory:');
|
||||
new SessionStore(db, { cloudSyncStatePath: join(tempDir, 'no-legacy.json') });
|
||||
new SessionStore(db);
|
||||
apply = new SyncApply(db, { deviceId: SELF });
|
||||
clients = [];
|
||||
});
|
||||
@@ -133,6 +134,7 @@ describe('SyncClient', () => {
|
||||
expect(state.requests[0].headers['Authorization']).toBe('Bearer test-token-1234');
|
||||
expect(state.requests[0].headers['X-User-Id']).toBe('user-42');
|
||||
expect(state.requests[0].headers['X-Device-Id']).toBe(SELF);
|
||||
expect(state.requests[0].headers['X-Device-Name']).toBe('test laptop');
|
||||
});
|
||||
|
||||
it('loops while more=true, presenting the advanced cursor each page', async () => {
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
|
||||
Production deployment steps for the two-lane sync hub (plan
|
||||
`plans/2026-07-17-phase5-two-lane-sync.md`). Everything below is a
|
||||
**production-only** action: local dev, vitest, and the e2e scripts need none
|
||||
of it (they run against `wrangler dev` with `.dev.vars`).
|
||||
**production-only** action: local Vitest and the canonical matrix E2E need none
|
||||
of it. The matrix starts an ephemeral local Miniflare Worker/SQLite Durable
|
||||
Object plus loopback-only verifier/projector sidecars.
|
||||
|
||||
Prime directive for every knob in this file: **cost guardrails are
|
||||
structural — watchdog trips → poll mode, never "stop working."** A tripped
|
||||
@@ -115,7 +116,33 @@ or endpoint that can enable deletion. Keep the full ordered log until a
|
||||
snapshot/reset bootstrap protocol exists, so a newly first-seen device at
|
||||
cursor `0` can always replay contiguous history.
|
||||
|
||||
### 1.4 Local Pro/Hub Miniflare E2E hook
|
||||
### 1.4 Device admission ceiling
|
||||
|
||||
The Durable Object admits at most 64 distinct device ids per user, atomically,
|
||||
across push, pull, and WebSocket touch paths. A 65th id on an admitting path
|
||||
returns `409 {"error":"device_limit_exceeded"}`; already-registered devices
|
||||
continue normally. Public status and internal metadata are read-only for an
|
||||
unknown id, so connectivity probes cannot consume or exhaust device slots;
|
||||
status may refresh last-seen/name only for an already-registered device.
|
||||
Unknown-device renames also create nothing. Treat the stable 409 from an
|
||||
admitting path as an account/device-management condition, not a retriable
|
||||
transport failure.
|
||||
|
||||
### 1.5 Local Pro/Hub Miniflare E2E hook
|
||||
|
||||
The canonical client matrix is the safe default from the repository root:
|
||||
|
||||
```sh
|
||||
npm run e2e:sync-matrix
|
||||
```
|
||||
|
||||
It drives exactly two file-backed client stacks against the actual bundled
|
||||
Worker and SQLite Durable Object. The Hub, token verifier, and projector all
|
||||
bind ephemeral `127.0.0.1` ports; runtime guards reject any non-loopback URL.
|
||||
It covers WebSocket hints plus HTTP authority, concurrent writes, offline
|
||||
retry, restart, delete/revive, all three mutation types, decimal cursors, and
|
||||
the projection checkpoint reaching the head. The runner owns clean shutdown
|
||||
and leaves no persistent Hub or client state.
|
||||
|
||||
`test/miniflare-pro-e2e.ts` exports
|
||||
`createSyncHubMiniflareE2EOptions(...)`. A sibling Pro Vitest config passes its
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
# SyncHub internal metadata contract
|
||||
|
||||
SyncHub owns device identity, last-seen state, sync cursors, and the
|
||||
authoritative Turbopuffer projection checkpoint. Pro reads this payload-free
|
||||
control-plane state instead of querying content tables or `pro_sync_state`.
|
||||
|
||||
Both routes require:
|
||||
|
||||
```http
|
||||
Authorization: Bearer <CMEM_INTERNAL_PROJECTOR_SECRET>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
Missing or incorrect credentials return `401`. Bodies are exact versioned
|
||||
objects: unknown fields return `400`, and non-`POST` methods return `405`.
|
||||
|
||||
## Read metadata
|
||||
|
||||
`POST /internal/v1/sync/metadata`
|
||||
|
||||
```json
|
||||
{ "protocol_version": 1, "user_id": "canonical-user-id" }
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"protocol_version": 1,
|
||||
"user_id": "canonical-user-id",
|
||||
"epoch": "1784531270123",
|
||||
"head_seq": "42",
|
||||
"projected_seq": "40",
|
||||
"projection_lag_ops": "2",
|
||||
"sync_health": "projector_lagging",
|
||||
"devices": [
|
||||
{
|
||||
"device_id": "a-stable-device-id",
|
||||
"name": "Alex's Laptop",
|
||||
"last_seen_at": "2026-07-20T12:00:00.000Z",
|
||||
"last_seen_epoch_ms": "1784548800000",
|
||||
"last_ack_seq": "39",
|
||||
"cursor_lag_ops": "3",
|
||||
"connection_state": "disconnected"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`epoch`, every sequence/cursor, and every lag are canonical unsigned decimal
|
||||
strings. They must never pass through a JavaScript `number`.
|
||||
|
||||
`sync_health` is `healthy` exactly when `projected_seq === head_seq`, and is
|
||||
`projector_lagging` otherwise. An offline device's cursor lag is informational
|
||||
and does not make projection unhealthy. Devices sort by most-recent last seen,
|
||||
then by device id. This response intentionally contains no content, content
|
||||
counts, local outbox depth, or migration/backfill telemetry.
|
||||
|
||||
Clients send `X-Device-Name` (trimmed, at most 80 characters) with their Hub
|
||||
requests. The first nonempty client name is retained; a dashboard rename is
|
||||
not overwritten by a later hostname header. `connection_state` reflects an
|
||||
accepted advisory WebSocket at read time. Correctness never depends on it.
|
||||
|
||||
## Device admission bound
|
||||
|
||||
Each user's Hub stores at most 64 distinct device ids. Device-admitting paths
|
||||
enforce the same bound: push, pull, and WebSocket upgrade (including their
|
||||
optional `X-Device-Name` header). At the limit, an existing device still
|
||||
updates last-seen/cursor state and continues to sync; a previously unseen id
|
||||
on one of those paths receives HTTP `409` with the stable body:
|
||||
|
||||
```json
|
||||
{ "error": "device_limit_exceeded" }
|
||||
```
|
||||
|
||||
Admission is one atomic SQLite statement inside the per-user Durable Object,
|
||||
so concurrent first-seen requests cannot overshoot 64. Metadata returns at
|
||||
most 64 devices. Metadata reads and every public status request are
|
||||
non-admitting: a known status device may refresh last-seen/name, while an
|
||||
unknown `X-Device-Id` is ignored for persistence. This keeps repeated
|
||||
authenticated connectivity probes from exhausting the cap. Renaming an
|
||||
unknown device also creates nothing and remains `404`.
|
||||
|
||||
## Rename a device
|
||||
|
||||
`POST /internal/v1/sync/device-name`
|
||||
|
||||
```json
|
||||
{
|
||||
"protocol_version": 1,
|
||||
"user_id": "canonical-user-id",
|
||||
"device_id": "a-stable-device-id",
|
||||
"name": "Desk Mac"
|
||||
}
|
||||
```
|
||||
|
||||
The name is trimmed and must contain 1–80 characters. The device id is trimmed
|
||||
and must contain 1–128 characters. A registered device returns:
|
||||
|
||||
```json
|
||||
{
|
||||
"protocol_version": 1,
|
||||
"user_id": "canonical-user-id",
|
||||
"device_id": "a-stable-device-id",
|
||||
"name": "Desk Mac"
|
||||
}
|
||||
```
|
||||
|
||||
An unknown device returns `404`; rename never creates a phantom device.
|
||||
@@ -25,6 +25,8 @@ const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
const MAX_PAGE = 500;
|
||||
const ADVANCE_MAX_OPS = 100;
|
||||
const ADVANCE_MAX_FRAME_BYTES = 262_144;
|
||||
export const MAX_DEVICES_PER_USER = 64;
|
||||
export const DEVICE_LIMIT_ERROR = "device_limit_exceeded";
|
||||
/** 45s Hub abort < 60s Pro platform ceiling < 90s fencing lease. */
|
||||
export const PROJECTION_LEASE_MS = 90_000;
|
||||
const encoder = new TextEncoder();
|
||||
@@ -45,12 +47,12 @@ export interface PushResult {
|
||||
head_seq: string;
|
||||
}
|
||||
|
||||
export interface PushRefusal {
|
||||
export interface HubRefusal {
|
||||
refused: true;
|
||||
error: string;
|
||||
}
|
||||
|
||||
export type PushOutcome = PushResult | PushRefusal;
|
||||
export type PushOutcome = PushResult | HubRefusal;
|
||||
|
||||
export interface ChangeOp {
|
||||
seq: string;
|
||||
@@ -67,6 +69,8 @@ export interface ChangesResult {
|
||||
more: boolean;
|
||||
}
|
||||
|
||||
export type ChangesOutcome = ChangesResult | HubRefusal;
|
||||
|
||||
export interface StatusResult {
|
||||
protocol_version: 2;
|
||||
epoch: string;
|
||||
@@ -76,6 +80,29 @@ export interface StatusResult {
|
||||
device_count: number;
|
||||
}
|
||||
|
||||
export type StatusOutcome = StatusResult | HubRefusal;
|
||||
|
||||
export interface DeviceMetadata {
|
||||
device_id: string;
|
||||
name: string | null;
|
||||
last_seen_at: string | null;
|
||||
last_seen_epoch_ms: string | null;
|
||||
last_ack_seq: string;
|
||||
cursor_lag_ops: string;
|
||||
connection_state: "connected" | "disconnected";
|
||||
}
|
||||
|
||||
export interface HubMetadata {
|
||||
protocol_version: 1;
|
||||
user_id: string;
|
||||
epoch: string;
|
||||
head_seq: string;
|
||||
projected_seq: string;
|
||||
projection_lag_ops: string;
|
||||
sync_health: "healthy" | "projector_lagging";
|
||||
devices: DeviceMetadata[];
|
||||
}
|
||||
|
||||
export interface ProjectionLease {
|
||||
acquired: boolean;
|
||||
lease_token?: string;
|
||||
@@ -112,6 +139,14 @@ function projectionError(message: string): Error {
|
||||
return new Error(`${PROJECTION_ERROR_PREFIX} ${message}`);
|
||||
}
|
||||
|
||||
function deviceLimitError(): Error {
|
||||
return new Error(DEVICE_LIMIT_ERROR);
|
||||
}
|
||||
|
||||
function isDeviceLimitError(error: unknown): boolean {
|
||||
return error instanceof Error && error.message === DEVICE_LIMIT_ERROR;
|
||||
}
|
||||
|
||||
interface ValidatedOp {
|
||||
body: CanonicalContentBody;
|
||||
serialized: string;
|
||||
@@ -204,6 +239,18 @@ export class SyncHub extends DurableObject<Env> {
|
||||
}
|
||||
const deviceId = (request.headers.get("X-Device-Id") ?? "").trim();
|
||||
if (deviceId.length === 0) return new Response("missing X-Device-Id header", { status: 400 });
|
||||
const deviceName = normalizeDeviceName(request.headers.get("X-Device-Name"));
|
||||
try {
|
||||
this.touchDevice(deviceId, deviceName);
|
||||
} catch (error) {
|
||||
if (isDeviceLimitError(error)) {
|
||||
return new Response(JSON.stringify({ error: DEVICE_LIMIT_ERROR }), {
|
||||
status: 409,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const [client, server] = Object.values(new WebSocketPair());
|
||||
this.ctx.acceptWebSocket(server);
|
||||
server.serializeAttachment({ device_id: deviceId });
|
||||
@@ -274,7 +321,7 @@ export class SyncHub extends DurableObject<Env> {
|
||||
// Canonical append path and client cursor reads.
|
||||
// ---------------------------------------------------------------------
|
||||
|
||||
async pushOps(deviceId: string, ops: PushOp[]): Promise<PushOutcome> {
|
||||
async pushOps(deviceId: string, ops: PushOp[], deviceName: string | null = null): Promise<PushOutcome> {
|
||||
let rows: ValidatedOp[];
|
||||
try {
|
||||
if (typeof deviceId !== "string" || deviceId.length === 0) throw invalid("deviceId must be non-empty");
|
||||
@@ -294,6 +341,7 @@ export class SyncHub extends DurableObject<Env> {
|
||||
if (error instanceof Error && error.message.startsWith(INVALID_OPS_PREFIX)) {
|
||||
return { refused: true, error: error.message };
|
||||
}
|
||||
if (isDeviceLimitError(error)) return { refused: true, error: DEVICE_LIMIT_ERROR };
|
||||
throw error;
|
||||
}
|
||||
|
||||
@@ -304,12 +352,7 @@ export class SyncHub extends DurableObject<Env> {
|
||||
const acked: AckedOp[] = [];
|
||||
try {
|
||||
this.ctx.storage.transactionSync(() => {
|
||||
sql.exec(
|
||||
`INSERT INTO devices (device_id, last_seen) VALUES (?, ?)
|
||||
ON CONFLICT(device_id) DO UPDATE SET last_seen = excluded.last_seen`,
|
||||
deviceId,
|
||||
now,
|
||||
);
|
||||
this.touchDevice(deviceId, normalizeDeviceName(deviceName), now);
|
||||
for (const row of rows) {
|
||||
const body = row.body;
|
||||
const head = sql.exec<HeadRow>(
|
||||
@@ -387,6 +430,7 @@ export class SyncHub extends DurableObject<Env> {
|
||||
if (error instanceof Error && error.message.startsWith(INVALID_OPS_PREFIX)) {
|
||||
return { refused: true, error: error.message };
|
||||
}
|
||||
if (isDeviceLimitError(error)) return { refused: true, error: DEVICE_LIMIT_ERROR };
|
||||
throw error;
|
||||
}
|
||||
|
||||
@@ -394,28 +438,38 @@ export class SyncHub extends DurableObject<Env> {
|
||||
return { acked, head_seq: this.headSeq() };
|
||||
}
|
||||
|
||||
getChanges(deviceId: string, sinceSeq: string, limit = MAX_PAGE): ChangesResult {
|
||||
getChanges(
|
||||
deviceId: string,
|
||||
sinceSeq: string,
|
||||
limit = MAX_PAGE,
|
||||
deviceName: string | null = null,
|
||||
): ChangesOutcome {
|
||||
if (typeof deviceId !== "string" || deviceId.length === 0) throw invalid("deviceId must be non-empty");
|
||||
const since = assertCanonicalDecimal(sinceSeq);
|
||||
const lim = Number.isFinite(limit) ? Math.min(MAX_PAGE, Math.max(1, Math.floor(limit))) : MAX_PAGE;
|
||||
const head = this.headSeq();
|
||||
const acknowledged = decimalMin(since, head);
|
||||
const sql = this.ctx.storage.sql;
|
||||
const existing = sql.exec<{ last_ack_seq: string }>(
|
||||
"SELECT last_ack_seq FROM devices WHERE device_id = ?",
|
||||
deviceId,
|
||||
).toArray()[0];
|
||||
const nextAck = existing && compareCanonicalDecimals(existing.last_ack_seq, acknowledged) > 0
|
||||
? existing.last_ack_seq
|
||||
: acknowledged;
|
||||
sql.exec(
|
||||
`INSERT INTO devices (device_id, last_seen, last_ack_seq) VALUES (?, ?, ?)
|
||||
ON CONFLICT(device_id) DO UPDATE SET last_seen=excluded.last_seen,
|
||||
last_ack_seq=excluded.last_ack_seq`,
|
||||
deviceId,
|
||||
Date.now(),
|
||||
nextAck,
|
||||
);
|
||||
try {
|
||||
this.ctx.storage.transactionSync(() => {
|
||||
this.touchDevice(deviceId, normalizeDeviceName(deviceName));
|
||||
const existing = sql.exec<{ last_ack_seq: string }>(
|
||||
"SELECT last_ack_seq FROM devices WHERE device_id = ?",
|
||||
deviceId.trim(),
|
||||
).one();
|
||||
const nextAck = compareCanonicalDecimals(existing.last_ack_seq, acknowledged) > 0
|
||||
? existing.last_ack_seq
|
||||
: acknowledged;
|
||||
sql.exec(
|
||||
"UPDATE devices SET last_ack_seq = ? WHERE device_id = ?",
|
||||
nextAck,
|
||||
deviceId.trim(),
|
||||
);
|
||||
});
|
||||
} catch (error) {
|
||||
if (isDeviceLimitError(error)) return { refused: true, error: DEVICE_LIMIT_ERROR };
|
||||
throw error;
|
||||
}
|
||||
const rows = sql.exec<{
|
||||
seq: string;
|
||||
body: string;
|
||||
@@ -453,7 +507,11 @@ export class SyncHub extends DurableObject<Env> {
|
||||
};
|
||||
}
|
||||
|
||||
getStatus(): StatusResult {
|
||||
getStatus(deviceId: string | null = null, deviceName: string | null = null): StatusOutcome {
|
||||
// Status is an authenticated read/probe, not device admission. A known
|
||||
// device may refresh its display metadata, but an arbitrary X-Device-Id
|
||||
// must not consume one of the account's 64 durable device slots.
|
||||
if (deviceId !== null) this.touchExistingDevice(deviceId, normalizeDeviceName(deviceName));
|
||||
const sql = this.ctx.storage.sql;
|
||||
return {
|
||||
protocol_version: 2,
|
||||
@@ -465,6 +523,63 @@ export class SyncHub extends DurableObject<Env> {
|
||||
};
|
||||
}
|
||||
|
||||
getMetadata(userId: string): HubMetadata {
|
||||
if (typeof userId !== "string" || userId.length === 0) throw invalid("user_id must be non-empty");
|
||||
const head = this.headSeq();
|
||||
const projected = this.projectedSeq();
|
||||
const connected = new Set<string>();
|
||||
for (const socket of this.ctx.getWebSockets()) {
|
||||
try {
|
||||
const attachment = socket.deserializeAttachment() as { device_id?: unknown } | null;
|
||||
if (typeof attachment?.device_id === "string") connected.add(attachment.device_id);
|
||||
} catch {}
|
||||
}
|
||||
const devices = this.ctx.storage.sql.exec<{
|
||||
device_id: string;
|
||||
name: string | null;
|
||||
last_ack_seq: string;
|
||||
last_seen: number | null;
|
||||
}>(
|
||||
`SELECT device_id, name, last_ack_seq, last_seen
|
||||
FROM devices
|
||||
ORDER BY last_seen IS NULL, last_seen DESC, device_id
|
||||
LIMIT ${MAX_DEVICES_PER_USER}`,
|
||||
).toArray().map((row) => {
|
||||
const lastSeen = row.last_seen === null ? null : String(row.last_seen);
|
||||
return {
|
||||
device_id: row.device_id,
|
||||
name: row.name,
|
||||
last_seen_at: row.last_seen === null ? null : new Date(row.last_seen).toISOString(),
|
||||
last_seen_epoch_ms: lastSeen,
|
||||
last_ack_seq: row.last_ack_seq,
|
||||
cursor_lag_ops: decimalLag(head, row.last_ack_seq),
|
||||
connection_state: connected.has(row.device_id) ? "connected" as const : "disconnected" as const,
|
||||
};
|
||||
});
|
||||
return {
|
||||
protocol_version: 1,
|
||||
user_id: userId,
|
||||
epoch: this.meta("epoch"),
|
||||
head_seq: head,
|
||||
projected_seq: projected,
|
||||
projection_lag_ops: decimalLag(head, projected),
|
||||
sync_health: head === projected ? "healthy" : "projector_lagging",
|
||||
devices,
|
||||
};
|
||||
}
|
||||
|
||||
renameDevice(deviceId: string, name: string): boolean {
|
||||
const normalizedId = deviceId.trim();
|
||||
const normalizedName = normalizeDeviceName(name);
|
||||
if (normalizedId.length === 0 || normalizedId.length > 128) throw invalid("device_id must be 1-128 characters");
|
||||
if (normalizedName === null) throw invalid("name must be 1-80 characters");
|
||||
return this.ctx.storage.sql.exec(
|
||||
"UPDATE devices SET name = ? WHERE device_id = ?",
|
||||
normalizedName,
|
||||
normalizedId,
|
||||
).rowsWritten > 0;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------
|
||||
// Authoritative projection checkpoint and short per-user lease.
|
||||
// ---------------------------------------------------------------------
|
||||
@@ -656,6 +771,45 @@ export class SyncHub extends DurableObject<Env> {
|
||||
private headSeq(): string { return this.meta("head_seq"); }
|
||||
private projectedSeq(): string { return this.meta("projected_seq"); }
|
||||
|
||||
private touchDevice(deviceId: string, name: string | null, now = Date.now()): void {
|
||||
const normalizedId = this.normalizeDeviceId(deviceId);
|
||||
const result = this.ctx.storage.sql.exec(
|
||||
`INSERT INTO devices (device_id, name, last_seen)
|
||||
SELECT ?, ?, ?
|
||||
WHERE EXISTS (SELECT 1 FROM devices WHERE device_id = ?)
|
||||
OR (SELECT COUNT(*) FROM devices) < ?
|
||||
ON CONFLICT(device_id) DO UPDATE SET
|
||||
name=COALESCE(devices.name, excluded.name),
|
||||
last_seen=excluded.last_seen`,
|
||||
normalizedId,
|
||||
name,
|
||||
now,
|
||||
normalizedId,
|
||||
MAX_DEVICES_PER_USER,
|
||||
);
|
||||
if (result.rowsWritten === 0) throw deviceLimitError();
|
||||
}
|
||||
|
||||
private touchExistingDevice(deviceId: string, name: string | null, now = Date.now()): void {
|
||||
const normalizedId = this.normalizeDeviceId(deviceId);
|
||||
this.ctx.storage.sql.exec(
|
||||
`UPDATE devices
|
||||
SET name = COALESCE(name, ?), last_seen = ?
|
||||
WHERE device_id = ?`,
|
||||
name,
|
||||
now,
|
||||
normalizedId,
|
||||
);
|
||||
}
|
||||
|
||||
private normalizeDeviceId(deviceId: string): string {
|
||||
const normalizedId = deviceId.trim();
|
||||
if (normalizedId.length === 0 || normalizedId.length > 128) {
|
||||
throw invalid("deviceId must be 1-128 characters");
|
||||
}
|
||||
return normalizedId;
|
||||
}
|
||||
|
||||
private meta(key: string): string {
|
||||
const row = this.ctx.storage.sql.exec<{ v: string }>("SELECT v FROM meta WHERE k = ?", key).toArray()[0];
|
||||
if (!row) throw new Error(`sync-hub invariant: missing meta ${key}`);
|
||||
@@ -678,3 +832,18 @@ export class SyncHub extends DurableObject<Env> {
|
||||
this.ctx.storage.sql.exec("DELETE FROM meta WHERE k = ?", key);
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeDeviceName(value: string | null): string | null {
|
||||
if (value === null) return null;
|
||||
const normalized = value.trim();
|
||||
if (normalized.length === 0) return null;
|
||||
if (normalized.length > 80) throw invalid("device name must be at most 80 characters");
|
||||
return normalized;
|
||||
}
|
||||
|
||||
function decimalLag(head: string, cursor: string): string {
|
||||
const canonicalHead = assertCanonicalDecimal(head);
|
||||
const canonicalCursor = assertCanonicalDecimal(cursor);
|
||||
if (compareCanonicalDecimals(canonicalCursor, canonicalHead) >= 0) return "0";
|
||||
return (BigInt(canonicalHead) - BigInt(canonicalCursor)).toString(10);
|
||||
}
|
||||
|
||||
+108
-10
@@ -22,10 +22,13 @@
|
||||
* authentication (canonical-userId binding) as
|
||||
* every other route; the socket itself carries
|
||||
* nothing durable — it is a downstream hint lane.
|
||||
* POST /internal/v1/sync/metadata — payload-free Hub state for Pro.
|
||||
* POST /internal/v1/sync/device-name — rename an existing Hub device.
|
||||
*/
|
||||
|
||||
import type { PushOp } from "./do/SyncHub";
|
||||
import {
|
||||
DEVICE_LIMIT_ERROR,
|
||||
INVALID_OPS_PREFIX,
|
||||
PROJECTION_LEASE_MS,
|
||||
PROJECTION_ERROR_PREFIX,
|
||||
@@ -99,6 +102,7 @@ interface AuthOk {
|
||||
ok: true;
|
||||
userId: string;
|
||||
deviceId: string | null;
|
||||
deviceName: string | null;
|
||||
}
|
||||
|
||||
interface AuthFail {
|
||||
@@ -209,6 +213,14 @@ export async function authenticateRequest(
|
||||
// missing ids). Canonical operation bodies themselves are never rewritten.
|
||||
const deviceIdTrimmed = (request.headers.get("X-Device-Id") ?? "").trim();
|
||||
const deviceId = deviceIdTrimmed.length > 0 ? deviceIdTrimmed : null;
|
||||
const deviceNameTrimmed = (request.headers.get("X-Device-Name") ?? "").trim();
|
||||
if (deviceId !== null && deviceId.length > 128) {
|
||||
return { ok: false, response: errorResponse(400, "X-Device-Id must be at most 128 characters") };
|
||||
}
|
||||
if (deviceNameTrimmed.length > 80) {
|
||||
return { ok: false, response: errorResponse(400, "X-Device-Name must be at most 80 characters") };
|
||||
}
|
||||
const deviceName = deviceNameTrimmed.length > 0 ? deviceNameTrimmed : null;
|
||||
|
||||
if (!authHeader.startsWith("Bearer ")) {
|
||||
return { ok: false, response: errorResponse(401, "missing bearer token") };
|
||||
@@ -231,7 +243,7 @@ export async function authenticateRequest(
|
||||
dependencies.logCacheFailure("get", error);
|
||||
}
|
||||
if (cached === "1") {
|
||||
return { ok: true, userId, deviceId };
|
||||
return { ok: true, userId, deviceId, deviceName };
|
||||
}
|
||||
|
||||
let verifyRes: Response;
|
||||
@@ -275,7 +287,7 @@ export async function authenticateRequest(
|
||||
// could not be populated. The next request will verify upstream again.
|
||||
dependencies.logCacheFailure("put", error);
|
||||
}
|
||||
return { ok: true, userId, deviceId };
|
||||
return { ok: true, userId, deviceId, deviceName };
|
||||
}
|
||||
if (verifyRes.status === 401 || verifyRes.status === 403) {
|
||||
return { ok: false, response: errorResponse(401, "invalid token") };
|
||||
@@ -291,6 +303,7 @@ async function handlePushOps(
|
||||
env: Env,
|
||||
userId: string,
|
||||
deviceId: string,
|
||||
deviceName: string | null,
|
||||
): Promise<Response> {
|
||||
const raw = await request.text();
|
||||
// Deliberate 413s (see the cap constants above): an oversize batch must
|
||||
@@ -324,9 +337,9 @@ async function handlePushOps(
|
||||
|
||||
const stub = env.SYNC_HUB.getByName(userId);
|
||||
try {
|
||||
const result = await stub.pushOps(deviceId, ops as PushOp[]);
|
||||
const result = await stub.pushOps(deviceId, ops as PushOp[], deviceName);
|
||||
if ("refused" in result) {
|
||||
return errorResponse(400, result.error);
|
||||
return errorResponse(result.error === DEVICE_LIMIT_ERROR ? 409 : 400, result.error);
|
||||
}
|
||||
const projection = await drainProjection(env, userId, result.head_seq);
|
||||
if (!projection.ok) {
|
||||
@@ -349,6 +362,7 @@ async function handleGetChanges(
|
||||
env: Env,
|
||||
userId: string,
|
||||
deviceId: string,
|
||||
deviceName: string | null,
|
||||
): Promise<Response> {
|
||||
const sinceRaw = url.searchParams.get("since") ?? "0";
|
||||
const limitRaw = url.searchParams.get("limit");
|
||||
@@ -366,24 +380,100 @@ async function handleGetChanges(
|
||||
|
||||
const stub = env.SYNC_HUB.getByName(userId);
|
||||
try {
|
||||
const result = await stub.getChanges(deviceId, sinceRaw, limit);
|
||||
const result = await stub.getChanges(deviceId, sinceRaw, limit, deviceName);
|
||||
if ("refused" in result) return errorResponse(409, result.error);
|
||||
return json(200, result);
|
||||
} catch (e) {
|
||||
return mapHubError(e);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleGetStatus(env: Env, userId: string): Promise<Response> {
|
||||
async function handleGetStatus(
|
||||
env: Env,
|
||||
userId: string,
|
||||
deviceId: string | null,
|
||||
deviceName: string | null,
|
||||
): Promise<Response> {
|
||||
const stub = env.SYNC_HUB.getByName(userId);
|
||||
try {
|
||||
const result = await stub.getStatus();
|
||||
const result = await stub.getStatus(deviceId, deviceName);
|
||||
if ("refused" in result) return errorResponse(409, result.error);
|
||||
return json(200, result);
|
||||
} catch (e) {
|
||||
return mapHubError(e);
|
||||
}
|
||||
}
|
||||
|
||||
function hasInternalCredential(request: Request, env: Env): boolean {
|
||||
const secret = env.CMEM_INTERNAL_PROJECTOR_SECRET ?? "";
|
||||
return secret.length > 0 && request.headers.get("Authorization") === `Bearer ${secret}`;
|
||||
}
|
||||
|
||||
function exactKeys(record: Record<string, unknown>, expected: string[]): boolean {
|
||||
const keys = Object.keys(record).sort();
|
||||
return keys.length === expected.length && keys.every((key, index) => key === expected[index]);
|
||||
}
|
||||
|
||||
async function readInternalBody(request: Request): Promise<Record<string, unknown> | null> {
|
||||
let value: unknown;
|
||||
try { value = await request.json(); } catch { return null; }
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value)
|
||||
? value as Record<string, unknown>
|
||||
: null;
|
||||
}
|
||||
|
||||
async function handleMetadataRead(request: Request, env: Env): Promise<Response> {
|
||||
if (!hasInternalCredential(request, env)) return errorResponse(401, "invalid internal credential");
|
||||
const body = await readInternalBody(request);
|
||||
if (
|
||||
body === null
|
||||
|| !exactKeys(body, ["protocol_version", "user_id"])
|
||||
|| body.protocol_version !== 1
|
||||
|| typeof body.user_id !== "string"
|
||||
|| body.user_id.trim().length === 0
|
||||
) {
|
||||
return errorResponse(400, "expected exactly {protocol_version:1,user_id}");
|
||||
}
|
||||
const userId = body.user_id.trim();
|
||||
try {
|
||||
return json(200, await env.SYNC_HUB.getByName(userId).getMetadata(userId));
|
||||
} catch (error) {
|
||||
return mapHubError(error);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDeviceRename(request: Request, env: Env): Promise<Response> {
|
||||
if (!hasInternalCredential(request, env)) return errorResponse(401, "invalid internal credential");
|
||||
const body = await readInternalBody(request);
|
||||
if (
|
||||
body === null
|
||||
|| !exactKeys(body, ["device_id", "name", "protocol_version", "user_id"])
|
||||
|| body.protocol_version !== 1
|
||||
|| typeof body.user_id !== "string"
|
||||
|| typeof body.device_id !== "string"
|
||||
|| typeof body.name !== "string"
|
||||
) {
|
||||
return errorResponse(400, "expected exactly {protocol_version:1,user_id,device_id,name}");
|
||||
}
|
||||
const userId = body.user_id.trim();
|
||||
const deviceId = body.device_id.trim();
|
||||
const name = body.name.trim();
|
||||
if (userId.length === 0) return errorResponse(400, "user_id must be non-empty");
|
||||
if (deviceId.length === 0 || deviceId.length > 128) return errorResponse(400, "device_id must be 1-128 characters");
|
||||
if (name.length === 0 || name.length > 80) return errorResponse(400, "name must be 1-80 characters");
|
||||
try {
|
||||
const renamed = await env.SYNC_HUB.getByName(userId).renameDevice(deviceId, name);
|
||||
if (!renamed) return errorResponse(404, "device not found");
|
||||
return json(200, { protocol_version: 1, user_id: userId, device_id: deviceId, name });
|
||||
} catch (error) {
|
||||
return mapHubError(error);
|
||||
}
|
||||
}
|
||||
|
||||
function mapHubError(e: unknown): Response {
|
||||
if (e instanceof Error && e.message.includes(DEVICE_LIMIT_ERROR)) {
|
||||
return errorResponse(409, DEVICE_LIMIT_ERROR);
|
||||
}
|
||||
if (e instanceof Error && e.message.includes(INVALID_OPS_PREFIX)) {
|
||||
return errorResponse(400, e.message);
|
||||
}
|
||||
@@ -738,6 +828,14 @@ export default {
|
||||
if (request.method !== "POST") return errorResponse(405, "use POST");
|
||||
return handleRepairDrain(request, env);
|
||||
}
|
||||
if (pathname === "/internal/v1/sync/metadata") {
|
||||
if (request.method !== "POST") return errorResponse(405, "use POST");
|
||||
return handleMetadataRead(request, env);
|
||||
}
|
||||
if (pathname === "/internal/v1/sync/device-name") {
|
||||
if (request.method !== "POST") return errorResponse(405, "use POST");
|
||||
return handleDeviceRename(request, env);
|
||||
}
|
||||
|
||||
if (
|
||||
pathname !== "/v1/sync/ops" &&
|
||||
@@ -801,18 +899,18 @@ export default {
|
||||
if (pathname === "/v1/sync/ops") {
|
||||
if (request.method !== "POST") return errorResponse(405, "use POST");
|
||||
if (!auth.deviceId) return errorResponse(400, "missing X-Device-Id header");
|
||||
return handlePushOps(request, env, auth.userId, auth.deviceId);
|
||||
return handlePushOps(request, env, auth.userId, auth.deviceId, auth.deviceName);
|
||||
}
|
||||
|
||||
if (pathname === "/v1/sync/changes") {
|
||||
if (request.method !== "GET") return errorResponse(405, "use GET");
|
||||
if (!auth.deviceId) return errorResponse(400, "missing X-Device-Id header");
|
||||
return handleGetChanges(url, env, auth.userId, auth.deviceId);
|
||||
return handleGetChanges(url, env, auth.userId, auth.deviceId, auth.deviceName);
|
||||
}
|
||||
|
||||
// /v1/sync/status
|
||||
if (request.method !== "GET") return errorResponse(405, "use GET");
|
||||
return handleGetStatus(env, auth.userId);
|
||||
return handleGetStatus(env, auth.userId, auth.deviceId, auth.deviceName);
|
||||
})();
|
||||
if (killSwitch.tripped) {
|
||||
response.headers.set(SYNC_MODE_HEADER, SYNC_MODE_POLL);
|
||||
|
||||
Vendored
+1
-1
@@ -8,7 +8,7 @@
|
||||
* watchdog treats absence as "unconfigured" and skips instead of crashing.
|
||||
*/
|
||||
interface Env {
|
||||
/** Shared Hub/Pro internal projector credential. */
|
||||
/** Shared Hub/Pro internal projector and payload-free metadata credential. */
|
||||
CMEM_INTERNAL_PROJECTOR_SECRET?: string;
|
||||
/**
|
||||
* Cloudflare API token for the GraphQL Analytics API.
|
||||
|
||||
@@ -151,7 +151,7 @@ describe("token-verdict cache behavior", () => {
|
||||
};
|
||||
|
||||
const result = await authenticateRequest(request, authEnv, dependencies);
|
||||
expect(result).toEqual({ ok: true, userId, deviceId: "dev-auth" });
|
||||
expect(result).toEqual({ ok: true, userId, deviceId: "dev-auth", deviceName: null });
|
||||
expect(verifyCalls).toBe(1);
|
||||
expect(putCalls).toBe(1);
|
||||
expect(logged).toEqual(["get"]);
|
||||
@@ -177,7 +177,7 @@ describe("token-verdict cache behavior", () => {
|
||||
};
|
||||
|
||||
const result = await authenticateRequest(request, authEnv, dependencies);
|
||||
expect(result).toEqual({ ok: true, userId, deviceId: "dev-auth" });
|
||||
expect(result).toEqual({ ok: true, userId, deviceId: "dev-auth", deviceName: null });
|
||||
expect(verifyCalls).toBe(1);
|
||||
expect(logged).toEqual(["put"]);
|
||||
});
|
||||
@@ -209,6 +209,7 @@ describe("token-verdict cache behavior", () => {
|
||||
ok: true,
|
||||
userId,
|
||||
deviceId: "dev-auth",
|
||||
deviceName: null,
|
||||
});
|
||||
expect(ttlWrites).toEqual([60]);
|
||||
revoked = true;
|
||||
|
||||
@@ -15,9 +15,15 @@ import {
|
||||
type CanonicalContentBody,
|
||||
} from "../src/canonical-content";
|
||||
import {
|
||||
DEVICE_LIMIT_ERROR,
|
||||
MAX_DEVICES_PER_USER,
|
||||
PROJECTION_LEASE_MS,
|
||||
type ChangesOutcome,
|
||||
type ChangesResult,
|
||||
type PushOutcome,
|
||||
type PushResult,
|
||||
type StatusOutcome,
|
||||
type StatusResult,
|
||||
type SyncHub,
|
||||
} from "../src/do/SyncHub";
|
||||
import {
|
||||
@@ -50,6 +56,16 @@ function ok(outcome: PushOutcome): PushResult {
|
||||
return outcome;
|
||||
}
|
||||
|
||||
function changes(outcome: ChangesOutcome): ChangesResult {
|
||||
if ("refused" in outcome) throw new Error(`unexpected refusal: ${outcome.error}`);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
function status(outcome: StatusOutcome): StatusResult {
|
||||
if ("refused" in outcome) throw new Error(`unexpected refusal: ${outcome.error}`);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
function refused(outcome: PushOutcome, pattern: RegExp): void {
|
||||
expect(outcome).toHaveProperty("refused", true);
|
||||
expect((outcome as { error: string }).error).toMatch(pattern);
|
||||
@@ -214,7 +230,7 @@ describe("shared canonical content-v2 contract", () => {
|
||||
expect(appended.acked[0].seq, `${vector.name}: ${path}`).toBe("1");
|
||||
expect(appended.acked[0].operation_sha256).toBe(paddedWrapper.operation_sha256);
|
||||
|
||||
const stored = await stub.getChanges("first-reader", "0", 500);
|
||||
const stored = changes(await stub.getChanges("first-reader", "0", 500));
|
||||
expect(stored.ops).toHaveLength(1);
|
||||
expect(stored.ops[0].seq).toBe("1");
|
||||
expect(stored.ops[0].body).toBe(paddedWrapper.body);
|
||||
@@ -241,7 +257,7 @@ describe("canonical reducer and entity-head ledger", () => {
|
||||
refused(await stub.pushOps("dev-a", [conflicting]), /revision_hash_conflict/);
|
||||
const second = ok(await stub.pushOps("dev-a", [await observationOp("1", "2")]));
|
||||
refused(await stub.pushOps("dev-a", [firstOp]), /stale_revision/);
|
||||
expect((await stub.getStatus()).head_seq).toBe(second.head_seq);
|
||||
expect(status(await stub.getStatus()).head_seq).toBe(second.head_seq);
|
||||
});
|
||||
|
||||
it("supports tombstone-before-create and a higher-revision revive", async () => {
|
||||
@@ -289,7 +305,7 @@ describe("canonical reducer and entity-head ledger", () => {
|
||||
entity_rev: "18446744073709551615",
|
||||
seq: "9223372036854775809",
|
||||
});
|
||||
const page = await stub.getChanges("dev-reader", "9223372036854775808", 500);
|
||||
const page = changes(await stub.getChanges("dev-reader", "9223372036854775808", 500));
|
||||
expect(page.ops.map((op) => op.seq)).toEqual(["9223372036854775809"]);
|
||||
expect(page.head_seq).toBe("9223372036854775809");
|
||||
});
|
||||
@@ -350,7 +366,7 @@ describe("projection checkpoint, lease fencing, and launch log retention", () =>
|
||||
]);
|
||||
});
|
||||
|
||||
const firstSeen = await stub.getChanges("brand-new-device", "0", 500);
|
||||
const firstSeen = changes(await stub.getChanges("brand-new-device", "0", 500));
|
||||
expect(firstSeen.ops.map((op) => op.seq)).toEqual(["1", "2"]);
|
||||
expect(firstSeen.ops.every((op, index) => BigInt(op.seq) === BigInt(index + 1))).toBe(true);
|
||||
expect(firstSeen.head_seq).toBe("2");
|
||||
@@ -421,7 +437,7 @@ describe("projection checkpoint, lease fencing, and launch log retention", () =>
|
||||
expect(page.ops.length).toBeLessThan(ops.length);
|
||||
const next = ok(await stub.pushOps("dev-a", []));
|
||||
expect(next.head_seq).toBe(pushed.head_seq);
|
||||
const allChanges = await stub.getChanges("boundary-reader", "0", 500);
|
||||
const allChanges = changes(await stub.getChanges("boundary-reader", "0", 500));
|
||||
const nextOp = allChanges.ops[page.ops.length];
|
||||
expect(projectionRequestBytes({
|
||||
userId,
|
||||
@@ -707,7 +723,7 @@ describe("large cursor pagination", () => {
|
||||
let cursor = "0";
|
||||
let count = 0;
|
||||
for (;;) {
|
||||
const page = await stub.getChanges("dev-reader", cursor, 500);
|
||||
const page = changes(await stub.getChanges("dev-reader", cursor, 500));
|
||||
for (const op of page.ops) {
|
||||
expect(BigInt(op.seq)).toBe(BigInt(cursor) + 1n);
|
||||
cursor = op.seq;
|
||||
@@ -817,3 +833,287 @@ describe("front Worker durability and repair", () => {
|
||||
expect(invalid.status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
describe("internal payload-free Hub metadata", () => {
|
||||
const base = "https://sync-hub.test";
|
||||
const internalHeaders = {
|
||||
Authorization: "Bearer test-projector-secret",
|
||||
"Content-Type": "application/json",
|
||||
};
|
||||
|
||||
it("reports device names, last seen, decimal cursors, projection lag, and health", async () => {
|
||||
const userId = "88888888-8888-4888-8888-888888888888";
|
||||
const stub = hub(userId);
|
||||
changes(await stub.getChanges("dev-dashboard", "0", 500));
|
||||
const clientStatus = await SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: {
|
||||
Authorization: `Bearer valid-for:${userId}`,
|
||||
"X-User-Id": userId,
|
||||
"X-Device-Id": "dev-dashboard",
|
||||
"X-Device-Name": " Alex's Laptop ",
|
||||
},
|
||||
});
|
||||
expect(clientStatus.status).toBe(200);
|
||||
|
||||
ok(await stub.pushOps("dev-writer", [
|
||||
await observationOp("1", "1", "dev-writer"),
|
||||
await observationOp("2", "1", "dev-writer"),
|
||||
], "Writer"));
|
||||
await stub.getChanges("dev-reader", "1", 500, "Reader");
|
||||
|
||||
const response = await SELF.fetch(`${base}/internal/v1/sync/metadata`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: userId }),
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
const body = await response.json() as {
|
||||
protocol_version: number;
|
||||
user_id: string;
|
||||
epoch: string;
|
||||
head_seq: string;
|
||||
projected_seq: string;
|
||||
projection_lag_ops: string;
|
||||
sync_health: string;
|
||||
devices: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(body).toMatchObject({
|
||||
protocol_version: 1,
|
||||
user_id: userId,
|
||||
head_seq: "2",
|
||||
projected_seq: "0",
|
||||
projection_lag_ops: "2",
|
||||
sync_health: "projector_lagging",
|
||||
});
|
||||
expect(body.epoch).toMatch(/^(?:0|[1-9][0-9]*)$/);
|
||||
expect(body).not.toHaveProperty("op_count");
|
||||
expect(body.devices).toHaveLength(3);
|
||||
const byId = new Map(body.devices.map((device) => [device.device_id, device]));
|
||||
expect(byId.get("dev-dashboard")).toMatchObject({
|
||||
name: "Alex's Laptop",
|
||||
last_ack_seq: "0",
|
||||
cursor_lag_ops: "2",
|
||||
connection_state: "disconnected",
|
||||
});
|
||||
expect(byId.get("dev-reader")).toMatchObject({
|
||||
name: "Reader",
|
||||
last_ack_seq: "1",
|
||||
cursor_lag_ops: "1",
|
||||
});
|
||||
for (const device of body.devices) {
|
||||
expect(device.last_seen_epoch_ms).toMatch(/^[1-9][0-9]*$/);
|
||||
expect(device.last_seen_at).toMatch(/Z$/);
|
||||
}
|
||||
});
|
||||
|
||||
it("renames only registered devices and preserves dashboard names over client headers", async () => {
|
||||
const userId = "99999999-9999-4999-8999-999999999999";
|
||||
const stub = hub(userId);
|
||||
changes(await stub.getChanges("dev-a", "0", 500, "Initial hostname"));
|
||||
|
||||
const rename = await SELF.fetch(`${base}/internal/v1/sync/device-name`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: userId, device_id: "dev-a", name: "Desk Mac" }),
|
||||
});
|
||||
expect(rename.status).toBe(200);
|
||||
expect(await rename.json()).toEqual({
|
||||
protocol_version: 1,
|
||||
user_id: userId,
|
||||
device_id: "dev-a",
|
||||
name: "Desk Mac",
|
||||
});
|
||||
|
||||
await stub.getStatus("dev-a", "Changed hostname");
|
||||
const metadata = await SELF.fetch(`${base}/internal/v1/sync/metadata`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: userId }),
|
||||
});
|
||||
const body = await metadata.json() as { sync_health: string; projection_lag_ops: string; devices: Array<{ name: string }> };
|
||||
expect(body.sync_health).toBe("healthy");
|
||||
expect(body.projection_lag_ops).toBe("0");
|
||||
expect(body.devices[0].name).toBe("Desk Mac");
|
||||
|
||||
const missing = await SELF.fetch(`${base}/internal/v1/sync/device-name`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: userId, device_id: "missing", name: "Nope" }),
|
||||
});
|
||||
expect(missing.status).toBe(404);
|
||||
});
|
||||
|
||||
it("fails closed on internal auth and rejects contract extensions", async () => {
|
||||
const denied = await SELF.fetch(`${base}/internal/v1/sync/metadata`, {
|
||||
method: "POST",
|
||||
headers: { Authorization: "Bearer wrong", "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: "user" }),
|
||||
});
|
||||
expect(denied.status).toBe(401);
|
||||
|
||||
const extended = await SELF.fetch(`${base}/internal/v1/sync/metadata`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: "user", include_content_counts: true }),
|
||||
});
|
||||
expect(extended.status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
describe("per-user device admission bound", () => {
|
||||
const base = "https://sync-hub.test";
|
||||
const internalHeaders = {
|
||||
Authorization: "Bearer test-projector-secret",
|
||||
"Content-Type": "application/json",
|
||||
};
|
||||
|
||||
function clientHeaders(userId: string, deviceId?: string, deviceName?: string): Record<string, string> {
|
||||
return {
|
||||
Authorization: `Bearer valid-for:${userId}`,
|
||||
"X-User-Id": userId,
|
||||
...(deviceId === undefined ? {} : { "X-Device-Id": deviceId }),
|
||||
...(deviceName === undefined ? {} : { "X-Device-Name": deviceName }),
|
||||
};
|
||||
}
|
||||
|
||||
async function metadata(userId: string): Promise<{
|
||||
devices: Array<{ device_id: string; name: string | null }>;
|
||||
head_seq: string;
|
||||
}> {
|
||||
const response = await SELF.fetch(`${base}/internal/v1/sync/metadata`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({ protocol_version: 1, user_id: userId }),
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
it("direct status refreshes known devices but never admits an unknown probe", async () => {
|
||||
const userId = "device-status-direct-read-only";
|
||||
const stub = hub(userId);
|
||||
|
||||
expect(status(await stub.getStatus("probe-only", "Must Not Persist")).device_count).toBe(0);
|
||||
expect((await metadata(userId)).devices).toEqual([]);
|
||||
|
||||
changes(await stub.getChanges("known-device", "0", 500));
|
||||
expect(status(await stub.getStatus("known-device", "Known Name")).device_count).toBe(1);
|
||||
expect(await metadata(userId)).toMatchObject({
|
||||
devices: [{ device_id: "known-device", name: "Known Name" }],
|
||||
});
|
||||
|
||||
expect(status(await stub.getStatus("another-probe", "Still Not Persisted")).device_count).toBe(1);
|
||||
expect((await metadata(userId)).devices.map((device) => device.device_id)).toEqual(["known-device"]);
|
||||
});
|
||||
|
||||
it("concurrent status probes create zero devices and cannot exhaust admission", async () => {
|
||||
const userId = "device-cap-concurrent";
|
||||
const probes = await Promise.all(Array.from({ length: 80 }, (_, index) => {
|
||||
const device = `device-${String(index).padStart(2, "0")}`;
|
||||
return SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: clientHeaders(userId, device, `Test ${index}`),
|
||||
});
|
||||
}));
|
||||
expect(probes.every((response) => response.status === 200)).toBe(true);
|
||||
expect((await metadata(userId)).devices).toEqual([]);
|
||||
|
||||
const admissions = await Promise.all(Array.from({ length: 80 }, (_, index) => {
|
||||
const device = `admitted-${String(index).padStart(2, "0")}`;
|
||||
return SELF.fetch(`${base}/v1/sync/changes?since=0`, {
|
||||
headers: clientHeaders(userId, device, `Admitted ${index}`),
|
||||
});
|
||||
}));
|
||||
const accepted = admissions.filter((response) => response.status === 200);
|
||||
const rejected = admissions.filter((response) => response.status === 409);
|
||||
expect(accepted).toHaveLength(MAX_DEVICES_PER_USER);
|
||||
expect(rejected).toHaveLength(80 - MAX_DEVICES_PER_USER);
|
||||
for (const response of rejected) expect(await response.json()).toEqual({ error: DEVICE_LIMIT_ERROR });
|
||||
|
||||
const state = await metadata(userId);
|
||||
expect(state.devices).toHaveLength(MAX_DEVICES_PER_USER);
|
||||
expect(new Set(state.devices.map((device) => device.device_id)).size).toBe(MAX_DEVICES_PER_USER);
|
||||
expect(state.devices.every((device) => device.name?.startsWith("Admitted "))).toBe(true);
|
||||
|
||||
const afterCapProbes = await Promise.all(Array.from({ length: 80 }, (_, index) =>
|
||||
SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: clientHeaders(userId, `post-cap-probe-${index}`),
|
||||
})
|
||||
));
|
||||
expect(afterCapProbes.every((response) => response.status === 200)).toBe(true);
|
||||
expect((await metadata(userId)).devices).toHaveLength(MAX_DEVICES_PER_USER);
|
||||
});
|
||||
|
||||
it("keeps existing devices writable/readable while new admitting paths are rejected at the cap", async () => {
|
||||
const userId = "device-cap-http-paths";
|
||||
for (let index = 0; index < MAX_DEVICES_PER_USER; index++) {
|
||||
const response = await SELF.fetch(`${base}/v1/sync/changes?since=0`, {
|
||||
headers: clientHeaders(userId, `device-${index}`, `Named ${index}`),
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
}
|
||||
|
||||
const existingStatus = await SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: clientHeaders(userId, "device-0", "Changed by client"),
|
||||
});
|
||||
expect(existingStatus.status).toBe(200);
|
||||
|
||||
const readOnlyNewStatus = await SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: clientHeaders(userId, "device-new"),
|
||||
});
|
||||
expect(readOnlyNewStatus.status).toBe(200);
|
||||
|
||||
const rejectedPull = await SELF.fetch(`${base}/v1/sync/changes?since=0`, {
|
||||
headers: clientHeaders(userId, "device-pull-new"),
|
||||
});
|
||||
expect(rejectedPull.status).toBe(409);
|
||||
expect(await rejectedPull.json()).toEqual({ error: DEVICE_LIMIT_ERROR });
|
||||
|
||||
const rejectedPush = await SELF.fetch(`${base}/v1/sync/ops`, {
|
||||
method: "POST",
|
||||
headers: { ...clientHeaders(userId, "device-push-new"), "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
protocol_version: 2,
|
||||
ops: [await observationOp("1", "1", "device-push-new")],
|
||||
}),
|
||||
});
|
||||
expect(rejectedPush.status).toBe(409);
|
||||
expect(await rejectedPush.json()).toEqual({ error: DEVICE_LIMIT_ERROR });
|
||||
|
||||
const acceptedPush = await SELF.fetch(`${base}/v1/sync/ops`, {
|
||||
method: "POST",
|
||||
headers: { ...clientHeaders(userId, "device-0"), "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
protocol_version: 2,
|
||||
ops: [await observationOp("1", "1", "device-0")],
|
||||
}),
|
||||
});
|
||||
expect(acceptedPush.status).toBe(200);
|
||||
const acceptedPull = await SELF.fetch(`${base}/v1/sync/changes?since=0`, {
|
||||
headers: clientHeaders(userId, "device-1"),
|
||||
});
|
||||
expect(acceptedPull.status).toBe(200);
|
||||
expect((await acceptedPull.json() as { ops: unknown[] }).ops).toHaveLength(1);
|
||||
|
||||
const readOnlyStatus = await SELF.fetch(`${base}/v1/sync/status`, {
|
||||
headers: clientHeaders(userId),
|
||||
});
|
||||
expect(readOnlyStatus.status).toBe(200);
|
||||
const unknownRename = await SELF.fetch(`${base}/internal/v1/sync/device-name`, {
|
||||
method: "POST",
|
||||
headers: internalHeaders,
|
||||
body: JSON.stringify({
|
||||
protocol_version: 1,
|
||||
user_id: userId,
|
||||
device_id: "unknown-rename",
|
||||
name: "Must Not Exist",
|
||||
}),
|
||||
});
|
||||
expect(unknownRename.status).toBe(404);
|
||||
|
||||
const state = await metadata(userId);
|
||||
expect(state.head_seq).toBe("1");
|
||||
expect(state.devices).toHaveLength(MAX_DEVICES_PER_USER);
|
||||
expect(state.devices.find((device) => device.device_id === "device-0")?.name).toBe("Named 0");
|
||||
expect(state.devices.some((device) => device.device_id === "unknown-rename")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
import { env, evictDurableObject, SELF } from "cloudflare:test";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { PushOutcome, PushResult } from "../src/do/SyncHub";
|
||||
import {
|
||||
DEVICE_LIMIT_ERROR,
|
||||
MAX_DEVICES_PER_USER,
|
||||
type ChangesOutcome,
|
||||
type ChangesResult,
|
||||
type PushOutcome,
|
||||
type PushResult,
|
||||
} from "../src/do/SyncHub";
|
||||
import { KILL_SWITCH_KEY } from "../src/kill-switch";
|
||||
import { observationOp } from "./content-v2-helpers";
|
||||
|
||||
@@ -36,6 +43,11 @@ function ok(outcome: PushOutcome): PushResult {
|
||||
return outcome;
|
||||
}
|
||||
|
||||
function changes(outcome: ChangesOutcome): ChangesResult {
|
||||
if ("refused" in outcome) throw new Error(`unexpected refusal: ${outcome.error}`);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
interface Connected {
|
||||
ws: WebSocket;
|
||||
messages: string[];
|
||||
@@ -80,6 +92,24 @@ describe("upgrade and auth", () => {
|
||||
const notUpgrade = await SELF.fetch(`${base}/v1/sync/ws`, { headers: headers("user-ws-426", "dev-a") });
|
||||
expect(notUpgrade.status).toBe(426);
|
||||
});
|
||||
|
||||
it("rejects a 65th WebSocket device but still upgrades an existing device", async () => {
|
||||
const user = "user-ws-device-cap";
|
||||
for (let index = 0; index < MAX_DEVICES_PER_USER; index++) {
|
||||
const response = await SELF.fetch(`${base}/v1/sync/changes?since=0`, {
|
||||
headers: headers(user, `dev-${index}`),
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
}
|
||||
const rejected = await SELF.fetch(`${base}/v1/sync/ws`, {
|
||||
headers: { ...headers(user, "dev-new"), Upgrade: "websocket" },
|
||||
});
|
||||
expect(rejected.status).toBe(409);
|
||||
expect(await rejected.json()).toEqual({ error: DEVICE_LIMIT_ERROR });
|
||||
|
||||
const existing = await connect(user, "dev-0");
|
||||
existing.ws.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("canonical advisory fan-out", () => {
|
||||
@@ -154,7 +184,7 @@ describe("canonical advisory fan-out", () => {
|
||||
replica.ws.close();
|
||||
const result = ok(await stub.pushOps("dev-a", [await observationOp("2")]));
|
||||
expect(result.acked).toHaveLength(1);
|
||||
expect((await stub.getChanges("dev-c", "0", 500)).ops).toHaveLength(2);
|
||||
expect(changes(await stub.getChanges("dev-c", "0", 500)).ops).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user