mirror of
https://github.com/thedotmack/claude-mem.git
synced 2026-09-20 04:23:02 +08:00
Alex Newman
d861f9d0e5
Lesson of the Jul 20-22 outage: Supabase paused the project silently and
signups failed for 52 hours with no page, while the static landing page
answered 200 throughout. This adds an external probe that lives in the
sync-hub Worker (outside the Vercel/Supabase failure domain) and checks a
DB-BACKED endpoint, never the landing page.
- Second cron "*/5 * * * *" in wrangler.jsonc; src/index.ts's scheduled
handler dispatches on event.cron ("7 * * * *" keeps reaching the
watchdog; the probe branch is wrapped in the one permitted top-level
try/catch so a probe bug can never escape the handler).
- src/control-plane-probe.ts: GET TOKEN_VERIFY_URL with a deliberately
bogus bearer token + all-zero user id, 10 s hard deadline. HEALTHY iff
the answer is 401/403 with a JSON body -- exactly the response that
requires the Pro app AND its Postgres pro_users lookup to be alive.
Unhealthy: network error, timeout, 5xx, non-JSON 401 (edge page for a
dead app), or any other status. A 2xx for the bogus token pages a
distinct SECURITY alert (auth bypass, not an outage).
- Anti-flap KV state in AUTH_CACHE under "control:uptime-probe": first
failure alerts only after one immediate in-run retry confirms it;
re-page at most every 30 min while failing; one green embed on
recovery; healthy steady state writes nothing to KV and posts nothing.
Operator silence marker {"silenced_until": ISO} documented in
DEPLOY.md section 7.
- src/discord.ts: shared embed-posting helper extracted from the
watchdog's inline code; both monitors now use it (identical wire
shape, no behavior change).
- No new secrets (reuses DISCORD_WEBHOOK_URL + TOKEN_VERIFY_URL), no
Durable Object involvement; Discord failures are swallowed with a log
and never block the KV state write.
- test/control-plane-probe.test.ts: 20 tests covering classification,
confirm-retry, anti-flap suppression/re-alert/recovery, security 2xx,
Discord-failure containment, maintenance silence, and cron dispatch
in both directions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JS4kwJj5GkVFEhe1X7XNox