skills/proffesor-for-testing__agentic-qe
1
0
Fork 0
mirror of https://github.com/proffesor-for-testing/agentic-qe.git synced 2026-09-19 08:45:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

OpenHuman QE analysis

Browse Source
This commit is contained in:
Dragan Spiridonov
2026-05-21 06:57:48 +00:00
parent 2c84e65465
commit c81078b7b7
8 changed files with 2291 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/qe-reports/openhuman/00-executive-summary.md
+128
View File
@@ -0,0 +1,128 @@
# OpenHuman — QE Executive Summary
**Subject**: `tinyhumansai/openhuman` (Early Beta)
**Snapshot**: 2026-05-20
**Method**: Seven specialist QE agents ran in parallel against `/tmp/openhuman` (cloned from `main`). Each produced an evidence-based report with file:line citations. This document is the cross-cutting synthesis.
**Source clone**: `/tmp/openhuman` — repo hash from cloned `main`.
---
## TL;DR
| Dimension | Score (1=worst, 10=best) | One-line verdict |
|---|---|---|
| **Code Quality** | 6/10 | Concentrated, mechanically fixable debt — five god-files do most of the damage |
| **Security** | 7.5/10 | Strong defenses overall; two one-line HIGH issues in the cloud-deploy path |
| **Performance** | 5/10 | Good hygiene in places; five systemic patterns that will hurt under real load |
| **QX (Quality Experience)** | 6.5/10 | Distinctive strengths in trust + boot UX; weakness in i18n delivery vs ambition |
| **Complexity** | 6/10 | Three named refactors would drop it to ~4/10 |
| **Dependencies** | 6/10 | Competent baseline; reproducibility, layering, and one license item to fix |
| **SFDIPOT (test surface)** | — | Test investment is happy-path-heavy; adversarial/concurrency/resource-pressure gaps |
**Composite QE verdict**: `6.2 / 10` — credible early-beta posture for a project of this surface area (a Tauri desktop AI assistant with 118 OAuth connectors, a wallet, embedded CEF, in-process skills sandbox, MCP client+server, and auto-update for both shell and core). The product **is not** in trouble, but it carries five named risks that will become release-blocking if not paid down before GA.
---
## Top 10 Cross-Cutting Risks (ranked by blast radius × likelihood)
| # | Risk | Severity | Where | Effort to Fix |
|---|------|---------|-------|---------------|
| 1 | **RPC mutations default-on** — `rpc_mutations_enabled = true`, makes `update.run` / `update.apply` callable by any bearer-authed client. On Docker/cloud `0.0.0.0` deployments this is a credible auth'd downgrade/force-restart vector. | **HIGH** | `src/openhuman/config/schema/update.rs:53-55` | One line |
| 2 | **Variable-time token comparison** — bearer-token compared with `==` on `&str`. Source code literally comments that the fix is "one-line." | **HIGH** | `src/core/auth.rs:210-212` | One line |
| 3 | **Mutex poison bricks IPC** — `std::sync::Mutex` with 17 production `.lock().unwrap()` in the most callback-heavy Tauri host file. A single panic in any DOM-driven callback poisons the mutex → all subsequent IPC fails. `parking_lot::Mutex` is already a dependency. | **HIGH** | `app/src-tauri/src/webview_accounts/mod.rs:23` (+17 call sites) | Hours — swap import |
| 4 | **CEF `--remote-debugging-port=9222`** exposes the embedded Chromium to any local process via CDP, including OAuth-webview cookies and privileged-frame JS. | **HIGH** | tauri.conf / vendored Tauri-CEF fork | Days — gate behind dev flag |
| 5 | **Prompt-injection surface vs detector** — 118 connectors funnel untrusted text into one detector module; very few negative tests in repo. | **HIGH** | (system-level) | Weeks — adversarial corpus + property tests |
| 6 | **Skills supply chain unsigned** — `SKILLS_REGISTRY_URL` accepts arbitrary HTTP URLs, `SKILLS_LOCAL_DIR` any local path, no signing observed. Skills run in-process with the wallet and 118 OAuth tokens. | **HIGH** | env config + skills loader | Weeks — sigstore/minisign + provenance |
| 7 | **Five god-files / god-functions** — `webview_account_open` 840 LOC, `Agent::turn` 967 LOC, `Conversations.tsx` 2,125 LOC + 39 hooks, `AIPanel.tsx` 2,696 LOC, `apply_env_overrides_from` 720 LOC. Single-render crash in `Conversations.tsx` blanks the whole app (only one root `Sentry.ErrorBoundary`). | **MED** | listed in report 01/06 | 1–2 weeks per file |
| 8 | **Performance landmines under load** — WhatsApp bulk ingest is per-row INSERT with no transaction/`prepare_cached` + correlated COUNT/MAX subqueries; `Conversations.tsx` recomputes derived state every render with only one `React.memo` in the entire codebase; audio transcription does `Array.from(Uint8Array(blob))` IPC (~8× payload bloat); `sync_vault` blocking I/O in async; no code-splitting (`three` ~600 KB in main bundle). | **MED** | see report 03 | 2–4 days |
| 9 | **Non-reproducible Rust build** — `whisper-rs-sys` is git-pinned to `branch = "main"` with **no rev** in both `Cargo.toml` files. Plus two independent `Cargo.lock` files (no workspace), and two pnpm lockfiles that disagree (`vite@8.0.10` vs `vite@7.3.2`). | **MED** | `Cargo.toml`, `app/src-tauri/Cargo.toml`, lockfiles | Days — unify workspace + pin revs |
| 10 | **No backup or migration rollback for `~/.openhuman`** — single SQLite holds the irreplaceable Memory Tree; no rollback path on failed migration; backfill binaries can mutate the same DB concurrently with the live desktop. | **MED** | (system-level) | Weeks — backup + migration safety harness |
---
## Where the Product is Strong (do not undo these)
These came up across multiple reports and represent positive signals worth preserving as you grow:
- **Trust posture, best-in-class.** Default-deny consent gate, PII-stripped Sentry, GA event allowlist, in-product `WhatLeavesLink` disclosure — `features/privacy/whatLeavesItems.ts:11-31`, `services/analytics.ts:5-26`. (Caveat: `OPENHUMAN_ANALYTICS_ENABLED=true` default contradicts the marketing — flip it.)
- **Boot UX & crash recovery.** `PersistRehydrationScreen.tsx:14,47-50` has a 10s deadline with a recovery CTA; `ErrorFallbackScreen.tsx` is self-contained with three real recovery actions.
- **SSRF guard with DNS-rebinding protection** — `src/openhuman/tools/impl/network/url_guard.rs`. Real defense, not theater.
- **Docker hardening** — `read_only`, `cap_drop: ALL`, `no-new-privileges` in compose.
- **Tauri capability scoping** — caller-label verification on `webview_recipe_event` is correctly done.
- **Real test density** — ~86 k LOC of tests, ~20 Rust integration + ~55 WDIO E2E specs. Adversarial gap, but the baseline exists.
- **A11y discipline** — zero `<div onClick>` across 382 `.tsx` files; real `useT()` adoption at most points that matter.
---
## Quick Wins (≤ 1 day each, high yield)
Pick these up before anything bigger.
1. **Fix SEC-02**: replace `==` with `subtle::ConstantTimeEq` in `src/core/auth.rs:210-212`. (1 line)
2. **Fix SEC-01**: flip `rpc_mutations_enabled` default to `false`; opt-in via env. `src/openhuman/config/schema/update.rs:53-55`. (1 line + docs)
3. **Replace `std::sync::Mutex` with `parking_lot::Mutex`** in `webview_accounts/mod.rs` — eliminates the panic-poison-bricks-IPC class entirely. `parking_lot` is already a dep.
4. **Flip `OPENHUMAN_ANALYTICS_ENABLED` default to `false`** in `.env.example` to match privacy marketing.
5. **Add `role="alert"`** to the chat send-error banner — `app/src/pages/Conversations.tsx:1959-1963` — and `aria-invalid` propagation in `components/ui/Input.tsx:20-23`. (~5 lines total)
6. **Stop the 25 MB IPC JSON transcode** — switch `Conversations.tsx:823` to the existing file-path audio API. (~10 lines)
7. **Pin `whisper-rs-sys`** to a specific git `rev` in both `Cargo.toml` files. (2 lines)
8. **Drop module-wide `#![allow(dead_code)]`** in `src/openhuman/mod.rs:16` — let the compiler tell you what's actually dead.
9. **Unblock the `pr-quality.yml` CI gate** — remove `continue-on-error: true` from at least the lint/typecheck/test jobs. They are currently advisory-only.
10. **Wrap the 5 most-likely-to-crash UI subtrees in their own `ErrorBoundary`** (Conversations thread list, composer, streaming pane, agent-profile editor, voice). Right now one Sentry boundary at the root means a Conversations render error blanks the entire app.
---
## Recommended 30 / 60 / 90 Day Plan
**30 days — stop the bleeding**
- Land all 10 quick wins above.
- Build an **adversarial corpus** for the prompt-injection detector (target ≥ 500 cases across 118 connectors); wire it as a CI gate.
- Add `cargo audit` to CI (the security scanner could not run it locally; it must run in your pipeline).
- Gate CEF remote-debugging behind a build flag — off in release.
- Make `OPENHUMAN_CORE_TOKEN` required at startup; refuse to listen on `0.0.0.0` without one. Fail closed.
**60 days — pay down the structural debt**
- Refactor `Conversations.tsx` (2,125 LOC, 39 hooks) into thread-list / composer / streaming / agent-profile / voice subtrees, each with its own `ErrorBoundary` and `React.memo` discipline.
- Refactor `Agent::turn` (967 LOC) by extracting the four obvious sub-state-machines.
- Decompose `webview_account_open` (840 LOC) and `apply_env_overrides_from` (720 LOC).
- Replace `Result<_, String>` with typed error enums in `core/observability.rs` and the 1,062 sites that follow.
- Migrate off `ethers-rs` (deprecated upstream) to `alloy`.
- Convert the repo to a real Cargo workspace; unify the two `Cargo.lock`s; resolve the `vite` lockfile disagreement.
**90 days — durable resilience**
- Backup + migration safety harness for `~/.openhuman` (single SQLite holding the Memory Tree); reject concurrent backfill-binary writes when the desktop has the DB open.
- Sign skills (sigstore/minisign + provenance); allowlist `SKILLS_REGISTRY_URL` schemes.
- Bring `prepare_cached` + transactional batching to all ingest paths (WhatsApp ingest is the worst, but it's a pattern).
- Add a **real-VM install test** to CI for both Linux and macOS — current `installer-smoke.yml` is `--dry-run` only.
- Adversarial / concurrency / resource-pressure tests for the JSON-RPC `/rpc` endpoint and the QuickJS skills sandbox.
- Verify Remotion license posture (`remotion/package.json` declares `UNLICENSED`; Remotion's commercial tier triggers above 3 staff).
---
## What We Did Not Cover (recommended follow-up)
- **`cargo audit`** — could not run in the agent environment; needs to run in your CI.
- **Mutation testing** — none performed; would clarify whether the 86 k LOC of tests actually exercise behavior or just types.
- **License audit at the transitive level** — name-heuristic scan only; no SBOM-grade verification (GTK/cairo/glib LGPL are fine if dynamically linked, but worth confirming for OpenSSL on the Tauri build).
- **Threat-model walkthrough** — the static security audit is necessary but not sufficient for a desktop AI app with this surface; a structured STRIDE/LINDDUN session on the IPC + skills + wallet boundary is the highest-leverage next QE activity.
---
## Detailed Reports
| # | File | Owner Agent | Key Finding |
|---|------|-------------|-------------|
| 01 | [`01-code-quality.md`](./01-code-quality.md) | qe-code-reviewer | Mutex-poison bricks IPC; five god-files concentrate the debt |
| 02 | [`02-security.md`](./02-security.md) | qe-security-scanner | 0 CRIT / 2 HIGH / 5 MED / 4 LOW / 5 INFO; strong baseline |
| 03 | [`03-performance.md`](./03-performance.md) | qe-performance-reviewer | Five systemic patterns; ~2 engineer-days for the top 10 |
| 04 | [`04-qx-experience.md`](./04-qx-experience.md) | qe-qx-partner | Best-in-class trust posture; i18n delivery mismatch |
| 05 | [`05-product-factors-sfdipot.md`](./05-product-factors-sfdipot.md) | qe-product-factors-assessor | Test investment is happy-path-heavy; adversarial gap dominates |
| 06 | [`06-complexity-hotspots.md`](./06-complexity-hotspots.md) | qe-code-complexity | 483 k production LOC; 3 named refactors → 4/10 |
| 07 | [`07-dependencies.md`](./07-dependencies.md) | qe-dependency-mapper | Non-reproducible Rust build; lockfile disagreements; ethers deprecated |
---
## Closing Note
OpenHuman is ambitious — a private, local-first, personal AI assistant with the surface area of a small OS. The QE evidence suggests a team that **knows what it's doing** (trust posture, SSRF guard, Tauri capability scoping, real test density) but is at the point in scale where the structural debt and a few sharp-edged defaults are starting to bite faster than they can be paid down by hand.
The top 10 quick wins are all under a day each. The 30-day plan is achievable by a small team. After that, the bigger refactors (`Conversations.tsx`, `Agent::turn`, repo workspace unification, adversarial test corpus) are the ones that determine whether OpenHuman holds together through GA.
docs/qe-reports/openhuman/01-code-quality.md
+252
View File
@@ -0,0 +1,252 @@
# OpenHuman — Code Quality & Code Smell Review
**Reviewer:** AQE v3 Code Reviewer (sampled, evidence-based)
**Scope sampled:** 25 files across `src/` (Rust core), `app/src-tauri/src/` (Tauri host), `app/src/` (React/TS)
**Method:** size-ranked sweeps + grep for smell indicators + targeted line reads. Not exhaustive.
---
## Executive Summary
Severity-ranked (Critical → Info):
1. **[HIGH] One Tauri host file is a 4,450-line god module.** `app/src-tauri/src/webview_accounts/mod.rs` owns process state, IPC commands, CDP wiring, notification routing, and webview lifecycle in a single file. The single function `webview_account_open` is **840 lines** (line 13033). This blocks safe parallel work, hides invariants, and is the single largest maintainability risk in the codebase.
2. **[HIGH] `std::sync::Mutex` + `.lock().unwrap()` pattern in the Tauri host.** `webview_accounts/mod.rs` uses `use std::sync::Mutex;` (line 23) and contains **17 production-path `.lock().unwrap()` calls** (e.g., lines 1119, 1636, 1845, 2655, 2669, 2677, 3040, 3054, 3073, 3083). A single panic while a lock is held permanently poisons it — every subsequent IPC command from React will then crash the host. `parking_lot::Mutex` (which doesn't poison) is **already a direct dep** (`Cargo.toml`) and is the obvious fix.
3. **[HIGH] The agent turn loop is a 967-line god-method.** `src/openhuman/agent/harness/session/turn.rs:71` — `Agent::turn` runs from line 71 to ~1037. The same file holds three other 250+-line methods (`inject_agent_experience_context` 345 lines L1038, `emit_progress` 265 lines L1383, `build_system_prompt` 308 lines L1838). The orchestrator-of-everything pattern makes it hard to test branches in isolation and concentrates ownership of unrelated concerns (KV-cache prefix policy, prompt construction, integration hydration, memory injection).
4. **[HIGH] React page-level components carry too much state.** `app/src/pages/Conversations.tsx` is a single 1,919-line component (L206 → L2124) with **29 hook calls** inside one function body. `app/src/components/settings/panels/AIPanel.tsx` is 2,696 lines containing a 690-line `BackgroundLoopControls` (L2948) and a 521-line `AIPanel` (L4109). These will not survive their next major refactor without regressions.
5. **[MEDIUM] Env-driven config schema loader is a 720-line method.** `src/openhuman/config/schema/load.rs:5732` — `apply_env_overrides_from` is 720 lines in one function. Any new env var lands in a fan-out of `if let Some(...)` arms with no test partitioning.
6. **[MEDIUM] Inconsistent error-type strategy.** 392 files use `anyhow`, only 8 use `thiserror`, and **1,062 functions return `Result<..., String>`** (greppable in `src/`). Stringly-typed errors propagate context loss and force callers into substring sniffing — this is exactly what `src/core/observability.rs::expected_error_kind` (L137) and `is_session_expired_message` (L227) are doing for ~700 lines: classifying *strings* into error kinds. This is the long-tail cost of stringly-typed errors made concrete.
7. **[MEDIUM] Pervasive `#![allow(dead_code)]` suppression.** `src/openhuman/mod.rs:16` blankets the entire `openhuman` module subtree with `#![allow(dead_code)]`. 37 occurrences of `#[allow(dead_code)]` and 110 total `#[allow(...)]` attributes across `src/` mean the compiler's own dead-code detector is silenced — actual dead code is invisible.
8. **[MEDIUM] Comment-driven feature toggles via `[#1123]`.** `app/src/App.tsx` has 7 commented-out blocks tagged `[#1123]` (L23, 48, 50, 125, 161, 194, 215) and the same pattern in `Conversations.tsx` (L14, 21, 23) and `pages/Accounts.tsx`. Dead code preserved as comments is harder to keep current than removing it and pulling from git history.
9. **[LOW] Hardcoded transport endpoints.** `src/openhuman/tools/impl/browser/types.rs:33` hardcodes `http://127.0.0.1:8787/v1/actions` as a default. `src/openhuman/webview_apis/client.rs:142` constructs `ws://127.0.0.1:{port}/` from an env var with no fallback validation. Both are leaky abstractions for tests/dev that have escaped into production defaults.
10. **[LOW] Documentation outweighs code in places.** `AGENTS.md` is 649 lines; `CLAUDE.md` is 311. Some module docs (`src/openhuman/mod.rs`, `app/src-tauri/src/webview_accounts/mod.rs`) are genuinely good; others (e.g., `src/openhuman/agent/harness/session/turn.rs:51-68`) read like docstrings written to compensate for function size rather than to document an interface.
**Top-level finding:** the codebase is *not* sloppy — naming is good, error handling is principled in most modules, doc-comments are present, and the test scaffolding is substantial (~85k LOC of test code vs ~360k LOC of prod Rust = ~24% test-LOC ratio, healthy for a Rust codebase). The structural problems are concentrated in a small number of hotspot files that have grown organically into god-objects.
---
## Quantitative Metrics
| Metric | Value | Note |
|---|---|---|
| Rust files (prod + test) | 1,323 | under `src/`, `packages/`, `app/src-tauri/` |
| TS/TSX files (prod + test) | 895 | under `app/src/`, `packages/` |
| Rust prod LOC | ~360k | excludes `*_test*.rs` and `tests.rs` |
| Rust test LOC | ~86k | `*_test*.rs` + `tests.rs` files |
| TS prod LOC | ~110k | excludes `__tests__/` and `*.test.*` |
| Workspace structure | **single crate** | no `[workspace]` in root `Cargo.toml`; 74 `pub mod` lines in one `openhuman/mod.rs` |
| Direct Rust deps | 144 | top-level entries in `Cargo.toml` |
| `Result<_, String>` signatures | 1,062 | non-test Rust |
| `anyhow::Result` users | 392 files | predominant |
| `thiserror` users | 8 files | the typed-error story is barely adopted |
| `.unwrap()` total | 3,009 | mostly inside `#[cfg(test)]` blocks (good) |
| `.unwrap()` in prod paths | ~120–150 | concentrated in webview_accounts (49) + a few benchmarks/schemas |
| `.expect(` total | 730 | most are fixture/test code |
| `panic!(` calls | 75 | none in critical hot paths sampled |
| `let _ = ` (discarded result) | 640 | many are fire-and-forget event emits — semi-legitimate |
| TS `as any` | 21 | mostly in `app/src/polyfills.ts` for global injection |
| TS `@ts-ignore` | 4 | all in test files |
| TS `console.*` in prod | 195 | predominantly `console.warn/error` — acceptable |
| TODO/FIXME/HACK | 20 | low density, mostly with linked issue numbers |
### Largest Rust production files (top 10)
```
4450 app/src-tauri/src/webview_accounts/mod.rs
3846 app/src-tauri/src/lib.rs
2702 src/core/observability.rs
2345 src/openhuman/memory/tree/read_rpc.rs
2181 src/openhuman/agent/harness/session/turn.rs
2093 app/src-tauri/src/whatsapp_scanner/mod.rs
2013 src/openhuman/config/schema/load.rs
2005 src/openhuman/inference/provider/compatible.rs
1889 src/openhuman/composio/ops.rs
1792 src/openhuman/channels/providers/web.rs
```
### Largest TS/TSX production files (top 10)
```
2696 app/src/components/settings/panels/AIPanel.tsx
2261 app/src/lib/i18n/en.ts (translation strings — legitimate)
2125 app/src/pages/Conversations.tsx
2111 app/src/lib/i18n/ko.ts (translation strings — legitimate)
1489 app/src/services/webviewAccountService.ts
997 app/src/pages/Skills.tsx
923 app/src/providers/ChatRuntimeProvider.tsx
920 app/src/components/composio/ComposioConnectModal.tsx
906 app/src/features/human/Mascot/yellow/MascotCharacter.tsx
857 app/src/components/settings/panels/VoicePanel.tsx
```
---
## Top 10 Specific Issues (file:line citations)
### 1. `webview_accounts/mod.rs` is a 4,450-line god module
`app/src-tauri/src/webview_accounts/mod.rs:1-3239` (prod section) + tests below.
The module docstring at lines 1-19 honestly admits the scope: "Hosts third-party web apps … recipe injection … per-account session isolation … notification bypass." That's at least five bounded contexts in one file:
- Provider URL registry (`provider_url` L48)
- CDP browser session management
- Notification forwarding (`forward_native_notification` L1112, 329 lines)
- Webview lifecycle commands (`webview_account_open` L13033, **840 lines**)
- DND / mute / focus preference state
The single function `webview_account_open` at 840 lines is the centerpiece — it spawns child webviews, injects scripts, registers CDP sessions, wires notification handlers, and stores state in three different `Mutex<HashMap<...>>` fields. This cannot be unit-tested.
**Fix:** Split into `mod.rs` (re-exports + state struct), `commands.rs` (Tauri `#[command]` entries), `lifecycle.rs` (open/close/prewarm), `notifications.rs` (forward + bypass prefs), `recipes.rs` (script injection). Keep the state struct as the only cross-cutting type.
---
### 2. `std::sync::Mutex` + `.lock().unwrap()` panic-amplifier in Tauri host
`app/src-tauri/src/webview_accounts/mod.rs:23` declares `use std::sync::Mutex;`. The production section (lines 1-3239, before `#[cfg(test)]`) contains **17 calls to `.lock().unwrap()`**:
```
1119: state.notification_bypass.lock().unwrap().clone();
1636: app_state.inner.lock().unwrap().get(account_id).cloned();
1845: state.inner.lock().unwrap();
2655: state.inner.lock().unwrap().remove(&args.account_id);
2669: state.browser_ids.lock().unwrap().remove(&args.account_id);
2677: state.cdp_sessions.lock().unwrap().remove(&args.account_id);
2745, 2755, 2763, 2896, 2937, 2993, 3010,
3040, 3054, 3073, 3083 (notification bypass prefs)
```
Pattern recurs in `app/src-tauri/src/lib.rs` (13 calls), `screen_capture/mod.rs` (6), `meet_call/mod.rs` (6).
**Risk:** If any closure invoked while holding `state.inner` panics — e.g., a logging macro chokes on a malformed UTF-8 payload from `webview_recipe_event` (which is fed by untrusted DOM content from third-party sites like Slack/Discord/WhatsApp) — the mutex is poisoned. Every subsequent IPC command that tries to lock it will panic at the `.unwrap()`. Recovery requires restarting the app process.
**Fix:** Replace `use std::sync::Mutex;` with `use parking_lot::Mutex;` (already in `Cargo.toml`). `parking_lot::Mutex::lock()` returns the guard directly — no `unwrap()` needed, no poisoning. This is a mechanical change with a high blast-radius improvement.
---
### 3. `Agent::turn` is a 967-line method
`src/openhuman/agent/harness/session/turn.rs:71` — `pub async fn turn(&mut self, user_message: &str) -> Result<String>` runs to ~line 1037.
Same file:
- L1038: `inject_agent_experience_context` — 345 lines
- L1383: `emit_progress` — 265 lines
- L1838: `build_system_prompt` — 308 lines
The docstring at L51-68 lists six numbered responsibilities (initialization, prompt construction, context injection, execution loop, synthesis, background tasks). That's a Single Responsibility Principle violation enumerated in the comment.
**Fix:** Extract each numbered step into a private method on `Agent` (e.g., `fn maybe_resume_transcript`, `fn maybe_bake_system_prompt`, `async fn execution_loop`). Each step is independently testable — and a 9-step state machine surfaces naturally.
---
### 4. `apply_env_overrides_from` is 720 lines in `config/schema/load.rs:5732`
`src/openhuman/config/schema/load.rs:5732` — `fn apply_env_overrides_from(&mut self, env: &(dyn EnvLookup + Send + Sync))` is **720 lines** long. With 144 direct Cargo deps and a sprawling settings surface, the codebase has a lot of env vars to handle — but doing them all in one method means no test partitioning, no documentation per group, and merge conflicts whenever two PRs add env vars.
**Fix:** Group env vars by domain (e.g., `apply_inference_env`, `apply_observability_env`, `apply_voice_env`) and have the top-level method delegate. Each sub-method can have a corresponding `#[test]` that pokes specific keys.
---
### 5. `pages/Conversations.tsx` is a 1,919-line single component with 29 hooks
`app/src/pages/Conversations.tsx:206` defines `const Conversations = ({...}) => {` and the body extends to L2124. Inside that single function:
- 29 hook calls (`useState`, `useEffect`, `useMemo`, `useCallback`, `useRef`) — counted via grep within the component body
- L2125: `export const AgentChatPanel = () => <Conversations variant="page" />;` exists *only* to expose a variant alias
When this component has 29 stateful concerns mixed with chat send/receive, MIC composer, autocomplete polling (`AUTOCOMPLETE_POLL_DEBOUNCE_MS` L100), thread management, agent profile editing, prompt-injection guard, voice STT, and Redux dispatch — every renderer rerun touches all of them. Memoization at the top level is masked.
**Fix:** Pull subsystems into `useConversationsChatState`, `useConversationsAutocomplete`, `useConversationsVoice` hooks under `app/src/pages/conversations/hooks/`. The shell component becomes a layout coordinator.
---
### 6. `AIPanel.tsx` packs 4 distinct components into 2,696 lines
`app/src/components/settings/panels/AIPanel.tsx`:
- L2650: `ProviderKeyDialog` — 127 lines
- L2948: `BackgroundLoopControls` — **690 lines**
- L3727: `CustomRoutingDialog` — 326 lines
- L4109: `AIPanel` — 521 lines
Plus two large hooks (`useAISettings` L272, `useOllamaStatus` L387, `useInstalledModels` L436) that should be in `hooks/` rather than embedded in the panel.
**Fix:** Split into a directory:
```
panels/ai/
AIPanel.tsx
ProviderKeyDialog.tsx
BackgroundLoopControls.tsx
CustomRoutingDialog.tsx
hooks/useAISettings.ts
hooks/useOllamaStatus.ts
hooks/useInstalledModels.ts
```
---
### 7. Stringly-typed errors infect the entire core
1,062 functions return `Result<..., String>` in non-test Rust code under `src/`. The cost is concretely visible:
`src/core/observability.rs:137-516` — there are **8 different "is_xxx_message"** classifier functions:
- `expected_error_kind` L137
- `is_session_expired_message` L227
- `is_loopback_unavailable` L271
- `is_network_unreachable_message` L299
- `is_transient_upstream_http_message` L347
- `is_backend_user_error_message` L384
- `is_provider_user_state_message` L420
- `is_local_ai_capability_unavailable_message` L516
Every one of them does substring matching on error *strings*. This is the dual of the upstream choice to throw away typed errors. The same substring-matching pattern appears in `is_transient_provider_http_failure` (L812) and `is_max_iterations_event` (L845) but reading Sentry events.
**Fix:** Introduce `thiserror` enums at the integration boundaries (provider errors, channel errors, memory errors, tool errors). The classifier file then matches on enum variants, not strings, and the matchers can never silently drift when a vendor changes a message.
---
### 8. Module-wide `#![allow(dead_code)]` defeats the compiler
`src/openhuman/mod.rs:16` — `#![allow(dead_code)]` at module level disables dead-code warnings for the entire 75-module subtree (74 `pub mod` lines in the same file). The justification comment ("Many types/functions are intended for future use or integration with the frontend") is the canonical anti-pattern: the compiler is the only tool that reliably detects truly-unreachable code, and you have turned it off everywhere.
37 separate `#[allow(dead_code)]` and 110 `#[allow(...)]` attributes underneath compound the issue — they're locally justified, but the umbrella `#![allow(dead_code)]` makes them redundant *and* prevents anyone from cleaning them up without a flag day.
**Fix:** Remove the module-level allow. Let the compiler emit warnings. Either delete what is unused, mark individual items `#[allow(dead_code)]` with a comment, or keep public exports referenced by a doctest. Yes, this is a flag-day cleanup, but the long-term cost of not having dead-code signal is steeper.
---
### 9. Commented-out code as feature flags (`[#1123]` markers)
`app/src/App.tsx` lines 23, 48, 50, 125, 161, 194, 215 — seven commented-out blocks tagged `[#1123]`. Same pattern in `app/src/pages/Conversations.tsx` L14, L21, L23, and `app/src/pages/Accounts.tsx` L8, L10, L111+.
The comment-tag is well-meant (each one references the issue removing welcome-agent onboarding), but git history already records that. Long-lived comments rot faster than code because nobody runs `cargo check` against them.
**Fix:** Delete the commented blocks. If a rollback is needed, `git revert` is one command.
---
### 10. Hardcoded loopback URLs/ports in production defaults
- `src/openhuman/tools/impl/browser/types.rs:33` — `endpoint: "http://127.0.0.1:8787/v1/actions".into()`. A hardcoded loopback URL with a magic port as a struct default. If two browser tools collide on 8787, the second one silently 502s.
- `src/openhuman/webview_apis/client.rs:142` — `let url = format!("ws://127.0.0.1:{port}/");` where `port` comes from `OPENHUMAN_WEBVIEW_APIS_PORT` (`webview_apis/mod.rs:10`). No bounds check on the env var; an empty string will panic the URL parse downstream.
**Fix:** Lift these into a `BrowserToolConfig` / `WebviewApisConfig` populated by the config loader (the same loader that already handles env overrides, see issue #4). Validate at parse time.
---
## Strengths Observed (what they're doing right)
1. **Test scaffolding is generous.** ~86k LOC of Rust test code against ~360k LOC of prod is healthy. Files like `src/openhuman/security/policy_tests.rs` (1,497 LOC) and `src/openhuman/composio/ops_test.rs` (1,534 LOC) show real coverage discipline.
2. **Module-level docstrings are real.** E.g., `app/src-tauri/src/webview_accounts/mod.rs:1-19` explains the architecture concisely with an ASCII flow diagram. `src/openhuman/agent/harness/session/turn.rs:1-18` lists the public surface up-front. Many crates would kill for this.
3. **TODOs reference issue numbers.** Of the 20 TODO/FIXME markers in the codebase, most have linked issue IDs: `TODO(#1339)`, `TODO(phase-5)`, `TODO(composio-retry-dedup)`. This is unusually disciplined.
4. **Domain segmentation is broadly correct.** 75 modules under `src/openhuman/` cleanly map to capability areas (memory, channels, composio, inference, voice, security). The boundaries are real even if individual modules have grown too large.
5. **Error-classification is centralized, not scattered.** Even though `observability.rs` is 2,702 lines, it is the *one* place where error-string heuristics live, rather than being duplicated across providers.
6. **TypeScript discipline is high.** Only 21 `as any` (most in `polyfills.ts` for legitimate global injection), 4 `@ts-ignore` (all in tests), and no `@ts-nocheck`. Compare to typical Electron+React apps where these run into hundreds.
7. **Notification routing is feature-flagged correctly.** `webview_accounts/mod.rs:1146-1158` — the `forward_native_notification` function gates on `NotificationSettingsState::enabled()` and exits early when off. Defensive coding done right.
8. **Test/prod split via `#[cfg(test)]` inline modules is consistent.** The 3,009 `.unwrap()` calls compress to ~120-150 in actual production paths once `#[cfg(test)]` blocks are excluded — the test scaffold is doing its job.
9. **No `unimplemented!()` or `todo!()` macros in prod.** Zero occurrences across `src/` and `app/src-tauri/src/`. This means no half-finished code paths can panic the agent loop.
10. **Inference provider module is decomposed.** Even though `compatible.rs` is 2,005 lines, the surrounding directory has nine sibling files (`compatible_parse.rs`, `compatible_stream.rs`, `compatible_types.rs`, `reliable.rs`, `router.rs`, `factory.rs`, etc.) — the directory was clearly designed with separation in mind even if `compatible.rs` itself never got split.
---
## Risk Score: **6 / 10**
The codebase is in the upper half of its peer group. Test density is good, naming is sound, TS strictness is enforced, and most modules show genuine architectural intent. The risk drivers are concentrated rather than diffuse: a small handful of god-files (`webview_accounts/mod.rs`, `agent/harness/session/turn.rs`, `pages/Conversations.tsx`, `panels/AIPanel.tsx`, `config/schema/load.rs`) plus one systemic choice (`Result<_, String>` everywhere, dead-code warnings disabled module-wide). These are addressable with mechanical refactors — no architectural rewrite needed. The single most acute *operational* risk is the `std::sync::Mutex` + `.lock().unwrap()` pattern in the Tauri host: a one-line panic in a third-party-DOM-driven callback poisons a lock and bricks the IPC layer until the user restarts the app. That alone justifies a sprint of focused cleanup before the next release.
docs/qe-reports/openhuman/02-security.md
+242
View File
@@ -0,0 +1,242 @@
# OpenHuman Security Audit
**Audit Date**: 2026-05-20
**Auditor**: V3 QE Security Scanner (agentic-qe)
**Project Version**: 0.54.3 (per `app/src-tauri/tauri.conf.json`)
**Scope**: OWASP Top 10 + AI/LLM-specific + Tauri/desktop + supply chain
**Methodology**: Static analysis (grep + targeted file reads), `pnpm audit`, manual code review of auth/IPC/network surfaces
---
## Executive Summary
OpenHuman is a Tauri-based desktop AI assistant with a Rust core (`openhuman-core` JSON-RPC server) and a TypeScript/React frontend. The codebase shows a **strong baseline security posture** with several pieces of evidence:
- **Per-process random 256-bit bearer tokens** for the core RPC, 0o600 file perms on Unix (`src/core/auth.rs:104-269`).
- **SSRF defence in depth** with DNS-rebinding protection on outbound HTTP tools (`src/openhuman/tools/impl/network/url_guard.rs`).
- **Prompt-injection detector + enforcement** wired into the agent bus, web channel, and local inference paths (`src/openhuman/prompt_injection/detector.rs`).
- **Tauri capabilities are explicitly scoped** to specific commands and webview labels; recipe events are bound to the caller webview label.
- **Hardened Docker compose**: `read_only`, `no-new-privileges`, `cap_drop: ALL`, tmpfs, mem/cpu limits (`docker-compose.yml:25-50`).
- **No secrets committed**: the suspicious 12.9KB `.env.example` is documentation only — every value is blank or commented out. `scripts/ci-secrets.example.json` is also all-blank.
**Critical findings**: **0**
**High findings**: **2**
**Medium findings**: **5**
**Low findings**: **4**
**Info / observations**: **5**
The two `HIGH` findings are not exploitable in isolation but combine into a credible remote-code-execution path when the core is deployed in Docker/cloud mode (the `0.0.0.0` bind path that the project explicitly supports).
---
## Findings Table
| ID | Severity | Category | File:Line | Description | Recommendation |
|----|----------|----------|-----------|-------------|----------------|
| SEC-01 | HIGH | Auth / Update / RCE | `src/openhuman/config/schema/update.rs:53-55`, `src/openhuman/update/ops.rs:21-39` | `rpc_mutations_enabled` defaults to `true` — any bearer-authenticated RPC client can call `update.apply`/`update.run`, downloading and executing a new core binary. On Docker/cloud deployments (`OPENHUMAN_CORE_HOST=0.0.0.0`) this is an authenticated remote-code-execution path. `.env.example:79-81` acknowledges the risk but does not change the default. | Flip the default to `false`. Keep desktop-Tauri (where the shell controls the token) opt-in via a feature flag or runtime check (`if bind_host == 127.0.0.1 { allow }`). |
| SEC-02 | HIGH | Auth / Timing side-channel | `src/core/auth.rs:210-212` | `bearer_matches` uses `==` (variable-time `&str` comparison). Token comparison is non-constant-time. The code comment even calls this out: *"adding constant-time semantics later is a one-line change."* For a 64-hex (256-bit) token over LAN/cloud this is a *theoretical* timing oracle, but it is reachable on `0.0.0.0` deployments. | Replace with `subtle::ConstantTimeEq` or `ring::constant_time::verify_slices_are_equal`. One-line fix. |
| SEC-03 | MEDIUM | XSS / Injection | `app/src/features/human/Mascot/backend/BackendMascot.tsx:144`, `:87` | `dangerouslySetInnerHTML` with backend-fetched SVG (`app/src/services/mascotService.ts:19-24`) plus a live `slot.innerHTML = inner` swap on viseme change. SVG can contain `<script>` and event handlers in HTML context. The code comment says *"Treated as trusted"*, but the trust boundary is `api.tinyhumans.ai` (one HTTP MITM or backend compromise away). | Sanitize via DOMPurify with `USE_PROFILES: { svg: true, svgFilters: true }`, or parse with `DOMParser` and reject anything outside an SVG-element whitelist. Also block `<foreignObject>`. |
| SEC-04 | MEDIUM | CSP | `app/src-tauri/tauri.conf.json:25-27` | CSP allows `'unsafe-inline'` in `default-src`, plus wildcard `https:`, `wss:`, `http:`, `ws:`, `data:`, `blob:` in `connect-src`. `connect-src` effectively allows any URL. `default-src` with `unsafe-inline` weakens script and style XSS defence. | Drop `'unsafe-inline'` from `default-src` (use nonces or move to explicit `script-src 'self'`). Narrow `connect-src` to the actual hosts the app uses (`https://api.tinyhumans.ai`, `https://staging-api.tinyhumans.ai`, IPC, loopback ports). |
| SEC-05 | MEDIUM | CORS | `src/core/jsonrpc.rs:616-635` | `Access-Control-Allow-Origin: *` is hardcoded on **every** response, including the authenticated `/rpc` and `/v1/chat/completions` endpoints. While bearer-in-header is not auto-attached by browsers, a malicious page can probe `/health`, `/schema`, and `/events` cross-origin, and any cached/stolen token allows full cross-origin RPC from a hostile page. | When the core binds to `0.0.0.0`, restrict CORS to `null`/explicit origins. When loopback-only, keep `*` if needed but at least skip the header on `POST /rpc`. |
| SEC-06 | MEDIUM | Supply chain | `package.json` → transitive `ws@8.18.3` (via `app > socket.io-client > engine.io-client > ws`) | `pnpm audit` reports GHSA-58qx-3vcg-4xpx (ws uninitialized memory disclosure, `>=8.0.0 <8.20.1`). The repo also has a direct dev-dep `ws@^8.20.0` (which is patched) but the transitive copy is still 8.18.3. | Add a pnpm `overrides` entry pinning `ws` to `>=8.20.1`, or upgrade `socket.io-client`/`engine.io-client` to a version that ships the patched transitive. |
| SEC-07 | MEDIUM | Supply chain | `Cargo.toml:203`, `app/src-tauri/Cargo.toml:192` | `whisper-rs-sys = { git = "https://github.com/tinyhumansai/whisper-rs-sys.git", branch = "main" }` — pinned to a mutable branch in a project-owned fork. Anyone with push access to that fork (or a fork takeover) silently changes what gets compiled into the core binary on the next `cargo update`. | Pin to a specific `rev = "<commit-sha>"` so the lockfile materially constrains the source. |
| SEC-08 | LOW | Crypto / Randomness | `app/src/utils/deviceFingerprint.ts:13`, `app/src/pages/Accounts.tsx:41`, `app/src/components/settings/panels/AIPanel.tsx:2275,2414`, `app/src/store/threadSlice.ts:202` | `Math.random()` used as fallback for IDs (`fp_*`, `acct-*`, message IDs, profile IDs). For toast/UI IDs this is fine, but `acct-` IDs become webview labels (`acct_<account_id>`) and route security-relevant `webview_recipe_event` checks (`app/src-tauri/src/webview_accounts/mod.rs:3107-3117`). Predictable IDs could theoretically allow forging targets. | Prefer `crypto.randomUUID()` (already used as the preferred path in `threadSlice.ts:202`) and remove the `Math.random()` fallback — `crypto.randomUUID` is universally available in any environment Tauri/modern browsers run. |
| SEC-09 | LOW | Command injection (defence-in-depth) | `src/openhuman/tools/impl/browser/browser_open.rs:144-147` | Windows path: `cmd.exe /C start "" brave <url>`. The URL is already validated against an allowlist with no whitespace, HTTPS-only, no userinfo, etc., so this is **not exploitable** today. However, `start` and `cmd /C` have well-known quoting quirks (the leading empty `""` is the window-title arg, and `&`/`^`/`%` semantics inside `start` are subtle). | Either use `tokio::process::Command::new("cmd").args(["/C", "start", "", "brave", url])` (already done — good) **and** add explicit assertion in the validator that the URL contains no `&`, `%`, `^`, `>` characters; or invoke brave directly via its registered protocol handler. |
| SEC-10 | LOW | CSP scope | `app/src-tauri/tauri.conf.json:26` | `frame-src 'self' https: data: blob:` allows arbitrary HTTPS framing inside the Tauri shell. Combined with the embedded webview-account child webviews (LinkedIn, Slack, WhatsApp, etc.) this is intentional, but the wide `https:` permits unexpected framing from any HTTPS origin. | Audit whether the main window actually needs `frame-src https:`. The child-webview model uses separate `acct_*` webviews with their own capability, so the main window may be able to drop `frame-src` entirely. |
| SEC-11 | LOW | Insecure default flag | `src/openhuman/tools/impl/browser/security.rs:131`, documented in `.env.example:87` | `OPENHUMAN_BROWSER_ALLOW_ALL=1` disables the browser-tool URL allowlist. Default is `0` (locked). Opt-in env var is acceptable, but if it ever ships set to `1` in a packaged config it bypasses SSRF defences. | Add a startup log warning at WARN level when this flag is enabled. Consider gating it behind a debug build. |
| SEC-12 | INFO | Auth design | `src/core/auth.rs:79`, `:186-193` | `/events/webhooks` accepts bearer via `?token=…` query param. Documented rationale (browser `EventSource` cannot set headers). Query tokens land in server logs, browser history, referer chains. The `http_request_log_middleware` already strips query (`?…` substitution at `:596`), but downstream proxies / Sentry might still see the URL. | Add explicit redaction for `?token=…` everywhere the request URI is logged or sent to Sentry. Already partially done — verify Sentry's HTTP integration also redacts. |
| SEC-13 | INFO | AI / Prompt injection | `src/openhuman/prompt_injection/detector.rs` + integrations | Detector is regex-based with `Allow / Review / Block` verdicts, leet-speak normalization, Cyrillic homoglyph mapping, zero-width-char stripping. Heuristic classifier is opt-in (`OPENHUMAN_PROMPT_INJECTION_CLASSIFIER=heuristic`). Wired into `agent::bus`, `inference::local`, `channels::providers::web`. **No ML classifier**, no embeddings detector. | Acceptable for current threat model. Consider adding a small classifier (TinyBERT or a hosted moderation API behind a feature flag) for higher-stakes flows like wallet operations and tool calls. |
| SEC-14 | INFO | Crypto | `src/openhuman/tools/impl/network/polymarket.rs:820-830` | Salt generated via `OsRng.fill_bytes` for CLOB order signing. Comment explicitly explains why `rand::random` is insufficient (replay/front-running). Correctly used. | No action. |
| SEC-15 | INFO | Dockerfile | `Dockerfile:76-77`, `docker-compose.yml:25-50` | Non-root UID 10001, `read_only` filesystem, `cap_drop: ALL`, `no-new-privileges`, tmpfs, mem/cpu limits. Healthcheck via `curl http://localhost:7788/health`. | Exemplary. Optionally pin base image by digest (`rust:1.93-bookworm@sha256:…`) for reproducible / supply-chain-hardened builds. |
---
## Detailed Write-up — HIGH Findings
### SEC-01: Authenticated RCE via `update.run`/`update.apply` defaults
**Files**: `src/openhuman/config/schema/update.rs:53-55`, `src/openhuman/update/ops.rs:21-39`, `src/openhuman/update/ops.rs:182-205`
**Reasoning**:
The update subsystem exposes two mutating JSON-RPC methods, `openhuman.update_apply` and `openhuman.update_run`, that download a new binary from GitHub Releases, stage it, and (optionally) self-restart. These are gated by `enforce_update_mutation_policy()`:
```
src/openhuman/update/ops.rs:29
if policy.rpc_mutations_enabled {
return Ok(policy);
}
```
`policy.rpc_mutations_enabled` is read from `UpdateConfig::rpc_mutations_enabled`, whose default is:
```
src/openhuman/config/schema/update.rs:53-55
fn default_rpc_mutations_enabled() -> bool {
true
}
```
The `.env.example` even warns:
```
.env.example:80-81
# [optional] Allow bearer-authenticated RPC callers to invoke update.apply/update.run
# Disable on exposed server deployments unless you explicitly want remote self-upgrade.
```
…but the user has to read that comment, set `OPENHUMAN_AUTO_UPDATE_RPC_MUTATIONS_ENABLED=false`, and rebuild their config to be safe.
**Threat model**: Any party with the bearer token + network reachability to `:7788` can ship code into the running process. On the desktop-Tauri path the token is in-memory and only loopback-reachable; **on the Docker/cloud path** documented in `docker-compose.yml` the core binds to `0.0.0.0:7788` and the token is provided via `.env`. If that token leaks (Sentry breadcrumb, log line, env-var dump, screenshot), an attacker with TCP reachability becomes root on the host — modulo the container's `cap_drop: ALL` (which limits damage but the attacker still owns the process and any mounted volumes).
**Severity**: HIGH — because the attack is post-auth, but the auth surface (a single bearer token, sometimes user-provisioned and sometimes pasted into a UI) is realistic to compromise.
**PoC reasoning**:
1. Attacker obtains the bearer token (env leak, support log, malicious extension reading clipboard at paste time, etc.).
2. `curl -X POST http://victim:7788/rpc -H "Authorization: Bearer $TOKEN" -d '{"jsonrpc":"2.0","id":1,"method":"openhuman.update_run","params":{}}'`
3. The core downloads the configured GitHub release artifact, stages it, and self-replaces.
4. Note: the **release URL is fixed** to the project's GitHub releases by `tauri.conf.json:78` (`https://github.com/tinyhumansai/openhuman/releases/latest/download/latest.json`) and downloads are minisign-verified by the Tauri updater pipeline. The core's `openhuman.update_apply` path uses the project's own GitHub asset list, so a third party cannot easily redirect to a malicious binary. **The exploit becomes a guaranteed-restart vector** (DoS, downgrade-to-buggy-version) rather than arbitrary RCE — but the auto-update pipeline is GitHub-released only.
5. Combined with a compromised maintainer or workflow secret, this becomes full RCE. As of this audit there is no evidence of GitHub workflow misconfiguration.
**Recommended fix**:
```rust
// src/openhuman/config/schema/update.rs
fn default_rpc_mutations_enabled() -> bool {
false // was: true
}
```
…plus add a runtime check in `enforce_update_mutation_policy` that auto-allows when `OPENHUMAN_CORE_HOST` resolves to a loopback address (so the desktop-Tauri flow keeps working out-of-the-box).
---
### SEC-02: Variable-time bearer-token comparison
**File**: `src/core/auth.rs:210-212`
**Reasoning**:
```rust
fn bearer_matches(supplied: &str, expected: &str) -> bool {
!supplied.is_empty() && supplied == expected
}
```
The code comment 7 lines above acknowledges:
> *"Hex tokens of fixed length make the comparison non-secret-shaped, but we still pin a deliberate helper so adding constant-time semantics later is a one-line change."*
For a 256-bit hex token, a remote timing oracle is extremely hard to exploit in practice (network jitter swamps the 100-ns-per-byte signal), but:
1. The cloud deployment path puts the core behind a Docker bridge on an arbitrary network. Local-network attackers can time-sample much faster than internet attackers.
2. Co-tenant attackers (e.g., another container on the same host, malicious IPC, kernel side-channels) can observe with high precision.
3. The fix is genuinely one line. There is no engineering cost to apply it.
**Recommended fix**:
```rust
use subtle::ConstantTimeEq;
fn bearer_matches(supplied: &str, expected: &str) -> bool {
if supplied.is_empty() { return false; }
bool::from(supplied.as_bytes().ct_eq(expected.as_bytes()))
}
```
---
## Dependency Audit Summary
### npm / pnpm (`pnpm audit --prod`)
```
1 vulnerabilities found
Severity: 1 moderate
```
| Severity | Package | Version found | Fix | Path |
|----------|---------|---------------|-----|------|
| Moderate | `ws` | `8.18.3` | `>=8.20.1` | `app > socket.io-client > engine.io-client > ws` (GHSA-58qx-3vcg-4xpx) |
The repo also has `ws@8.20.0` as a direct dev dep (patched). Only the transitive copy is vulnerable.
**Recommendation**: add to root `package.json`:
```json
"pnpm": { "overrides": { "ws": ">=8.20.1" } }
```
(or `resolutions` field if migrating tooling — `resolutions` is currently used for `@tauri-apps/api`.)
### Cargo (`cargo audit`)
**Not run** — `cargo` was not available in the audit environment (`cargo: command not found`), and the constraint was to skip if installation took >60s. The `Cargo.lock` contains 914 crates. Manual review of pinned versions showed current minor versions for crypto-critical crates:
- `openssl@0.10.79` — current, no open RUSTSEC
- `ring@0.17.14` — current
- `rustls@0.23` — current
- `reqwest@0.12` — current
**Recommendation**: run `cargo audit` in CI on a schedule (weekly) — the project has 914 transitive crates and audit results drift constantly.
### Git dependencies in Cargo manifests
| Dep | Pinning | Risk |
|-----|---------|------|
| `whisper-rs-sys` | `branch = "main"` (project-owned fork) | **SEC-07 / MEDIUM** — branch pin is mutable |
| `tauri-plugin-opener` | `rev = "c6561ab6..."` | OK — commit-pinned |
| `tauri-plugin-deep-link` | `rev = "c6561ab6..."` | OK — commit-pinned |
| `tauri-plugin-global-shortcut` | `rev = "c6561ab6..."` | OK — commit-pinned |
| `tauri-plugin-single-instance` | `rev = "c6561ab6..."` | OK — commit-pinned |
---
## Areas Checked With No Findings
To make the audit's negative space explicit:
- **Hardcoded secrets in source**: no real secrets found. All "matches" for `sk-…`, `AKIA…`, `-----BEGIN PRIVATE KEY-----`, `Bearer xyz`, `xoxb-…` were either (a) inside `src/openhuman/memory/safety/mod.rs` regex patterns that *detect* secrets, (b) inside `src/openhuman/agent_experience/types.rs` and `src/openhuman/memory/store/unified/fts5.rs` test-redaction fixtures, or (c) inside `tests/fixtures/composio_github.json` example values published by the upstream provider. The 12.9KB `.env.example` is documentation only — every variable is blank or commented.
- **SQL injection**: ripgrep across all `format!()` calls touching SQL keywords found only one site (`src/openhuman/memory/tree/read_rpc.rs:1278,1452`) which uses a `TABLES: &[&str]` *constant* allowlist as the interpolated value, not user input. Other SQL goes through `rusqlite::params!` / parameterized queries.
- **Command injection**: every `std::process::Command::new(...)` site found (~40 sites) uses fixed program names with separately-passed arguments (no shell). Validated URLs flow into argument position only. AppleScript escaping in `src/openhuman/voice/text_input.rs:155-158` escapes `\` and `"` correctly.
- **`eval` / `new Function`**: zero matches in `app/src/**` source.
- **Weak hashes (MD5/SHA-1)**: SHA-1 used only for the WebSocket protocol handshake (`scripts/mock-api/socket/websocket.mjs:42-45`) — required by RFC 6455. No MD5 / SHA-1 used for security purposes.
- **TLS skip-verification**: zero matches for `rejectUnauthorized: false`, `verify=False`, `InsecureSkipVerify`, `danger.*=true` related to TLS. The one `skip_verify` match (`src/bin/gmail_backfill_3d.rs:279`) is a chunk-file integrity check flag, not a TLS bypass.
- **Tauri allowlist drift**: capabilities are explicitly scoped per-window and per-webview-label; `webview-accounts` capability is bound to remote URLs and exposes only `webview_recipe_event` + screen-share session commands. The recipe-event handler verifies caller-label match before processing (`app/src-tauri/src/webview_accounts/mod.rs:3107-3117`).
- **Open redirect / SSRF**: `src/openhuman/tools/impl/network/url_guard.rs` implements full SSRF defence — HTTP(S)-only, allowlist required, no whitespace, no userinfo, no IPv6 literal, blocklist for loopback/RFC1918/link-local/multicast/carrier-grade-NAT, **plus DNS-resolution rebinding check** before issuing the request.
- **AI/LLM prompt injection**: `src/openhuman/prompt_injection/detector.rs` enforces with verdict Allow/Review/Block, normalises leet-speak, Cyrillic homoglyphs, full-width ASCII, zero-width/bidi formatting chars. Enforcement is wired into agent bus, web channel, and local inference paths (3 distinct integration points).
- **`dangerouslySetInnerHTML`**: only one production usage, in `BackendMascot.tsx` for backend-supplied SVG. Flagged as SEC-03 above.
- **`target="_blank"` without `rel`**: every match has `rel="noreferrer"` or `rel="noopener noreferrer"`.
- **Tauri `withGlobalTauri`**, dangerous config flags: none enabled.
- **`tauri-plugin-opener`** auto-injection: explicitly disabled (`open_js_links_on_click(false)`) in `app/src-tauri/src/lib.rs:2277-2280`.
---
## Overall Security Posture
**Score**: **7.5 / 10**
**Rationale**:
- **+** Strong baseline: SSRF protection with DNS rebinding, prompt-injection detection wired in three places, hardened Docker, scoped Tauri capabilities, sealed `.env.example`, properly-permissioned token files, MIT-vetted dependency pins.
- **+** Defensive-coding evidence throughout: token redaction in tool outputs, scrubbing logic in `agent/harness/credentials.rs`, deliberate non-CSPRNG comments where used (with reasons), commented rationale on every security-relevant decision.
- **+** Architecturally sound auth: per-process bearer token, headers-only for `/rpc`, query-token only for SSE/WS where browsers can't set headers.
- **-** Two HIGH issues (`SEC-01`, `SEC-02`) are one-line fixes in security-critical paths. The fact that they are still open suggests the security review process for the cloud-deploy code path hasn't received the same attention as the desktop-Tauri path.
- **-** Permissive CSP (`SEC-04`) and wildcard CORS (`SEC-05`) reflect an understandable desktop-app trust model but expand the attack surface unnecessarily when the same code runs in the cloud-deploy path.
- **-** One transitive `ws` vuln (`SEC-06`) and one branch-pinned Cargo git dep (`SEC-07`) — both easy to fix.
- **-** SVG-injection vector via backend manifest (`SEC-03`) — small impact, but not zero, and DOMPurify is one import away.
**With the HIGH findings remediated (default-deny on `rpc_mutations_enabled`, constant-time token comparison) the score moves to 8.5. With also the CSP tightened, ws override, and SVG sanitized, 9.0+.**
---
## Suggested Remediation Order (by ROI)
1. **SEC-02** (1-line change, eliminates timing oracle): `subtle::ConstantTimeEq`. **15 minutes.**
2. **SEC-06** (1 JSON change): add `pnpm.overrides.ws: ">=8.20.1"`. **5 minutes.**
3. **SEC-01** (3-line change + one runtime guard): default `rpc_mutations_enabled` to `false`, auto-allow on loopback bind. **30 minutes.**
4. **SEC-07** (1-line change): pin `whisper-rs-sys` to a `rev = "..."`. **5 minutes.**
5. **SEC-03** (one DOMPurify import + sanitize call): protects against compromised backend / MITM SVG. **1 hour.**
6. **SEC-04, SEC-05** (CSP + CORS tightening): more invasive, requires per-deployment thought. **2-4 hours.**
7. **SEC-08** (replace `Math.random()` ID fallbacks with `crypto.randomUUID()`): mechanical. **30 minutes.**
Everything else is INFO / hardening.
docs/qe-reports/openhuman/03-performance.md
+414
View File
@@ -0,0 +1,414 @@
# OpenHuman — Performance Review
**Scope reviewed**: `src/` (Rust, 1,345 .rs files), `app/src/` (TypeScript/React, 377 .tsx + 291 .ts files)
**Date**: 2026-05-20
**Reviewer**: V3 QE Performance Reviewer
**Posture score**: **5 / 10** — solid foundations (SSE streaming, HTTP client cache, `spawn_blocking` discipline in some hot tools), but several systemic patterns will hurt under realistic load (bulk WhatsApp ingest, large conversations, voice transcription, vault sync).
---
## Executive summary
OpenHuman has good performance hygiene in some places (`build_runtime_proxy_client` caches `reqwest::Client`s by service key; `GrepTool` correctly wraps its sync walker in `tokio::task::spawn_blocking`; SSE streaming uses `bytes_stream` not full-buffer reads) but is undermined by **five systemic issues**:
1. **No SQLite connection pooling and no transactions for bulk writes.** Stores like `whatsapp_data/store.rs` and `vault/store.rs` open a fresh `Connection` per call and execute INSERT/UPDATE rows in a `for` loop with no transaction and no `prepare_cached`. The WhatsApp message upsert path is particularly bad: O(N) `INSERT ON CONFLICT` statements + an UPDATE with two correlated COUNT(*)/MAX subqueries per affected chat.
2. **Voice transcription transcodes audio to JSON `number[]` and ships it across Tauri IPC.** `Array.from(new Uint8Array(...))` blows up a binary blob ~8× in memory and JSON serialization cost. A 30-second 48kHz mono clip (~1MB raw) becomes a ~8MB JS array, then a ~25MB JSON string.
3. **No code-splitting in the React app.** Every route — including the WebGL `three.js` welcome screen, the `remotion`/`MascotCharacter` page, `react-joyride`, and full `react-markdown` — is imported at top level and ships in the main bundle. Vite has no `manualChunks` configured.
4. **The largest page component (`Conversations.tsx`, 2125 LOC, 39 hook calls, ~30 `useState`) recomputes derived data on every render.** `visibleMessages`, `latestVisibleAgentMessage`, `activeToolTimelineEntry`, `selectedThreadParent`-via-`find` and several `[...arr].reverse().find(...)` clones run *every render*, none are memoized. Message bubbles (`BubbleMarkdown` → `react-markdown`) are NOT wrapped in `React.memo`, so every keystroke in the composer re-renders every visible message.
5. **Sync `std::fs` I/O inside `async fn` without `spawn_blocking`** in `vault/sync.rs` (the entire vault walk + per-file read is inline-blocking inside `pub async fn sync_vault`). One file-system tree walk = one tokio worker blocked for the full duration.
The codebase shows comments that indicate past perf surgery (the `html2md` removal note in `Cargo.toml` is exemplary), so the team is performance-aware — these findings are about the next wave of fixes.
---
## Findings by category
### 1. Algorithmic complexity / hot loops with allocations
#### 1.1 Streaming SSE buffer re-allocates per chunk — **MEDIUM**
**File**: `src/openhuman/inference/provider/compatible.rs:911-923`
```rust
let mut bytes_stream = response.bytes_stream();
let mut buffer = String::new();
while let Some(item) = bytes_stream.next().await {
let bytes = item?;
buffer.push_str(&String::from_utf8_lossy(&bytes)); // L916: allocates Cow<str>→String per chunk
while let Some(sep_idx) = buffer.find("\n\n") {
let event = buffer[..sep_idx].to_string(); // L922: full copy of event
buffer.drain(..sep_idx + 2); // L923: O(n) shift
...
```
Three allocations per SSE event in the chat-streaming hot path. `from_utf8_lossy` allocates a `String` even when the bytes are valid UTF-8 (the common case). `buffer.drain` shifts the remaining tail on every event. For a long streaming completion (10K+ events for a multi-thousand-token response) this is noticeable GC/alloc pressure.
**Fix**: Use a `BytesMut`/`bytes::Bytes` ring buffer or read into a `Vec<u8>` and split on `\n\n` boundaries before doing UTF-8 decode once; reuse the buffer instead of `drain`. Or use the `eventsource-stream` crate.
**Impact**: Reduces allocations roughly N×events. Streaming feels less janky on long completions on low-RAM laptops.
#### 1.2 Conversations page recomputes derived state every render — **HIGH**
**File**: `app/src/pages/Conversations.tsx:1036-1047`
```tsx
const visibleMessages = messages.filter(msg => !msg.extraMetadata?.hidden); // alloc Vec
const latestVisibleMessage = visibleMessages[visibleMessages.length - 1] ?? null;
const latestVisibleAgentMessage = [...visibleMessages] // FULL CLONE
.reverse()
.find(msg => msg.sender === 'agent');
const activeSubagentTimelineEntry = selectedThreadToolTimeline.find(...);
const activeToolTimelineEntry = [...selectedThreadToolTimeline] // FULL CLONE
.reverse()
.find(entry => entry.status === 'running' && !entry.name.startsWith('subagent:'));
```
Every render of this 2125-LOC component clones the entire `visibleMessages` array and the entire `selectedThreadToolTimeline` array just to find the *last* matching element. With 30+ `useState` calls in the same component and Redux events arriving every SSE chunk, this runs constantly. Each clone forces a new array identity, so any downstream `useEffect`/`useMemo` keyed on these values re-fires.
**Fix**: Wrap each derived value in `useMemo` keyed on the source array; replace `[...arr].reverse().find(p)` with `findLast(p)` (ES2023, no allocation).
**Impact**: HIGH — this is the most visible chat-interaction surface. Removing the clones cuts per-render allocator pressure dramatically and stabilizes hook deps.
#### 1.3 WhatsApp upsert: N+1 + correlated subqueries + no transaction — **HIGH**
**File**: `src/openhuman/whatsapp_data/store.rs:184-249`
```rust
fn upsert_messages_inner(&self, account_id: &str, msgs: &[IngestMessage]) -> Result<usize> {
let conn = self.open_conn()?; // L181: NEW connection
for m in msgs {
conn.execute("INSERT INTO wa_messages ... ON CONFLICT ... DO UPDATE ...", ...)?; // L195-228: per-row
}
if count > 0 {
conn.execute("UPDATE wa_chats SET message_count = (SELECT COUNT(*) FROM wa_messages WHERE ...), last_message_ts = COALESCE((SELECT MAX(timestamp) FROM wa_messages WHERE ...), ...) WHERE account_id = ?2", ...)?; // L234-247: scan-per-chat
}
}
```
And in `prune_old_messages_inner` (L273-319): same pattern but the post-prune UPDATE is run **once per affected chat** (`for (acct, chat_id) in &affected`), each containing two correlated subqueries that scan `wa_messages`.
Plus zero `prepare_cached` calls anywhere in the repo (`grep -rn prepare_cached src/openhuman` = 0 matches), so every iteration re-parses SQL.
**Fix**:
- Wrap the loop in `conn.transaction()` (or `unchecked_transaction`) — single fsync instead of N
- `prepare_cached` the INSERT statement and reuse across the loop
- For the chat-stats refresh: do a single set-based UPDATE … FROM (SELECT … GROUP BY) instead of one UPDATE per chat
- Use a shared connection pool (`r2d2_sqlite` or a single long-lived connection + WAL) instead of `open_conn` per call
**Impact**: For an initial WhatsApp backfill of 50K messages across 200 chats, current code does ~50K + 1 statements plus 200 multi-subquery updates. A batched transactional version drops this to ~50K statements in ONE transaction (typically 20-100× faster on SSD, more on HDD) plus a single aggregate UPDATE.
#### 1.4 Vault per-file read: in-loop sync I/O on async fn — **HIGH**
**File**: `src/openhuman/vault/sync.rs:78` (signature: `pub async fn sync_vault`) + `:130` (sync `WalkDir`) + `:217` (sync `std::fs::read_to_string`)
`sync_vault` is `async fn` but contains a sync `walkdir::WalkDir` iteration plus `std::fs::read_to_string` per file (and a `sha256_hex` over the contents). Nothing in `src/openhuman/vault/sync.rs` calls `tokio::task::spawn_blocking` (`grep -rn spawn_blocking src/openhuman/vault` = no matches). For a 10K-file vault, this monopolizes a tokio worker for seconds-to-minutes.
**Fix**: Wrap the entire walk/read body in `spawn_blocking`, or move the walk to a dedicated thread that streams `(path, hash, mtime)` tuples back to the async side via a bounded channel.
**Impact**: Today, a vault sync can starve every other tokio task on the same worker thread (default tokio multi-thread runtime has `num_cpus` workers — on a 4-core laptop that's 25% of compute frozen).
#### 1.5 Grep tool single-threaded inside spawn_blocking — **MEDIUM**
**File**: `src/openhuman/tools/impl/filesystem/grep.rs:118-122` + `:154-202`
Correctly wraps in `spawn_blocking` (good) but `scan_for_matches` walks one file at a time and reads contents synchronously per file. On a workspace with 5–10K files this is bottlenecked on serial syscalls.
**Fix**: Use `ignore::WalkBuilder::threads(num_cpus)` (from `ripgrep`'s `ignore` crate, which the team is already comfortable with given the lint:commands-tokens script) or `rayon::par_bridge()` on `WalkDir`. Same regex, parallel reads.
**Impact**: Multi-core speedup on agent grep tool calls — practically MEDIUM because matches are usually rare hot calls but each can be slow.
---
### 2. Async / concurrency
#### 2.1 `std::fs::write` / `std::fs::read_to_string` inside async paths — **MEDIUM**
**Files (production code, not tests)**:
- `src/openhuman/workspace/ops.rs:23` — `std::fs::write(&path, contents)` (workspace file write)
- `src/openhuman/tree_summarizer/cli.rs:135` — `std::fs::read_to_string(path)` (called from `block_on`)
- `src/openhuman/tree_summarizer/store.rs:175`, `:197`, `:260` — sync `fs::write`, `fs::read_to_string`, `fs::read_dir`
- `src/openhuman/tools/impl/browser/image_output.rs:41` — sync `fs::write` after browser screenshot
- `src/openhuman/vault/sync.rs:217` — see 1.4
- `src/openhuman/webhooks/router.rs:51` — sync read at construction (acceptable — startup)
`webhooks/router.rs:535` correctly offloads to `tokio::task::spawn_blocking` (good pattern). The others don't.
**Fix**: Migrate to `tokio::fs` or wrap in `spawn_blocking`.
#### 2.2 `block_in_place` + `block_on` from inside async — **MEDIUM**
**File**: `src/openhuman/agent/harness/session/builder.rs:1549-1581` (`prefetch_tool_memory_rules_blocking`)
The code is *careful* (checks `runtime_flavor() != MultiThread` and returns empty rather than panicking), but the entire prefetch is on the session-startup hot path. Every chat session spawn pays this cost on the calling thread. It also defeats the `Memory` trait's async-ness — under heavy load (many concurrent sessions starting) all of them serialize through the calling worker.
**Fix**: Make `build_session` (or wherever `prefetch_tool_memory_rules_blocking` is called) genuinely async; `await` the rules. The "no runtime → empty Vec" fallback is fine, but the multi-threaded path should not `block_in_place`.
#### 2.3 Unbounded mpsc channels in hot paths — **MEDIUM**
**Files**:
- `src/openhuman/memory/ingestion/queue.rs:106` — `mpsc::unbounded_channel::<IngestionJob>()`
- `src/openhuman/voice/server.rs:587` — unbounded for voice events
- `src/openhuman/voice/hotkey.rs:196` — unbounded for hotkey events
- `src/openhuman/service/restart.rs:194` — unbounded for restart events
`memory/ingestion/queue.rs` is the most concerning — under a backfill burst (Gmail 3-day backfill binary `gmail-backfill-3d` exists per `Cargo.toml`), an unbounded queue means producer outpaces consumer → unbounded memory growth → OOM. Voice/hotkey/restart are lower volume so practical risk is smaller.
**Fix**: `mpsc::channel(N)` with `N` sized for expected concurrency, plus `try_send` with explicit backpressure logging on the producer side.
#### 2.4 Sequential embedding calls — **HIGH** (during backfills)
**File**: `src/openhuman/memory/tree/score/embed/ollama.rs:151-205`
`embed()` accepts one `&str`, makes one HTTP round-trip. All callsites (`tree_global/digest.rs`, `tree_global/seal.rs`, `tree_source/bucket_seal.rs`, `retrieval/source.rs`) call it inside sequential `await`s. For a multi-chunk seal or a backfill that needs to embed N docs, this is `O(N × RTT)`.
Ollama supports batch via passing `prompt` as an array (or `/api/embed` with `input: [...]`) — the embedder doesn't expose that. Even without batch, `futures::stream::iter(items).buffer_unordered(8)` would parallelize. There's no `join_all`/`FuturesUnordered`/`buffer_unordered` anywhere in `memory/tree/` for embeds.
**Fix**:
- Add `embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>` to the `Embedder` trait
- For Ollama, use `/api/embed` (plural) with `input: [...]`
- For cloud, batch using the Voyage `inputs: [...]` form
- At callsites that loop, switch to `stream::iter(...).buffer_unordered(8).try_collect()`
**Impact**: HIGH for backfills. Today, embedding 200 chunks at ~80ms each = 16s sequential. Even simple `buffer_unordered(8)` drops this to ~2s.
#### 2.5 Embedder rebuilt per call — **LOW**
**File**: `src/openhuman/memory/tree/score/embed/factory.rs:60` — `build_embedder_from_config` is called from every seal/digest. `OllamaEmbedder::new` (`embed/ollama.rs:75`) calls `reqwest::Client::builder().connect_timeout(...).build()` — a new HTTP client every time. `reqwest::Client` is cheap to clone but not free to build (connection pool init, TLS config). The comment at `factory.rs:58` calls this "cheap" but it isn't free in a tight loop.
**Fix**: Cache the embedder in a `OnceCell`/`once_cell::sync::Lazy` keyed on config hash, or pass `Arc<dyn Embedder>` down instead of constructing per call.
---
### 3. Resource management
#### 3.1 Per-request `reqwest::Client::new()` — **MEDIUM**
Several production paths construct a new HTTP client per instance rather than going through the cached `build_runtime_proxy_client`:
- `src/openhuman/channels/providers/telegram/channel_core.rs:32` — `client: reqwest::Client::new()` in `Self { ... }` (per `TelegramChannel`). Fine if the channel is long-lived; but the same file at L72 also has `fn http_client(&self) -> reqwest::Client { build_runtime_proxy_client("channel.telegram") }` — so requests go through *two* client paths inconsistently. The `client` field appears unused for sending if `http_client()` is the canonical method.
- `src/openhuman/channels/providers/linq.rs:25` — `client: reqwest::Client::new()` — no proxy support, doesn't honour `runtime_proxy_config`.
- `src/openhuman/tools/ops.rs:79` — `reqwest::Client::new()` passed into `NodeBootstrap::new(...)` — bootstrap downloads node distros. Probably fine (rare event).
- `src/openhuman/inference/local/service/bootstrap.rs:69` — fallback client construction.
- `src/openhuman/memory/tree/score/embed/ollama.rs:83` — fallback for builder failure (rare path).
- `src/openhuman/routing/health.rs:62` — fallback for builder failure.
`build_runtime_proxy_client` (`config/schema/proxy.rs:438`) caches clients by service key — that's the good pattern. Channels should use it.
**Fix**: Route Linq and Telegram through `build_runtime_proxy_client("channel.linq")` / `"channel.telegram"` for consistent proxy + connection pooling. Drop the per-struct `client` field.
#### 3.2 SQLite `Connection::open` per request, no pool — **HIGH** (under load)
**Files**: 14+ files open a fresh connection per operation:
- `src/openhuman/whatsapp_data/store.rs:88` (`open_conn`)
- `src/openhuman/vault/store.rs:19`
- `src/openhuman/subconscious/store.rs:27`
- `src/openhuman/subconscious/situation_report/hotness.rs:198`
- `src/openhuman/redirect_links/store.rs:208`
- `src/openhuman/notifications/store.rs:69`
- `src/openhuman/embeddings/store.rs:88`
- `src/openhuman/cron/store.rs:565`
- `src/openhuman/memory/tree/read_rpc.rs:1382`
- `src/openhuman/people/store.rs:56`
- `src/openhuman/migration/core.rs:154`
Each call re-runs `PRAGMA journal_mode=WAL`, `busy_timeout`, and pays the open() cost (~1–5ms per open on a warm cache, more if the DB file isn't in the OS page cache). With 10K SQL operations during a backfill that's 10–50s of pure connection-open overhead.
**Fix**: Use `r2d2_sqlite::SqliteConnectionManager` + `r2d2::Pool` with `max_size = num_cpus`. Or hold a single `Arc<Mutex<Connection>>` for writes (SQLite WAL allows concurrent readers without locking). The codebase already has `parking_lot::Mutex` and `tokio::sync::Mutex` so the dependency is in place.
#### 3.3 `redux-persist` serializableCheck on every action — **LOW (dev only)**
**File**: `app/src/store/index.ts:156-166`
`serializableCheck` is left at default (enabled) for dev; with chat-runtime state containing turn timelines this walks a deep object tree on every dispatch. Production builds disable it by default in `@reduxjs/toolkit`, but dev experience is degraded.
**Fix**: `serializableCheck: false` or scope to ignored paths if devs are noticing dev slowdown.
---
### 4. React rendering
#### 4.1 Only ONE `React.memo` in the entire app — **HIGH**
**Measured**: `grep -rn "React.memo\|memo(" app/src --include="*.tsx" --include="*.ts"` returns 1 match (`src/components/intelligence/MemoryResultList.tsx` comment only; no actual `React.memo` call). Zero memoized components.
In a chat UI streaming tokens via Redux dispatch (one `setStreamingAssistantForThread` per SSE chunk), this means **every visible message re-renders on every chunk**. `BubbleMarkdown` (`app/src/pages/conversations/components/AgentMessageBubble.tsx:40-78`) wraps `react-markdown` which is CPU-expensive (full Markdown → AST → React tree per render).
**Fix**:
- `export const AgentMessageBubble = React.memo(function AgentMessageBubble(...) { ... })`
- `export const BubbleMarkdown = React.memo(function BubbleMarkdown(...) { ... })` keyed on `content`
- Same for thread sidebar items (currently inline `sortedThreads.map(thread => (...))` at `Conversations.tsx:1279`)
**Impact**: HIGH — each token streamed today re-Markdowns every visible bubble. For a chat with 50 visible bubbles, that's 50× wasted work per SSE chunk.
#### 4.2 No virtualization for messages or thread list — **MEDIUM**
**Files**:
- `app/src/pages/Conversations.tsx:1279` (sidebar `sortedThreads.map`)
- `app/src/pages/Conversations.tsx:1555` (messages `visibleMessages.map`)
- `app/src/components/intelligence/MemoryResultList.tsx:9-10` (explicit comment "intentionally non-virtualized for now")
`grep -rn "react-window\|react-virtuoso\|virtualized"` returns no library usage. A long conversation (200+ messages) or a power user with 500+ threads will render all DOM nodes.
**Fix**: Add `react-virtuoso` (TS-friendly, simple API). Apply to message list and thread sidebar first.
#### 4.3 Inline new objects/arrays in `useAppSelector` defaults — **MEDIUM**
**File**: `app/src/pages/Conversations.tsx`
- L1031-1032: `toolTimelineByThread[selectedThreadId] ?? []` — new `[]` reference on every render
- L1034: `taskBoardByThread[selectedThreadId] ?? null` — fine for null, but pattern applies elsewhere
- L263: `state.locale?.current ?? 'en'` — fine (primitive)
These aren't *inside* the selector but the `?? []` literal creates a new array reference at render time, which then feeds into the timeline render below. Downstream `useEffect([selectedThreadToolTimeline, ...])` would fire every render.
**Fix**: Move the `?? []` into `useMemo`, or use a module-level `EMPTY_TIMELINE = Object.freeze([])` constant.
#### 4.4 `selectedThreadParent` uses `Array.find` on every render — **LOW**
**File**: `app/src/pages/Conversations.tsx:1217-1226` — IS wrapped in `useMemo([threads, selectedThreadId])`, which is good. But `threads.find(...)` runs twice (current + parent). For a user with 1000 threads, that's 2000 ops on every selection change. Build a `Map<id, Thread>` once.
**Fix**: `const threadById = useMemo(() => new Map(threads.map(t => [t.id, t])), [threads])`, then `threadById.get(id)`.
#### 4.5 Top-level state from `useAppSelector(state => state.thread)` — **MEDIUM**
**File**: `app/src/pages/Conversations.tsx:219` — `} = useAppSelector(state => state.thread);` destructures the entire `thread` slice. Any modification to any field (e.g. setting a single thread's title) re-renders all 2125 lines of this component. With `react-redux` v9 the default equality is reference, so this is acceptable IF the slice carefully maintains stable identity for unchanged sub-slices — but with `redux-persist`, rehydration mutates the whole slice.
**Fix**: Split into field-level selectors: `useAppSelector(s => s.thread.threads)`, `useAppSelector(s => s.thread.selectedThreadId)`, etc.
---
### 5. Database / persistence
#### 5.1 No prepared-statement cache — **MEDIUM**
**Measured**: `grep -rn "prepare_cached" src/openhuman --include="*.rs"` = **0 matches** in production code.
Every `conn.execute("INSERT ... ", ...)` re-parses the SQL on each call. For tight loops (WhatsApp, vault, embeddings store) this is wasted work. `rusqlite::Connection::prepare_cached` exists exactly for this and is the idiomatic fix.
#### 5.2 Correlated subqueries in chat-stats refresh — **MEDIUM**
See §1.3 — `whatsapp_data/store.rs:234-247` and `:298-312`. Two subqueries per row.
**Fix**: One aggregate-based UPDATE: `UPDATE wa_chats SET (message_count, last_message_ts) = (SELECT COUNT(*), MAX(timestamp) FROM wa_messages WHERE wa_messages.account_id = wa_chats.account_id AND wa_messages.chat_id = wa_chats.chat_id), updated_at = ?` keyed on the affected `(account_id, chat_id)` set.
#### 5.3 Bundled subquery in `list_vaults` — **LOW**
**File**: `src/openhuman/vault/store.rs:77-100`
```sql
SELECT v.id, ..., (SELECT COUNT(*) FROM vault_files vf WHERE vf.vault_id = v.id AND vf.status = 'ok') AS file_count
FROM vaults v
```
For each vault row, a COUNT(*) scan over its files. With many vaults this is N+1-shaped (correlated scalar subquery per row). Use a single `LEFT JOIN ... GROUP BY` for one pass.
#### 5.4 No `LIMIT` on many `SELECT` paths — **LOW**
Several `SELECT account_id, chat_id, message_id, sender, ... FROM wa_messages WHERE ...` queries in `whatsapp_data/store.rs:371-484` don't show a LIMIT — they may be intentional dump queries, but for any UI-facing path they should be paginated. Quick audit needed.
---
### 6. LLM / streaming
#### 6.1 Streaming buffer allocations — see §1.1
#### 6.2 Sequential embeds — see §2.4
#### 6.3 Non-SSE fallback buffers full response — **EXPECTED**
`compatible.rs:893` — `response.bytes().await?` when content-type isn't SSE — correct behavior; logged as warning. No action.
#### 6.4 No cancellation on stream — **MEDIUM**
**File**: `src/openhuman/inference/provider/compatible.rs:911-` (main streaming loop)
The `while let Some(item) = bytes_stream.next().await { ... }` loop has no cooperative-cancel checkpoint. If the user hits cancel mid-stream, the only path to stop is dropping the future. That works but until cancellation lands, more chunks keep getting parsed and dispatched. Check for an `AbortHandle` / `CancellationToken` and bail explicitly on each iteration.
---
### 7. Build / startup — bundle size
#### 7.1 No code-splitting — **HIGH**
**Files**: `app/vite.config.ts:107-201` (no `build.rollupOptions.output.manualChunks`), `app/src/AppRoutes.tsx:1-17` (all 13 routes imported eagerly).
Top-level imports include:
| Dep | Used by | Why heavy |
|-----|---------|-----------|
| `three` | `RotatingTetrahedronCanvas.tsx` (Welcome only) | ~600 KB gzipped |
| `remotion` + `@remotion/player` + `@remotion/zod-types` | Mascot, MascotFrameProducer | ~250 KB combined |
| `react-joyride` | Walkthrough only | ~75 KB |
| `react-markdown` | Chat | ~60 KB + unified/remark dep tree |
| `lottie-react` | Mascot animations | ~250 KB |
| `socket.io-client` | Always on | ~100 KB |
| `redux-logger` | Dev only, but is in `dependencies` not `devDependencies` | Tree-shaken via `IS_DEV` guard but still imported |
| `@sentry/react` | Telemetry | ~80 KB |
| `@noble/curves`, `@noble/secp256k1`, `@scure/bip32`, `@scure/bip39` | Wallet only | ~100 KB combined |
| `react-ga4` | Analytics | small but still ships |
**Fix**:
- `const Welcome = React.lazy(() => import('./pages/Welcome'))` etc. for every route in `AppRoutes.tsx`; wrap `<Routes>` in `<Suspense fallback={...}>`
- Dynamic `import('three')` inside `RotatingTetrahedronCanvas` so it ships only when the user lands on `/welcome`
- Dynamic `import('react-joyride')` inside `AppWalkthrough`
- `import('@noble/secp256k1')` inside wallet flows only
- `redux-logger` → move to `devDependencies` (it's in dependencies at `app/package.json:94`)
- Configure `build.rollupOptions.output.manualChunks` to extract `react-markdown`, `socket.io-client`, `@sentry/react` into separate vendor chunks for better caching
**Impact**: HIGH for cold-start of the welcome route in particular; bundle audit should show 30-50% reduction in initial JS payload.
#### 7.2 `redux-logger` in production `dependencies` — **LOW**
**File**: `app/package.json:94` — `"redux-logger": "^3.0.6"` is in `dependencies` (used only in dev per `store/index.ts:162`). Tree-shaken if the bundler proves the branch is dead, but `IS_DEV` is a runtime check — Vite *should* drop it via `import.meta.env.PROD` substitution, but worth confirming with a bundle analyzer.
**Fix**: Move to `devDependencies` AND guard the import: `if (IS_DEV) { const { createLogger } = await import('redux-logger'); ... }`.
#### 7.3 `nodePolyfills` includes `crypto` and `stream` — **MEDIUM**
**File**: `app/vite.config.ts:132-139` — `nodePolyfills` adds shims for `buffer`, `process`, `util`, `os`, `crypto`, `stream`. The wallet code (`@noble/*`) uses these for compatibility but the polyfills add weight to the main bundle if not tree-shaken precisely. Worth a `vite-bundle-visualizer` pass.
---
### 8. I/O patterns
#### 8.1 Audio binary → `number[]` → JSON → IPC — **HIGH**
**File**: `app/src/pages/Conversations.tsx:823` + `app/src/utils/tauriCommands/voice.ts:155-165`
```tsx
const audioBytes = Array.from(new Uint8Array(await blob.arrayBuffer()));
// ...
openhumanVoiceTranscribeBytes(audioBytes, extension, context); // number[] across IPC
```
A 30-second 48 kHz 16-bit mono blob ≈ 2.9 MB binary. `Array.from(new Uint8Array(...))` produces a JS array where each byte becomes a Number (boxed, ~8 bytes each in V8's small-int form, but the array literal expansion for JSON serialization writes `[0,12,255,...]` — typically ~4 chars/byte) → ~12 MB JSON string. For 1-minute clips this is 25 MB+. Then crosses the Tauri IPC bridge (postMessage), gets JSON-parsed on the Rust side.
**Fix**: Use Tauri's binary IPC path — write the blob to a tempfile via `convertFileSrc` + `writeFile` plugin, then pass the path (this codebase already has `openhumanVoiceTranscribe(audio_path: ...)` at `voice.ts:150` — use it instead of `*Bytes`). Or use the Tauri 2.x `ArrayBuffer` transport directly without the `Array.from` round-trip.
**Impact**: HIGH — for a 60-second recording this swings from ~25 MB JS heap allocation + 25 MB IPC + 25 MB JSON parse on Rust side down to a tiny path string. Also cuts perceived latency by ~100-500 ms.
#### 8.2 `serde_json::Value` on hot LLM paths — **LOW**
**File**: `src/openhuman/inference/provider/traits.rs:296-301` — Anthropic/Gemini/OpenAI tool definitions stored as `Vec<serde_json::Value>`. `serde_json::Value` is convenient but allocation-heavy (every string is a `String`, every object a `Map<String, Value>`). For frequent serialization of tool specs at chat turn start, prefer typed structs.
Not urgent — tool sets are typically <20 entries and serialized once per turn.
---
## Bundle / dependency size observations
(Confirmed via reading `app/package.json` and grep; no `npm` install was run.)
Heavy deps shipping in main bundle (no code-splitting):
- `three` (~600 KB gz) — used only on `/welcome` (`RotatingTetrahedronCanvas`)
- `remotion` + `@remotion/player` + `@remotion/zod-types` (~250 KB gz) — used only on `/human` (Mascot)
- `lottie-react` (~250 KB gz) — used in Mascot
- `react-joyride` (~75 KB gz) — used only when walkthrough is active
- `@noble/curves` + `@noble/secp256k1` + `@scure/bip32` + `@scure/bip39` (~100 KB gz combined) — wallet only
- `react-markdown` + remark/rehype/unified tree (~60 KB gz) — chat (always needed, but message components not memoized — see §4.1)
- `socket.io-client` (~100 KB gz) — always-on
- `@sentry/react` (~80 KB gz) — telemetry
- `redux-logger` (~5 KB but should be devDep)
Cargo / Rust:
- `whisper-rs = "0.16"` (with `metal` on macOS) — large native dep; pinned per-platform, that's correct
- `socketioxide` server + `socket.io-client` browser — full bidi socket support, expected for the use case
- `sentry = "0.47.0"` — `default-features = false` already with curated feature set (the Cargo comment notes the actix bloat the team explicitly avoided — good hygiene)
- `matrix-sdk`, `whatsapp-rust`, `fantoccini`, `pdf-extract` all properly behind feature flags — good
- The `html2md` removal note (`Cargo.toml:36-45`) documents an exemplary perf fix (894 MB heap → linear-time stripper). Keep this culture.
No build-time embedded models found (good — `whisper-rs` bundles the runtime but not the model weights).
---
## Top 10 quick wins (high impact, low effort)
| # | Fix | File:Line | Effort | Impact |
|---|-----|-----------|--------|--------|
| 1 | Wrap WhatsApp `upsert_messages_inner` / `upsert_chats_inner` in `conn.transaction()` + `prepare_cached` | `src/openhuman/whatsapp_data/store.rs:180-230` and `:136-164` | 1 hr | 20-100× faster bulk ingest |
| 2 | Memoize `Conversations.tsx` derived state (`visibleMessages`, `latestVisibleAgentMessage`, `activeToolTimelineEntry`) in `useMemo`; replace `[...arr].reverse().find()` with `findLast()` | `app/src/pages/Conversations.tsx:1036-1047` | 30 min | Cuts per-keystroke allocs; smoother streaming UI |
| 3 | Wrap `BubbleMarkdown` and `AgentMessageBubble` in `React.memo` | `app/src/pages/conversations/components/AgentMessageBubble.tsx:40,80` | 15 min | 50× cheaper streaming render for long chats |
| 4 | Switch voice transcribe call from `*TranscribeBytes(number[])` to file-path version | `app/src/pages/Conversations.tsx:823-833` | 30 min | Cuts ~25 MB allocation per 1-min recording |
| 5 | Add `tokio::task::spawn_blocking` wrapper around `sync_vault`'s walk + read body | `src/openhuman/vault/sync.rs:78-260` | 30 min | Stops vault sync from starving tokio workers |
| 6 | Lazy-load route components in `AppRoutes.tsx` via `React.lazy` + `Suspense` | `app/src/AppRoutes.tsx:1-17` | 1 hr | 30-50% smaller initial bundle, faster cold start |
| 7 | Dynamic-import `three` inside `RotatingTetrahedronCanvas` (used only on `/welcome`) | `app/src/components/RotatingTetrahedronCanvas.tsx:3` | 30 min | Drops ~600 KB from main bundle |
| 8 | Parallelize embedding calls during seal/digest with `stream::iter(...).buffer_unordered(8)` | `src/openhuman/memory/tree/tree_global/digest.rs`, `tree_source/bucket_seal.rs`, `tree_global/seal.rs` | 2 hrs | ~8× faster batch embeds; HIGH for backfills |
| 9 | Route Linq + Telegram channels through `build_runtime_proxy_client` instead of bare `reqwest::Client::new()` | `src/openhuman/channels/providers/linq.rs:25`, `telegram/channel_core.rs:32` | 30 min | Shared connection pool + proxy support consistency |
| 10 | Move `redux-logger` to `devDependencies` and dynamic-import in dev | `app/package.json:94`, `app/src/store/index.ts:2` | 15 min | Smaller prod bundle; guarantees no shipping |
---
## What's already good (worth preserving)
- `build_runtime_proxy_client` HTTP client cache (`src/openhuman/config/schema/proxy.rs:438-451`)
- `GrepTool` correctly uses `spawn_blocking` (`src/openhuman/tools/impl/filesystem/grep.rs:118-122`)
- SSE streaming with `bytes_stream` (`src/openhuman/inference/provider/compatible.rs:911`) — not buffering full response
- `WebhookRouter::persist` offloads to `spawn_blocking` when in tokio runtime (`webhooks/router.rs:547-558`)
- `RoutingHealthChecker` caches `Ollama /api/tags` probe results with TTL (`src/openhuman/routing/health.rs:74-99`)
- Static `Lazy<Regex>` everywhere for redaction patterns (`memory/safety/mod.rs:48-141`, `memory/tree/score/extract/regex.rs`)
- `Rc<str>` for shared markdown heading in `Chunk` (`memory/chunker.rs:20`) — good memory-conscious choice
- `memory/tree/store.rs:323,1218` use `conn.unchecked_transaction()` for batch writes — proves the team knows the pattern, just hasn't applied it to whatsapp_data/vault stores
- `Cargo.toml` comments documenting the `html2md` removal and Sentry feature pruning — institutional perf memory
---
## Performance posture: **5 / 10**
- **+** Structured streaming, careful HTTP client caching, explicit spawn_blocking in some hot tools, regex statics, documented perf wins.
- **−** No connection pool, no bulk transactions in 2 of the busiest stores, no React memoization, no code-splitting, blocking I/O inside one major `async fn`, audio IPC anti-pattern that allocates ~10× the necessary memory, sequential embedding during ingest.
The fixes are well-bounded — none require architectural rework. The top-10 quick wins above are mostly mechanical and should land in two engineer-days of focused work. Once #1, #2, #3, #4, and #6 ship, the score moves to ~7-8 with no other changes.
docs/qe-reports/openhuman/04-qx-experience.md
+270
View File
@@ -0,0 +1,270 @@
# OpenHuman — Quality Experience (QX) Analysis
**Subject:** OpenHuman desktop AI assistant
**Repo path:** `/tmp/openhuman` (commit at time of audit: see `.git/HEAD`)
**Surface analyzed:** React 19 / TypeScript frontend under `app/src/` (382 `.tsx` files), Tauri shell, README, install scripts, gitbook docs, `.env.example`.
**Method:** static read of source, no runtime execution. Findings quote `file:line` where load-bearing.
**Stance:** opinionated. This is a product review through a QA + UX lens, not a code review.
---
## Executive Summary
OpenHuman is unusually mature for an "Early Beta" desktop AI app. The product has a coherent, opinionated point of view (privacy-first, local-by-default, mascot-led) and that point of view is *consistently expressed in code*, not just on the README. The team has clearly thought about the things most AI desktop apps fumble — error recovery, consent, multi-language users, boot stalls, mic permissions, OS indicators.
Where it falls down is **breadth of polish vs. ambition**. The product markets itself in five languages on the README but only ships translations for six (and not the same six). It funnels everything through a 2,125-line `Conversations.tsx` god-component with one Error Boundary at the root. Accessibility is patchy — strong in some surfaces (mic composer, error fallback, dialogs), weak in others (chat error banner has no `role="alert"`, language selector's accessible name is hardcoded English even when the UI is in Hindi).
The trust posture is the strongest part of the product. The `WhatLeavesLink` + `WhatLeavesMyComputerSheet` pattern, the explicit consent gate in `analytics.ts`, the honest "what leaves" copy that refuses to claim "100% local", and the Sentry `beforeSend` filter that strips PII at the boundary — these are signals of a team that takes "Private" seriously, not as a marketing word.
**Headline verdict:** the product has good bones and good intentions. The gaps are mostly fit-and-finish, not architectural. Most of them are 1-day fixes that would push the experience meaningfully forward.
**QX maturity score: 6.5 / 10** (rationale at the bottom).
---
## User Journey Snapshot
From `app/src/AppRoutes.tsx:1-143` and the Tauri shell, the in-app journey looks like this:
1. **First launch** → `BootCheckGate` (`App.tsx:107`) runs prerequisite checks → `PersistRehydrationScreen` (`components/PersistRehydrationScreen.tsx`) covers Redux rehydration with a **10-second deadline + recovery CTA** (`REHYDRATION_WARN_TIMEOUT_MS = 10_000`, line 14). Best-in-class boot UX detail.
2. **Welcome** (`/`) → `pages/Welcome.tsx`. Public route, redirects to `/home` if already authed.
3. **Onboarding** (`/onboarding/*`) → `pages/onboarding/Onboarding.tsx`. Linear stepper: `welcome → runtime-choice → (cloud → /home | custom → inference → voice → oauth → /home)`. Forcibly gated by `onboardingPending` in `App.tsx:121-138` — you cannot escape onboarding by URL-hacking.
4. **Home** (`/home`) → mascot, banners (`HomeBanners.tsx`), 3-way connectivity status (`Home.tsx:73-105`, internet vs core vs backend — *not* a conflated "offline").
5. **Chat** (`/chat`) → `pages/Accounts.tsx` (alias) — the heavy lifter is `pages/Conversations.tsx` (2,125 lines).
6. **Adjacent surfaces** → `/human` (mascot), `/intelligence` (memory tree), `/skills`, `/channels`, `/notifications`, `/rewards`, `/settings/*`, `/invites`.
7. **Walkthrough** — post-onboarding Joyride tour (`components/walkthrough/AppWalkthrough.tsx`), persisted via localStorage with try/catch around every storage call.
8. **Catastrophic crash** → `Sentry.ErrorBoundary` → `ErrorFallbackScreen.tsx`, which is exemplary: three recovery actions (Try Recover / Reload App / Download Latest), error name + message visible, component stack hidden in `<details>`. Self-contained (no Redux, no Router, no context) so a render error in any provider doesn't take the fallback with it.
The structure is sensible. The shape is "one big chat surface with sidebars + a settings flyout" — closer to Discord than to ChatGPT. There is no global keyboard shortcut surface I can see (no command palette wired into `CommandProvider` from a user-discoverable place), though `cmdk` is in dependencies, suggesting it exists somewhere.
---
## Findings by QX Dimension
### 1. Empty / Loading / Error States — **Good, but uneven**
**What works:**
- Empty states are i18n'd and have a `title + hint` pattern with a CTA seed:
- `components/intelligence/MemoryEmptyPlaceholder.tsx:13-15` — `memory.empty` + `memory.emptyHint`, meditative no-CTA approach (deliberate per the file comment).
- `components/notifications/NotificationCenter.tsx:201-202` — `notifications.center.empty` + `emptyHint`.
- `components/intelligence/IntelligenceTasksTab.tsx:140-142` — same pattern.
- `components/webhooks/WebhookActivity.tsx:38`, `components/skills/SkillResourceTree.tsx:85`, `components/webhooks/TunnelList.tsx:129`, `pages/Notifications.tsx:101` — empty states everywhere they're needed.
- Chat loading uses inline skeleton bars: `pages/Conversations.tsx:1506-1517` — alternating left/right pulse blocks that match the message layout. Good.
- Boot/rehydration: `PersistRehydrationScreen` with a 10s recovery deadline (`PersistRehydrationScreen.tsx:14, 47-50`). This is **better than most production apps**, which leave users staring at a frozen splash forever.
- Error → settings deep-link: when `sendError.code` matches a setup error, the banner offers a one-click jump to `/settings/voice` (`Conversations.tsx:1965-1980`). Errors point to the fix.
**What's missing:**
- Only **one** `Skeleton` usage across the whole app (`grep -rEn "Skeleton" app/src --include="*.tsx" | wc -l` → 1). There's no reusable `<Skeleton>` primitive in `components/ui/` (which contains only `Button`, `Card`, `Input`). Loading shimmers are inlined ad-hoc.
- Only **one ErrorBoundary**, at the app root (`App.tsx:84`). A render error inside `Conversations.tsx` (2,125 lines, complex state) crashes the entire app to the fallback screen. There are no route-level boundaries to keep the rest of the app alive.
- The chat-send error banner (`Conversations.tsx:1959-1988`) is rendered as `<p className="text-xs text-coral-500">` with **no `role="alert"`, no `aria-live`**. Screen readers will not announce it.
### 2. Accessibility (a11y) — **Above average for a beta, gaps are concentrated**
**Strong evidence:**
- 102 `aria-label`, 62 `aria-hidden`, 47 `aria-checked`, 17 `aria-modal`, 14 `aria-pressed`, 12 `aria-live`, 12 `aria-labelledby` (`grep -rEoh 'aria-[a-z]+' app/src --include="*.tsx" --include="*.ts"`).
- 18 `role="dialog"`, 14 `role="switch"`, 11 `role="status"`, 9 `role="alert"`, 4 `role="radiogroup"` + 4 `role="radio"`, 3 `role="tab"` + 2 `role="tablist"`, 2 `role="progressbar"`.
- **Zero `onClick` on `<div>`** across `app/src`. This is the single biggest a11y anti-pattern in React, and OpenHuman avoids it completely. Everything clickable is a `<button>`.
- `components/ui/Button.tsx:14-16` has `focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-primary-500/25 focus-visible:ring-offset-2` baked into BASE — keyboard focus is **always visible**, no exceptions.
- Chat send button is properly labeled: `Conversations.tsx:2030-2031` — `aria-label={t('chat.send')} title={t('chat.send')}`.
- Mic composer has the correct labels and toggles: `features/human/MicComposer.tsx:437,452` — `aria-label="Microphone device"` (selector) and `aria-label={isRecording ? t('mic.stopRecording') : t('mic.startRecording')}` (button).
- Sentry ErrorBoundary fallback uses a `<details>` for the stack trace (`ErrorFallbackScreen.tsx:71-80`) — keyboard-operable and progressively disclosed.
**Gaps:**
- `components/ui/Input.tsx:20-23` has an `invalid` prop that styles a red border but **does not propagate `aria-invalid`**. Screen-reader users have no signal that the field is invalid. One-line fix.
- `components/LanguageSelect.tsx:40,49` — `ariaLabel = 'Language'` default is **hardcoded English**, never localized. A Hindi user on this picker hears "Language" announced in English. Should be `t('settings.language.ariaLabel')`.
- Chat send-error banner (`Conversations.tsx:1959-1988`) has no `role="alert"` / `aria-live="polite"`. SR users miss every send failure.
- Only **5 files use `tabIndex`** — most components rely on natural tab order, which is fine, but a few custom widgets (e.g. the streaming preview bubble) may not be keyboard-reachable. Worth a focused audit.
- No skip-to-content link visible at the app root (`App.tsx:209-237`) — desktop apps get away with this more than the web, but it's still a gap.
- No evidence of a high-contrast mode beyond Tailwind dark-mode classes. Color-only signals (e.g. `text-coral-500` for errors, `text-stone-400` for hints) need a secondary cue for users with low vision. The `ErrorFallbackScreen` does pair color with an icon + heading, so the pattern exists — it's just not enforced.
### 3. Internationalization (i18n) — **Ambitious infrastructure, dishonest marketing**
This is the most important QX finding in the report.
**What's there:**
- A custom i18n system (`app/src/lib/i18n/I18nContext.tsx`) with `useT()` hook, RTL handling (`I18nContext.tsx:38`, `RTL_LOCALES = ['ar']`), `<html lang>` + `dir` mirroring (lines 71-78), defensive CJS/ESM unwrap (lines 41-55), and English fallback for missing keys.
- 12 locales declared in `app/src/lib/i18n/types.ts:1-13`: `en, zh-CN, hi, es, ar, fr, bn, pt, ru, id, it, ko`.
- **Real, populated translation catalogs** in `app/src/lib/i18n/chunks/`. Key counts (`grep -c "': '"`):
- `en`: 1,739 keys
- `zh-CN`: 1,848 keys
- `ar`: 1,792 keys
- `es`: 1,732 keys
- `ko`: 1,687 keys
- `fr`: 1,530 keys
- Other locales (`bn, pt, ru, hi, id, it`): chunks exist, key counts not validated here but they exist in the build.
- `LanguageSelect.tsx:8-21` lists 12 languages with native labels (한국어, हिन्दी, العربية, বাংলা, etc.).
- `useT()` is widely adopted — seen in `WelcomeStep.tsx:10`, `ErrorFallbackScreen.tsx:29`, `RouteLoadingScreen.tsx:7`, `MicComposer.tsx:65`, `Home.tsx:54`, all settings panels, the empty-state components, etc. This is not aspirational i18n — it's wired in.
**The painful gap:**
- **The README is published in EN, ZH, JA, KO, DE** (`README.md:33`).
- **The app does not ship Japanese (`ja`) or German (`de`).** They are not in the `Locale` union type (`types.ts:1-13`), not in `LanguageSelect`'s `LOCALE_OPTIONS` (`LanguageSelect.tsx:8-21`), and there are no `ja-*.ts` or `de-*.ts` chunks under `app/src/lib/i18n/chunks/`.
- **The reverse is also true:** the app ships Arabic, Bengali, Hindi, Indonesian, Portuguese, Russian, Italian, and Spanish — none of which have README translations. A Bangla speaker who installs the app sees a localized UI but cannot read the README that brought them there.
This is a **classic marketing/product mismatch**. A German user reading `README.de.md` who installs the app gets an English UI with no German option in the picker. The trust cost of "you advertised support you don't have" is real.
**Other i18n bugs to fix:**
- `pages/Home.tsx:67` — `welcomeVariants` is hardcoded English: `` [`Welcome, ${userName} 👋`, `Let's cook, ${userName} 🧑‍🍳.`, `Time to Zone In 🧘🏻`] ``. A Korean user opening Home gets the localized nav and three English welcome variants typed character-by-character.
- `LanguageSelect.tsx:40` — default `ariaLabel = 'Language'` (English, not localized).
- No `Intl.PluralRules` / ICU MessageFormat usage detected. Pluralization is faked with `.replace('{n}', String(min))` (`pages/Notifications.tsx:21-25`). This produces "1 minutes ago" in English and breaks worse in Russian/Polish/Arabic (which have multi-form plurals). The i18n script `scripts/i18n-coverage.ts` exists but no plural infrastructure.
- RTL is wired (`I18nContext.tsx:38`) but only Arabic is marked RTL. Hebrew, Persian, Urdu would all need to be added if they're ever shipped.
### 4. Trust & Transparency — **Strongest part of the product**
OpenHuman markets itself as "Private, Simple and extremely powerful" (`README.md:20`). Most products that say this are lying. OpenHuman appears to back it up.
- `features/privacy/whatLeavesItems.ts:11-31` — the file is *labeled in code* as the "honest list" with the comment `// Copy source: repo README + handoff doc. Do not soften this list — the point is to not lie about "100% local".` That is a team that's internalized the trust posture.
- The three items: Cloud AI Inference (only when a feature needs it), Third-party integrations (only with permission), Sentry + GA (opt-out, no PII, no content). Headline: `"Local by default. Cloud when you ask."` (line 32).
- `features/privacy/WhatLeavesLink.tsx` — a *reusable* privacy disclosure trigger that the onboarding `WelcomeStep.tsx:24-26` opts into. The principle is "invisible when not needed, one click away when it is" (file comment line 11-13). This is a UX pattern more apps should copy.
- `services/analytics.ts:5-26` documents the privacy guarantees in the module docstring:
- Sentry: no breadcrumbs / extras / contexts, no frame-locals, no source-context, anonymous user id only, `sendDefaultPii: false`.
- GA4: explicit allowlist (`GA_ALLOWED_EVENTS`), no content/messages/credentials/PII, ad personalization off, skipped in dev.
- Consent gating: `analytics.ts:47` — `let gaEnabled = false;` (default-deny). `setAnalyticsConsent` re-syncs both Sentry and GA (lines 215-235). Toggleable from Settings → Privacy & Security.
- `SECURITY.md` is short but complete: scope, disclosure email path, safe harbor, OS-level credential storage (Keychain / Windows Credential Manager), "message content is processed on request and not retained for training or long-term storage".
The one thing missing is a per-message "this went to the cloud" indicator. The `WhatLeavesLink` tells you *that* cloud calls happen; it doesn't tell you *when this specific reply* used one. For an app whose differentiator is "local by default", showing the user which messages stayed local vs. used cloud inference would close the loop.
### 5. Onboarding & Discoverability — **Short and gated, with one big gotcha**
- The flow is short: `welcome → runtime-choice → (cloud → home | custom → inference → voice → oauth → home)` (`pages/onboarding/Onboarding.tsx:18-26`). Cloud users get one click to working.
- `OnboardingNextButton` is a single component shared across steps — consistent visual rhythm.
- `WelcomeStep.tsx:24-26` puts `WhatLeavesLink` directly under the welcome CTA. Setting the privacy frame *before* the user hands over an OAuth token is exactly the right placement.
- Hard gate: `App.tsx:121-138` forces any non-`/onboarding` route back to onboarding while incomplete. You cannot skip it by URL. Once complete, `/onboarding` redirects to `/home`. Idempotent and safe.
- Post-onboarding Joyride walkthrough (`AppWalkthrough.tsx`) for *existing-user upgrades* (`isWalkthroughPending(userIsOnboarded)`, line 21-29). Migration from non-walkthrough versions is handled. Try/catch around every localStorage call (lines 25-29, 47-51, 60-66) — robust to private-browsing / quota.
- README install path: one curl/irm command (`README.md:53-60`) or a website download. **Friction here is low** — the `.env.example` (260 lines) is a contributor concern, not a desktop-user concern. Desktop users never touch it.
**Friction points:**
- `RuntimeChoicePage` (custom route → 4 sub-pages: inference, voice, oauth, ~~search~~, ~~memory~~) shows the team has *already cut* steps from the custom flow. Two more are commented out in `Onboarding.tsx:9-11,38-39`. Cutting custom-flow steps is the right instinct.
- The "Configure later" callout exists (`components/ConfigureLaterCallout.tsx`) — good escape hatch.
- The README's "first 5 minutes" is reasonable but the comparison table (`README.md:122-133`) is a marketing artifact, not a getting-started cue. A newcomer wanting to *try the thing* has to scroll past Discord/Reddit/X/Docs links, badges, a feature dump, and a competitor comparison before seeing the install command. That's normal for OSS but it's not lean.
**Beta posture (good):** `README.md:37,47` — both a badge and a callout: `> **Early Beta**: Under active development. Expect rough edges.` Honest. Sets expectations.
### 6. Error Messaging Quality — **Structured at the source, mediocre at the surface**
The structured-error pattern (`chat/chatSendError.ts:1-25`) is excellent:
```ts
export type ChatSendErrorCode =
| 'socket_disconnected' | 'local_model_failed' | 'cloud_send_failed'
| 'voice_transcription' | 'stt_not_ready' | 'voice_synthesis' | 'tts_not_ready'
| 'microphone_unavailable' | 'microphone_recording' | 'microphone_access'
| 'voice_playback' | 'safety_timeout' | 'usage_limit_reached'
| 'prompt_blocked' | 'prompt_review';
```
- Stable `code`s mean tests can assert error states deterministically (and analytics can aggregate them).
- The chat-send error banner (`Conversations.tsx:1961`) renders `data-chat-send-error-code={sendError.code}` — testable from outside.
- Contextual recovery: voice/STT/TTS errors render a "Setup" CTA that navigates to `/settings/voice` (`Conversations.tsx:1965-1980`).
- `ErrorFallbackScreen.tsx:35-36, 64-69` shows the actual `errorName` and `errorMessage` to the user — not a generic "Something went wrong." That's honest, and useful for users filing issues.
But:
- The banner is just `<p className="text-xs text-coral-500">`. No `role="alert"`. SR users miss the failure entirely (covered in §2).
- Some `setSendError` calls pass raw English strings instead of i18n keys: `Conversations.tsx:910` (`'Microphone recording failed.'`), `:925` (`` `Microphone access failed: ${message}` ``), `:961` (`'Failed to play voice reply.'`). A Korean user sees Korean UI, then an English error.
- The `componentStack` shown in `ErrorFallbackScreen` is dev-friendly text. For a non-engineer who hits a render crash, seeing `at AppShell at App at Provider` is noise. Hide it behind "Show technical details" with friendlier framing.
### 7. Performance Perception — **Good streaming bones, missing skeletons**
- Streaming is wired: `streamingAssistantByThread` (`Conversations.tsx:269-270`), inference lifecycle (`'started' → 'streaming'`) tracked per-thread (`:772`, `:1103`). The streaming preview bubble takes over from the 3-dot placeholder once tokens arrive (`:1721-1723`).
- Loading skeletons exist for chat history (`Conversations.tsx:1506-1517`).
- Optimistic UI for user-sent messages isn't explicitly visible in the snippets read, but `streamingAssistantByThread` + a `pendingTurn` pattern is the right shape.
- The `Sentry.ErrorBoundary` is split out into a standalone screen that *doesn't* depend on Redux, Router, or i18n provider (`ErrorFallbackScreen.tsx:8-12` comment). That means even a Provider-time crash renders correctly.
Missing:
- No global `<Skeleton>` primitive. Loading shimmers are inlined per-feature.
- The streaming preview suppresses the 3-dot placeholder once a token arrives (`Conversations.tsx:1721-1723`), but the transition from "nothing" to "first token" is the longest-perceived part of a chat turn and could use a discrete "thinking" affordance. The mascot animation may already cover this — not verified from code alone.
### 8. Documentation Experience — **Forking paths, mostly clear**
- `gitbooks/SUMMARY.md` is a proper TOC: Overview, Features (12+ subpages), Developing (10+ subpages), Legal. 48 markdown files total.
- `CONTRIBUTING.md` (320 lines) for general contributors; `CONTRIBUTING-BEGINNERS.md` (378 lines) for first-timers, with a copy-paste AI-agent prompt option (`README.md:81`). This is a thoughtful split — most projects have one wall-of-text CONTRIBUTING that scares new contributors off.
- `CLAUDE.md` (25KB) sits at the root for AI-coding-agent guidance. Whether you like that or not is a values question; from a QX perspective it signals the team is intentional about AI-assisted contribution.
- The README itself does triple duty: marketing pitch, install instructions, contributor quickstart, competitive comparison. It's long (12.6KB) and the "I want to try this" person has to skim past a lot of context to find the curl line.
**Friction:**
- `docs/` (repo-internal) vs `gitbooks/` (user-facing) are not labeled as such anywhere I can see. A new user clicking into `docs/PROMPT_INJECTION_GUARD.md` or `docs/AGENT_SELF_LEARNING.md` is getting team-internal notes, not product docs. The split is visible to readers but not signposted.
- No standalone "Quickstart" page in `gitbooks/overview/` (just `getting-started.md`). The README *is* the quickstart, and it's overloaded.
### 9. Sample Component Polish
Quick rotation of components, rated 1-5 for fit and finish:
| File | Lines | Polish | Notes |
|------|-------|--------|-------|
| `components/ErrorFallbackScreen.tsx` | 105 | 5/5 | Icon + heading + subhead + hint + collapsible stack + three action buttons. Self-contained. |
| `components/PersistRehydrationScreen.tsx` | ~120 | 5/5 | 10s timeout → recovery CTA. Defensive against stuck boots. |
| `components/walkthrough/AppWalkthrough.tsx` | ~200 | 4/5 | Try/catch around all storage; clean Joyride wiring. Wins by being defensive. |
| `components/ui/Button.tsx` | 67 | 4/5 | 4 variants × 5 sizes, focus-visible ring built in. No `loading` prop or `aria-busy` — every caller renders its own spinner (see chat send button, `Conversations.tsx:2037-2052`). |
| `components/ui/Input.tsx` | 36 | 3/5 | `invalid` prop styles but doesn't propagate `aria-invalid`. No label association helper. |
| `components/LanguageSelect.tsx` | 60 | 3/5 | Native labels + flags is great. Hardcoded English `aria-label` default and 12 languages vs. README's 5 markets is a coordination smell. |
| `features/human/MicComposer.tsx` | ~470 | 4/5 | Releases mic on unmount (`disposedRef` lines 79-95), guards re-tap during `getUserMedia`, prefers AAC-in-MP4 with documented reason. Comments explain the *why*. |
| `pages/Conversations.tsx` | 2,125 | 2/5 | God-component. Mixes thread list, composer, streaming, error UI, voice, agent-profile editing, kanban. The pieces *inside* are well-built; the file is just too large. |
| `pages/Home.tsx` | 80+ | 3/5 | Hardcoded English `welcomeVariants` (line 67) in an otherwise i18n'd page. 3-way connectivity status is the redeeming detail. |
---
## Top 10 QX Improvements, Ranked by User Impact
| # | Improvement | User Impact | Effort | Where |
|---|-------------|-------------|--------|-------|
| 1 | **Reconcile README languages with app languages.** Either ship `ja-*.ts` + `de-*.ts` chunks (and add to `Locale` type + `LanguageSelect`), or remove the JA/DE README translations and the language badges. Add README translations for the locales that exist (AR, ES, PT, BN, HI). | A user installs in their language because the README promised it. Today: trust hit on first launch. | Add chunks: ~1 week per language (LLM-translate + review). Or trim README: 1 day. | `README.md:33,42-46` + `app/src/lib/i18n/types.ts` + `LanguageSelect.tsx:8-21` |
| 2 | **Add `role="alert"` / `aria-live="polite"` to the chat send-error banner.** SR users currently get no signal when a send fails. | Every blind / low-vision user who hits a send error is left wondering why nothing happened. | 1-line. | `pages/Conversations.tsx:1959-1963` |
| 3 | **Add route-level Error Boundaries.** One Sentry boundary at the root means a render error inside `Conversations.tsx` blanks the whole app. Wrap each route in `AppRoutes.tsx` with a smaller boundary. | A crash on /chat still lets the user reach /settings or /home to recover. Today: full app reload. | Half-day. Sentry's `withErrorBoundary` HOC + a route-scoped fallback. | `app/src/AppRoutes.tsx`, new `components/RouteErrorBoundary.tsx` |
| 4 | **Localize the hardcoded `welcomeVariants` on Home.** A non-English user sees "Welcome, 김민준 👋" then "Let's cook…" typed character by character. Move to `t('home.welcomeVariant.1')`, etc. | Daily friction for every non-English user. | 1 hour. | `pages/Home.tsx:67` |
| 5 | **Decompose `Conversations.tsx`.** 2,125 lines is a maintainability and quality risk. Extract `ChatComposer`, `ChatErrorBanner`, `MessageList`, `AgentProfileEditor`, `KanbanPanel`. Each extracted piece gets its own route-level Error Boundary too. | Improves stability and lets each surface get tested independently. Indirect but large. | 2-3 days. | `pages/Conversations.tsx` |
| 6 | **Propagate `aria-invalid` from `Input`'s `invalid` prop.** Currently the prop styles a red border with no SR signal. | All form errors silently invisible to SR users. | 1-line. | `components/ui/Input.tsx:20-23` |
| 7 | **Localize chat error messages.** `Conversations.tsx:910,925,961` pass raw English strings to `setSendError`. Replace with `t()` keys. | Non-English users see EN errors interrupting an otherwise localized flow. | Half-day (find all hardcoded `setSendError(... 'string' ...)` calls). | `pages/Conversations.tsx:910,925,961` and similar |
| 8 | **Per-message "stayed local / used cloud" indicator.** OpenHuman markets "local by default, cloud when you ask" but the user has no per-turn proof. A small badge on each assistant message (🏠 local / ☁ cloud) closes the loop. | Converts trust-by-promise into trust-by-evidence. Differentiating. | 2-3 days (the routing layer already knows). | New `components/chat/InferenceProvenanceBadge.tsx` |
| 9 | **Add a real `<Skeleton>` primitive in `components/ui/` and migrate inline skeletons to it.** Currently 4 inlined skeleton bars (`Conversations.tsx:1508-1516`) and no shared component. Other surfaces (Notifications, Memory, Skills) just go from spinner → content with no transition. | Perceived performance + consistent visual rhythm. | 1 day. | New `components/ui/Skeleton.tsx` |
| 10 | **Localize `LanguageSelect`'s `ariaLabel`.** The accessible name of the language picker is hardcoded English. A Bangla SR user hears "Language combobox" announced in English while the rest of the UI is Bangla. | Small, but symbolic — it's the *language picker*, of all things. | 1-line. | `components/LanguageSelect.tsx:40,49` |
---
## Trust / Privacy Posture Assessment
**Score: 9/10 — best-in-class for the category.**
| Dimension | Evidence | Verdict |
|-----------|----------|---------|
| Honest marketing copy | `features/privacy/whatLeavesItems.ts:18` — "Core assistant features run locally by default. Cloud inference is only used when a feature explicitly needs stronger hosted models or network-backed services." Refuses the "100% local" lie. | Strong |
| Default-deny consent | `services/analytics.ts:47` — `let gaEnabled = false;`. GA + Sentry both skip until consent is set (lines 122-145, 215-235). | Strong |
| PII stripping at the boundary | `services/analytics.ts:7-13` — Sentry `beforeSend` strips breadcrumbs, extras, contexts, source-context, anonymizes user. `sendDefaultPii: false`. | Strong |
| Allowlist not denylist | `GA_ALLOWED_EVENTS` Set (`analytics.ts:64+`) — only declared events ship to GA. Anything else dropped with a warning. | Strong |
| In-product disclosure | `features/privacy/WhatLeavesLink.tsx` — reusable inline trigger, shown at onboarding welcome (`WelcomeStep.tsx:24-26`). Comment line 11-13: "Invisible when not needed, one click away when it is." | Strong |
| Toggle-able in Settings | Mentioned in `whatLeavesItems.ts:26` ("Toggle anytime in Settings → Privacy & Security"). Need to verify the toggle is actually wired and respected. | Verified via `setAnalyticsConsent` flow. |
| Per-message provenance | No per-turn local/cloud indicator visible. | **Gap** — see improvement #8. |
| Crash-report opt-out before first crash | Sentry initializes pre-consent for the **smoke-test event** only (`SENTRY_SMOKE_TEST`, gated explicitly). Real errors gated on consent. | Strong, but worth verifying the smoke-test bypass cannot leak. |
| OS-level credentials | `SECURITY.md:48-52` — macOS Keychain, Windows Credential Manager. No plain-text secrets claim. | Strong |
| Content retention | `SECURITY.md:50` — "Message content is processed on request and not retained for training or long-term storage." | Strong claim. (Out of scope to verify here.) |
The team understands trust UX. The one missing piece is **per-message provenance** — turn the abstract promise into a per-turn observable.
---
## QX Maturity Score: 6.5 / 10
**Breakdown:**
| Dimension | Score | Why |
|-----------|-------|-----|
| Boot & recovery UX | 9/10 | 10s rehydration deadline, structured `ErrorFallbackScreen`, defensive localStorage in walkthrough. Best-in-class. |
| Onboarding | 7/10 | Short, gated, escape hatch present. Privacy framing in the right place. Step count already reduced once. |
| Empty / loading / error states | 7/10 | Empty states are i18n'd and pervasive. Skeleton is inlined ad-hoc. Error UI doesn't announce to SR. |
| Accessibility | 6/10 | Zero clickable divs, real ARIA usage, focus-visible everywhere. Specific gaps: `aria-invalid`, `role="alert"` on send error, localized aria-labels. |
| Internationalization | 5/10 | Solid infrastructure, real catalogs in 6+ languages. Marketing/product mismatch (DE/JA missing, but advertised). Hardcoded EN strings in Home, error messages, aria-labels. No plural-form handling. |
| Error messaging | 7/10 | Structured error codes with contextual recovery CTAs. Some unlocalized raw strings leak through. SR-invisible. |
| Performance perception | 7/10 | Streaming wired, inline skeleton for chat history. No shared `<Skeleton>`. |
| Trust & privacy posture | 9/10 | Honest copy, default-deny, PII stripped, allowlist GA, in-product disclosure. Missing per-message provenance. |
| Documentation experience | 6/10 | Two CONTRIBUTING tiers is smart. README is overloaded. `docs/` vs `gitbooks/` boundary unsignposted to outsiders. |
| Component polish | 6/10 | UI primitives are minimal (Button, Card, Input). `Conversations.tsx` is a 2.1k-line god component. Most other components are clean. |
**Weighted overall: 6.5 / 10.** This is a *high* score for an "Early Beta" — most apps at this label sit at 3-4. OpenHuman's distinctive strengths are the trust posture and the resilient boot UX. Its distinctive weakness is the gap between i18n ambition and i18n delivery, compounded by one big component that's accreting features faster than it's being decomposed.
**Path to 8/10:** ship items 1-7 from the improvement list. None of them are architectural rewrites. The product is already most of the way there — what's left is fit-and-finish.
---
## Notes on Method
- All findings are based on static reading of source files in `/tmp/openhuman`. No runtime execution.
- File references use the project-relative path under `/tmp/openhuman/`. Line numbers reflect the snapshot read at the time of analysis.
- "Polish" ratings are subjective and based on a 5-10 file sample per surface — they signal direction, not certainty.
- The "QX maturity score" weighting is opinion. Reasonable QX reviewers could land 0.5-1.0 in either direction.
- No browser automation was used (the report is a static-only audit). The next pass should drive the actual desktop binary with Vibium or Appium and verify keyboard-only navigation, screen-reader announcement order, and the per-locale visual rendering.
docs/qe-reports/openhuman/05-product-factors-sfdipot.md
+479
View File
@@ -0,0 +1,479 @@
# 05 – OpenHuman: Product Factors (SFDIPOT) Test Strategy
**Framework:** James Bach's Heuristic Test Strategy Model (HTSM) — SFDIPOT product factors
**Project:** OpenHuman (`/tmp/openhuman`)
**Version observed:** 0.54.3 (core + Tauri shell), README badge: "Early Beta"
**Assessor scope:** Configs, workspace layout, CI workflows, top-level subsystem directories. Source files were not deeply read — risks are inferred from product surface.
---
## Executive Summary
OpenHuman is an **agentic local-first desktop assistant** built as a single Tauri 2.x application on a vendored **Chromium Embedded Framework (CEF) runtime**, with a **Rust core** (`openhuman-core`, ~95 subsystem dirs) embedded **in-process** in the Tauri shell. The shell exposes a **JSON-RPC server on `127.0.0.1:7788`** (also reachable in headless Docker mode) plus a **Socket.IO** event stream and a **WebSocket bridge to CEF** for connector recipes (Gmail, WhatsApp Web, Slack, Discord, iMessage, Google Meet). Persistent state lives in **bundled SQLite** (`rusqlite`) under a per-user workspace, with **AES-GCM / ChaCha20-Poly1305 + Argon2** at-rest encryption and OS-keychain credentials. The app ships **118+ OAuth integrations** (via Composio + native scanners), an **MCP client and server**, a **QuickJS skills sandbox**, **local AI** (Whisper STT, Piper TTS, Ollama/LM Studio), **embeddings + vector store** for the Memory Tree, a **cron-based scheduler**, a **wallet** (EVM/BTC/Solana/Tron), and **auto-update** for both the Tauri bundle and the core. The product is multilingual (5 README locales, i18n CI gate), targets macOS/Windows/Linux desktops plus a self-hosted Docker server, and is openly labelled "expect rough edges." Existing tests: ~20 Rust integration tests, ~55 WDIO/Appium E2E specs, frontend Vitest, installer Pester/bash tests, plus 20 CI workflows including a release pretest gate. Major **untested surface**: the QuickJS skills sandbox isolation, prompt-injection detector under adversarial inputs, wallet RPC against rogue chains, CEF intercept and credential-store recovery on locked keyrings, auto-fetch 20-minute loop under partial outage, and the proliferation of webview scanners (`*_scanner` modules) reading from third-party UIs that change without notice.
---
## S — Structure
### Observed (what the product is made of)
- **Cargo workspace root** with one root crate `openhuman` (v0.54.3) producing 5 binaries: `openhuman-core` (main JSON-RPC server), `slack-backfill`, `gmail-backfill-3d`, `memory-tree-init-smoke`, `inference-probe`.
- **`app/src-tauri/`** — the Tauri shell crate `OpenHuman` v0.54.3 (`staticlib`, `cdylib`, `rlib`), embedding `openhuman_core` *in-process* (Cargo.toml: `openhuman_core = { path = "../.." }`). Per the comment in `app/src-tauri/Cargo.toml`, sidecar was removed in PR #1061; core now runs as a tokio task inside the Tauri host.
- **`app/src-tauri/vendor/tauri-cef/`** — vendored fork of Tauri on a `feat/cef-notification-intercept` branch, plus a vendored `tauri-plugin-notification`. CEF version is pinned to `=146.4.1`.
- **`pnpm-workspace.yaml`** lists only `"app"` — the React/TS frontend (`app/src/`) is the lone JS workspace member; `remotion/` (mascot renderer) and `scripts/agent-batch/` have their own `package.json` outside the workspace.
- **Frontend** (`app/src/`): React 19 + Vite 8 + Redux Toolkit + Tailwind 3 + Sentry + Three.js (mascot 3D) + Remotion player + react-router-dom 7 + Tauri APIs.
- **`src/openhuman/`** has **~95 subsystem directories** including `agent/`, `agentmemory backend`, `audio_toolkit`, `autocomplete`, `billing`, `channels`, `composio`, `credentials`, `cron`, `desktop_companion`, `doctor`, `embeddings`, `encryption`, `inference`, `integrations`, `learning`, `mcp_client`, `mcp_server`, `meet`, `meet_agent`, `memory`, `migration`, `migrations`, `notifications`, `overlay`, `people`, `prompt_injection`, `referral`, `routing`, `runtime_node`, `runtime_python`, `scheduler_gate`, `screen_intelligence`, `security`, `service`, `skills`, `socket`, `subconscious`, `team`, `threads`, `todos`, `tokenjuice`, `tool_registry`, `tools`, `tree_summarizer`, `update`, `vault`, `voice`, `wallet`, `webhooks`, `webview_accounts`, `webview_apis`, `webview_notifications`, `whatsapp_data`.
- **`packages/`** — distribution packaging: `deb`, `homebrew`, `homebrew-core`, `npm`.
- **`remotion/`** — separate Remotion project for mascot asset rendering.
- **`scripts/`** — ~80+ utility scripts (release, debug-*, mock-api, agent-batch, deep-work, rabbit code-review CLI, weekly-code-review, install.sh/ps1, tauri DMG signing, sentry symbol upload).
- **Git submodules** declared (`.gitmodules`) — README step 2 says `git submodule update --init --recursive` is mandatory before `pnpm install`; the Tauri-CEF fork is one of them.
- Rust toolchain pinned to **1.93.0** with a comment: "Pin below 1.94 until matrix-sdk resolves recursion limit overflow"; Node ≥ 24; pnpm 10.10.0.
- `[patch.crates-io]` rewrites **all `tauri-*` crates** and `whisper-rs-sys` to forks/vendored paths. Plugins are pinned to a specific commit on `plugins-workspace@feat/cef`.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| S1 | **Single-process model with embedded core**: a panic in any of 95 subsystem dirs takes down the GUI and silences the JSON-RPC server. Sentry capture is best-effort. | H |
| S2 | **Vendored Tauri-CEF fork on `feat/cef-notification-intercept`** is critical-path: any upstream Tauri security fix must be manually back-ported. The fork sets `macOSPrivateApi: true` and has `--remote-debugging-port=9222` exposed to the bridge. | H |
| S3 | **Five binaries** (`openhuman-core`, `slack-backfill`, `gmail-backfill-3d`, `memory-tree-init-smoke`, `inference-probe`) each have independent CLI surfaces; only `openhuman-core` appears in `app/scripts/e2e-*`. The backfill binaries can mutate the same workspace SQLite while the desktop is running. | H |
| S4 | **CEF binary pin** `=146.4.1` exact. A forced-pinned binary blob means CVE response time for embedded Chromium is whatever the maintainers' upgrade cadence is. The `.github/workflows/tauri-cef-pin-guard.yml` indicates this is known-fragile. | H |
| S5 | **Optional features compile-gated** (`whatsapp-web`, `channel-matrix`, `peripheral-rpi`, `browser-native`, `rag-pdf`, `sandbox-landlock`) — feature matrix multiplies failure modes. Most CI runs likely use default features only. | M |
| S6 | **`e2e-test-support` feature** flips `openhuman.test_reset` RPC on — comment in Cargo.toml says shipped binaries don't have it, but the safety depends entirely on the build script never accidentally enabling it. | M |
| S7 | Submodules required before install means a fresh contributor clone without `--recursive` will produce a broken Cargo build with confusing errors. | L |
| S8 | `app/src-tauri/Cargo.lock` exists alongside the root `Cargo.lock`. Diverging lockfiles across two workspaces is a class of "works on my machine" supply-chain risk. | M |
### Test Ideas (Structure)
1. **Build the same SHA twice with and without `--recursive`** submodule fetch; capture the failure mode and time-to-failure of `cargo check -p openhuman --lib` to assess developer ergonomics.
2. **Force a panic inside `src/openhuman/voice/`** via a malformed Whisper model path and observe whether the JSON-RPC server on :7788 stays up, restarts, or wedges; capture Sentry breadcrumbs and core.token rewrite timing.
3. **Compile-matrix sweep**: run `cargo check` across all 6 optional features (`sandbox-landlock`, `channel-matrix`, `peripheral-rpi`, `browser-native`, `rag-pdf`, `whatsapp-web`) in pairs; count compile errors and unused-dependency warnings.
4. **Build a release binary with `e2e-test-support` accidentally enabled**, then attempt `openhuman.test_reset` over RPC from a third-party client; record whether anything other than the absent feature flag blocks the wipe.
5. **Run `slack-backfill` and the desktop concurrently** against the same workspace; trigger overlapping SQLite writes and inspect `PRAGMA integrity_check`, lock contention, and final row counts vs expected.
6. **Diff `Cargo.lock` (root) against `app/src-tauri/Cargo.lock`** for shared crates; flag any version skew, then try to reproduce the build from each lockfile in isolation.
7. **Upgrade the Tauri-CEF submodule pin by one commit on `feat/cef`** without changing the patch table; capture every callsite that fails to compile to size the upgrade surface.
---
## F — Function
### Observed (what the product does)
Inferred from the 95 subsystem dirs, README, and E2E spec names:
**User-facing capabilities** (one E2E spec per area, mostly):
- Chat & agent harness (send / stream / cancel / subagent / scroll / wallet flow).
- Onboarding (modes, judge, stress, chat).
- 118+ third-party integration connectors via Composio + per-service native scanners (Gmail, Slack, Discord, Telegram, WhatsApp Web, iMessage, Google Messages, Notion, Reddit, GitHub, Drive, Sheets, Facebook, Instagram).
- Google Meet *participation* — the mascot joins meetings as a real participant with fake-camera SVG-to-Y4M frames piped into CEF's `--use-file-for-fake-video-capture`, plus audio out via meet_audio/meet_call.
- Voice: STT via whisper-rs (Metal on macOS), TTS via ElevenLabs + Piper, dictation hotkeys, mascot lip-sync.
- Memory Tree (canonicalize → chunk → score → fold into hierarchical summaries) + Obsidian-compatible `.md` vault export.
- Skills system: discovery, install, OAuth, multi-round execution, socket-reconnect handling — registry can be remote URL or local dir.
- MCP client (stdio servers) and MCP server (exposing OpenHuman as MCP to other agents).
- Webhooks (ingress + tunnel), cron jobs, scheduler with battery/idle gating, subconscious background ticks every 20 minutes.
- Wallet: EVM, BTC, Solana, Tron — ABI execution, RPC fallback to public endpoints.
- Auto-update for Tauri bundle AND core in lockstep.
- Multi-locale UI (en, zh-CN, ja-JP, ko, de — checked by `pnpm i18n:check`).
- Native overlay window, command palette, autocomplete, screen intelligence (vision-based).
- Card and crypto payment flows (Stripe-ish + crypto).
**Internal capabilities**:
- JSON-RPC dispatch with bearer-token auth (`OPENHUMAN_CORE_TOKEN`), structured errors, RPC log.
- Socket.IO event bus over `socketioxide`.
- Model routing (low/medium/high tier presets) + provider quality + telemetry.
- TokenJuice compression layer for every tool call/scrape/email/search payload.
- Prompt-injection detector (`src/openhuman/prompt_injection/`).
- Tool registry with user-filter, orchestrator tools, schema validation.
- Encryption at rest (AES-GCM + Argon2 KDF + ChaCha20-Poly1305).
- Sandboxing options: Landlock (Linux), Bubblewrap, Firejail, macOS sandbox profiles, Docker — `src/openhuman/security/` has files for each.
- Migration framework with versioned migrations (`migrations/`, `migration/`).
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| F1 | **Prompt injection from any of 118+ integration data sources** (e.g., a Notion page, a Gmail email body, a Slack message) can hijack the agent. The detector is one Rust module against an unbounded adversarial surface. | H |
| F2 | **Auto-fetch every 20 minutes** silently pulls fresh data from every connected service into memory — a single misbehaving connector can DoS the device, fill disk, or leak data into Memory Tree. | H |
| F3 | **Wallet** with private-key signing in the same process as 118 OAuth tokens, embedded Chromium, and untrusted skills (QuickJS) — process compromise = wallet drain. Wallet defaults to public RPC endpoints (publicnode.com, blockstream.info, mainnet-beta.solana.com) which an attacker on the network path could MITM. | H |
| F3a | **Wallet `execution.rs` + `abi.rs`**: ABI decoding bugs or wrong-chain-id replay against forks can sign transactions the user did not intend. | H |
| F4 | **Skills runtime is QuickJS sandbox**; `src/openhuman/skills/inject.rs` exists. Sandbox-escape from skill code into core gives the skill full agent powers (memory, wallet, OAuth tokens). | H |
| F5 | **`SKILLS_LOCAL_DIR` and `SKILLS_REGISTRY_URL`** can point to a local file path or arbitrary HTTP URL — supply-chain attack vector for any user who copy-pastes an env var from a "tutorial". No signing observed for skills. | H |
| F6 | **Google Meet agent** sends *the user's* audio/video into Meet sessions and joins as a participant. A bug here = silent eavesdropping on the user's own meetings or unauthorized join. | H |
| F7 | **MCP server (`mcp_server/`)** exposes OpenHuman tools to other agents; if no authn or weak authn, any local process can drive OpenHuman. | H |
| F8 | **Update flow**: `OPENHUMAN_AUTO_UPDATE_RPC_MUTATIONS_ENABLED` toggles whether bearer-auth callers can invoke `update.apply` — the env-comment itself flags "disable on exposed server deployments." Default not visible from .env.example. | H |
| F9 | **Subconscious/scheduler ticks** keep the agent thinking in the background. A loop bug = CPU drain, battery destruction, or runaway LLM bill (every tick is a potential model call). | M |
| F10 | **Migrations** (`migrations/phase_out_profile_md.rs`, `retire_chat_v1_model.rs`, `unify_ai_provider_settings.rs`) run on workspace upgrade; a failed migration mid-flight on the bundled SQLite = data loss. No visible rollback story. | H |
| F11 | **TokenJuice** is run on *every* LLM input — a compression bug can corrupt prompts (lose CJK grapheme boundaries despite the claim, drop URLs, change PII handling) and the user never sees the original was mangled. | M |
| F12 | **Approval flow** (`src/openhuman/approval/`) is the last gate before destructive actions. Bypass = silent execution of arbitrary tools without user consent. | H |
### Test Ideas (Function)
1. **Inject a prompt-injection payload** into a Notion page, a Gmail subject line, and a Slack DM, then trigger auto-fetch; assert via `rpc_log` whether the agent attempts the injected action and whether `prompt_injection::detector` flagged it. Try base64-wrapped, zalgo-Unicode, and ASCII-art variants.
2. **Disconnect the network during an auto-fetch tick**, then drop a 50 MB attachment into Gmail's IMAP feed and reconnect; measure disk growth in `~/.openhuman`, memory-tree row count, and whether the next tick double-ingests.
3. **Sign a transaction with the wallet** while a MITM proxy returns a different chain-id from the configured `OPENHUMAN_WALLET_RPC_EVM`; observe whether the signed payload protects against replay (EIP-155) and whether the user sees the chain mismatch.
4. **Install a hostile skill** that calls `oauth.fetch` for a service the user did not authorize; assert the bearer-token scope-check (`session_support.rs`) rejects the call rather than returning the token.
5. **Set `SKILLS_REGISTRY_URL` to an attacker HTTP server** that serves a registry pointing to a malicious `index.js`; run `skill install`, then capture which paths in the workspace the skill touches.
6. **Start the desktop, then `curl -X POST http://127.0.0.1:7788/rpc -d 'update.apply'`** without the bearer token; capture status, and repeat with a stolen token from a different session — does revocation work?
7. **Schedule a Meet join while the user is in a separate Meet** call; observe whether the audio pipeline mixes streams, switches cameras, or refuses cleanly with a recoverable error.
8. **Send 1000 RPC frames in 10 seconds** to `/rpc` and `/socketio`; count rejected vs accepted, then assert no inbound frame survives in `rpc_log` with truncated/un-parsed JSON that could pivot to deserialization-confusion bugs.
9. **Run a migration mid-flight power kill**: start `openhuman-core run` while a `migrations/` step is executing, SIGKILL the process, restart and observe whether the SQLite is left mid-migration (corrupt) or rolls forward.
10. **Feed TokenJuice an input mixing 4-byte CJK, ZWJ emoji families, RTL Hebrew, and base64-encoded payloads**, then diff the model-bound prompt vs the original byte-for-byte; verify no graphemes are split and no URL is silently rewritten.
---
## D — Data
### Observed (what it stores and processes)
- **Bundled SQLite** via `rusqlite = { features = ["bundled"] }`. Single workspace dir (`~/.openhuman` or `~/.openhuman-staging` based on `OPENHUMAN_APP_ENV`).
- **Encryption at rest**: `aes-gcm`, `chacha20poly1305`, `argon2`, `sha2`, `hmac`, `ring`. Module `src/openhuman/encryption/` plus `ops.rs`. Memory backend likely encrypts blobs; key derivation via Argon2 from a user-supplied key or device-derived material.
- **`src/openhuman/credentials/`** — separate cred store (`core.rs`, `ops.rs`, `profiles.rs`, `responses.rs`, `schemas.rs`, `session_support.rs`). SECURITY.md says OS-level keychain (Keychain, Credential Manager). On Linux likely libsecret/`keyring`.
- **`src/openhuman/memory/`** has 7 modules: `conversations/`, `ingestion/`, `tree/`, `store/`, `safety/`, `schemas/`, `tool_memory/`, `stm_recall/`, `sync_status/`. Store split into `agentmemory/` (optional proxy backend) and `unified/` (local SQLite with `fts5`, `kv`, `graph`, `documents`, `events`, `segments`, `query`, `profile`).
- **Tree store**: `canonicalize`, `chunk`, `content_store`, `jobs`, `retrieval`, `score`, `tree_global`, `tree_source`, `tree_topic`. README claims ≤3k-token Markdown chunks.
- **Obsidian vault export** to `.md` files — second copy of memory data outside SQLite.
- **`postgres = "0.19"` dependency** in core Cargo.toml — surprising for a local-first app. Possibly used by `agentmemory` backend or for the optional cloud backend.
- **FTS5** virtual table (`memory/store/unified/fts5.rs`) for full-text search.
- **Embeddings** stored via `embeddings/store.rs` with providers (`cloud.rs`, `ollama.rs`, `openai.rs`, `noop.rs`).
- **Vault** module (`src/openhuman/vault/`) — likely secrets management separate from `credentials/`.
- **WhatsApp data** is mentioned in a dedicated subsystem (`whatsapp_data/`) — likely encrypted session blobs for the wa-rs client.
- **iMessage scanner** reads `~/Library/Messages/chat.db` *read-only* on macOS (per Cargo.toml comment).
- **Gmail-backfill** binary fetches and stores 3 days of mail; `slack-backfill` similar.
- Per-OS workspace path via `directories = "5"`/"6"`.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| D1 | **Single SQLite for everything** (memory, jobs, credentials cache, kv, graph, fts5, segments, events) — a corruption event = unrecoverable. No visible automatic backup story. | H |
| D2 | **Encryption key management**: Argon2 KDF parameters not visible from outside; if the key derivation uses a device-bound secret (TPM/Keychain) without a recovery path, OS reinstall = data loss. If it uses a weak passphrase, brute force is trivial. | H |
| D3 | **Two copies of memory** (SQLite + Obsidian `.md` vault) — divergence inevitable. The `.md` files are plaintext on disk and not encrypted (contradicting "everything is encrypted at rest"). | H |
| D4 | **Memory Tree compresses email and chat into ≤3k-token chunks** — irreversible information loss; if the compression rule overlay (TokenJuice) drops a phone number or contract clause, the user has no way to retrieve the original. | M |
| D5 | **Gmail + Slack backfill binaries** mutate the same SQLite as the live core. If the user runs them while the desktop is open, write-locks can starve real-time ingestion. | H |
| D6 | **Embeddings** can leak conversation content to OpenAI/cloud providers if `provider = "openai"` (cloud.rs/openai.rs) — opposed to the "private, on-device" marketing. Easy to set wrong; no banner observed. | H |
| D7 | **PII boundary unclear**: README says "workflow data stays on device", but Composio is a SaaS connector hub, Sentry receives crash reports, Seltz/SearXNG receive search queries, ElevenLabs receives TTS text. Each is a separate data egress path. | H |
| D8 | **iMessage `chat.db` read-only** — but read-only is at the rusqlite level; macOS Full-Disk-Access TCC grant is broad. If the agent later writes back (regression), Apple's chat.db can be corrupted with no rollback. | M |
| D9 | **Migrations** rewrite schemas. The CLAUDE.md project instructions in *this* repo describe data-loss risk patterns that apply: no visible row-count verification step in OpenHuman's `migrations/` module surface. | H |
| D10 | **FTS5 indexes** can grow unboundedly with auto-fetch; no visible quota or pruning. | M |
| D11 | **Multi-byte text claim** ("CJK, emoji preserved grapheme-by-grapheme") — `unicode-segmentation` is in the dep tree, but actual coverage is unproven on user data. A regression silently mangles non-Latin text. | M |
| D12 | **`postgres` crate** in dependencies but not in `.env.example` — dead code, or a hidden cloud backend toggled by `BACKEND_URL`. Either way, surprise data egress. | H |
### Test Ideas (Data)
1. **Run `gmail-backfill-3d` while the desktop is open and actively ingesting Slack**; after both finish, run `sqlite3 workspace.db "PRAGMA integrity_check; SELECT COUNT(*) FROM memory_segments;"` and compare to expected counts; assert no `database disk image is malformed`.
2. **Fill the workspace SQLite to 90% disk** with synthetic email, then trigger auto-fetch; capture the failure mode (graceful pause? crash? silent data loss?) and whether the user sees a notification.
3. **Connect Gmail with `embeddings.provider = "openai"`**, then ingest a folder containing the word "PRIVATE-TEST-TOKEN-12345"; use a network tap to confirm whether the literal string (or vector encoding of it) leaves the device.
4. **Diff the Obsidian `.md` vault and the SQLite memory store** after 10 conversations; flag any chunk present in one but not the other, then attempt to recover by editing the `.md` and confirm the SQLite ingests the edit.
5. **Force-kill the process during a `unify_ai_provider_settings.rs` migration**, then restart; capture whether the workspace is mid-state, rolled back, or wedged on retry.
6. **Generate a 50 KB email body of mixed Hangul, ZWJ-emoji families, RTL Arabic, and Devanagari ligatures**, push it through TokenJuice, then compare grapheme cluster counts before/after via `unicode-segmentation`.
7. **Delete the `~/.openhuman/core.token` file** while the core is running and a session is active; observe whether next RPC call regenerates safely or wedges.
8. **Reinstall the OS (simulate by deleting `~/Library/Keychains` on macOS)** while preserving `~/.openhuman`; observe whether memory becomes unreadable, partially readable, or whether there's a recovery path.
9. **Set `BACKEND_URL` to a self-hosted server** and observe outbound traffic; characterize what *core* sends home even when "analytics disabled" (`OPENHUMAN_ANALYTICS_ENABLED=false`).
10. **Use a malformed `chat.db`** in `~/Library/Messages/` (simulate corruption) and start the iMessage scanner; confirm the read-only constraint plus structured-error path.
---
## I — Interfaces
### Observed (how it connects)
- **JSON-RPC** at `POST /rpc` on `127.0.0.1:7788` (default), bearer-token auth (`OPENHUMAN_CORE_TOKEN`). `src/core/jsonrpc.rs`, `src/rpc/dispatch.rs`, `src/rpc/structured_error.rs`.
- **REST** at `/health`, `/?` via `src/api/rest.rs`. JWT via `src/api/jwt.rs`.
- **Socket.IO** event bus via `socketioxide` (`src/api/socket.rs`, `src/openhuman/socket/`).
- **WebSocket bridge** server in the Tauri shell on 127.0.0.1 accepting JSON-RPC frames from the core, so core-side handlers can drive the live CEF webview connectors via CDP (Chrome DevTools Protocol). The CEF instance exposes `--remote-debugging-port=9222`.
- **Tauri commands** (per `app/src-tauri/src/`): `companion_commands.rs`, `core_process.rs`, `core_rpc.rs`, `cdp/`, `webview_apis/`, `webview_accounts/`, `screen_capture/`, scanners for Discord/Slack/Telegram/iMessage/Google Meet/WhatsApp/Google Messages.
- **MCP client** (`src/openhuman/mcp_client/`) talks to external MCP servers over stdio; runs them under managed Python (`runtime_python`) or Node (`runtime_node`).
- **MCP server** (`src/openhuman/mcp_server/`) exposes OpenHuman's tools to other agents over stdio (`protocol.rs`, `tools.rs`).
- **Webhooks** at `src/openhuman/webhooks/` (router, bus, types) — inbound HTTP webhooks. Tunnel (`webhooks-tunnel-flow.spec.ts`) likely uses Ngrok-style relay.
- **Composio** connector hub (`src/openhuman/composio/`) for OAuth-mediated third-party APIs (118+ services). Auth retry, error mapping, googlecalendar arg adapters, periodic syncs.
- **CDP** (`app/src-tauri/src/cdp/`) for Chrome DevTools Protocol over `tokio-tungstenite`.
- **Voice interfaces**: STT via whisper-rs (Metal on macOS), TTS via Piper (binary lookup `PIPER_BIN`) + ElevenLabs (cloud). Audio capture via `cpal` 0.15. Dictation listener + hotkey (`rdev`).
- **Deep-link scheme**: `openhuman://` registered via `tauri-plugin-deep-link`.
- **CLI**: `clap` 4.5 derive — subcommands include at least `serve` (Docker default), `run`, `core run`, plus per-binary helpers.
- **Auto-updater** endpoint: `https://github.com/tinyhumansai/openhuman/releases/latest/download/latest.json` signed with a minisign pubkey baked in `tauri.conf.json`.
- **External services**: Sentry (multiple DSNs), Seltz, SearXNG, Composio, LM Studio (`http://localhost:1234/v1`), Ollama, OpenAI/Anthropic/etc. via routing.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| I1 | **CSP in `tauri.conf.json` is wide open**: `connect-src 'self' ipc: http://ipc.localhost http://127.0.0.1:* http://localhost:* http: ws://127.0.0.1:* ws://localhost:* ws: https: wss: data: blob:` — accepts any HTTP/WS/HTTPS/WSS. `frame-src 'self' https: data: blob:`. Effectively no XSS-fed-via-network protection. | H |
| I2 | **`--remote-debugging-port=9222`** on the embedded CEF is reachable from any local process (binds to localhost by default, but any malware running as the user can talk CDP and execute arbitrary JS in OpenHuman's webviews — including pages that hold OAuth tokens, wallet state, etc.). | H |
| I3 | **JSON-RPC server on 0.0.0.0** in Docker mode (`OPENHUMAN_CORE_HOST=0.0.0.0`). If the bearer token is weak, missing, or leaked, anyone on the network controls the agent. Docker compose comment explicitly notes: "REQUIRED. Generate with `openssl rand -hex 32`." Easy to miss. | H |
| I4 | **No visible authn on the WebSocket bridge** (127.0.0.1) between core and Tauri shell. Any local process can connect and impersonate either side. | H |
| I5 | **MCP server over stdio** — stdio is inherently local but MCP tool calls are unrestricted; a hostile MCP client (or a misconfigured one) can fire arbitrary tools. | M |
| I6 | **Deep-link `openhuman://` scheme** — browser-side phishing can craft `openhuman://...` URLs that trigger second-launch handlers; the single-instance plugin forwards payloads to the running instance. | H |
| I7 | **Composio** is a third-party SaaS — every integration token round-trips through their service. If Composio is breached, all 118 connectors compromise simultaneously. | H |
| I8 | **CDP-driven scanners** (Gmail, WhatsApp, etc.) silently scrape from sites whose DOM can change daily. A breaking change = silent ingestion failure with no user-visible signal. | M |
| I9 | **Updater pubkey is baked in `tauri.conf.json`** — pubkey rotation requires a binary update. If the signing key is lost or compromised, all installed clients are stuck. | M |
| I10 | **Auto-updater endpoint is GitHub Releases over HTTPS** — if GitHub serves a malicious binary (account compromise, MITM with cert pinning bypass), every installed client pulls it. Mitigated by minisign, but TOCTOU on signature verification needs proof. | H |
| I11 | **Multiple binary discovery via `WHISPER_BIN`, `PIPER_BIN`, `OLLAMA_BIN` env vars** — symlink an attacker binary, agent executes it as the user. Standard PATH-hijack with extra steps. | M |
| I12 | **Webhooks tunnel** opens an inbound HTTP listener accessible externally; if the route table (`router.rs`) has any path that reflects input into responses, instant SSRF/XSS. | H |
### Test Ideas (Interfaces)
1. **From a non-admin local process**, connect to `ws://127.0.0.1:9222`, list CEF targets via CDP, then call `Runtime.evaluate` to read `document.cookie` from the OAuth webview; verify whether OpenHuman blocks, logs, or grants the read.
2. **Spin up `docker compose up` with `OPENHUMAN_CORE_TOKEN=`** (empty); send `POST /rpc` with no Authorization header from a sibling container; capture the response code and whether the RPC was executed.
3. **Send 10 KB of JSON garbage** to `POST /rpc` with a valid token: malformed nesting, oversized strings, JSON-bomb (10MB of `[[[[[...]]]]]` arrays); observe memory growth, response time, and whether any deserialization panics.
4. **Trigger an `openhuman://` deep link via the browser** that points at a path with `../` traversal in its payload; assert the single-instance handler rejects rather than passing it to the primary instance.
5. **Replace the `PIPER_BIN` env var** with a shell script that prints to stderr "OWNED"; trigger TTS and check whether OpenHuman executes the script and whether stderr leaks into a notification or log.
6. **Stand up a fake Composio API server** that returns an OAuth token to OpenHuman for a service the user did not authorize; record whether OpenHuman accepts and stores the token without UI confirmation.
7. **Connect a hostile MCP server** (advertises 50 tools, one named `system.shell` with arbitrary command exec) via stdio; check whether the user is prompted before tools are registered and whether the approval flow gates execution.
8. **Serve a poisoned `latest.json`** from a local proxy and point the updater endpoint at it; confirm minisign rejects (capture the verification window — does the binary download first or check signature first?).
9. **Open three concurrent Socket.IO connections** with the same token, then revoke the token in the credentials store; verify all three drop within N seconds.
10. **Webhook ingress** — send a POST to `/webhooks/<path>` with `<script>` in the body; navigate any UI page that displays webhook history and check for stored XSS.
---
## P — Platform
### Observed (what it depends on / runs on)
- **OS targets**: macOS (≥ 10.15, `.app` + DMG with codesign + notarization scripts, Apple silicon + Intel matrices), Windows (NSIS + MSI, codesign, install.ps1), Linux (deb + AppImage; deb depends on `libgtk-3-0`, `libwebkit2gtk-4.1-0`, `libx11-6`, `libgdk-pixbuf-2.0-0`, `libglib2.0-0`).
- **Rust 1.93.0** pinned (constrained by matrix-sdk recursion bug).
- **Node ≥ 24**, **pnpm 10.10.0** exact.
- **CEF 146.4.1** binary blob downloaded by `cef-dll-sys` build script on first build; can also be pre-cached at `$HOME/Library/Caches/tauri-cef`.
- **Docker base**: `rust:1.93-bookworm` builder → `debian:bookworm-slim` runtime. System deps: `libssl3`, `libasound2`, `libxdo3`, `libxtst6`, `libx11-6`, `libevdev2`, `curl`, `gosu`. Runs as UID 10001, `read_only: true`, `cap_drop: ALL`, `no-new-privileges`.
- **Bundled SQLite** (`rusqlite` "bundled" feature) — does not depend on system libsqlite.
- **Audio**: `cpal` (cross-platform), `whisper-rs` with Metal on macOS, `hound` for WAV.
- **Input**: `enigo` (keyboard simulation), `arboard` (clipboard), `rdev` (global hotkey).
- **System detection**: `sysinfo`, `starship-battery` (laptop throttling), `hostname`, `dirs/directories`.
- **Linux-only**: `landlock = "0.4"` (optional), `rppal` (Raspberry Pi GPIO! — confirms `peripheral-rpi` feature), `notify-rust` for dbus notifications.
- **macOS-only**: `objc2`, `objc2-foundation`, `objc2-contacts`, `objc2-app-kit`, `objc2-web-kit`, `block2`, `mac-notification-sys`. Reads Contacts framework (`CNContactStore`).
- **Windows-only**: `windows-sys` for Console reattach, EnumWindows/ShowWindow, CreateMutexW (single-instance guard).
- **External services** (cloud-side): Composio (118 integrations), Sentry (3 DSNs: core, Tauri, frontend), Seltz, SearXNG, OpenAI / Anthropic / etc. via routing, ElevenLabs (TTS), Ollama (local), LM Studio (local).
- **Update channel**: GitHub releases over HTTPS, minisign-signed.
- **Submodules**: Tauri-CEF fork + others.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| P1 | **CEF download on first build** — pulls a binary blob from somewhere over HTTPS. Build supply-chain risk; cache poisoning at `~/Library/Caches/tauri-cef` is local privilege escalation. | H |
| P2 | **Three sandboxing options on Linux** (Landlock, Bubblewrap, Firejail) — sandbox-bubblewrap is on by default? sandbox-landlock is feature-gated and OFF by default. Most Linux users will run unsandboxed. | H |
| P3 | **macOS TCC permissions** required for: full disk access (iMessage), accessibility (autocomplete + screen intelligence + dictation hotkeys), screen recording (screen_intelligence + screen_capture), camera/mic (Meet agent), notifications, contacts (objc2-contacts). Loss/regrant scenarios are fragile. | H |
| P4 | **Windows `windows-subsystem` quirk** — main binary is windowed; `core` subcommand reattaches to parent console via `AttachConsole`. Easy to break with non-default consoles (Windows Terminal, ConEmu, ssh sessions). | M |
| P5 | **Linux** requires `libwebkit2gtk-4.1-0` per the deb manifest, but the runtime is CEF — confusing dependency; if the system has 4.0 only, install fails opaquely. | M |
| P6 | **Docker container is read-only** but writes to a named volume `openhuman-workspace`. If the volume host is on a filesystem without `O_TMPFILE`/proper case-sensitivity (e.g., macOS Docker Desktop default), SQLite WAL behaviour and Argon2 atomic rename can break in subtle ways. | M |
| P7 | **`mem_limit: 4g`, `cpus: 2.0`** in compose — embedding models + voice + memory tree on a 4GB / 2-core container will OOM under load. | M |
| P8 | **Rust 1.93.0 pin** means contributors with `rustup default stable` will see a downgrade prompt; CI uses a specific container image (`ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0`) which is yet another point of supply-chain risk. | M |
| P9 | **Five system libs on Linux** (xdo, xtst, x11, evdev, asound) are unconditional dependencies even when the corresponding features are off (per the Dockerfile comment). Headless server deploys still pull GUI libs. | L |
| P10 | **Battery-aware scheduler** (`starship-battery`) — unmaintained-fork dependency, decisions about throttling background LLM jobs ride on a crate's ABI. | L |
| P11 | **macOS minimum 10.15** — Apple silicon Macs ship with 11+, but `10.15` means Intel Catalina, which is past Apple security updates. | L |
### Test Ideas (Platform)
1. **Strip TCC permissions** mid-session (System Settings → Privacy → revoke Full Disk Access for OpenHuman.app), then trigger iMessage scan; capture whether the structured error matches the user-visible message and whether the agent retries until the user re-grants.
2. **Run `docker compose up` with `mem_limit: 1g`** and trigger memory-tree initialization on 10k mock conversations; capture OOM-kill behaviour, observe whether the volume is left in a consistent state.
3. **Install on Ubuntu 22.04 with only `libwebkit2gtk-4.0`** (not 4.1); run the deb install and the launcher; capture the exact failure message.
4. **Launch on Windows from PowerShell, Windows Terminal, ConEmu, and Git Bash**, then call `OpenHuman.exe core run --help`; assert console output appears in each and `AttachConsole` is wired correctly.
5. **Build CEF from a cache poisoned with a 1-byte modified binary** at `$HOME/Library/Caches/tauri-cef`; observe whether `cef-dll-sys`'s build script detects mismatched checksum (or whether it just compiles and links).
6. **Run on a battery-only laptop at 15% battery**; trigger a memory-tree compaction job and capture whether `scheduler_gate::signals` correctly throttles via `starship-battery` or whether the job runs and drains.
7. **macOS Sequoia (15.x) first-run**: launch the unsigned dev DMG and capture every Gatekeeper / quarantine / TCC consent dialog in order; map them to the onboarding screens to detect any sequence regression.
8. **Run two instances of `cargo tauri dev`** at the same time on macOS; per the single-instance plugin comment, the second should hand off argv. Verify the second exits cleanly and doesn't trigger `cef::initialize(...) != 1`.
---
## O — Operations
### Observed (how it is used and operated)
- **Install**: `curl … install.sh | bash` on Mac/Linux, `irm install.ps1 | iex` on Windows. Downloads from `tinyhumans.ai/openhuman` or GitHub Releases. DMG drag-install on macOS, MSI on Windows, deb/AppImage on Linux.
- **Update**: dual-path — Tauri shell via `tauri-plugin-updater` (minisign), core via `src/openhuman/update/` with restart strategies (`self_replace` | `supervisor`). RPC mutations to `update.apply` gated by `OPENHUMAN_AUTO_UPDATE_RPC_MUTATIONS_ENABLED`.
- **Observability**:
- Sentry x3 projects (core, Tauri shell, frontend). DSNs in env vars, version+SHA in release tag.
- OpenTelemetry traces + metrics (OTLP HTTP) via `opentelemetry-otlp = "0.32"`.
- Prometheus metrics (`prometheus` crate, no client server module obvious).
- `tracing-subscriber` + `tracing-appender` + `env_logger`.
- `OPENHUMAN_ANALYTICS_ENABLED=true` default.
- **Backup**: not visible. No `backup.rs`, no compose-level backup mention. Volume is named `openhuman-workspace`.
- **Doctor**: `src/openhuman/doctor/` — likely a diagnostic CLI.
- **Health**: `/health` endpoint on the core, Docker `HEALTHCHECK` defined.
- **Logging**: `src/core/logging.rs`, `src/core/rpc_log.rs`. RUST_LOG configurable. Tracing-appender suggests file rotation. Sentry capture before_send filter (`tests/observability_smoke.rs`).
- **CI / CD**: **20 GitHub workflows**:
- `build-desktop.yml`, `build-windows.yml`, `build.yml`
- `coverage.yml`, `test.yml`, `test-reusable.yml`, `typecheck.yml`
- `e2e.yml`, `e2e-reusable.yml`, `e2e-agent-review.yml`
- `deploy-smoke.yml`, `installer-smoke.yml`
- `docker-ci-image.yml`
- `release-packages.yml`, `release-production.yml`, `release-staging.yml`
- `tauri-cef-pin-guard.yml`
- `pr-quality.yml` (soft checks — checklist, coverage-matrix, lychee link check; all `continue-on-error`)
- `contributor-rewards.yml`
- `weekly-code-review.yml`
- **Husky pre-push**: `format`, `lint`, `compile`, `rust:check`, `lint:commands-tokens`. Auto-fixes formatting/lint, fails on TS/Rust/tokens.
- **`scripts/install.sh`** has a unit-test harness (`scripts/test_install.sh`) — exercised by `installer-smoke.yml`.
- **`scripts/install.ps1`** has Pester tests (`scripts/tests/OpenHumanWindowsInstall.Tests.ps1`).
- **Onboarding**: judge + stress test scripts (`scripts/test-onboarding-{chat,judge,stress}.mjs`), 4 onboarding modes from spec list.
- **Approval flow** + **prompt-injection detector** + **tool-policy** form the user-facing safety surface.
- **Recovery**: `process_recovery.rs`, `process_kill.rs` in the Tauri shell handle crashes/zombie processes.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| O1 | **No visible backup mechanism** for `~/.openhuman` (or the Docker volume). Memory tree is the user's life corpus; a single corruption = total loss. The CLAUDE.md in *this assessor repo* identifies the same anti-pattern. | H |
| O2 | **`curl … | bash` install**: standard but high-risk if `tinyhumans.ai` or the GitHub raw content endpoint is ever compromised. No GPG/minisign sig on the install script itself. | H |
| O3 | **Sentry, OTLP, analytics all opt-out** — `OPENHUMAN_ANALYTICS_ENABLED=true` default. README emphasizes privacy, but defaults phone home. Default-on telemetry contradicts marketing. | H |
| O4 | **Update restart contract** has two strategies (`self_replace`, `supervisor`); failure mode of a partial update (Tauri shell updated, core not) is implicit. | H |
| O5 | **Auto-fetch every 20 min + subconscious ticks + cron jobs + scheduler_gate** = three overlapping scheduling systems. Behaviour during sleep/wake, dock/undock, VPN flap is a combinatorial nightmare. | H |
| O6 | **`pr-quality.yml` jobs are `continue-on-error: true`** for the first ~2 weeks — comment says "flip to hard-fail once stable". If never flipped, soft gates are not gates. | M |
| O7 | **Five binaries shipped**, but only `openhuman-core` has a Docker image. Backfill binaries deployed to a server need separate management. | M |
| O8 | **Husky pre-push auto-fixes formatting**, then asks user to re-commit. If the user pushes anyway (no re-commit), they push unformatted code — but the hook said it auto-fixed. Confusing. | L |
| O9 | **`weekly-code-review.yml`** runs an automated code review (perhaps the `rabbit` CLI). If the review surfaces secrets in PR descriptions, escalation path unclear. | L |
| O10 | **`installer-smoke.yml`** is `pull_request` + `push:main` — exercises `--dry-run`, not actual install. Production install on a fresh VM is not in CI. | H |
| O11 | **Symbol/source upload to Sentry** is in scripts/upload_sentry_symbols.sh — if this step fails on release, debugging production crashes becomes blind. No visible retry/alarm. | M |
| O12 | **Logs may contain user data** by default — `OPENHUMAN_LOG_PROMPTS=0` is the flag; if a user turns this on for support, the rotating log captures plaintext LLM prompts (i.e., user PII). | M |
### Test Ideas (Operations)
1. **Install via `install.sh` from a local HTTP server** that serves a script signed differently than expected; verify whether the user is warned about the signature mismatch (or whether there is no signature at all).
2. **Trigger the `tauri-plugin-updater` and then SIGKILL during the binary swap** on macOS; restart and assert the app is in either old or new state, never half-replaced.
3. **Start the Tauri shell, update the core only** (manually replace `openhuman-core` on disk), restart core only; capture whether the shell detects version skew and refuses or retries handshake.
4. **Delete `~/.openhuman/` while the app is running**; capture data loss, crash, or graceful re-initialization.
5. **Run the agent through a 24-hour stress** with all 118 connectors active and subconscious ticks on; sample memory, CPU, disk, network outflow every hour. Plot for leaks.
6. **Suspend the laptop for 4 hours** mid-cron job; on wake, assert whether the cron `cron` crate fires the missed jobs once, all at once, or skips.
7. **Submit a PR that intentionally fails the soft `pr-quality.yml` checks** — confirm the merge button is not blocked (verifying that "soft" is actually soft).
8. **Run `openhuman doctor`** with intentional environmental gaps: missing Whisper binary, expired OAuth token, full disk, no internet; capture which diagnostics are accurate, missing, or misleading.
9. **Send a malformed log line** (10MB single-line, embedded null byte, ANSI escapes) to the `tracing-appender` target; assert the appender rotates correctly and does not crash the process.
10. **Verify `OPENHUMAN_ANALYTICS_ENABLED=false` actually disables Sentry/OTLP at runtime** — run with that flag, then network-trace and assert zero outbound to Sentry/OTLP endpoints over a 10-min window with real activity.
---
## T — Time
### Observed (when things happen)
- **Streaming LLM responses**: `chat-harness-send-stream.spec.ts`, `chat-harness-cancel.spec.ts` indicate streaming + cancellation in the chat harness.
- **20-minute auto-fetch loop** per active connector (README).
- **Subconscious ticks**: `src/openhuman/subconscious/`, `scripts/test-subconscious-ticks.sh`.
- **Cron jobs**: `cron = "0.12"` crate + `src/openhuman/cron/`, spec `cron-jobs-flow.spec.ts`.
- **Scheduler gate**: `src/openhuman/scheduler_gate/signals` watches battery + idle to decide when to run background LLM work.
- **Skill + agent tool execution timeout**: `OPENHUMAN_TOOL_TIMEOUT_SECS` default 120s, max 3600s.
- **Web search timeout**: `OPENHUMAN_WEB_SEARCH_TIMEOUT_SECS` default 10s.
- **`wait-timeout = "0.2"`** crate guards `node --version` probe.
- **JWT token** (`src/api/jwt.rs`) — implies session expiry/refresh.
- **Health check**: 30s interval, 5s timeout, 3 retries, 10s start-period (Docker).
- **Single-instance lock** (Tauri-side mutex) — gates concurrent launches.
- **Periodic Composio sync** (`composio/periodic.rs`).
- **Cooldown / retry**: `composio/auth_retry.rs`, `auth_retry_tests.rs`.
- **`tokio-stream`**, **`futures-util`**, **`async-imap`**, **`tokio-tungstenite`** — heavy async surface.
- **Idle watchdog** for CDP sessions (`cdp/session.rs`, comment references `start_paused = true` tokio test).
- **Approval timeout** likely (not confirmed but implied by approval module).
- **Concurrency primitives**: `parking_lot`, `once_cell`, `Arc<RwLock<...>>` patterns inferred from Tokio "full" feature set, `sync` feature.
### Quality Risks
| ID | Risk | Severity |
|----|------|---|
| T1 | **Cancellation races**: streaming responses + tool calls — if the user cancels mid-stream, partial tool side effects (file writes, OAuth API calls, wallet signs) may already be in flight. The cancel signal does not undo network sends. | H |
| T2 | **Auto-fetch + cron + subconscious overlap**: three schedulers can hit the same connector simultaneously; rate-limit blowups, double-ingestion, and lock contention on the SQLite. | H |
| T3 | **JWT expiry**: `JWT_TOKEN` in env is used by skills sandbox for `oauth.fetch` proxy. Expiry handling not documented — if the token is stale on a long-running session, all skill OAuth calls fail silently. | M |
| T4 | **Sleep/wake**: macOS App Nap, Windows modern standby — `cron` crate computes next fire time from wall clock; system-time jumps backward (NTP, DST, manual change) can fire every job at once or skip all. | H |
| T5 | **Single-instance plugin lock**: the comment in `Cargo.toml` documents that the mutex must be acquired *before* `tauri::Builder` work — race condition between launches can still hit `cef::initialize(...) != 1` if the order is wrong. Sentry tag `OPENHUMAN-TAURI-A`. | H |
| T6 | **CEF idle watchdog**: CDP sessions can leak if the websocket disconnects without a close frame; `tokio::test(start_paused = true)` exists, but coverage depends on how mocked time matches reality. | M |
| T7 | **Tool timeout default 120s, max 3600s**: a hostile/buggy MCP tool can occupy the agent for 1 hour. No per-tool budget visible. | M |
| T8 | **Health-check Docker (30s interval)**: between checks, a deadlocked core looks healthy. Orchestrators (Kubernetes) won't restart. | M |
| T9 | **OAuth refresh tokens** — Composio mediates, but expirations span weeks. Token refresh during auto-fetch + sleep cycle = race on which scheduler retries first. | M |
| T10 | **Time-of-check / Time-of-use** in updater (sig verify then apply): signature verified on download, apply happens later. If `/tmp` is writable by other users, swap binary between verify and apply. | H |
| T11 | **Whisper streaming + Meet audio + dictation hotkey**: three audio-capture paths can collide via `cpal` — overlapping device locks lead to dropped audio or silent device-busy errors. | M |
| T12 | **Battery-aware scheduling latency**: when battery drops below threshold, jobs throttle. If the throttle fires *during* an LLM stream (high cost call mid-response), aborting now wastes the tokens already paid for. | L |
### Test Ideas (Time)
1. **Send a chat that triggers a 10-tool-call agent loop with a 30s sleep tool**; cancel after 5s and inspect `rpc_log` + memory store for partial side effects (writes that hit disk before cancel propagated).
2. **Force-shift system time backward by 25 hours** while the desktop runs an active cron job; assert no job double-fires and the scheduler does not lock up.
3. **Suspend macOS for 30 minutes** while a 20-minute auto-fetch tick is pending; on wake, count fired ticks (expect 1, not 2 or 0).
4. **Launch the desktop binary 5 times in 2 seconds**; assert only one survives, the rest exit < 500ms with exit-code 0, and Sentry has 0 `cef::initialize` panics.
5. **Set `OPENHUMAN_TOOL_TIMEOUT_SECS=2`**, then run an MCP tool that sleeps 10s; assert the tool is killed at 2.0s ± 100ms and the agent receives a structured timeout error.
6. **Open a CDP session, drop the underlying TCP connection without close frame** (firewall block), and watch the idle watchdog — measure leak in tokio task count via `tokio_metrics`.
7. **Acquire the keychain lock from another process** so the credentials read blocks for 10s; observe whether RPC handlers serialize behind it or whether unrelated handlers stay live.
8. **Boot OpenHuman with a clock 5 years in the past** (sandboxed VM with `faketime`); assert TLS cert validation rejects updates and the agent doesn't sign wallet transactions with rotted timestamps.
9. **Trigger a Meet join while dictation is active and Whisper STT is streaming**; assert `cpal` either negotiates shared access cleanly or surfaces a structured "device busy" without crashing.
10. **Pump 100 cron-jobs to fire in the same 1-minute window** (synthetic config); measure executor saturation, queue depth, and whether any are silently dropped.
---
## Top 20 Prioritized Test Ideas (Across All Factors)
Risk ranking combines severity, blast radius, and existing-test gap.
| # | Pri | Factor | Test Idea | Automation Fitness |
|---|-----|--------|-----------|----|
| 1 | P0 | F/I | Inject prompt-injection payloads (base64, zalgo, ASCII-art) through Gmail, Notion, Slack into auto-fetch; observe whether the agent attempts the injected action and whether `prompt_injection::detector` flagged it. | Integration + Human exploration |
| 2 | P0 | I | From a non-admin local process, connect to `ws://127.0.0.1:9222`, list CEF targets via CDP, and call `Runtime.evaluate` to read `document.cookie` from the OAuth webview. | Integration |
| 3 | P0 | F | Sign a wallet transaction while a MITM proxy returns a forked chain-id; assert EIP-155 replay protection and visible chain-mismatch warning. | Integration |
| 4 | P0 | F | Install a hostile skill from a fake `SKILLS_REGISTRY_URL` and trace which paths it touches and which OAuth tokens it can reach. | Integration |
| 5 | P0 | D | Run `gmail-backfill-3d` and the desktop concurrently; afterward run `PRAGMA integrity_check` and assert no `database disk image is malformed`. | Integration |
| 6 | P0 | F | Force-kill the process during a `migrations/` step and assert workspace state is forward-rollable, never half-migrated. | Integration |
| 7 | P0 | I | `docker compose up` with `OPENHUMAN_CORE_TOKEN=` empty; send unauthenticated RPC and capture whether it executes. | Unit + Integration |
| 8 | P0 | T | Cancel an active streaming agent loop with 10 tool calls mid-flight; audit partial side effects (file writes, OAuth API calls) and assert idempotency. | Integration |
| 9 | P0 | O | Verify `OPENHUMAN_ANALYTICS_ENABLED=false` produces zero outbound to Sentry/OTLP over 10 min of real activity. | Integration |
| 10 | P0 | I | Trigger `openhuman://` deep link with `../` traversal in payload; assert rejection by the single-instance handler. | Unit |
| 11 | P1 | D | Ingest a folder with literal `PRIVATE-TEST-TOKEN-12345` while `embeddings.provider = "openai"`; confirm via network tap whether the string or its vector leaves the device. | Integration + Human |
| 12 | P1 | F | Force a panic inside `voice/` and observe whether the JSON-RPC server stays up, restarts, or wedges. | Unit + Integration |
| 13 | P1 | T | Shift system time backward 25 hours mid-cron; assert no double-fire and scheduler does not lock. | Integration |
| 14 | P1 | I | Send 10 KB of JSON-bomb (`[[[[[…]]]]]`), oversized strings, malformed nesting to `/rpc`; observe memory growth and deserialization safety. | Integration |
| 15 | P1 | F | Schedule a Meet join while the user is in a separate live Meet; observe audio/camera arbitration. | Human exploration |
| 16 | P1 | P | Strip Full-Disk-Access TCC mid-session and trigger iMessage scan; capture structured-error and recovery prompt. | Human exploration |
| 17 | P1 | F | 20-minute auto-fetch tick during a network outage with a 50MB attachment in the IMAP feed; measure disk growth, memory-tree rows, double-ingestion on reconnect. | Integration |
| 18 | P1 | O | Delete `~/.openhuman/` while running; capture data-loss surface and recovery path. | Integration |
| 19 | P1 | D | Generate 50 KB of mixed CJK + ZWJ-emoji + RTL + Devanagari; push through TokenJuice and diff grapheme counts before/after. | Unit |
| 20 | P1 | I | Stand up a fake Composio API that returns an unauthorized OAuth token; observe whether the desktop silently stores it. | Integration |
Distribution: P0 = 10 (50%), P1 = 10 (50%) — reflecting "early beta" status; lower priorities exist in the per-factor lists.
---
## Coverage Matrix: Existing Tests vs SFDIPOT Gaps
Sources counted: `/tmp/openhuman/tests/` (20 Rust integration tests), `/tmp/openhuman/app/test/e2e/specs/` (55 WDIO E2E specs), `/tmp/openhuman/app/test/` (~8 frontend unit), `/tmp/openhuman/scripts/tests/` (1 Pester), inline `*_tests.rs` files (extensive).
| Factor | Existing Coverage Examples | Gap Severity | Notes |
|--------|----------------------------|--------------|-------|
| **S** Structure | `app/test/info-plist-required-keys.test.ts`, inline `_tests.rs` modules, `linux_cef_deb_runtime_e2e.rs`, `tauri-cef-pin-guard.yml` | **HIGH gap** | No feature-matrix sweep; no failure-injection on subsystem panic; no submodule-missing build test; no `Cargo.lock` divergence test. |
| **F** Function | Extensive Rust unit + 55 E2E specs covering chat, skills, OAuth flows, voice, wallet, webhooks, channels, etc.; `agent_*` and `memory_*` E2E exist | **MEDIUM gap** | Happy paths well covered. Adversarial input, prompt injection, skill sandbox escape, wallet MITM, migration crash all missing. |
| **D** Data | `memory_roundtrip_e2e.rs`, `memory_graph_sync_e2e.rs`, `agentmemory_backend.rs`, `agent_memory_loader_public.rs`, `agent_retrieval_e2e.rs`, `autocomplete_memory_e2e.rs`, `inline tests` for migrations + ops | **HIGH gap** | No concurrency-on-same-SQLite (binaries vs desktop), no disk-full, no encryption-key-loss recovery, no PII egress check, no FTS5-growth bound, no plaintext-vault leak check. |
| **I** Interfaces | `json_rpc_e2e.rs`, `webview_apis_bridge.rs`, `live_routing_e2e.rs`, `tauri-commands.spec.ts`, `webhooks-*.spec.ts`, `tool-*-flow.spec.ts`, `inference_provider_e2e.rs` | **HIGH gap** | No fuzzing of `/rpc`, no port-9222 CDP-takeover test, no CSP review/test, no malformed deep-link, no PATH-hijack on `WHISPER_BIN`/`PIPER_BIN`/`OLLAMA_BIN`, no Composio-impersonation. |
| **P** Platform | `linux_cef_deb_runtime_e2e.rs`, `installer-smoke.yml` (dry-run only), `OpenHumanWindowsInstall.Tests.ps1` (MSI args), `build-desktop.yml` matrix, container image `openhuman_ci:rust-1.93.0` | **HIGH gap** | No fresh-VM install verification, no TCC revoke test, no CEF cache poisoning, no `mem_limit` OOM behaviour, no real Windows console matrix, no Ubuntu 22.04 with libwebkit 4.0 missing. |
| **O** Operations | 20 workflows, `coverage.yml`, `observability_smoke.rs`, `tokenjuice_integration.rs`, `pr-quality.yml` (soft) | **MEDIUM gap** | No backup/restore test, no analytics-off telemetry verification, no Doctor diagnostic correctness, no real install.sh end-to-end (only dry-run), no partial-update recovery, no log-rotation under load. |
| **T** Time | `subconscious_e2e.rs`, `chat-harness-cancel.spec.ts`, `chat-harness-send-stream.spec.ts`, `cron-jobs-flow.spec.ts`, `composio/auth_retry_tests.rs`, `skill-socket-reconnect.spec.ts`, `cdp/session.rs` paused-time unit | **MEDIUM gap** | No clock-skew test, no system-time-jump, no overlapping-schedulers race, no Meet+dictation+STT collision, no sleep/wake mid-cron, no Docker healthcheck false-positive (deadlock detection). |
**Top hidden-gap themes** (cross-cutting):
1. **Adversarial input handling** — almost zero coverage. All E2E flows use clean fixtures.
2. **Concurrency between binaries and desktop** — backfill binaries are not tested against a running core.
3. **Sandbox escape** — QuickJS skill runtime has no negative tests visible.
4. **Real install / real upgrade** — only dry-runs in CI.
5. **Privacy claims under audit** — no test asserts "analytics off = nothing leaves the box".
---
## Strategic Recommendations: Where to Invest Test Effort First
### Tier 1 — Investment in next sprint (highest yield per hour)
1. **Build an adversarial-input corpus** for prompt injection, ingest it through every connector (Gmail, Notion, Slack, Telegram, etc.) and gate releases on the detector blocking ≥ a defined %. This single corpus pays back across F, I, and D factors. *Existing surface: `src/openhuman/prompt_injection/tests.rs` exists but coverage breadth is unknown.*
2. **Port 9222 / CDP isolation hardening + tests** — restrict the debug port or auth it; add a test that asserts a non-OpenHuman local process *cannot* drive the CEF webviews. This closes the single largest local-privilege gap.
3. **`/rpc` fuzzer** — JSON-RPC under cargo-fuzz / `wiremock` adversarial frames. Cheap, fast, catches deserialization panics that crash the embedded core (and thus the GUI).
4. **Migration crash-safety harness** — power-off SQLite mid-migration on every migration in `src/openhuman/migrations/`. Migration bugs are silent and catastrophic.
5. **Analytics-off audit** — a recurring CI step that runs the app with `OPENHUMAN_ANALYTICS_ENABLED=false` and asserts pcap-level egress is zero. Marketing parity check; cheap to automate.
### Tier 2 — Investment in next quarter
6. **Real install on fresh VMs** — Lima/Tart/Vagrant/Windows Sandbox images for macOS, Ubuntu 22.04, Ubuntu 24.04, Windows 11. Run actual `install.sh` and `install.ps1` end-to-end, including update path. `installer-smoke.yml` is dry-run only today.
7. **Wallet adversarial suite** — MITM RPC, replay, chain-id confusion, malformed ABI, sign-without-confirmation. Wallet drain is the highest-impact single bug class.
8. **Skill sandbox negative tests** — install a malicious skill, audit which APIs it can reach. Pair with a documented "skill trust tier" model if it doesn't exist.
9. **Concurrent-binary test rig** — run `gmail-backfill-3d` + `slack-backfill` + desktop concurrently with synthetic workloads, run `PRAGMA integrity_check` on exit. Three commands of CI yield.
10. **Auto-fetch chaos** — partition network, fill disk, return giant payloads from connectors during the 20-min loop. Throughput/resilience characteristics matter for "always running" desktop apps.
### Tier 3 — Investment when stable
11. **TokenJuice grapheme/CJK regression suite** — fixed corpus of edge-case Unicode, diffed before/after compression. Cheap to maintain, prevents marketing-claim regressions.
12. **macOS TCC matrix** — revoke each entitlement (Full Disk, Accessibility, Screen Recording, Camera, Mic, Contacts, Notifications) and capture user-visible state. Manual but high-signal.
13. **Sleep/wake/clock-skew matrix** — drive faketime-style tests through `scheduler_gate`, `cron`, `composio/periodic`, `subconscious`, `auto-fetch`.
14. **Privacy egress map** — for each external service (Composio, Sentry x3, Seltz, SearXNG, ElevenLabs, OpenAI, Anthropic, Ollama-default, LM-Studio-default, Tinyhumans backend) document and test what data goes out, when, and under what flag. Make this a maintained matrix in `docs/`.
15. **Doctor correctness** — programmatic faults (missing binary, expired token, full disk, no internet) cross-checked against Doctor's output.
16. **Convert `pr-quality.yml` from `continue-on-error: true` to hard-fail** — the comment in the workflow itself says to do this; until done, the gates are theatre.
### Cross-cutting observation: "Early beta" + 95 subsystems + 118 integrations + wallet + auto-update
OpenHuman has the surface area of a **mid-sized OS** and the maturity label of a beta. The current test suite is dense in *happy-path E2E* and thin in *adversarial, resource-pressure, and concurrency* tests. Investment should shift from adding more E2E specs (already 55) toward **negative testing, resource-pressure, and the trust boundaries** (skills, wallet, RPC, CEF, update). The smallest test effort with the largest expected risk reduction is *adversarial-corpus-driven testing against the prompt-injection detector and the JSON-RPC interface* — both have a single owning module and a finite input space, but block the largest blast radius.
docs/qe-reports/openhuman/06-complexity-hotspots.md
+177
View File
@@ -0,0 +1,177 @@
# OpenHuman — Code Complexity Hotspots
**Scope analysed:** 2,282 source files (1,345 Rust + 937 TS/TSX), excluding `node_modules/`, `target/`, `dist/`, `build/`, `.git/`, declaration files.
**Aggregate:** 581,955 LOC (all), 483,916 LOC (production, ex-tests).
**Method:** LOC via `wc -l`, function spans via `awk` between `fn`/`function`/arrow declarations, branch counts via `grep -cE`, coupling via `use`/`import` line counts, indent depth via leading-space buckets (16 / 20 / 24 spaces).
**Note on churn:** TODO/FIXME density across the codebase is extremely low (max single-file count = 3). Churn-via-comment is **not a meaningful signal** here, so the report omits it from final ranking.
---
## 1. Top 30 largest source files (production + tests)
| Rank | LOC | File | Role |
|----:|----:|------|------|
| 1 | 6,306 | `tests/json_rpc_e2e.rs` | JSON-RPC end-to-end test harness (test code) |
| 2 | 4,450 | `app/src-tauri/src/webview_accounts/mod.rs` | Franz-style embedded webview hosting (WhatsApp/Slack/Discord/…) — Tauri child webview lifecycle, recipe injection, per-account session isolation |
| 3 | 3,846 | `app/src-tauri/src/lib.rs` | Tauri command/plugin wiring + app bootstrap |
| 4 | 2,702 | `src/core/observability.rs` | Tracing/metrics/logging plumbing |
| 5 | 2,696 | `app/src/components/settings/panels/AIPanel.tsx` | AI settings UI (providers, models, planner controls) |
| 6 | 2,345 | `src/openhuman/memory/tree/read_rpc.rs` | Memory-tree read-side RPC surface |
| 7 | 2,261 | `app/src/lib/i18n/en.ts` | English translation table (data) |
| 8 | 2,181 | `src/openhuman/agent/harness/session/turn.rs` | Per-turn agent loop (provider call, tool dispatch, progress emission) |
| 9 | 2,125 | `app/src/pages/Conversations.tsx` | Conversations page (threads, composer, message dispatch) |
| 10 | 2,111 | `app/src/lib/i18n/ko.ts` | Korean translation table (data) |
| 11 | 2,093 | `app/src-tauri/src/whatsapp_scanner/mod.rs` | WhatsApp DOM scraper / sync |
| 12 | 2,013 | `src/openhuman/config/schema/load.rs` | Config loader + env-var overrides + migration |
| 13 | 2,005 | `src/openhuman/inference/provider/compatible.rs` | OpenAI-compatible provider client (chat + streaming) |
| 14 | 1,889 | `src/openhuman/composio/ops.rs` | Composio integration ops |
| 15 | 1,834 | `src/openhuman/channels/providers/telegram/channel_tests.rs` | Telegram channel tests (test code) |
| 16 | 1,792 | `src/openhuman/channels/providers/web.rs` | Web channel provider |
| 17 | 1,717 | `src/openhuman/agent/harness/test_support_test.rs` | Agent test support (test code) |
| 18 | 1,699 | `src/openhuman/config/schema/load_tests.rs` | Config loader tests (test code) |
| 19 | 1,679 | `src/openhuman/agent/harness/subagent_runner/ops.rs` | Sub-agent inner tool-call loop |
| 20 | 1,659 | `app/src-tauri/src/discord_scanner/mod.rs` | Discord DOM scraper / sync |
| 21 | 1,656 | `src/openhuman/agent/harness/session/builder.rs` | Session/system-prompt builder |
| 22 | 1,617 | `src/openhuman/memory/tree/store.rs` | Memory-tree write-side store |
| 23 | 1,534 | `src/openhuman/composio/ops_test.rs` | Composio tests (test code) |
| 24 | 1,497 | `src/openhuman/security/policy_tests.rs` | Policy tests (test code) |
| 25 | 1,489 | `app/src/services/webviewAccountService.ts` | Front-end webview-account service |
| 26 | 1,488 | `src/openhuman/channels/runtime/dispatch.rs` | Channel message dispatch / routing |
| 27 | 1,483 | `src/core/jsonrpc.rs` | JSON-RPC core |
| 28 | 1,463 | `src/openhuman/tokenjuice/reduce_tests.rs` | TokenJuice reduce tests (test code) |
| 29 | 1,459 | `src/openhuman/inference/local/service/ollama_admin.rs` | Local Ollama service admin (install, diagnostics) |
| 30 | 1,422 | `src/openhuman/agent/prompts/mod_tests.rs` | Prompt tests (test code) |
Production-only top three (ignoring `*_test*`, `tests/`): `webview_accounts/mod.rs` (4,450), `app/src-tauri/src/lib.rs` (3,846), `core/observability.rs` (2,702).
---
## 2. Top 15 likely-most-complex files (combined score)
Score combines: LOC, branch-keyword density per 1k LOC, deep-nest line counts (≥16 / ≥20 / ≥24 leading spaces), match-arm density (`=>` count), imports/uses, and largest single-function span.
| Rank | File | LOC | Branches | Branches/1k | `=>` arms | 16sp | 20sp | 24sp | Imports | Biggest fn (LOC) | Verdict |
|----:|------|----:|---------:|------------:|---------:|----:|----:|----:|--------:|-----------------:|---------|
| 1 | `app/src-tauri/src/webview_accounts/mod.rs` | 4,450 | 302 | 67.9 | 86 | 318 | 163 | 71 | 17 | `webview_account_open` (839) | God-module: IPC + webview lifecycle + recipe injection + per-OS shims |
| 2 | `app/src-tauri/src/lib.rs` | 3,846 | 269 | 69.9 | 79 | — | — | — | 29 | `run` (255), `drop` impl block (1,051 spans) | Tauri bootstrap dumping-ground |
| 3 | `src/openhuman/agent/harness/session/turn.rs` | 2,181 | 150 | 68.8 | 47 | 300 | 261 | 146 | 26 | `turn` (~967) | Single function = 44% of file; high-density nesting |
| 4 | `src/openhuman/inference/provider/compatible.rs` | 2,005 | 152 | 75.8 | 33 | 229 | 225 | 146 | 9 | `stream_native_chat` (421) | Streaming SSE state machine; deepest 24-sp count of all hotspots |
| 5 | `src/openhuman/config/schema/load.rs` | 2,013 | 259 | **128.7** | 86 | 244 | 81 | 60 | 12 | `apply_env_overrides_from` (719) | Highest branch density in repo — huge env-var override switch |
| 6 | `src/openhuman/agent/harness/subagent_runner/ops.rs` | 1,679 | 126 | 75.0 | 39 | 207 | 154 | 99 | 22 | `run_typed_mode` (740), `run_inner_loop` (576) | Two giant loops in one file |
| 7 | `src/openhuman/channels/runtime/dispatch.rs` | 1,488 | 121 | 81.3 | 44 | 132 | 68 | 34 | 22 | (top non-test fn 133) | High coupling + high branch density |
| 8 | `src/openhuman/agent/harness/session/builder.rs` | 1,656 | 113 | 68.2 | — | 159 | 112 | 66 | 18 | — | Builder for the system prompt + tool list |
| 9 | `src/openhuman/inference/local/service/ollama_admin.rs` | 1,459 | 128 | **87.7** | — | — | — | — | 16 | `download_and_install_ollama` (445) | Installer + diagnostics + lifecycle in one |
| 10 | `src/openhuman/memory/tree/read_rpc.rs` | 2,345 | 94 | 40.1 | 14 | 245 | 118 | 35 | 22 | `delete_chunk_rpc` (166) | Large but flatter; per-RPC functions stay <170 LOC |
| 11 | `app/src/components/settings/panels/AIPanel.tsx` | 2,696 | 97 | 36.0 | — | 213 | 150 | 24 | 13 | `BackgroundLoopControls` (689), `runPlannerNow` (678) | Multi-feature panel; nested components |
| 12 | `app/src/pages/Conversations.tsx` | 2,125 | 109 | 51.3 | — | 122 | 87 | 45 | 35 (TS imports) | `selectedThreadParent` (908), `resolveThreadDisplayTitle` (~930 span heuristic) | Top TS coupling; mixes thread state + composer + display logic |
| 13 | `src/core/jsonrpc.rs` | 1,483 | — | — | 69 | — | — | — | 18 | — | High match-arm density (69) over manageable LOC |
| 14 | `app/src-tauri/src/whatsapp_scanner/mod.rs` | 2,093 | — | — | — | — | — | — | 14 | — | DOM scraper; similar shape to discord/slack scanners |
| 15 | `src/openhuman/composio/ops.rs` | 1,889 | — | — | — | — | — | — | 15 | — | Integration ops surface |
Other notable coupling outliers (imports/uses ≥ 30, even where LOC is modest):
- `src/openhuman/channels/runtime/startup.rs` — **46 uses** (highest in repo, file not even in size top-30)
- `src/openhuman/memory/tree/jobs/handlers/mod.rs` — 41 uses, 1,224 LOC
- `app/src/pages/Settings.tsx` — 41 imports
- `app/src/App.tsx` — 31 imports
- `app/src/pages/Skills.tsx` — 30 imports
---
## 3. Specific refactor recommendations — top 5 hotspots
### 3.1 `app/src-tauri/src/webview_accounts/mod.rs` (4,450 LOC, biggest fn 839)
**Evidence:** 172 `fn` declarations, 153 `if`, 86 `=>` arms, 318 lines indented ≥16 spaces, single function `webview_account_open` spans lines 1,787 → 2,626 (839 LOC).
**What it does:** IPC commands for opening/closing/sizing child webviews, per-provider recipe injection, OS-specific shims (`#[cfg(windows)]` / `#[cfg(target_os = "linux")]`), drop/cleanup logic.
**Refactor:**
1. **Split by responsibility into a submodule directory** `webview_accounts/{ipc.rs, recipe.rs, lifecycle.rs, platform_linux.rs, platform_macos.rs, platform_windows.rs}`. The OS-cfg blocks are an obvious seam.
2. **Decompose `webview_account_open`** — at 839 LOC it should be a thin orchestrator calling: `resolve_recipe()`, `build_webview_config()`, `attach_event_bridge()`, `register_navigation_handlers()`, `persist_account_metadata()`.
3. **Extract recipe injection** into its own type with a small trait (`RecipeInjector { fn script(&self) -> Cow<str>; fn initialization_args(&self) -> Value; }`) — currently per-provider branching is inline.
### 3.2 `app/src-tauri/src/lib.rs` (3,846 LOC, `run` = 255 LOC)
**Evidence:** 152 `fn`, 109 `if`, 79 `=>` arms, 29 imports. Contains the Tauri bootstrap plus dozens of `#[tauri::command]` handlers.
**Refactor:**
1. **Move all `#[tauri::command]` functions** into per-domain modules (`commands/accounts.rs`, `commands/updates.rs`, `commands/data.rs`, …) and have `lib.rs` only call `.invoke_handler(generate_handler![...])`.
2. **Extract update flow** — `apply_app_update` (119), `download_app_update` (92), `reset_local_data` (92) belong in an `app_update` submodule.
3. **Cap `lib.rs` at ~500 LOC** (boilerplate only).
### 3.3 `src/openhuman/agent/harness/session/turn.rs` (`turn` = ~967 LOC, lines 71 → 1,038)
**Evidence:** 300 lines indented ≥16 spaces (highest in the report), 261 at ≥20, 146 at ≥24. The single `turn` function is ~44% of the file.
**Refactor:**
1. **Extract a state machine.** A turn has clear phases: `LoadOrResumeTranscript → BuildOrReuseSystemPrompt → InvokeProvider → DispatchToolCalls → EmitProgress → PersistTurn`. Encode them as enum variants of `TurnPhase` and let `turn()` be a loop over `next_phase()`.
2. **Pull tool-iteration logic** (the inner loop that drives `max_tool_iterations`) into a `tool_iteration::drive(…)` helper. That alone should kill 200+ LOC of nesting from `turn`.
3. **Move logging/progress emission** behind a small `TurnObserver` so the happy path reads like the doc comment at the top of the file.
### 3.4 `src/openhuman/inference/provider/compatible.rs` (2,005 LOC, `stream_native_chat` = 421)
**Evidence:** Branch density 75.8/1k LOC, deepest 24-sp count of any production file (146). Already partially split via `#[path]` into `compatible_parse.rs`, `compatible_stream.rs`, `compatible_dump.rs`, `compatible_types.rs` — the seam exists but the orchestrator file is still huge.
**Refactor:**
1. **Move `stream_native_chat`** wholesale into `compatible_stream.rs` and expose it via the existing module split. The 421-LOC streaming state handler does not belong in the trait-impl file.
2. **Collapse `chat` / `chat_with_system` / `chat_with_history` / `stream_chat_with_system`** (233 + 166 + 100 + 156 LOC) — they almost certainly share request-building boilerplate. Extract a `ChatRequestBuilder` and a single `dispatch(builder, mode: Streaming | OneShot)`.
3. **Type the deep nesting away.** `serde_json::Value` walking is what produces the 24-space indents; introduce typed structs for the OpenAI/Responses API payloads (Serde derives) so the parser flattens.
### 3.5 `src/openhuman/config/schema/load.rs` (2,013 LOC, `apply_env_overrides_from` = 719)
**Evidence:** Highest branch density in the entire repo: **128.7 branches per 1k LOC**, 200 `if`, 25 `match`, 86 `=>` arms. `apply_env_overrides_from` is a 719-LOC if/else cascade.
**Refactor:**
1. **Replace the cascade with a declarative table.** Define `static ENV_OVERRIDES: &[EnvBinding]` where each row is `(env_var_name, path_in_schema, parser_fn)`, and have one generic applier walk the table. Each row becomes one line; the file shrinks by an order of magnitude.
2. **Split migration logic** (`migrate_cloud_provider_slugs` = 120, plus `decrypt_config_secrets` = 92) into `schema/migrate.rs` and `schema/secrets.rs`.
3. **Parallel benefit for tests:** `load_tests.rs` is 1,699 LOC — table-driven overrides will let the test file shrink in proportion.
---
## 4. Module-level coupling observations
**Re-export-heavy mod.rs files** (≥30% lines are `pub use`):
| File | LOC | `pub use` lines | % |
|------|----:|----------------:|--:|
| `src/openhuman/channels/mod.rs` | 65 | 37 | 57% |
| `src/openhuman/security/mod.rs` | 41 | 13 | 32% |
| `src/openhuman/config/schema/mod.rs` | 85 | 26 | 31% |
These are healthy façade modules, not monoliths. The actual monoliths are the implementation `mod.rs` files in `app/src-tauri/`:
| File | LOC | Shape |
|------|----:|-------|
| `app/src-tauri/src/webview_accounts/mod.rs` | 4,450 | Single-file module — no submodule split |
| `app/src-tauri/src/whatsapp_scanner/mod.rs` | 2,093 | Single-file scraper |
| `app/src-tauri/src/discord_scanner/mod.rs` | 1,659 | Single-file scraper |
| `app/src-tauri/src/screen_capture/mod.rs` | 1,066 | Single-file |
| `app/src-tauri/src/slack_scanner/mod.rs` | 1,049 | Single-file |
| `src/openhuman/agent/prompts/mod.rs` | 1,392 | Prompt assembly logic in `mod.rs` instead of submodules |
| `src/openhuman/memory/tree/jobs/handlers/mod.rs` | 1,224 | 41 imports (highest of any handler-style file) |
**Pattern:** the `*/mod.rs` files in `app/src-tauri/src/{provider}_scanner/` consistently exceed 1k LOC. They likely share scaffolding (login detection, DOM polling, event piping). A common `scanner_core` crate-internal module would extract the boilerplate and let each `mod.rs` focus on provider-specific selectors.
**Front-end coupling concentrates in `app/src/pages/`:**
| File | Imports | Note |
|------|--------:|------|
| `app/src/pages/Settings.tsx` | 41 | Aggregates every settings panel |
| `app/src/pages/Conversations.tsx` | 35 | Largest TS file in scope (also: function `selectedThreadParent` spans 908 lines per arrow-block heuristic — likely fronted by closures, but warrants a hand-look) |
| `app/src/App.tsx` | 31 | Router + provider tree |
| `app/src/pages/Skills.tsx` | 30 | Skill catalog UI |
These page-level components are the typical SPA "container" anti-pattern: pull every hook + every component into one file. Recommendation: each `pages/X.tsx` should be ≤300 LOC and only compose smaller components from `components/X/`.
---
## 5. Overall complexity score: **6 / 10**
| Dimension | Score | Reasoning |
|-----------|------:|-----------|
| File size distribution | 6 | 30+ files over 1k LOC; 4 files over 2.5k LOC. Heavy tail. |
| Function size | **8** | At least 6 production functions exceed 400 LOC; one exceeds 950 LOC (`turn::turn`). This is the worst single dimension. |
| Branch density | 6 | Median around 70/1k LOC; outliers at 128/1k (`config/schema/load.rs`) and 88/1k (`ollama_admin.rs`). |
| Nesting depth | 5 | Deep blocks exist (146 lines at 24+ spaces in `compatible.rs` and `turn.rs`) but they are localized to a handful of files. |
| Coupling | 4 | Most files import 5–15 things. A few page/runtime files reach 35–46. No widespread god-deps. |
| Module structure | 5 | Façade `mod.rs` re-exports look clean. Implementation `mod.rs` files in `app/src-tauri/*_scanner/` and `webview_accounts/` are not subdivided — those drag the score. |
| TODO/FIXME churn proxy | 2 | Negligible (max 3 per file across entire codebase). Code is curated, not abandoned. |
| Tests | 4 | Heavy test suites alongside hotspots (`config/schema/load_tests.rs` = 1,699 LOC, `composio/ops_test.rs` = 1,534). Hotspots are exercised; that lowers risk. |
**Verdict:** Roughly **6/10** — *Medium complexity, well-trafficked but with concentrated debt*. The codebase is not unmaintainable; the hot-spot pattern is the classic "five files do 80% of the heavy lifting and got too big." Three changes would move the score to ~4/10:
1. Decompose `webview_account_open`, `turn::turn`, `apply_env_overrides_from`, `stream_native_chat`, `run_typed_mode` + `run_inner_loop`. (Five functions = >3,700 LOC of god-functions.)
2. Split `app/src-tauri/src/lib.rs` and `app/src-tauri/src/webview_accounts/mod.rs` into submodule directories.
3. Replace the `apply_env_overrides_from` if/else cascade with a declarative table — single highest-leverage refactor for branch density.
Risk-weighted priority for QE focus: **`turn.rs`** (agent loop = correctness-critical) > **`compatible.rs`** (LLM I/O, deep nesting on hot path) > **`subagent_runner/ops.rs`** (recursion + budgets) > **`webview_accounts/mod.rs`** (cross-platform shims, OS-conditional code = brittle) > **`config/schema/load.rs`** (config drift = silent data corruption).
docs/qe-reports/openhuman/07-dependencies.md
+329
View File
@@ -0,0 +1,329 @@
# OpenHuman — Dependency Surface Map
**Scope:** `/tmp/openhuman` @ working copy (project version `0.54.3`, Rust toolchain pinned at `1.93.0`)
**Method:** static manifest + lockfile inspection. No `cargo build` / `pnpm install` run.
**Sources of truth:** 2 `Cargo.toml`, 2 `Cargo.lock`, 4 `package.json`, 2 `pnpm-lock.yaml`.
---
## 1. Workspace structure overview
OpenHuman is **NOT a Cargo workspace**. The root `Cargo.toml` has no `[workspace]` table — it is a single crate (`openhuman` v0.54.3) that exposes `lib.rs` + 5 binaries, with `app/src-tauri/Cargo.toml` declaring a **separate, non-member crate** (`OpenHuman` v0.54.3) that depends on the core via `openhuman_core = { path = "../..", package = "openhuman" }`. The two crates therefore have **two independent `Cargo.lock` files** (914 packages in root, 988 in tauri, 786 unique combined).
The pnpm side **is** a workspace, but a trivially small one — `pnpm-workspace.yaml` lists only `"app"`. There are 4 `package.json` files but only 2 are workspace members (`./` and `./app`). `remotion/` and `packages/npm/` are unmanaged and not picked up by pnpm.
### Members at a glance
| Manifest | Kind | Version | Direct deps (runtime / dev) | Role |
|---|---|---|---|---|
| `Cargo.toml` | crate `openhuman` | 0.54.3 | 98 + 2 = **100** declared (105 resolved in lock) | Core sidecar: RPC server, agents, memory, integrations, embedded HTTP/JSON-RPC, audio (whisper), wallet (ethers), email (lettre+imap), Matrix, WhatsApp |
| `app/src-tauri/Cargo.toml` | crate `OpenHuman` | 0.54.3 | 33 + 2 = **35** | Desktop shell. Embeds `openhuman_core` in-process, wraps CEF via `tauri-runtime-cef`, plus deep-link / single-instance / updater plugins |
| `package.json` | workspace root | — | 1 + 3 = **4** | Pnpm pass-through scripts; husky/tsx/ws toolchain |
| `app/package.json` | `openhuman-app` | 0.54.3 | 37 + 41 = **78** | React 19 + Vite + Redux Toolkit frontend; Tauri JS API; WDIO E2E |
| `remotion/package.json` | `remotion` | 1.0.0 | 9 + 6 = **15** | Mascot video assets pipeline (separate from pnpm workspace) |
| `packages/npm/package.json` | `openhuman` | 0.0.0 | 0 + 0 = **0** | Tiny postinstall installer stub for `npm i -g openhuman` |
### Source size (for context, not strictly dependencies)
- **Rust core:** 1,217 `.rs` files, **~415k LOC** across `src/`.
- **Frontend:** 817 `.ts`/`.tsx` files, **~158k LOC** across `app/src/`.
- **Vendored Rust forks** sit in `app/src-tauri/vendor/{tauri-cef, tauri-plugin-notification}` (git-submodule pinned).
### High-level graph (Rust side, by file count)
```
openhuman_core (1 crate, ~150 modules)
├── memory (62k LOC, 146 incoming imports — central hub)
├── agent (45k LOC, 220 outgoing imports — the orchestrator)
├── tools (38k LOC, 224 outgoing imports — most outgoing of any module)
├── inference (30k LOC, 114 incoming imports — second hub)
├── channels (33k LOC, providers for discord/telegram/web/...)
├── composio (26k LOC, third-party integrations)
└── ~140 leaf modules
app/src-tauri (separate crate)
└── openhuman_core (path dep) + tauri/CEF/plugins
```
---
## 2. Top 25 Rust dependencies (by importance / weight)
Selected from the **105 direct deps of `openhuman` in `Cargo.lock`**, prioritising heavy, sensitive, or risk-flagged crates. Versions are exact pins from `Cargo.lock`.
| # | Crate | Declared | Resolved | Role | Risk notes |
|---|---|---|---|---|---|
| 1 | `tokio` | `"1"` features = `["full","sync"]` | **1.52.3** | Async runtime — pulled in by basically everything | Heavy but safe. `features=full` ships all sub-features. |
| 2 | `reqwest` | `"0.12"` 8 features incl. both `rustls-tls` AND `native-tls` | **0.12.28** | HTTP client | Enabling BOTH `rustls-tls` and `native-tls` in one binary is a smell — usually one or the other. |
| 3 | `axum` | `"0.8"` default-features-off | **0.8.9** | RPC server (the `openhuman.*` JSON-RPC surface) | Current. |
| 4 | `rusqlite` | `"0.37"` features = `["bundled"]` | **0.37.0** | Embedded SQLite — backs `memory/store`, `memory/tree`, credentials | `bundled` ships an in-tree SQLite (build size +~3MB but no system dep). Also depended on transitively by `matrix-sdk-sqlite` and `whatsapp-rust-sqlite-storage`. |
| 5 | `postgres` | `"0.19"` | (resolved 0.19.x) | Sync Postgres client — present alongside SQLite | Unusual: sync `postgres` crate while everything else is async. Used where? worth a follow-up. |
| 6 | `rustls` | `"0.23"` features=`["ring"]` | **0.23.40** | TLS | Current. |
| 7 | `tokio-rustls` | `"0.26.4"` | 0.26.4 | TLS for tokio sockets | Current. |
| 8 | `ring` | `"0.17"` | 0.17.x | Crypto primitives (also via rustls) | OK. |
| 9 | `aes-gcm` | `"0.10"` | 0.10.x | Symmetric crypto (likely vault/credential encryption) | Standard RustCrypto. |
| 10 | `chacha20poly1305` | `"0.10"` | 0.10.x | Symmetric crypto (alt path) | Two AEADs in one binary — duplication worth checking. |
| 11 | `argon2` | `"0.5"` | 0.5.x | Password hashing | OK. |
| 12 | `whisper-rs` | `"0.16"` (+ git-patched `whisper-rs-sys`) | 0.16.0 | Local speech-to-text via whisper.cpp | **Heavy.** macOS path enables `metal` feature. `whisper-rs-sys` is forked at `tinyhumansai/whisper-rs-sys` (branch=`main`, no rev pin) for an MSVC `/MT` CRT fix on Windows — unstable. **No commit pin = non-reproducible build** when upstream branch advances. |
| 13 | `whisper-rs-sys` (`patch.crates-io`) | git branch=main | n/a | C++ bindings to whisper.cpp | Same fork issue as above. |
| 14 | `matrix-sdk` | `"0.16"` optional, e2e-encryption + rustls-tls + markdown | **0.16.1** | Optional Matrix channel | Pulls **massive** transitive set: `matrix-sdk-base/common/crypto/sqlite/indexeddb/store-encryption`, `vodozemac`, `ruma-signatures`, `matrix-pickle`. Causes the **Rust 1.94 pin** (rust-toolchain.toml says: "Pin below Rust 1.94 until matrix-sdk resolves recursion limit overflow in async — issue 6254"). Real blocker on toolchain upgrades. |
| 15 | `whatsapp-rust` | `"0.5"` optional | **0.5.0** | Optional WhatsApp Web channel | Mid-popularity ecosystem (`whatsapp-rust-sqlite-storage`, `wacore-*`, `wacore-libsignal`). Single-org maintainership. Comment in Cargo.toml admits this is a recent migration from a 0.2 fork (`wa-rs`). Watch for regressions. |
| 16 | `wacore` | `"0.5"` optional | 0.5 | WhatsApp protocol core | Same risk family as `whatsapp-rust`. |
| 17 | `ethers-core` + `ethers-signers` | `"2.0.14"` default-features-off | 2.0.14 | Wallet/signing for EVM keys | `ethers-rs` is **deprecated** upstream (the ethers-rs project recommends migration to `alloy`). Last release 2.0.x is old. Still functional but a known migration debt. |
| 18 | `lettre` | `"0.11.19"` rustls-tls only | 0.11.19 | SMTP send | OK. |
| 19 | `mail-parser` | `"0.11.2"` | 0.11.2 | RFC 5322 email parsing | OK; tracked by Stalwart team. |
| 20 | `async-imap` | `"0.11"` runtime-tokio | 0.11 | IMAP fetch | OK. |
| 21 | `socketioxide` | `"0.15"` features=`["extensions"]` | 0.15 | Socket.IO server | Small ecosystem, primarily one maintainer. Used by `webview_apis`/`socket`. |
| 22 | `tauri-runtime-cef` (in tauri lock only) | path = `vendor/tauri-cef/...` | **0.1.0** | CEF runtime fork | **Vendored as git submodule**; CEF support lives on `feat/cef` branch of `tauri-apps/tauri`. The fork is `tinyhumansai/tauri-cef` on `feat/cef-notification-intercept`. Submodule commit IS the pin, but if a contributor forgets `--recurse-submodules` they get nothing. |
| 23 | `cef` (tauri lock only) | `"=146.4.1"` exact pin | 146.4.1+146.0.9 | Chromium Embedded Framework Rust bindings | `cef-dll-sys` auto-downloads ~200MB of Chromium runtime on first build. Heavy. |
| 24 | `sentry` | `"0.47.0"` (root) and same (tauri) | 0.47.0 | Error reporting | Three separate Sentry projects (core, tauri shell, frontend). |
| 25 | `cpal` + `hound` + `enigo` + `rdev` + `arboard` | various 0.x | various | OS input/output: audio capture, keystroke synthesis, global hotkeys, clipboard | Heavy native FFI. `rdev` and `enigo` are notorious for permissions/security issues on macOS (require Accessibility/Input Monitoring grants); `rdev` historically has had silent input-event leakage on macOS — review usage. |
**Honourable mentions (not in top 25 but worth naming):**
- `prost = "0.14"` — protobuf, used by `opentelemetry-otlp`.
- `opentelemetry / opentelemetry_sdk / opentelemetry-otlp = "0.32"` — observability stack.
- `prometheus = "0.14"` — metrics endpoint.
- `fantoccini = "0.22.0"` (optional, behind `browser-native` feature) — WebDriver client.
- `pdf-extract = "0.10"` (optional, behind `rag-pdf`) — pulls `lopdf`, `adobe-cmap-parser`.
- `objc2 / objc2-contacts / objc2-app-kit / block2` — macOS Contacts framework + WKWebView.
- `landlock = "0.4"` (optional, linux-only) — sandbox.
- `starship-battery = "0.10"` — battery probe. Comment in Cargo.toml notes it's a **maintained fork** of the abandoned `battery` crate. Documented risk.
- `nu-ansi-term = "0.46"` and `wait-timeout = "0.2"` — both single-maintainer.
---
## 3. Top 25 Node dependencies (by importance / weight)
From `app/package.json` (the only manifest with substantive deps). Resolved versions from the **root** `pnpm-lock.yaml` (the active workspace lockfile).
| # | Package | Declared | Resolved | Role | Risk notes |
|---|---|---|---|---|---|
| 1 | `react` | `^19.1.0` | **19.2.5** | UI runtime | Current. Multiple peer-pinned copies in the graph (`19.2.5`, `19.2.3`). |
| 2 | `react-dom` | `^19.1.0` | **19.2.5** | DOM renderer | Same. |
| 3 | `@reduxjs/toolkit` | `^2.11.2` | **2.11.2** | State management | Current. |
| 4 | `react-redux` | `^9.2.0` | **9.2.0** | React bindings | Current. |
| 5 | `redux-persist` | `^6.0.0` | **6.0.0** | Persisted store | Project effectively unmaintained (last release 2022). Common churn target. |
| 6 | `redux-logger` | `^3.0.6` | **3.0.6** | Dev-time logger | Unmaintained (last release 2017). Should probably be devDep only. |
| 7 | `react-router-dom` | `^7.13.0` | **7.14.2** | Routing | Current (Remix-era). |
| 8 | `@sentry/react` | `^10.38.0` | **10.49.0** | FE error reporting | Drift accepted. Heavy: pulls `@sentry-internal/replay`, `replay-canvas`, `feedback`, `browser-utils`. |
| 9 | `@sentry/vite-plugin` (dev) | `^2.22.6` | 2.23.1 | Source maps upload | Pulls all 7 platform-specific `@sentry/cli-*` binaries (linux-x64/arm64/arm/i686, win32-x64/i686, darwin) into the lockfile — large npm footprint but only one runs on a given host. |
| 10 | `@tauri-apps/api` | `^2.10.0` (root pins `2.10.1`) | **2.10.1** | Tauri JS bindings | `resolutions` in root forces exact 2.10.1. |
| 11 | `@tauri-apps/plugin-deep-link` | `^2` | 2.4.8 | Deep links | OK. |
| 12 | `@tauri-apps/plugin-opener` | `^2` | 2.5.3 | Open URL/file | OK. |
| 13 | `@tauri-apps/plugin-os` | `^2.3.2` | 2.3.2 | OS info | OK. |
| 14 | `@tauri-apps/cli` (dev) | `2.10.0` (exact) | 2.10.0 | Build CLI | Bundles 10 platform-specific binaries (`@tauri-apps/cli-{darwin-arm64, darwin-x64, linux-arm-gnueabihf, linux-arm64-gnu, linux-arm64-musl, linux-riscv64-gnu, linux-x64-gnu, linux-x64-musl, win32-*}`). Normal for native CLIs. |
| 15 | `remotion` | `4.0.454` (exact) | **4.0.454** | Video rendering for the mascot | **HEAVY.** Pulls `@remotion/player` (also `4.0.454`) and `@remotion/zod-types`. Remotion is **commercial-licensed for companies >3 people**. Worth verifying license posture for the company shipping OpenHuman. The remotion package is **only** a dependency via the `mascot` feature — `remotion/package.json` says `"license": "UNLICENSED"`. |
| 16 | `@remotion/player` | `4.0.454` (exact) | 4.0.454 | Embedded player | Same license posture as `remotion`. |
| 17 | `three` | `^0.183.2` | **0.183.2** | 3D rendering (mascot?) | Current. Tree-shakable, but ESM ergonomics often pull more than needed. |
| 18 | `@types/three` (dev) | `^0.183.1` | 0.183.1 | Three.js types | OK. |
| 19 | `lottie-react` | `^2.4.1` | 2.4.1 | Lottie animations (mascot?) | OK. |
| 20 | `cmdk` | `^1.1.1` | **1.1.1** | Command palette UI primitive | OK. |
| 21 | `react-joyride` | `^3.1.0` | **3.1.0** | Onboarding tours | Less active maintenance; older deps internally. |
| 22 | `react-markdown` | `^10.1.0` | **10.1.0** | Markdown renderer | Current. |
| 23 | `socket.io-client` | `^4.8.3` | **4.8.3** | Socket.IO transport | Pairs with the Rust `socketioxide` server. |
| 24 | `@radix-ui/react-dialog` | `^1.1.15` | (resolved 1.1.x) | Accessible dialog primitive | Current. |
| 25 | `@noble/curves` + `@noble/secp256k1` + `@noble/hashes` + `@scure/bip32` + `@scure/bip39` + `@scure/base` | `^2.x` / `^3.x` | `2.2.0` / `3.1.0` / `2.2.0` / `2.2.0` / `2.2.0` / `2.2.0` | Crypto primitives (wallet, mnemonics, BIP32/39) | All `paulmillr/noble-*` and `paulmillr/scure-*`. Trusted, audited. Single-maintainer concentration is a *known and accepted* risk in the JS crypto ecosystem — note it. |
**Honourable mentions:**
- **Test stack (dev only):** `vitest@4.1.5`, `@vitest/coverage-v8@4.1.5`, `vite@8.0.10`, `@vitejs/plugin-react@6.x`, `jsdom@28.1.0`, `@testing-library/{react,dom,jest-dom,user-event}`.
- **WDIO/Appium for desktop E2E:** `@wdio/cli`, `@wdio/appium-service`, `@wdio/local-runner`, `@wdio/mocha-framework`, `@wdio/spec-reporter` — all on `9.27.0` family. Heavy dev-time graph (pulls `@puppeteer/browsers`).
- **Polyfills:** `buffer`, `process`, `os-browserify`, `util`, `vite-plugin-node-polyfills` — needed because some deps assume Node globals in browser. Adds bundle size; check whether all are still required after React 19.
- **Tooling-only:** `knip@6.6.2`, `eslint@9.39.4`, `prettier@3.8.3`, `husky@9.1.7`, `tsx@4.21.0`, `cross-env@10.1.0`, `tailwindcss@3.4.19`, `autoprefixer@10.5.0`, `postcss@8.5.10`.
**Resolved/declared drift worth a follow-up:** none significant in the active root lockfile — every `^X` we sampled landed on a sane minor. (Earlier confusion was from `app/pnpm-lock.yaml`, a stale secondary lockfile.)
---
## 4. Heavy / unusual / risky dependencies (flagged)
### Compile-/runtime-heavy
1. **`cef@=146.4.1+146.0.9`** + `cef-dll-sys` — downloads ~200MB Chromium on first build; pinned via exact `=` constraint (good).
2. **`tauri-runtime-cef`** + 7 vendored tauri-cef path crates — entire Tauri fork checked out as submodule (`vendor/tauri-cef`). If the submodule isn't fetched, the build silently fails to compile any of `tauri{,-build,-utils,-macros,-runtime,-runtime-wry,-plugin}`.
3. **`whisper-rs` + `whisper-rs-sys`** — whisper.cpp via FFI. Forked at `tinyhumansai/whisper-rs-sys`, **branch=`main` with NO commit pin**. Non-reproducible the moment upstream pushes.
4. **`matrix-sdk@0.16.1`** — ~12 transitive crates (`matrix-sdk-base/common/crypto/sqlite/indexeddb`, `vodozemac`, `ruma-signatures`, etc.). Documented blocker on Rust toolchain upgrades.
5. **`whatsapp-rust@0.5`** + 6 `wacore-*` transitives + own SQLite store. Mid-popularity, recent fork migration.
6. **`remotion@4.0.454`** — heavy renderer with commercial-license tier above 3 staff; package.json declares `"license": "UNLICENSED"` on the `remotion/` workspace.
### Sensitive (input/output/secret-touching)
- **Crypto:** `aes-gcm`, `chacha20poly1305`, `argon2`, `sha2`, `hmac`, `ring`, `rustls` (root). On the JS side: `@noble/*`, `@scure/*`, `ethers-core` (Rust), all signing keys.
- **Wallet/keys:** `ethers-core` + `ethers-signers` are **deprecated** upstream — migrate to `alloy`.
- **Input control:** `rdev`, `enigo`, `arboard` — privileged on macOS (TCC prompts), historically suspect for input leakage. Confirm scoping.
- **Sandbox:** `landlock` (opt-in, linux-only). Cargo features `sandbox-landlock`, `sandbox-bubblewrap` exist — feature gating is intentional.
- **Secrets in env:** `dotenvy` — fine.
### Forked / git deps (non-reproducible if not pinned)
| Crate | Source | Pin | Risk |
|---|---|---|---|
| `whisper-rs-sys` | `tinyhumansai/whisper-rs-sys`, **branch = main** | **NONE (HEAD floats)** | **HIGH — non-reproducible** |
| `tauri-plugin-opener` | `tauri-apps/plugins-workspace` | `rev = c6561ab6b4f9e7f650d4fc8c53fd8acc9b65b9b2` | OK, full SHA |
| `tauri-plugin-deep-link` | same | same rev | OK |
| `tauri-plugin-global-shortcut` | same | same rev | OK |
| `tauri-plugin-single-instance` | same | same rev | OK |
| `tauri`, `tauri-build`, `tauri-utils`, `tauri-macros`, `tauri-runtime`, `tauri-runtime-wry`, `tauri-plugin` | path = `vendor/tauri-cef/crates/*` (submodule) | submodule commit pin | OK if submodule fetched |
### Single-maintainer / abandoned-risk crates
- `starship-battery` — **explicit fork** of abandoned `battery` (acknowledged in comment).
- `nu-ansi-term`, `wait-timeout`, `enigo`, `rdev`, `fantoccini`, `socketioxide`, `whatsapp-rust` family — small core team / single-org maintainership.
- `redux-logger` and `redux-persist` (JS) — both effectively unmaintained.
### Drift / version-pin oddities
- `vite "^8.0.0"` in `app/package.json` → resolves to `vite@8.0.10` in root `pnpm-lock.yaml`. Fine. **However**, `app/pnpm-lock.yaml` is a **stale, parallel lockfile** still pinning `vite@7.3.2`. **Either delete `app/pnpm-lock.yaml` or document why two pnpm locks coexist** — having two is a footgun and an out-of-sync `app/` lockfile WILL eventually get picked up by some script.
- Two separate `Cargo.lock` files (root + `app/src-tauri`). They agree on every package they share (sampled: `tokio@1.52.3`, `reqwest@0.12.28`, `serde@1.0.228`, `axum@0.8.9`, `hyper@1.9.0`, `rustls@0.23.40`), but **divergence is possible** because they are independent resolutions. 200 crates exist only in tauri's lock, 159 only in root's.
### Engines / toolchain pins
- Rust: **`1.93.0` pinned** in `rust-toolchain.toml` — explicit upper bound because `matrix-sdk` recursion bug at ≥1.94 (issue 6254). **Real toolchain debt.**
- Node: `app/package.json` requires `node >= 24.0.0`. `packages/npm/package.json` requires `node >= 18`. Mismatched floors across packages.
- pnpm: `package.json` pins `pnpm@10.10.0` exactly via `packageManager` (good).
---
## 5. License posture
No `cargo-deny` or `license-checker` runs were performed; this is a heuristic name-based scan of `Cargo.lock` and `pnpm-lock.yaml`.
| Concern | Status |
|---|---|
| **AGPL transitive Rust deps** | None found (heuristic name match). |
| **GPL transitive Rust deps** | None found (heuristic name match). |
| **LGPL (e.g., GTK)** in tauri lock | `atk-sys`, `cairo-rs`, `gdk-*`, `gio-*`, `glib`, `gtk`, `gtk-sys`, `pango`, `soup3` — all present on the Linux build path via `tauri-runtime-wry`. These are LGPL and **dynamically linked** (system libraries) — fine for proprietary distribution. |
| **`webp-converter` (npm, in `remotion/`)** | Bundles native binaries; verify license file in dist. |
| **`remotion@4.0.454`** | **The biggest license question.** Remotion's license switched to dual personal/commercial; the `remotion/package.json` in this repo says `"license": "UNLICENSED"`. If the entity shipping OpenHuman has >3 employees, a Remotion company licence is required. **Action item: get sign-off from legal.** |
| **`@puppeteer/browsers`** (dev only) | Apache-2.0, OK. |
| **`openssl-sys`** | Present in tauri lock (transitive). OpenSSL's "OpenSSL License" is Apache-2.0 since 3.0; the root `openhuman` prefers `rustls`. Worth confirming OpenSSL isn't linked into shipped binaries. |
| **`whisper-rs-sys` fork** | Whisper.cpp upstream is MIT; fork commit history not audited. |
**Summary:** no obvious GPL/AGPL contamination of the Rust core. The two real license items are (a) **Remotion commercial-tier eligibility** and (b) confirming the OpenSSL/GTK linkage on Linux is dynamic, not static.
---
## 6. Internal coupling observations
### Rust — module-level import counts (`use crate::openhuman::{module}` matches)
Outgoing (afferent) = how many files in module X import from another module.
Incoming (efferent) = how many files outside module X import from X.
| Module | Total LOC | Outgoing imports | Incoming imports | Notes |
|---|---:|---:|---:|---|
| `memory` | 62,134 | 78 | **146** | Central hub. Most-depended-on module. Stable abstraction layer. |
| `agent` | 44,683 | **220** | 90 | The orchestrator. Imports more than anyone else. |
| `tools` | 37,638 | **224** | 60 | Heaviest outgoing. Reaches into `memory` (32 files), `agent` (31), `inference` (4). |
| `inference` | 29,794 | 55 | 114 | Second hub. Many depend on it; it depends on few. Healthy. |
| `channels` | 32,524 | 84 | 8 | Self-contained provider layer (telegram, discord, web). Healthy fan-out. |
| `composio` | 26,105 | 71 | 18 | External SaaS integrations (gmail, slack, notion, github). Healthy. |
### Cross-module edge matrix (Rust)
`use crate::openhuman::{tgt}` occurrences in files under `src/openhuman/{src}`:
| from \ to | memory | agent | tools | channels | inference | composio |
|---|---:|---:|---:|---:|---:|---:|
| **memory** | — | 5 | 1 | 2 | 4 | 2 |
| **agent** | 34 | — | 36 | 0 | 40 | 7 |
| **tools** | 32 | 31 | — | 0 | 4 | 2 |
| **channels** | 6 | 14 | 7 | — | 13 | 1 |
| **inference** | 0 | 2 | 1 | 0 | — | 0 |
| **composio** | 19 | 6 | 2 | 0 | 0 | — |
### Cycle-shaped relationships (Rust)
These are **bidirectional at module granularity**. They're not strict cycles at file granularity (no `A.rs` ↔ `B.rs` direct pair found), but they make refactoring risky:
1. **`agent ↔ tools`** — agent imports tools 36×, tools imports agent 31×. Heavy mutual coupling; the two modules effectively co-design each other. Most concerning structural finding.
2. **`agent ↔ memory`** — agent imports memory 34×, memory imports agent 5×. Mostly one-way but the back-edge exists.
3. **`tools ↔ memory`** — tools imports memory 32×, memory imports tools 1×. Effectively one-way with a small leak.
4. **`agent ↔ inference`** — agent imports inference 40×, inference imports agent 2×. One-way modulo a tiny leak.
**Net:** the structural seam between `agent` and `tools` is the riskiest place to refactor without breaking compilation in many files. `memory` is the most stable abstraction (146 incoming, 78 outgoing — high I = `Ce/(Ca+Ce)` = ~0.35, healthy).
### TypeScript — module-level import counts (relative `../{module}/` matches)
| from \ to | components | lib | features | services | store | pages | hooks | providers | utils |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| **components** | — | 170 | 15 | 66 | 53 | 1 | 46 | 19 | 125 |
| **lib** | — | — | — | 5 | 16 | — | — | — | 9 |
| **features** | 1 | 5 | — | 9 | 11 | 1 | — | 5 | 7 |
| **services** | — | 7 | 2 | — | 20 | — | — | — | 28 |
| **store** | 1 | 4 | 1 | 9 | — | — | — | — | 4 |
| **pages** | 102 | 42 | 5 | 33 | 32 | — | 14 | 4 | 27 |
| **hooks** | — | 2 | 5 | 20 | 8 | — | — | 6 | 11 |
| **providers** | — | 3 | — | 13 | 23 | — | 3 | — | 6 |
| **utils** | — | 1 | — | 28 | 5 | — | — | — | — |
### TS bidirectional pairs (apparent cycles at module granularity)
- **`components ↔ store`** (53 ↔ 1) — mostly one-way, but stores import back from components.
- **`components ↔ features`** (15 ↔ 1) — mostly one-way.
- **`services ↔ store`** (20 ↔ 9) — **bidirectional**, this is a smell. Services should not depend on stores; stores should consume services.
- **`lib ↔ store`** (16 ↔ 4) — bidirectional; stores reach into `lib`, but `lib` also reads `store`. Layering inversion.
- **`utils ↔ services`** (28 ↔ 28) — fully symmetric; utils should be leaf-only. **Concerning.**
- **`utils ↔ store`** (5 ↔ 4) — same pattern.
- **`pages ↔ features`** (5 ↔ 1) — mostly one-way.
**Net:** the `services / store / utils / lib` quadrant has multiple back-edges that imply unclear ownership. `components` and `pages` are mostly downstream consumers (healthy).
---
## 7. Dependency-health score
**Score: 6 / 10**
Strengths:
- Direct deps are mostly current; lock files actually resolve to recent minor versions.
- TLS, async runtime, JSON, observability stacks are conventional and modern (axum 0.8 / tokio 1.52 / rustls 0.23 / sentry 0.47 / opentelemetry 0.32).
- All Tauri plugin git deps are pinned by full SHA.
- Exact pin on `cef = "=146.4.1"`.
- Rust toolchain pin is explicit and documented.
- License posture is OK on the Rust side (no GPL/AGPL).
- No GitHub-tarball deps in npm; no `file:` deps.
Deductions:
- **−1** `whisper-rs-sys` git dep pinned to **branch=main, no rev**. Non-reproducible builds whenever upstream pushes. Easy to fix.
- **−1** `ethers-rs` is the deprecated path; significant migration debt to `alloy`.
- **−1** `agent ↔ tools` heavy bidirectional coupling (36/31), plus three TS cycles (`services↔store`, `utils↔services`, `lib↔store`).
- **−0.5** Two `Cargo.lock` files independently resolving the same crates (root + `app/src-tauri`). No active divergence today, but it's fragile.
- **−0.5** Two pnpm lockfiles (`pnpm-lock.yaml` and `app/pnpm-lock.yaml`), and they **DO disagree** (root: vite 8.0.10; app: vite 7.3.2). Delete the stale one.
- **−0.5** Remotion license posture unverified (`UNLICENSED` declared on `remotion/package.json`, commercial tier above 3 staff).
- **−0.5** Single-maintainer/abandoned npm deps: `redux-logger` (2017), `redux-persist` (2022). Behavioural debt.
- **−0.5** Rust toolchain **pinned below 1.94** due to `matrix-sdk` recursion bug. Real upgrade blocker.
Net: a competent dependency posture with a few concrete, fixable issues. Most are documented in code comments — which is itself a positive signal that the maintainers are aware.
---
## 8. Quick wins (sorted by ratio of effort to risk reduction)
1. **Pin `whisper-rs-sys` to a commit SHA** instead of `branch = "main"`. 1-line change in two `Cargo.toml` files.
2. **Delete `app/pnpm-lock.yaml`** (the stale duplicate) or document the workflow that re-creates it. Right now the two files actively disagree on `vite`.
3. **Resolve `app/pnpm-lock.yaml` vs root** — pick one, document, enforce in CI.
4. **License audit for Remotion** before next release. One legal email.
5. **Migration plan for `ethers-rs` → `alloy`** as a tracked tech-debt ticket.
6. **Decide whether `redux-logger` should be devDependency-only**.
7. **Refactor the `agent ↔ tools` seam**: extract a shared trait crate or a `agent_tool_interface` module so the back-edges go through one defined surface.
8. **Workspace-ify Cargo**: if `openhuman_core` and `OpenHuman` (tauri) shared a `[workspace]`, you'd have one lockfile, one resolution, less drift potential. The blocker is currently the `[patch.crates-io]` divergence (root patches `whisper-rs-sys` only; tauri patches whisper + 7 tauri crates), so it's a real refactor, not trivial — but a workspace would catch a class of bugs.
9. **Add `cargo-deny` + `license-checker` to CI** so the GPL/AGPL question is answered by tooling, not heuristics.
---
## Notes on method (transparency)
- All Rust dep counts are from manifest sections (`[dependencies]`, `[dev-dependencies]`) and `Cargo.lock` `name = ` entries. No `cargo tree` ran.
- All Node dep counts are JSON-parsed counts of `dependencies` + `devDependencies`.
- Internal coupling counts are `grep` over `use crate::openhuman::X` (Rust) and `from "../X/..."` / `from "../../X/..."` (TS). These match the *file count of an import edge*, not the unique-pair edge count, so the matrix shows "import statement intensity" — useful for spotting hot seams, but not a substitute for an AST-based analysis.
- "Single-maintainer" classification is heuristic (crate ecosystem familiarity), not pulled from `crates.io` API.
- The `app/pnpm-lock.yaml` discrepancy is real and reproducible — `grep -E "^ vite@" /tmp/openhuman/app/pnpm-lock.yaml` shows `vite@7.3.2`, but the root lock has `vite@8.0.10`.
## Source files referenced
- `/tmp/openhuman/Cargo.toml`
- `/tmp/openhuman/Cargo.lock`
- `/tmp/openhuman/app/src-tauri/Cargo.toml`
- `/tmp/openhuman/app/src-tauri/Cargo.lock`
- `/tmp/openhuman/package.json`
- `/tmp/openhuman/app/package.json`
- `/tmp/openhuman/remotion/package.json`
- `/tmp/openhuman/packages/npm/package.json`
- `/tmp/openhuman/pnpm-lock.yaml` (active workspace lock)
- `/tmp/openhuman/app/pnpm-lock.yaml` (stale; out of sync)
- `/tmp/openhuman/pnpm-workspace.yaml`
- `/tmp/openhuman/rust-toolchain.toml`