mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
Edward Cheng-I Wu
75070eec84
* feat(evals): #653 reviewer-calibration suite scaffolding — corpus assembler, isolated dispatcher, deterministic scorer, pre-registered rubric/RUN_PLAN (corpus freeze pending PDF access) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2iNYa6YYYaPUwD2Z2Jr5e * feat(evals): #653 freeze the ICLR 2026 calibration corpus manifest (12 papers) + shared PDF-text normalization Corpus freeze (PR-A of #653): `corpus/papers.json` (label-free, 6+6 ICLR 2026 papers by the pre-registered seed; pypdf 6.11.0; pool hashes unchanged from the 2026-08-07 selection) and `manifests/gold_labels.json` (public Decision note ids + strings). No page-cap exclusion fired; `verify` PASS. First real-PDF contact found a hashing defect: pypdf emits lone UTF-16 surrogates from math fonts (61 in one sampled manuscript) and strict UTF-8 encoding raised, so `extracted_text_sha256` was uncomputable. The normalization now lives in one shared module (`scripts/_calibration_pdf_text.py`: NFC + lone-surrogate -> U+FFFD), imported by both the assembler and the dispatcher so freeze/verify/dispatch hash identical bytes; the rule is recorded in the manifest's `extraction.text_normalization` and `verify` fails hard on rule drift (a rule, not a version). Two tests added (41 total). `scripts/fetch_calibration_corpus.py` is the authenticated OpenReview operator tool that produces the freeze input, so the "third-party reconstruction" claim in the README is backed by a runnable path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * refactor(evals): #653 simplify pass — shared hashing/fence/git-state, contract 1.1 docs /simplify findings applied (reuse, simplification, efficiency, altitude): - `_calibration_pdf_text.py` owns `sha256_hex` + `pdf_facts` (bytes hashed and parsed from one read via BytesIO; `extract_text=False` lets `verify` skip extraction when the pypdf version cannot be compared); surrogate replacement is one `re.sub` pass. Both the assembler and the dispatcher import it. - dispatcher reuses E4's closed data-fence grammar (`_delimited`), `_git_state` (declares unknown provenance dirty instead of raising), and the evidence path guard (`assert_plain_file`: rejects symlinked parent components, not just the leaf); one `_prepare` preamble for both stages; a text-hash mismatch now names its cause (installed vs manifest pypdf version). - assembler: exclusion rows stay dicts, `pool_list_mismatches` shared by freeze/verify, exclusion set built once. - scorer: `confusion`/`bootstrap_ci` take (predicted, gold) pairs (same RNG stream as before), `Counter` for the exact-mode vote, dead `_path` dropped. - RUN_PLAN/README: measurement contract 1.0 is closed to new rows (#664); the run publishes under 1.1 with its pre-registration record + write-once execution manifest (dispatcher/scorer support lands with the scored run). Re-freeze after the refactor reproduces papers[] and gold_labels byte-for-byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * fix(evals): #653 Iron Rule #7 at the two whole-file call boundaries + paper-id shape check Security review round 1 (first-party) found two below-threshold gaps and both are verified real: - The calibration dispatcher omitted E4's `DATA_BOUNDARY` sentence on the field-analyst call (the one E4 call that carries it, because `field_analyst_agent.md` states no untrusted-material rule of its own). Restored, and a fitted `REPORT_BOUNDARY` added on the synthesizer call, whose agent file is likewise dispatched whole with no such rule. Pinned by a transport-capture test that checks both sentences precede their fence. - Paper ids are spliced into file names (`<id>.pdf`, `cards/<id>/`) but `load_pool` accepted any non-empty string. Ids now must match `^[A-Za-z0-9_-]+$` (OpenReview's forum-id shape) in the assembler and the fetch tool; test pins the refusal. 43 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * fix(evals): #653 codex round 1 — dispatch/verify invariant parity, card-path guard, scorer completeness Codex round 1 (gpt-6-astra xhigh) findings 2-7, 10, 11 and the cheap half of 9, each re-verified first-party before the change: - dispatcher: frozen cards go through the same plain-file guard as PDFs and agent files (a symlinked card1.md -> gold_labels.json was readable); the manifest's text_normalization rule and page_count are checked before dispatch, so dispatch and verify enforce the same manuscript invariants; transport-failure artifacts keep the partial stdout and stderr verbatim; every call attempt records RFC-3339 start/complete and prompt/output hashes into the panel record and cards freeze (the per-call evidence the heldout-measurement/1.1 execution manifest is built from). - verify: label must match decision_raw under the label transform; paper count and per-class label counts must equal the recorded quotas (synchronized paper+label removal no longer passes). - scorer: a second record for the same paper/replicate is a hard error, not a silent overwrite; a gold paper with no complete ensemble blocks the full tier; an A1 override needs its verbatim `raw` excerpt present in synthesis.md. Nine regression tests added (52 total). Real-corpus verify still PASS. Not addressed here (need a decision): finding 1 (camera-ready format leaks the accept label) and finding 8 (numeric seat scores vs categorical seat contract); finding 9's manifest/row builders land with the scored run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * fix(evals): #653 drop the numeric score axis — protocol Phase 2 forbids AUC, seats are categorical Codex round 1 finding 8, verified against the source: the seat contract (eic/methodology/... agents) emits criterion-bound categorical judgements and states "Do not total, weight, average"; `calibration_mode_protocol.md` Phase 2 says "Do not report AUC: there is no continuous rubric score." The scorer nevertheless extracted a `Weighted Average` figure (a retired field) and RUN_PLAN promised AUC + score variance, so a conforming run would have published null numerics against a plan that promised them. The scorer now reports only what the protocol's full-tier table names: confusion matrix, balanced accuracy, FNR, FPR (bootstrap CIs), exact-label agreement (count/share/target-set size, with the binary-gold caveat), and replicate stability as categorical agreement (on side, on exact label). AUC is emitted as an explicit NOT REPORTED line. RUN_PLAN and the test fixtures follow. 52 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * docs(evals): #653 mark the 2026-09-06 corpus SUPERSEDED (layout leaks the label, #828); RUN_PLAN model currency - README/RUN_PLAN: the frozen ICLR 2026 corpus is a harness-rehearsal corpus only — camera-ready replacement makes accepted PDFs visibly different from rejected submission PDFs (6/6 + 6/6; 30/30 in a fresh accepted-pool sample). No profile or measurement row may be published from it; the gold corpus becomes an ICLR 2027 submission-time capture. The "Why ICLR 2026" rationale is kept as pre-registered and annotated with the two facts that now cut against it (layout leak; Fable 5.1's 2026-06 cutoff covers the decisions). - RUN_PLAN + dispatcher default: subject `claude-fable-5` -> `claude-fable-5-1`, judge `gpt-5.6-sol` -> `gpt-6-astra` (provisional, #783 policy). Pre-dispatch edits, not amendments. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EehvYnmG8Xym5tXhTLfVm1 * feat(evals): #828 layout-tell guard at corpus freeze — refuse a corpus whose page-1 layout is not constant `assemble_calibration_corpus.py freeze` now reads page 1 of every cached PDF and evaluates four venue-template signals (published-as header, under-review header, "Anonymous authors", >=10 bare three-digit line numbers). Any signal that is not constant across the whole corpus refuses the freeze with the per-class counts; a uniform corpus records `layout_tell_check` in papers.json. `verify` recomputes the same check (skipped with a warning when a PDF is not cached; a manifest without the block warns). On the superseded 2026-09-06 ICLR 2026 corpus every signal is 6/0, so `verify` now FAILs on it by design. Shared `_open_reader` + `first_page_text` in the PDF helper. Six tests (signal detection, full and partial separation refused, uniform freeze + verify round-trip, missing-PDF skip, pre-check manifest warning). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * feat(evals): #653/#828 rehearsal fixes + heldout-measurement/1.1 manifest and row builders Rehearsal 2026-09-06 (2 papers x 1 replicate, blocked at the first call by a rejected API key) exposed three dispatcher gaps, all fixed with tests: - credential preflight: zero-cost `GET /v1/models` before the first billed call; a definitive 401/403 refuses (key never echoed), network trouble is `inconclusive` and proceeds; outcome recorded in every record - credential rejection mid-run (`Failed to authenticate` / `API Error: 401` / `Not logged in`) is never retried (`CredentialRejected`); other transport failures keep the single retry - an aborted cards stage writes `runs/blocked-cards-<paper>.json` with its per-call rows instead of losing them; both stages share one record writer 1.1 contract substrate (RUN_PLAN "pre-registration record + execution manifest" item): - `dispatch_calibration_panel.py --stage manifest` folds the completed call rows of one attempt (frozen cards + panel records; `load_attempt` refuses mixed attempt identities) into a write-once, schema-validated `execution-manifest.json` - `build_calibration_measurement_row.py` composes the 1.1 row: plan and rubric hashed and compared against `frozen_commit` (drift refuses; dirty commit refuses), manifest re-derived from the records and compared field-for-field, judge rows required (no judges, no row), agreement recomputed by the checker's own `judge_divergence` (extracted from `check_heldout_measurement_report.py`, behaviour unchanged), validated by the checker before a write-once write - adjudication rubric gains `## Resolution direction` (flags_only, I13 lower-bound labelling); README tooling section; RUN_PLAN names the row builder; DATA_FLOWS names the dispatcher's preflight touchpoint; scorer docstring de-staled (no score axis); pytest manifest +1 No calibration number is recorded anywhere in the repository. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * fix(evals): #653/#828 codex round 2 — bind every row input to its attempt, harden the guards 12 of 13 findings applied (gpt-6-astra xhigh, read-only exec): - P1 foreign metrics: scorer output is bound to the attempt (per_panel keys == the complete panel records here, attempt ids match, n_papers matches) - P1 raw drift: record admission re-hashes every completed call's raw output against output_sha256 (manifest stage and row builder alike); prompts are not retained (they embed the manuscript) - P1 preflight redirects: the probe uses a no-redirect opener (a 3xx is `inconclusive`) and skips a non-https ANTHROPIC_BASE_URL - P2 estimand: class-A adjudication is now pre-registered as bidirectional (every synthesis decision transcribed blind and compared with the grammar), so the row publishes a point_estimate instead of an I13 "lower bound" that only meant audit coverage - P2 pre-write parity with R5: manifest timestamps parsed and ordered before the write; declared claims checked against the local manifest - P2 strict JSON: inputs parsed with the checker's strict loader, outputs serialized with allow_nan=False and round-tripped - P2 judge failures: `--blocked-run` ledger entries merge into attempts.blocked_runs (I11) - P2 admission by content: suite/stage/status/provenance from the record body, never the filename; blocked records are identity-checked too - P2 cards re-run: a reused evidence dir refuses (write-once stage records) - P2 auth signature: anchored at the start of stdout/stderr and limited to exit-code failures; a timeout's partial prose is never a credential error - P2 layout signals: phrase tests run on whitespace-folded text - P2 partial PDF cache: verify checks every cached PDF (can refuse, cannot clear) instead of skipping the guard - P3 real `git show` test for sha256_at_commit on a temporary repository Partially applied: "distinguish unobservable signals from absence" (not built; the constancy rule is pre-registered as stricter by design). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * fix(evals): #653/#828 shared transport — capture every assistant message, fence the subject's config Rehearsal take 2 (2026-09-06/07, 8 billed calls on the first paper) found two transport defects in `ClaudeCliTransport`, shared by the E4 and the calibration dispatchers: - text-mode `claude -p` prints only the LAST assistant message: the first paper's synthesis (long enough to be continued) came back starting mid-table, with the Editorial Decision Letter and its `### Decision:` line in the missing head. The transport now runs `--output-format stream-json --verbose` and concatenates the text blocks of every assistant message; an error result or an unreadable stream is a TransportFailure that keeps the raw bytes. - `--bare` does not fence the subject: a two-call probe on 2.1.260 showed the operator's whole global CLAUDE.md arriving as a system-reminder, plus `settings.json` `language` and the output style (the seats appended Traditional-Chinese "plain-language summary" sections). The subject now runs with an allowlisted environment (PATH/HOME/LANG/TMPDIR/TERM/USER/ SHELL + ANTHROPIC_*; no CLAUDE_* inherited from a parent session) and a per-transport empty `CLAUDE_CONFIG_DIR`; the same probe then reported no instruction beyond the SDK identity line and the date. E4 tests: one fake updated to emit stream-json; five new tests (message joining, error/junk results, unreadable-stream failure with bytes, environment allowlist, argv/env of a live call). Calibration docs and the panel record's `dispatch` field describe the new recipe (pre-dispatch change, no amendment). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * fix(evals): #653/#828 codex round 3 on the shared transport — eviction signals, LF framing, network env, failure evidence Five P2 findings (gpt-6-astra xhigh, read-only exec), all applied: - refusal-fallback eviction: assistant `supersedes` and system `model_refusal_fallback.retracted_message_uuids` (wire fields verified in the installed CLI 2.1.260) drop retracted partials before concatenation - NDJSON split on LF only (`str.splitlines` also splits on U+0085 / U+2028 / U+2029 inside a JSON string); CRLF tolerated - environment allowlist keeps documented network/TLS inputs (proxies, NODE_EXTRA_CA_CERTS, SSL_CERT_*, CLAUDE_CODE_CLIENT_*); an apiKeyHelper that needs more is documented as unsupported behind the fence - transport failures carry assistant TEXT in `stdout` and the raw stream in `raw_stdout`; a framing-only stream is "no model response" (E4 no longer writes stream metadata as a partial response); both dispatchers preserve the raw stream as `*.transport-stream.jsonl` - a structured error result (`[TRANSPORT: result <subtype>]`, diagnostic in stdout) is classified by the calibration retry loop like the plain-text startup failure: a credential rejection is never retried E4 tests +6 (256), calibration +1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * feat(evals): #653/#828 keep the raw stream of successful calls as evidence `ClaudeCliTransport.last_raw_stdout` exposes the stream-json framing of the most recent successful call; the calibration dispatcher writes it next to the text as `<label>.transport-stream.jsonl`, so the next rehearsal shows how many assistant messages a deliverable spanned (the 2026-09-06 synthesis lost its head to exactly that). Probe 2026-09-07: a 12,000-line reply at effort low arrived as ONE text message after a thinking-only message, so the head loss is attributed to multiple text messages in one turn (likely interleaved thinking at xhigh), not to an output-length continuation; the parser covers both. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bJSoFkGeRaWTMJodk4VGh * fix(evals): #653/#828 allow requiring a successful credential preflight * fix(calibration): bind audited decisions and preserve failed dispatch evidence * fix(transport): retain truncated UTF-8 output as byte evidence --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
123 lines
12 KiB
Markdown
123 lines
12 KiB
Markdown
# Data Flows: What Leaves the Machine, What Is Stored, and for How Long
|
|
|
|
**Purpose.** ARS touches the network in a small number of places and persists a small
|
|
number of local stores. Each is documented at its own feature page; this file is the
|
|
single user-facing map — one row per network touchpoint and one row per local store.
|
|
|
|
**Origin.** ISO/IEC 42001-spirit gap assessment
|
|
([`audits/iso42001-spirit-gap-assessment-2026-08-17.md`](../audits/iso42001-spirit-gap-assessment-2026-08-17.md),
|
|
finding T-7, [#758](https://github.com/Imbad0202/academic-research-skills/issues/758)).
|
|
Transparency here is one of this repo's distilled operating principles (with
|
|
informative anchors to ISO/IEC 42001) — not an ISO-mandated artifact.
|
|
|
|
## Scope
|
|
|
|
This page covers the network calls and stores that **ARS's own scripts** perform. The
|
|
boundaries around that scope:
|
|
|
|
- **The Claude session itself is not on this map.** Everything you type, every file the
|
|
session model reads, and any web search/fetch the model performs while executing the
|
|
research skills travels over your Claude platform connection under your Anthropic
|
|
account settings. That path exists with or without ARS and is governed by the
|
|
platform, not by this repo.
|
|
- **Maintainer/evaluation harnesses are not user paths.** A few repo scripts exist
|
|
only for maintainers running measurements (e.g. `scripts/dispatch_e4_panel.py`
|
|
and `scripts/dispatch_calibration_panel.py` through `claude -p` — the latter
|
|
also sends the operator's own `ANTHROPIC_API_KEY` to the Anthropic API's
|
|
`GET /v1/models` as a zero-cost credential preflight before the first billed
|
|
call — `scripts/run_review_criteria_constructive_value.py` through
|
|
the Codex CLI, `scripts/check_ranking_lift.py` through `gh api`). They send
|
|
content through locally authenticated CLIs when a maintainer invokes them, are
|
|
never triggered by any user-facing feature, and are deliberately excluded from
|
|
the touchpoint tables below.
|
|
- **Nothing here publishes.** No ARS component is designed to submit, post, or upload
|
|
your work anywhere autonomously — every network row below is a *lookup* (sending
|
|
queries or citation metadata to read public indexes), an *update check*, or an
|
|
*explicitly consented* verification call. This is the same first-party scope boundary
|
|
stated in [`POSITIONING.md`](../POSITIONING.md), and it inherits that page's
|
|
qualifier: a scope boundary and review criterion, not a runtime guarantee.
|
|
|
|
## Network touchpoints
|
|
|
|
### The citation-verification gate (four bibliographic indexes)
|
|
|
|
The deterministic citation-existence gate (#182) fires at the Stage 2.5 / 4.5
|
|
integrity gates or on standalone `verify_passport.py` / `verify_citation` calls,
|
|
cache-through by default. Each resolver sends the same payload class — identifiers
|
|
(DOIs/arXiv ids) and title query strings of your references; author/year metadata is
|
|
used locally for matching, not transmitted — and all four work
|
|
without any account or key; keeping the gate key-free is a deliberate reproducibility
|
|
choice. Off switch: don't run script-backed verification; prompt-only modes make no
|
|
calls.
|
|
|
|
| Resolver | Endpoint | Credentials (optional) | Per-index note |
|
|
|---|---|---|---|
|
|
| `scripts/semantic_scholar_client.py` | `api.semanticscholar.org` | `S2_API_KEY` (raises rate limit) | |
|
|
| `scripts/openalex_client.py` | `api.openalex.org` | `OPENALEX_API_KEY` | Polite-pool email sent if you configure one |
|
|
| `scripts/crossref_client.py` | `api.crossref.org` | none (polite email optional) | Polite-pool email rides the `User-Agent` header |
|
|
| `scripts/arxiv_client.py` | `export.arxiv.org` | none | ToU-aligned ≥3 s pacing |
|
|
|
|
### Everything else
|
|
|
|
| Touchpoint | When it fires | What is sent | Recipient | Credentials | Off switch |
|
|
|---|---|---|---|---|---|
|
|
| Chinese-literature resolver (`scripts/chinese_literature_client.py`, #595) | **Callable client only (no CLI wrapper) — deliberately NOT wired into the four-index verification gate** (the same reproducibility choice) | DOI prefixes / DOIs of the works you resolve; the PubMed path additionally sends your NCBI contact email (required by E-utilities) and bibliographic search coordinates (journal/volume/page, author/year) | `doi.org` (RA lookup + resolution), `hdl.handle.net`, NCBI E-utilities (`eutils.ncbi.nlm.nih.gov`) | NCBI email required for the PubMed path; NCBI API key optional (does not relax the client's polite pacing) | Don't call the client |
|
|
| Claim-standing discovery adapters (`scripts/claim_standing_discovery.py`, #655) | Only under an explicit consent-bound query plan; the evaluation substrate is offline by default | **Claim-derived search query strings** (they can derive from unpublished claims — hence the consent gate and the per-transmission ledger) + date filters | The same four indexes above | none (fixed User-Agent; the resolver clients' env keys are not consumed) | No consent → no calls; every transmission is ledgered |
|
|
| Timeline bootstrap (`scripts/bootstrap_timeline_yaml.py`, v3.9.4 opt-in) | Standalone CLI you invoke to seed `timeline.yaml` from a literature corpus; `--dry-run` makes no calls | DOIs of your corpus entries | `api.crossref.org` | none (needs the optional `requests` package; absent, lookups are treated as an outage) | Don't run it, or pass `--dry-run` |
|
|
| Cross-model verification transport | Only when `ARS_CROSS_MODEL` is configured **and** you give explicit per-session consent — the env var is configuration, not consent ([`shared/cross_model_verification.md`](../shared/cross_model_verification.md)) | Up to **manuscript content**: integrity-gate samples, the blind Devil's-Advocate critique input, the full paper for the consent-gated Reviewer-2 seat, checkpoint judgments | `api.openai.com`, `generativelanguage.googleapis.com`, or the OpenAI-compatible base URL you set (`ARS_OPENAI_COMPAT_BASE_URL`, e.g. DeepSeek) | `OPENAI_API_KEY` / `GOOGLE_AI_API_KEY` / `ARS_OPENAI_COMPAT_API_KEY` (required for this feature) | Leave `ARS_CROSS_MODEL` unset (zero calls), or decline consent per session |
|
|
| Codex audit wrapper (`scripts/run_codex_audit.sh`) | Only when a human, CI step, or SubagentStop hook invokes it (same-session in-LLM invocation is forbidden by its header contract); `--dry-run` writes and sends nothing | Audit prompts containing the **deliverable and supporting files' contents** | OpenAI, through the local Codex CLI login | Codex CLI auth | Don't invoke it |
|
|
| ChatGPT-subscription citation transport (`scripts/cross_model_codex_transport.py`, #630) | Only when `ARS_CROSS_MODEL_TRANSPORT=codex`; citation-integrity calls **only** (never DA / reviewer / judgment calls) | One reference's citation text plus its exact `citation_context` — the sentence where it is cited, which can contain unpublished manuscript text — sent through the local Codex CLI in a read-only sandbox with an auth-only ephemeral home | OpenAI, through your Codex CLI ChatGPT login | Codex CLI subscription login | Unset the transport selector; any other value fails visibly, no fallback |
|
|
| SessionStart update check (`scripts/ars_update_check.sh`, #544; plugin installs only) | At most one *successful* check per 24 h (a failed attempt writes no state and may retry at the next session start); 3-second total ceiling, redirects followed, silent on failure | **No user data** — fetches a public `plugin.json` and compares versions | `raw.githubusercontent.com` by default; `ARS_UPDATE_CHECK_REMOTE_URL` overrides the endpoint | none | `ARS_UPDATE_CHECK=0` |
|
|
| Manual smoke tests (`scripts/cross_model_smoke_test.sh`, `scripts/cross_model_smoke_test_codex.sh`) | Only when you run them by hand; CI never does | Public sample citation metadata | The provider under test | The provider's key / login | Don't run them |
|
|
|
|
Notes:
|
|
|
|
- **Agent-side lookups use the same indexes.** Outside the script clients, the
|
|
research/verification agents (`bibliography_agent`, `source_verification_agent`,
|
|
`integrity_verification_agent`) query the same four indexes at ingest and
|
|
verification time, following the per-index API protocol docs under
|
|
`deep-research/references/`. Same payload class (citation metadata), same
|
|
endpoints; executed through the session's tooling.
|
|
- Resolver clients never log or echo credentials; polite-pool emails and API keys are
|
|
stripped from error messages (see each client's redaction comments).
|
|
- CI runs against checked-in synthetic fixtures
|
|
(`scripts/test_transport_fixture_citation_gate.py`); no CI job performs live
|
|
resolver or provider calls.
|
|
|
|
## Local stores
|
|
|
|
| Store | Path | Content | Lifetime | How to delete |
|
|
|---|---|---|---|---|
|
|
| Citation-verification cache | `~/.cache/ars/verification.db` (override: `ARS_VERIFICATION_CACHE_PATH`) | Per-citation resolver outcomes (SQLite) | 90-day TTL per entry — expiry means a cache miss, not deletion; expired rows persist on disk until overwritten by a later re-verification, invalidated, or the file is deleted. Staleness advisory after `ARS_CACHE_STALE_ADVISORY_DAYS` (default 30), live re-validation via `ARS_CACHE_REVALIDATE=1` | `/ars-cache-invalidate <citation_key>` per key, or delete the file |
|
|
| Update-check state | `~/.cache/ars/` (override: `ARS_UPDATE_CHECK_STATE_DIR`) | A state label (`UP_TO_DATE` / `UPDATE_AVAILABLE`) plus installed and remote version strings | Re-fetched when older than 24 h | Delete the directory; `ARS_UPDATE_CHECK=0` stops new writes |
|
|
| Retraction-status cache (`scripts/retraction_status.py`) | A caller-supplied SQLite path (no default location) | DOI-keyed resolver observations with timestamps | No automatic expiry; observations older than the 30-day threshold are marked stale, not deleted | Delete the file |
|
|
| Material Passport + project ledgers | The passport path **you** name per run (never a hidden global location) | Your research content: corpus entries, read-attestation ledger, reset boundaries, compliance history, claim-standing consent receipts / transmission ledgers, rejection logs | No TTL — user-owned project files | Delete with your project |
|
|
| Codex transport working dir | A per-call `ars-codex-citation-*` temporary directory | Auth-only ephemeral home, empty working root | Removed automatically when the call returns | Automatic |
|
|
|
|
The tunable numbers above (TTLs, thresholds) are documented where they are set —
|
|
[`SETUP.md`](SETUP.md) § Citation verification cache and § Optional environment flags
|
|
are the user-facing authority for changing them.
|
|
|
|
The suite itself contains no telemetry and no analytics endpoint, and requires no
|
|
account with the ARS project; the inventory above is exhaustive for this repository's
|
|
own scripts as of this revision. A CI lint (`scripts/check_data_flows.py`) holds one
|
|
direction of that mechanically — a script that gains a *direct* network import, or a
|
|
shell script that gains a `curl`, fails CI until it has a row here. Paths that reach
|
|
the network indirectly — through a spawned CLI (the codex transport rows), or through
|
|
the session's own tooling following a protocol doc — are held by review, not by the
|
|
lint.
|
|
|
|
## Related
|
|
|
|
- [`SECURITY.md`](../SECURITY.md) — data exfiltration and credential leakage are
|
|
explicitly in scope for vulnerability reports; anything beyond this map is
|
|
report-worthy.
|
|
- [`THIRD_PARTY.md`](../THIRD_PARTY.md) — community projects around ARS and their own
|
|
policies (this map covers the core suite only).
|
|
- [`docs/SETUP.md`](SETUP.md) — installation, environment flags, and the canonical
|
|
home of the cache and cross-model configuration this map summarizes.
|
|
- [`shared/cross_model_verification.md`](../shared/cross_model_verification.md) — the
|
|
consent boundary and provider table for the cross-model rows.
|
|
- `docs/CONTROL_AVAILABILITY.md` (lands with PR #768) — which of these code paths even
|
|
exist in your install channel.
|