* Rewrite the case-studies README around the science, not the verification The README read like a test report. Its first substantive sentence was about matching published values, it carried three Published-vs-Observed tables, and the longest section was forensics on a ChEMBL outage. Someone arriving wants to run a scientific investigation, not audit a regression run. Restructure around what each case actually does: the question, where the evidence turns, and what the workflow concludes when two valid results point opposite ways. Each case now shows the pivotal moment as real output — the DepMap dependency that reframes BLM, the mechanism check that disqualifies nolasiban, the adjusted odds ratio and what it does not support. The verification material stays, but as a short "Reading the output" section: how the published: annotations work, which values are expected to drift and which are not, and the ChEMBL outage in three lines instead of thirty. 233 lines to 151. Every number shown is copied from an actual run. * Restructure the case-studies README as input, process, output The previous rewrite told each case as a narrative, which read better than the test report it replaced but still left the mechanics implicit: what you hand the script, what it does with it, what comes back. Give every case the same three headings. Input states the identifier or the data files and the question. Process is a table of steps with the tools each one calls and what it establishes, with the pivotal step marked. Output is the verdict, with the actual printed result above it. The header carries a diagram of that shape and a summary table, so the pattern is visible before reading any single case. Step and tool counts in the tables are extracted from the scripts rather than written by hand: 7 steps/15 tools, 6 steps/15 tools, 3 steps/1 tool in three modes. Every value quoted comes from a recorded run. * Stop pinning Case 2 to an unstable affinity value ChEMBL came back up, so the two steps that had never run against the live service finally did. Case 1's analog comparison reproduced exactly (DILI 0.987-0.992). Case 2's affinity lookup did not: it reported 0.5 nM against a published "Ki as low as 3.2 nM". The cause is not the data. ChEMBL returns activities unsorted, so the minimum over one page depends on which page you fetch: 0.50 nM at offset 0, 1.0 at offset 100, 0.09 at offset 200. Annotating that number against one published value makes the case appear to fail at random. Report what actually reproduces instead — that the receptor has nanomolar ligands — and label the sampled minimum as page-dependent rather than as a value being matched. The README's ChEMBL note is updated too: it claimed both steps were unconfirmed, which is no longer true. * Find the real lowest Ki for OXTR instead of sampling one page Case 2 reported 0.5 nM where the write-up says "Ki as low as 3.2 nM". Neither is the lowest measured affinity. ChEMBL returns activities unsorted and paginated, so the minimum over any single page is an artefact of which page was fetched: 0.50 nM at offset 0, 1.0 at offset 100, 0.09 at offset 200. ChEMBL holds 812 Ki values in nM for CHEMBL2049. The actual minimum is 0.025 nM (CHEMBL2037514, no validity flag, not a duplicate), and 182 of the 812 sit below the published 3.2 nM. The published figure is understated by roughly 130x because it too came from a partial sample. Page through the whole set rather than one page, restrict to Ki in nM, drop rows carrying a data_validity_comment, and report the count alongside the minimum so the figure is interpretable. The case's conclusion is unchanged, and better supported: OXTR has well-characterised high-affinity ligands.
Three research questions, answered end to end
Each script takes a research question, chains tools across databases, structures, screens and predictive models to answer it, and prints the evidence as it arrives. This is the same sequence the AI scientist ran in the write-up.
Every case has the same shape:
input process output
───────────────────── ────────────────────────────────── ──────────────────
a question + an N steps, each one or more tool a verdict, with
identifier or a file calls, each step's result feeding every number
the next traceable to a call
| Input | Process | Output | |
|---|---|---|---|
| 1 | gene BLM |
7 steps, 15 tools | is it a target, and is ML216 developable? |
| 2 | gene OXTR |
6 steps, 15 tools | is the chemistry right for the autism hypothesis? |
| 3 | 2 trial tables (CSV) | 3 steps, 1 tool in 3 modes | does BCG associate with worse adverse events? |
Setup
python -m venv .venv && source .venv/bin/activate # or: uv venv .venv
pip install 'tooluniverse[ml]' # ml extra = ADMET-AI models
git clone https://github.com/mims-harvard/ToolUniverse.git
cd ToolUniverse/examples/case_studies
pip install -r requirements.txt
No API keys. Every database used — Open Targets, ClinVar, GeneBe, AlphaFold, PDBe, Europe PMC, PubChem, ChEMBL — is open, and ADMET-AI runs locally.
python case1_blm_target_assessment.py # ~2 min, mostly ADMET-AI model load
python case2_oxtr_druggability.py # ~2 min
python download_bcg_data.py # ~4 MB of trial tables from Zenodo, CC0
python case3_bcg_ae_severity.py # seconds
Case 1 — is BLM a target?
Input
gene: BLM (a DNA-repair helicase; losing it causes cancer)
question: can blocking it nevertheless treat cancer,
and is its inhibitor ML216 developable?
Process
| Step | Tools | What it establishes |
|---|---|---|
| 1 Resolve the gene | OpenTargets_multi_entity_search_by_query_string OpenTargets_get_target_gene_ontology_by_ensemblID |
it is a genome-maintenance helicase |
| 2 Cancer-risk genetics | OpenTargets_get_diseases_phenotypes_by_target_ensembl |
464 disease associations, led by Bloom syndrome |
| 3 Classify a variant | ClinVar_search_variants GeneBe_classify_variant |
c.520C>T is pathogenic — evidence against inhibiting |
| 4 Dependency screen | OpenTargets_get_target_depmap_essentiality |
the turn: not pan-essential, but concentrated in specific tumours |
| 5 Structure | alphafold_get_summary PDBe_get_uniprot_structure_coverage |
use experimental structures, not the mixed-confidence prediction |
| 6 Precedent | EuropePMC_search_articles |
paralog WRN is already a validated synthetic-lethal target |
| 7 Triage the chemistry | OpenTargets_get_chemical_probes_by_target_ensemblID PubChem_* ADMETAI_* ChEMBL_search_similar_molecules |
ML216 is drug-like but carries a liability |
Step 4 is why the case exists — it contradicts step 3:
mean gene effect: -0.18
lines dependent at < -0.5: 94/1258 (7.5%)
Most-dependent cancer types (mean gene effect, n>=5 lines):
Mature T and NK Neoplasms -0.47 (n=8)
Cutaneous Squamous Cell Carcinoma -0.44 (n=5)
Output
Not a target — a biomarker-defined hypothesis. The germline genetics say protect BLM; the screen says it is a selective vulnerability in defined contexts. Both are right, so the workflow returns the contexts worth testing rather than a yes/no. ML216 is a probe, not a lead: its predicted liver-injury signal stays near maximum across every close analog, implicating the scaffold rather than the molecule.
Case 2 — does OXTR chemistry fit the hypothesis?
Input
gene: OXTR (oxytocin receptor, a GPCR)
question: is it druggable, and does its pharmacology offer a
starting point for the emerging autism association?
Process
| Step | Tools | What it establishes |
|---|---|---|
| 1 Resolve + structure | OpenTargets_multi_entity_search_by_query_string OpenTargets_get_target_gene_ontology_by_ensemblID alphafold_get_summary PDBe_get_uniprot_structure_coverage |
active and inactive experimental structures both exist |
| 2 Disease landscape | OpenTargets_get_diseases_phenotypes_by_target_ensembl |
reproductive indications, plus an emerging autism link |
| 3 Tractability + drugs | OpenTargets_get_target_tractability_by_ensemblID OpenTargets_get_associated_drugs_by_target_ensemblID |
druggable GPCR, 9 known agents, approved ones are peptides |
| 4 Mechanism | OpenTargets_get_drug_mechanisms_of_action_by_chemblId |
already drugged in both directions |
| 5 Measured affinity | ChEMBL_search_targets ChEMBL_get_target_activities |
nanomolar ligands exist |
| 6 CNS + mechanism | EuropePMC_search_articles PubChem_* ADMETAI_predict_BBB_penetrance OpenTargets_get_drug_mechanisms_of_action_by_chemblId |
the turn: the brain-penetrant candidate acts the wrong way |
Nolasiban passes every property filter — then fails the one that matters:
nolasiban action type: ['ANTAGONIST']
Read: brain-penetrant non-peptide OXTR chemistry is attainable,
but this chemotype BLOCKS the receptor, and the pro-social
hypothesis needs it ACTIVATED. Wrong direction.
Output
Tractable target, wrong chemotype. The objective is stated precisely instead of a repurposing suggestion: a brain-penetrant agonist or positive allosteric modulator, selective over the vasopressin receptors. Screening on affinity and brain penetrance alone would have produced a confident, wrong recommendation.
Case 3 — did BCG worsen adverse events?
Unlike the others, this one computes on raw data rather than querying databases.
Input
data: TASK008_BCG-CORONA_DM.csv demographics, 1000 subjects
TASK008_BCG-CORONA_AE.csv adverse events, 2694 records
(public BCG-CORONA trial, Zenodo 12737228, CC0 — download_bcg_data.py fetches it)
question: is BCG associated with higher adverse-event severity,
after adjusting for how often participants saw patients?
Process
One tool, clinical_trial_ae_severity_test, in three modes:
| Step | Mode | What it does |
|---|---|---|
| 1 | prepare |
inner-join AE onto DM by USUBJID, reduce each subject to their max AESEV → 791 evaluable subjects |
| 2 | chi-square |
unadjusted treatment × severity association |
| 3 | ordinal |
proportional-odds regression adjusting for patients_seen, expect_interact, work_hours |
Output
evaluable subjects: 791
chi-square: 10.12 dof: 3 p-value: 0.018
OR for higher severity with BCG: 1.53
95% CI: (1.16, 2.01)
p-value: 0.0024
A significant association that survives adjustment. Two things the script is careful about, and you should be too:
- Direction. The tool reports the odds ratio against its own reference level (BCG=0, Placebo=1 alphabetically), so the raw 0.65 is Placebo-vs-BCG and the 1.53 above is its reciprocal. Reporting the raw number as the BCG effect would invert the finding.
- What it supports. The cohort is restricted to participants with at least one adverse event, and severity is a derived per-subject maximum. That makes this a reproducible association from a secondary analysis — not evidence that vaccination changes severity.
Reading the output
Every value prints with the originally reported number beside it:
DILI (liver injury): 0.988 (published: 0.99)
So a run tells you at a glance whether the science still holds against live
databases, and exactly which number moved if it doesn't. Values marked
published: come from the write-up or from the supplementary note it condenses
(arXiv:2509.23426) — the note carries the
intermediate ones such as ACMG scores, PDB accessions and per-context DepMap
effects.
Expect some movement, and know which kind matters:
- Live databases get re-scored. OXTR disease associations were 393 at the time of writing and read 465 today; Open Targets re-scores every release. That is a release difference, not a failed reproduction, and no conclusion rests on the count.
- Case 3 should never move. Frozen dataset, deterministic model, identical on pandas 2 and 3. If its numbers change, something in your environment did.
- One published number is understated. The write-up reports OXTR Ki "as low as 3.2 nM". ChEMBL returns activities unsorted and paginated, so a single page gives an arbitrary minimum — 0.50, 1.0 and 0.09 nM at offsets 0, 100 and 200. Case 2 now pages through all 811 Ki measurements and finds the real figure, 0.025 nM, about 130× lower. The conclusion the case draws from it, that OXTR has well-characterised high-affinity ligands, is unaffected — if anything it is stronger.
- ChEMBL's data API goes down periodically (HTTP 500 upstream, known
issue). When
it does, the two steps that need it print
[unavailable]and the case carries on rather than failing. Everything else is unaffected.