Prospective · Evidence-grounded · Temporal holdout · Deterministic scoring
Can an AI find a new use for an existing drug — before the outcome is known?
Drug Repurposing and Reformulation (DRR) does not aim to discover new drugs. Instead, it seeks to identify new therapeutic indications and optimized procedure design for existing approved or clinically characterized drugs.
Why it matters
Credible new indications are buried in fragmented evidence
Because repurposed existing compounds often come with existing knowledge of human safety, pharmacology, dosing and manufacturing, repurposing can shorten development timelines, reduce cost and risk, and accelerate treatments for rare diseases and conditions with limited options.
But finding a credible new indication isn’t retrieving a known drug–disease association. The relevant evidence is fragmented across clinical trials, biomedical databases, molecular and genetic studies, safety records, perturbational experiments, and a fast-moving literature. Sources are incomplete or contradictory, and a drug with a compelling mechanism may still fail because of toxicity, inadequate exposure, patient heterogeneity, or poor trial design.
This benchmark tests whether AI systems can reason through that uncertainty and make prospective, evidence-grounded decisions — evaluated against outcomes that only became public after the model’s information cutoff.
100 / 200
A 100-pair public development set for transparent experimentation, plus a 200-pair hidden test set for blind evaluation.
Prospective
Predictions are scored against clinical-trial and regulatory outcomes that surfaced after the prediction cutoff — testing reasoning, not memorization.
Auditable
Every prediction carries a scientific rationale tying it to mechanisms, supporting and contradictory evidence, and unresolved uncertainties.
The task
A prediction, its confidence, and an auditable rationale
For each drug–disease pair, the system can use only information available before a fixed temporal cutoff. It produces three primary outputs.
Primary endpoint
Promisingness score
A value from 0 to 1 representing the pair’s predicted position on the clinical-development and approval spectrum.
Output 2
Confidence estimate
A value from 0 to 1 expressing how strongly the available evidence supports the forecast.
Output 3
Auditable rationale
A scientific explanation connecting the prediction to mechanisms, supporting and contradictory evidence, and open uncertainties.
Separate evaluation
Development procedure
A complementary task evaluates target population, biomarker strategy, combination therapy, and formulation or delivery.
Temporal separation is the point. Predictions are evaluated against clinical-trial and regulatory outcomes that became public only after the cutoff — testing prospective scientific reasoning rather than recall of historical outcomes.
Outcome scale
A clinically meaningful five-tier spectrum
Outcomes map to a fixed, nonuniform scale: 0.00, 0.15, 0.45, 0.65, and 1.00, from preclinical-only or failed Phase I through approval for the target indication.
The scale was fixed from the empirical distribution of 10,427 concluded historical programs; it is not refit for individual systems or runs. Exact labels penalize error symmetrically. Successful unapproved and ongoing programs set a lower bound, so only predictions below the stage already attained are penalized.
The primary score preserves continuous prediction error, then anchors it so that 0 represents the expected knowledge-free random predictor and 100 represents perfect prediction. Categorical accuracy is reported as a complementary nearest-tier measure.
Scoring · spec
PREDICTION
promisingness ∈ [0,1] + confidence + rationale
GROUND TRUTH
5-tier approval spectrum 0.00 → 1.00, post-cutoff outcomes
LABEL SEMANTICS
terminal outcomes are exact · ongoing or successful unapproved programs are lower bounds
NORMALIZATION
0 · random anchor: raw 0.86925
100 · perfect prediction
SETS
100-pair public dev · 200-pair hidden test
Executable environment
Apodex-Env: an executable world for DRR
We call the executable evidence and tool layer Apodex-Env. The Public-100 study compares the same backbone under two matched conditions: a single-response, no-retrieval baseline and iterative use of Apodex-Env.
Apodex-Env adds structured biomedical knowledge, literature-derived evidence, and scientific tools. The backbone, benchmark items, output schema, and scoring stay fixed, and neither condition can access held-out outcomes.
Apodex-Env · isolation
AGENT CANNOT SEE
ground-truth outcomes · private annotations · post-cutoff evidence
RECORDED
full trajectory, for reproducible grading
NO ENV
single response · no retrieval, executable tools, or network access
WITH APODEX-ENV
iterative use of biomedical knowledge, evidence, and scientific tools
Public-100 results · three independent runs
Apodex-Env improves frontier-model performance
Across GPT-5.5 and GPT-5.6-sol, Apodex-Env raises normalized continuous score, categorical accuracy, and raw continuous score. Values below are means over three runs; error terms are sample standard deviations.
Normalized continuous score
RANDOM ANCHORED: 0 = RANDOM PREDICTOR · 100 = ORACLE
GPT-5.5 · no env
53.78
GPT-5.5 · Apodex-Env
56.30
GPT-5.6-sol · no env
54.21
GPT-5.6-sol · Apodex-Env
61.81
0 · random
50
100 · perfect
LARGEST NORMALIZED GAIN
+7.60
GPT-5.6-sol with Apodex-Env
It also reaches the highest observed normalized score and categorical accuracy in the comparison.
Top normalized continuous score
61.81
Top categorical accuracy
85.00%
Top raw continuous score
0.95007
BACKBONE
CONDITION
NORMALIZED SCORE
RAW SCORE
CATEGORICAL ACCURACY
GPT-5.5
No env
53.78 ± 3.22
0.93957 ± 0.00421
81.00 ± 1.73%
With Apodex-Env
56.30 ± 1.56
0.94287 ± 0.00204
83.67 ± 1.15%
GPT-5.6-sol
No env
54.21 ± 2.20
0.94013 ± 0.00287
80.67 ± 1.15%
With Apodex-Env
61.81 ± 3.92
0.95007 ± 0.00512
85.00 ± 1.73%
Interpretation boundary. These are descriptive Public-100 development-set results from three runs, not definitive estimates of hidden-test performance.
HDS6 trajectory analysis · 0–4 scale
Does the reasoning process support the final answer?
HDS6 evaluates six capabilities in recorded reasoning and tool trajectories while keeping the held-out outcome hidden. Scores range from absent (0) through good (3) to excellent (4), with critical-failure caps retained.
T
Tools
Selecting, calling and correctly interpreting external tools
R
Repair
Locating and correcting its own errors once feedback arrives
A
Alternatives
Laying out competing hypotheses and keeping or discarding them as evidence accumulates
C
Coherence
Holding state, constraints and logic intact across a long chain of work
E
Evidence
Grounding every conclusion in observation, data, experiment or citation
S
Scope
Stating the conditions under which a conclusion holds, and where it does not apply
Process profile with Apodex-Env
Cell = Apodex-Env mean score / paired gain over no env · 0 absent · 2 insufficient · 4 excellent
DIMENSION
GPT-5.5
GPT-5.6-SOL
T · Tools
3.64
+2.64
3.56
+2.57
R · Repair
3.45
Apodex-Env only
3.83
Apodex-Env only
A · Alternatives
2.69
+0.34
2.72
+0.25
C · Coherence
2.76
+1.13
2.81
+1.14
E · Evidence
3.10
+1.58
3.10
+1.68
S · Scope
2.96
+0.24
3.09
+0.24
Where Apodex-Env helps most. Tool use, evidence fidelity, and long-horizon coherence show the largest gains across backbones. Repair gains are unavailable because repair was not observable in no-env trajectories. This matched comparison is a post-integrity-gate survivor analysis, so the process findings remain descriptive.
Procedure design · semantic evaluator, 1–5
Structured trial context sharpens patient-population design
The procedure task scores four clinically actionable fields against registered-trial references. The knowledge-augmented condition revises the initial prediction using structured eligibility context, which is available at sufficient coverage only for population specification.
Mean field scores
1 = irrelevant or contradictory · 3 = directionally correct · 5 = key clinical content agrees
LM only
LM + context
Target population
+0.89
LM only
2.89
+ context
3.78
Biomarker strategy
+0.00
LM only
4.00
+ context
4.00
Drug combination
+0.00
LM only
3.75
+ context
3.75
Formulation / delivery
+0.00
LM only
4.22
+ context
4.22
MACRO-AVERAGE
LM only 3.72 → LM + context 3.94 · +0.22
What this comparison measures. Because the retrieved eligibility record also supplies the population reference, the +0.89 population gain reflects evidence retrieval and synthesis rather than a fully prospective procedure-generation test. The other three fields are unchanged between conditions.