Molecular discovery

MolP2SBench leaderboard

Compare seven models across eleven property channels, using results reported in MolP2SBench v1.

This table reproduces the paper’s results. New score submissions are not open.

Property-Grounded Recovery

Within each eligible family, original recovery and mean counterfactual recovery each contribute half. PGR averages these family scores.

Ranks use the paper’s rounded point estimates; confidence intervals are not shown. See the paper’s family-subsampling analysis.

Showing Dipole magnitude: 7 models, ranked by PGR.

Recover a masked fragment using the molecule’s computed dipole magnitude.

Dipole magnitude. Paper-reported Property-Grounded Recovery, percent. Higher is better.
RankModelPGR (%) Higher is better
1GPT-6-AstraClosed weights
69.4 percent
2GPT-5.6-SolClosed weights
64.6 percent
3GPT-5.6-LunaClosed weights
62.2 percent
4Claude Opus 5Closed weights
59.7 percent
5Gemma-4-31B-itOpen weights
45.5 percent
6Qwen3.8-27BOpen weights
44.2 percent
7ChemDFM-v2.0-14BOpen weights
37.5 percent
Uniform random37.9%Expected score when choosing among the available candidates at random.
Scoring and evaluation

PGR gives equal weight to original and counterfactual recovery. For each molecular family, score the original question and average the scores of all alternative-candidate substitutions. Average these two quantities, then average across eligible families.

Models share the same input-validated, complete-coverage families within each channel. Invalid terminal answers count as failures. The paper evaluates models without tools, retrieval, task-specific training or solved examples.

Ranks follow the reported one-decimal point estimates; matching values share a rank. Small differences require uncertainty analysis. Each channel has its own ranking and a fixed 0–100% scale.

These are the source paper’s model evaluations. For RSI studies, record how a revised improvement process is reused. Compare paired results on independent cases under matched budgets, including a fixed-improver control. Evaluation protocol

From a score to a better solver

Use the benchmark’s controlled cases to investigate weak points, then test a revision on fresh molecular families.