Benchmarks and demos
See a benchmark in use: inspect scientific inputs, check an answer and plan a revision. Below are a molecular walkthrough, a synthetic checker example and proposed research studies.
Early collection. One molecular teaching example and a synthetic checker fixture are available. We are reviewing sources and building the wider collection.
Molecular property-to-structure recovery
MolP2SBench
A known structure becomes a question we can check.
A fragment is masked from a molecular structure. Choose its completion using computed molecular-property evidence, then inspect the reference and plan a diagnostic test.
- 2 observations
- 2 candidates
- 1 teaching family
Each property channel of the original benchmark has 136 source problems, with two to four candidates per task. The teaching pair illustrates its format and one controlled change in evidence.
Xie et al. (2026). Read the original paper · Citation & sources
Which structure fits the evidence?
1.45724 D
5.37590 D
Two questions, the same candidates. Compare each choice with its reference to see where predictions succeed or fail.
Try this exampleOne teaching family · computed dipole magnitudesA minimal checker example
The synthetic oscillator package shows how a task, predictions and a checker fit together. Use it to learn the package format before adapting your own scientific data.
Synthetic teaching fixture. It is separate from the scientific dataset catalog.
Contribute a scientific dataset
Share original data or suggest an existing resource. Source listings are reviewed for provenance, scientific use and access terms; runnable task packages also need checker and scientific review. A paper is optional.
Contribute a datasetProposed studies and improvement areas
Experimental conditions and outcomes
Traceable experimental records could support tests of how an AI planner selects its next experiment. An early source contribution would include conditions, measured outcomes, costs and independent experimental series. Read the proposed study.
- Data-building strategies
Improve how a system selects evidence, constructs scientific cases and finds gaps in its own tests.
- Experiment planning
Improve how a system designs experiments to compare proposed changes to its scientific methods.
- Scientific verification and diagnostics
Improve tools for testing assumptions, diagnosing scientific errors and checking proposed methods.