Realistic simulators for spatial transcriptomics

New methods for spatial transcriptomics are often developed and tested on simulated data, so those tests are only as good as the simulator. This benchmark asks for a simulator whose synthetic data match real spatial datasets, judged on many statistical properties and on how downstream analyses behave.

Task

A method receives a real spatial transcriptomics dataset: counts per spot or cell, with spatial coordinates. It must return a simulated dataset of the same form. The simulated data are compared with the real data, and with a matched single-cell reference, on 23 metrics: distributions of gene- and spot-level statistics, spatial statistics, and the results of downstream tasks run on real versus simulated data.

Background

Simulation lets developers test methods on data with known structure, but many simulators were built for dissociated single-cell data and ignore space. SpatialSimBench (Liang et al., 2025) assessed 13 simulation methods on ten spatial datasets and introduced simAdaptor, which extends single-cell simulators with spatial variables. Its authors, Xiaoqi Liang, Yue Cao and Jean Yang, brought the framework to Open Problems as a living benchmark.

Data and evaluation

• Data: ten public spatial datasets, each paired with a single-cell reference of similar tissue: mouse brain, olfactory bulb, gastrulation and regenerating hindlimb muscle, a mouse fibrosarcoma sample from the DestVI study, a seqFISH+ cortex dataset, and human breast cancer, osteosarcoma, pancreatic ductal adenocarcinoma and prostate. Files are in s3://openproblems-data/resources/task_spatial_simulators/datasets/. The task code is MIT-licensed; dataset licences are not restated on the task page.
• Split: there is no train/test split; each simulator is fitted to and compared with the same real dataset.
• Metric: 23 metrics in release v0.2.0. Most are kernel-density two-sample statistics (ks::kde.test, lower is better) comparing real and simulated distributions of, for example, library size, fraction of zeros, mean–variance relation, gene–gene correlation, Moran's I and nearest-neighbour correlation. The rest score downstream tasks: cell-type deconvolution error (Jensen–Shannon divergence, RMSE), spatially variable gene recall and precision, and spatial cross-correlation (cosine, Mantel). Each is rescaled per dataset between the worst and best control method (a positive control and two negative controls that shuffle the data or draw normal noise), $s=\min\{1,\max\{0,(x-c_{\text{worst}})/(c_{\text{best}}-c_{\text{worst}})\}\}$, and the overall score is the mean over the 10 × 23 pairs, with failed runs counted as 0. The leaderboard shows it to two decimals.
• Code: openproblems-bio/task_spatial_simulators.

What is known

On the v0.2.0 leaderboard (as of 2026-10-20):

• SRTsim, a simulator built for spatial data, leads at 0.90 (0.9021 before rounding, recomputed from the published results).
• scDesign3 with a negative binomial model follows at 0.84, then scDesign2 (0.79), zinbwave (0.77) and SPARsim (0.75).
• Splatter (0.59) and SymSim (0.57), two single-cell simulators, come last.
• The positive control scores 0.98 and the shuffling control 0.41, which bounds the room for improvement.

What would count as progress

• An overall score above 0.9021 on the v0.2.0 datasets and metrics, computed with the task's run_benchmark workflow, with code that reproduces it.
• A simulator that beats SRTsim on the downstream-task metrics (deconvolution, spatially variable genes, cross-correlation), which show how the simulated data behave when used to test analysis methods.
• A gain that holds on each of the ten datasets, not only on average.

References

1. Liang X, Torkel M, Cao Y, Yang JYH. Multi-task benchmarking of spatially resolved gene expression simulation models. Genome Biology 26, 57 (2025). doi:10.1186/s13059-025-03505-w
2. Zhu J, Shang L, Zhou X. SRTsim: spatial pattern preserving simulations for spatially resolved transcriptomics. Genome Biology 24, 39 (2023). doi:10.1186/s13059-023-02879-z
3. Song D, Wang Q, Yan G, et al. scDesign3 generates realistic in silico data for multimodal single-cell and spatial omics. Nature Biotechnology 42, 247–252 (2024). doi:10.1038/s41587-023-01772-1
4. Luecken MD, Gigante S, Burkhardt DB, Cannoodt R, et al. Defining and benchmarking open problems in single-cell analysis. Nature Biotechnology 43, 1035–1040 (2025). doi:10.1038/s41587-025-02694-w

Source

Imported from Open Problems in Single-Cell Analysis: Spatial Simulators (task_spatial_simulators), curated by the Open Problems in Single-Cell Analysis consortium. Open according to the source as of 2026-10-20 (leaderboard release v0.2.0). The statement above is written in our own words; see the source for the original wording, history and discussion.